Teleoperation system for multiple robots with intuitive hand recognition interface DOI Creative Commons

Lucas Alexandre Zick,

Dieisson Martinelli,

André Schneider de Oliveira

и другие.

Scientific Reports, Год журнала: 2024, Номер 14(1)

Опубликована: Дек. 4, 2024

Abstract Robotic teleoperation is essential for hazardous environments where human safety at risk. However, efficient and intuitive human–machine interaction multi-robot systems remains challenging. This article aims to demonstrate a robotic system, denominated AutoNav, centered around autonomous navigation gesture commands interpreted through computer vision. The central focus on recognizing the palm of hand as control interface facilitate in context multi-robots. MediaPipe framework was integrated implement recognition from USB camera. system developed using Robot Operating System, employing simulated environment that includes Gazebo RViz applications with multiple TurtleBot 3 robots. main results show reduction approximately 50% execution time, coupled an increase free time during teleoperation, reaching up 94% total time. Furthermore, there decrease collisions. These effectiveness practicality algorithm, showcasing its promise managing teleoperations across study fills knowledge gap by developing gesture-based more safer teleoperation. findings enhance complex operations. A video showing working available https://youtu.be/94S4nJ3IwUw .

Язык: Английский

Network Latency in Teleoperation of Connected and Autonomous Vehicles: A Review of Trends, Challenges, and Mitigation Strategies DOI Creative Commons
Sidharth Bhanu Kamtam, Qian Lu, Faouzi Bouali

и другие.

Sensors, Год журнала: 2024, Номер 24(12), С. 3957 - 3957

Опубликована: Июнь 18, 2024

With remarkable advancements in the development of connected and autonomous vehicles (CAVs), integration teleoperation has become crucial for improving safety operational efficiency. However, faces substantial challenges, with network latency being a critical factor influencing its performance. This survey paper explores impact along state-of-the-art mitigation/compensation approaches. It examines cascading effects on communication links (i.e., uplink downlink) how delays data transmission affect real-time perception decision-making operators. By elucidating challenges available mitigation strategies, offers valuable insights researchers, engineers, practitioners working towards seamless evolving landscape CAVs.

Язык: Английский

Процитировано

5
Comparing dynamic viewpoint control techniques for teleoperated robotic welding in construction DOI
Sungboo Yoon, Moonseo Park, Changbum R. Ahn

и другие.

Automation in Construction, Год журнала: 2025, Номер 172, С. 106053 - 106053

Опубликована: Фев. 16, 2025

Язык: Английский

Процитировано

0
Estimating strawberry weight for grading by picking robot with point cloud completion and multimodal fusion network DOI Creative Commons
Yiming Chen, Wei Wang, Junchao Chen

и другие.

Scientific Reports, Год журнала: 2025, Номер 15(1)

Опубликована: Апрель 2, 2025

Strawberry grading by picking robots can eliminate the manual classification, reducing labor costs and minimizing damage to fruit. size or weight is a key factor in grading, with accurate estimation being crucial for proper classification. In this paper, we collected 1521 sets of strawberry RGB-D images using depth camera manually measured strawberries construct training dataset regression model. To address issue incomplete caused environmental interference cameras, study proposes multimodal point cloud completion method specifically designed symmetrical objects, leveraging RGB guide same scene. The follows process locating pixel regions, calculating centroid coordinates, determining symmetry axis via PCA, completing image. Based on approach, fusion model estimation, named MMF-Net, developed. uses completed image as inputs, extracts features from EfficientNet PointNet, respectively. These are then integrated at feature level through gradient blending, realizing combination strengths both modalities. Using Percent Correct Weight (PCW) metric evaluation standard, compares performance four traditional machine learning methods, Support Vector Regression (SVR), Multilayer Perceptron (MLP), Linear Regression, Random Forest cloud-based deep models, PointNet++, PointMLP, Point Cloud Transformer, well an image-based model, ResNet, single-modal datasets. results indicate that among SVR achieved best accuracy 77.7% ([email protected]). Among obtained highest accuracy, reaching 85% ([email protected]), while PointNet + demonstrated 54.3% proposed 87.66% significantly outperforming methods models terms precision.

Язык: Английский

Процитировано

0
Neural dynamics of delayed feedback in robot teleoperation: insights from fNIRS analysis DOI Creative Commons
Tianyu Zhou, Yang Ye, Qi Zhu

и другие.

Frontiers in Human Neuroscience, Год журнала: 2024, Номер 18

Опубликована: Июнь 17, 2024

As robot teleoperation increasingly becomes integral in executing tasks distant, hazardous, or inaccessible environments, operational delays remain a significant obstacle. These delays, inherent signal transmission and processing, adversely affect operator performance, particularly requiring precision timeliness. While current research has made strides mitigating these through advanced control strategies training methods, crucial gap persists understanding the neurofunctional impacts of efficacy countermeasures from cognitive perspective. This study addresses by leveraging functional Near-Infrared Spectroscopy (fNIRS) to examine implications simulated haptic feedback on activity motor coordination under delayed conditions. In human-subject experiment (N = 41), sensory was manipulated observe its influences various brain regions interest (ROIs) during tasks. The fNIRS data provided detailed assessment cerebral activity, ROIs implicated time perception execution precise movements. Our results reveal that anchoring condition, which immediate with visual cue, significantly optimized neural functions related coordination. condition also improved performance compared asynchronous where were misaligned. findings provide empirical evidence about basis enhanced synthetic force presence delays. highlights potential for mitigate adverse effects thereby improving critical applications.

Язык: Английский

Процитировано

1
Exploratory study on time-delayed excavator teleoperation in virtual lunar construction simulation: Task performance and operator behavior DOI

Miran Seo,

Samraat Gupta,

Youngjib Ham

и другие.

Automation in Construction, Год журнала: 2024, Номер 168, С. 105871 - 105871

Опубликована: Ноя. 12, 2024

Язык: Английский

Процитировано

1
Motion capture and AR based programming by demonstration for industrial robots using handheld teaching device DOI Creative Commons
Guoliang Liu,

Wenlei Sun,

Pinwen Li

и другие.

Scientific Reports, Год журнала: 2024, Номер 14(1)

Опубликована: Окт. 6, 2024

In the industrial robots field, efficient and convenient programming methods have been a hot research topic. recent years, immersive simulation technology has developing rapidly in many fields, which provides new horizons for development of robots. This paper presents HTC VIVE laser scan motion capture Holohens augmented reality (AR) based interactive Programming by Demonstration (PbD) system robot. A portable Handheld Teaching Device (HTD) its calibration algorithm are designed system. The HTD is tracked can be viewed as an AR robot end-effector to teach paths. Meanwhile, simulated real time during programing. addition, reproducing operator's actions at same position space focus programming. So, Multi-system registration proposed determine relationship between systems, systems virtual systems. path planning convert captured raw points into robot-executable code. For unskilled operators, they easily perform complex using HTD. skilled senior workers, their skills quickly learned

Язык: Английский

Процитировано

0
Brain functional connectivity under teleoperation latency: a fNIRS study DOI Creative Commons
Ye Yang, Tianyu Zhou, Qi Zhu

и другие.

Frontiers in Neuroscience, Год журнала: 2024, Номер 18

Опубликована: Ноя. 13, 2024

Introduction Long-distance robot teleoperation faces high latencies that pose cognitive challenges to human operators. Latency between command, execution, and feedback in can impair performance affect operators’ mental state. The neural underpinnings of these effects are not well understood. Methods This study aims understand the impact latency related mitigation methods, using functional Near-Infrared Spectroscopy (fNIRS) analyze connectivity. A subject experiment ( n = 41) a simulated remote manipulation task was performed. Three conditions were tested: no latency, with visual haptic latency. fNIRS data recorded analyzed. Results presence significantly increased connectivity within prefrontal motor cortexes. Maintaining while providing real-time reduced average all cortical networks showed different ratio networks. results worst all-delayed condition best condition, which echoes activity patterns. Conclusion provides neurological evidence increases load, anxiety, motion planning control. Real-time feedback, however, positively influences pathways cognition, decision-making, sensorimotor processes. research inform design ergonomic systems mitigate

Язык: Английский

Процитировано

0
Teleoperation system for multiple robots with intuitive hand recognition interface DOI Creative Commons

Lucas Alexandre Zick,

Dieisson Martinelli,

André Schneider de Oliveira

и другие.

Scientific Reports, Год журнала: 2024, Номер 14(1)

Опубликована: Дек. 4, 2024

Abstract Robotic teleoperation is essential for hazardous environments where human safety at risk. However, efficient and intuitive human–machine interaction multi-robot systems remains challenging. This article aims to demonstrate a robotic system, denominated AutoNav, centered around autonomous navigation gesture commands interpreted through computer vision. The central focus on recognizing the palm of hand as control interface facilitate in context multi-robots. MediaPipe framework was integrated implement recognition from USB camera. system developed using Robot Operating System, employing simulated environment that includes Gazebo RViz applications with multiple TurtleBot 3 robots. main results show reduction approximately 50% execution time, coupled an increase free time during teleoperation, reaching up 94% total time. Furthermore, there decrease collisions. These effectiveness practicality algorithm, showcasing its promise managing teleoperations across study fills knowledge gap by developing gesture-based more safer teleoperation. findings enhance complex operations. A video showing working available https://youtu.be/94S4nJ3IwUw .

Язык: Английский

Процитировано

0