Vision-based tactile sensing: From performance parameters to device design DOI

Yi‐Hang Xin,

Kai‐Ming Hu,

Rui-Jia Xiang

et al.

Applied Physics Reviews, Journal Year: 2025, Volume and Issue: 12(2)

Published: April 28, 2025

By integrating the virtues of vision and touch, vision-based tactile sensors (VBTSs) achieve an artificial capability that transcends natural, demonstrating superior performance unattainable through either sense alone. VBTS, as innovative sensor, boasts commendable metrics has found extensive applications across various domains. Nevertheless, a comprehensive synthesis regarding perceptual achievable by VBTS is currently lacking. Moreover, parameter evaluation systems for are not standardized, strategies enhancing these remain unclear. Here, significant advancements in over recent years summarized, from sensing mechanisms to application scenarios. The review particularly focuses on parameters assess novel hardware design aimed at improving parameters, including key indicators (e.g., range, spatial resolution, sensitivity), along with error, temporal miniaturization, stability. This also discusses scenarios, such item identification, grasp control, material property detection, multimodal perception. Finally, perspectives provided. We expect this enables researchers rapidly comprehend capabilities offers references selection or its different modules.

Language: Английский

Marker Displacement Method Used in Vision-Based Tactile Sensors—From 2-D to 3-D: A Review DOI
Mingxuan Li, Tiemin Li, Yao Jiang

et al.

IEEE Sensors Journal, Journal Year: 2023, Volume and Issue: 23(8), P. 8042 - 8059

Published: March 15, 2023

The vision-based tactile sensor has been proven to be a promising device for sensing information. Among such sensors, the marker displacement method (MDM) is most common used in sensors representing and extracting contact It uses position field of array characterize original information further achieves multimodal perception through processing. This article first classify MDM into three typical categories based on dimensionality perspective: 2-D MDM, 2.5-D 3-D MDM. A comparison study presented with focus principles, characteristics, applications, distinctions these methods. latest literature also researched as arguments. Finally, summary helpful reference.

Language: Английский

Citations

32
TIRgel: A Visuo-Tactile Sensor With Total Internal Reflection Mechanism for External Observation and Contact Detection DOI
Shixin Zhang, Yuhao Sun, Jianhua Shan

et al.

IEEE Robotics and Automation Letters, Journal Year: 2023, Volume and Issue: 8(10), P. 6307 - 6314

Published: Aug. 18, 2023

This letter proposes a vision-based tactile sensor named TIRgel, leveraging visual integration to simplify the sensing system. First, achieves conversion of and modality via focus adjustment. Under far-focus imaging, camera can observe external environments; near-focus we perform total internal reflection (TIR) within lenticular elastomer visualize features. The effectiveness mechanism is verified MPM (Material Point Method) simulation. Second, TIRgel capture extract micron-scale details based on luminosity information. Tactile imaging has natural fusion information, where boundary reflects texture feature, background contains object's appearance Inherent data simplifies multimodal acquisition reduces information processing difficulty. Finally, resist interference ambient brightness illumination higher adaptive capacity environment than unidirectional perspective VBTSs. We believe that it potential provide visual-tactile for robot operation.

Language: Английский

Citations

23
PFS 1.0: A development tool applied to vision-based tactile sensor process formulation and fabrication DOI
Shixin Zhang, Yuhao Sun, Fuchun Sun

et al.

Sensors and Actuators A Physical, Journal Year: 2024, Volume and Issue: 367, P. 115090 - 115090

Published: Jan. 30, 2024

Language: Английский

Citations

9
A practical tactile exploration with enhanced sampling for fast shape estimation DOI

Feng Luan,

Chengjin Wang, Zhipeng Wang

et al.

Measurement, Journal Year: 2025, Volume and Issue: unknown, P. 117350 - 117350

Published: March 1, 2025

Language: Английский

Citations

1
Tacchi: A Pluggable and Low Computational Cost Elastomer Deformation Simulator for Optical Tactile Sensors DOI
Zixi Chen, Shixin Zhang, Shan Luo

et al.

IEEE Robotics and Automation Letters, Journal Year: 2023, Volume and Issue: 8(3), P. 1239 - 1246

Published: Jan. 16, 2023

Simulation is widely applied in robotics research to save time and resources. There have been several works simulate optical tactile sensors that leverage either a smoothing method or Finite Element Method (FEM). However, elastomer deformation physics not considered the former method, whereas latter requires massive amount of computational resources like computer cluster. In this work, we propose pluggable low cost simulator using Taichi programming language for simulating sensors, named as Tacchi . It reconstructs particles, which allows deformed surfaces be rendered into images reveals contact information without suffering from high costs. Tacchi facilitates creating realistic simulation, e.g., ones capture wear-and-tear defects on object surfaces. addition, proposed can integrated with simulators robot system simulation. Experiment results showed produce better similarity real achieved higher Sim2Real accuracy compared existing methods. Moreover, it connected MuJoCo Gazebo only requirement 1G memory space GPU cluster FEM. With Tacchi, physical simulation becomes possible.

Language: Английский

Citations

22
Simulation, Learning, and Application of Vision-Based Tactile Sensing at Large Scale DOI Creative Commons
Quan Khanh Luu, Nhan Huu Nguyen, Van Anh Ho

et al.

IEEE Transactions on Robotics, Journal Year: 2023, Volume and Issue: 39(3), P. 2003 - 2019

Published: Feb. 28, 2023

Large-scale robotic skin with tactile sensing ability is emerging the potential for use in close-contact human–robot systems. Although recent developments vision-based and related learning methods are promising, they have been mostly designed small-scale use, such as by fingers hands, manipulation tasks. Moreover, perception devices demands a huge dataset, which complicates data collection process. To address this, this study introduces multiphysics simulation pipeline, called SimTacLS , considers not only mechanical properties of external physical contact but also realistic rendering images environment. The system utilizes obtained including virtual deformation, to train deep neural network extract high-level information. we adopt generative minimize sim2real inaccuracy, preserving simulation-based performance. Last least, showcase method our large-scale sensor ( xmlns:xlink="http://www.w3.org/1999/xlink">TacLink ) demonstrating its two trial cases, namely, whole-arm nonprehensile intuitive motion guidance, using custom-built robot arm integrated TacLink. This article opens new possibilities transferable tactile-driven robotics tasks from worlds actual scenarios without compromising accuracy.

Language: Английский

Citations

21
A Novel Vision-Based Tactile Sensor Using Lamination and Gilding Process for Improvement of Outdoor Detection and Maintainability DOI
Shixin Zhang, Yiyong Yang, Jianhua Shan

et al.

IEEE Sensors Journal, Journal Year: 2023, Volume and Issue: 23(4), P. 3558 - 3566

Published: Jan. 4, 2023

This article presents a novel vision-based tactile sensor (VBTS) using pioneering process route to solve the problem of high cost and low maintainability. We adopt acrylic double-sided adhesive prepare high-quality elastomer through lamination metal foil fabricate semimirror coating gilding process. Our scheme does not rely on any equipment enabling hardware be fabricated or replaced manually both indoors outdoors. Compared with current preparation schemes, preparation/repair an takes only 10/5 min costs 0.055/0.035 USD. Through imaging tests, can obtain high-resolution mapping comparable that representative VBTSs. Depending high-efficiency preparation, we leverage advantages compensate for lack durability. work also explores feasibility outdoor detection. apply it detect fossils first time establish deep-learning model achieves 100% accuracy in identifying six kinds paleontological plant fossils. The results show our has potential application

Language: Английский

Citations

19
Visuo-tactile sensor development and its application for non-destructive measurement of peach firmness DOI

Chan Ma,

Yibin Ying, Lijuan Xie

et al.

Computers and Electronics in Agriculture, Journal Year: 2024, Volume and Issue: 218, P. 108709 - 108709

Published: Feb. 8, 2024

Language: Английский

Citations

8
A Compact Visuo-Tactile Robotic Skin for Micron-Level Tactile Perception DOI
Shixin Zhang, Yiyong Yang, Fuchun Sun

et al.

IEEE Sensors Journal, Journal Year: 2024, Volume and Issue: 24(9), P. 15273 - 15282

Published: March 18, 2024

Visuo-tactile skins (VTSs) are classified into coated-type, marker-type, and thermochromic-type. Coating wear resistance spatial resolution central issues for the coated-type VTS. This article proposes a wire-drawing process applied to VTS enhance coating robustness. The has more stable mechanical properties, including high adhesion, adaptive deformation, rupture resistance. Under different degrees of it can remain continuous undamaged. In addition, nanoscale particle distribution, whose microstructures similar spraying provide micron-scale resolution. Via test, higher than coating. Further, combined with multitask learning model identify textures clothing properties 18 fabrics 98% recognition accuracy. results show that using new highly learnable tactile data.

Language: Английский

Citations

6
Digital twin-enabled grasp outcomes assessment for unknown objects using visual-tactile fusion perception DOI

Zhuangzhuang Zhang,

Zhinan Zhang, Lihui Wang

et al.

Robotics and Computer-Integrated Manufacturing, Journal Year: 2023, Volume and Issue: 84, P. 102601 - 102601

Published: June 10, 2023

Language: Английский

Citations

13