Simulation of Vision-based Tactile Sensors with Efficiency-tunable Rendering DOI
Yuhao Sun, Shixin Zhang, Jianhua Shan

et al.

2021 IEEE International Conference on Robotics and Biomimetics (ROBIO), Journal Year: 2023, Volume and Issue: unknown, P. 1 - 6

Published: Dec. 4, 2023

Vision-based tactile sensors (VBTS) leverages visual modality to present high-resolution information. The vision-based sensing mechanism has good adaptability robot manipulation because of available information capture and data processing. However, the simulation VBTS is a challenging problem it deals with computer graphics elasto-plastic deformation. In this paper, we propose approach for using rendering based on path tracing. method can be integrated existing robotic simulators systems. addition, control efficiency quality by controlling number voxels in fitting deformation area meet requirements efficient training. experimental results verify that our provide high-quality images at low or reduce image improve efficiency. Our work potential advance sim-to-real research VBTS.

Language: Английский

TIRgel: A Visuo-Tactile Sensor With Total Internal Reflection Mechanism for External Observation and Contact Detection DOI
Shixin Zhang, Yuhao Sun, Jianhua Shan

et al.

IEEE Robotics and Automation Letters, Journal Year: 2023, Volume and Issue: 8(10), P. 6307 - 6314

Published: Aug. 18, 2023

This letter proposes a vision-based tactile sensor named TIRgel, leveraging visual integration to simplify the sensing system. First, achieves conversion of and modality via focus adjustment. Under far-focus imaging, camera can observe external environments; near-focus we perform total internal reflection (TIR) within lenticular elastomer visualize features. The effectiveness mechanism is verified MPM (Material Point Method) simulation. Second, TIRgel capture extract micron-scale details based on luminosity information. Tactile imaging has natural fusion information, where boundary reflects texture feature, background contains object's appearance Inherent data simplifies multimodal acquisition reduces information processing difficulty. Finally, resist interference ambient brightness illumination higher adaptive capacity environment than unidirectional perspective VBTSs. We believe that it potential provide visual-tactile for robot operation.

Language: Английский

Citations

23

PFS 1.0: A development tool applied to vision-based tactile sensor process formulation and fabrication DOI
Shixin Zhang, Yuhao Sun, Fuchun Sun

et al.

Sensors and Actuators A Physical, Journal Year: 2024, Volume and Issue: 367, P. 115090 - 115090

Published: Jan. 30, 2024

Language: Английский

Citations

9

A Compact Visuo-Tactile Robotic Skin for Micron-Level Tactile Perception DOI
Shixin Zhang, Yiyong Yang, Fuchun Sun

et al.

IEEE Sensors Journal, Journal Year: 2024, Volume and Issue: 24(9), P. 15273 - 15282

Published: March 18, 2024

Visuo-tactile skins (VTSs) are classified into coated-type, marker-type, and thermochromic-type. Coating wear resistance spatial resolution central issues for the coated-type VTS. This article proposes a wire-drawing process applied to VTS enhance coating robustness. The has more stable mechanical properties, including high adhesion, adaptive deformation, rupture resistance. Under different degrees of it can remain continuous undamaged. In addition, nanoscale particle distribution, whose microstructures similar spraying provide micron-scale resolution. Via test, higher than coating. Further, combined with multitask learning model identify textures clothing properties 18 fabrics 98% recognition accuracy. results show that using new highly learnable tactile data.

Language: Английский

Citations

7

PaLmTac: A Vision-Based Tactile Sensor Leveraging Distributed-Modality Design and Modal-Matching Recognition for Soft Hand Perception DOI
Shixin Zhang, Yiyong Yang, Jianhua Shan

et al.

IEEE Journal of Selected Topics in Signal Processing, Journal Year: 2024, Volume and Issue: 18(3), P. 288 - 298

Published: April 1, 2024

This paper proposes a vision-based tactile sensor (VBTS) embedded into the soft hand palm, named PaLmTac. We adopt distributed modality design instead of overlaying function layers. On one hand, problem unrelated integration (texture and temperature) is solved. other combining regional recognition can avoid mixed information. Herein, Level-Regional Feature Extraction Network (LRFE-Net) presented to match design. leverage feature mapping, convolution, vectorization construct mechanism, which extract features in parallel control fusion degrees. The level mechanism balances learning difficulty each modality. Compared with existing VBTSs, PaLmTac optimizes reduces interference. provides novel idea multimodal VBTS sensing expected be applied human-computer interaction scenarios based on fusion.

Language: Английский

Citations

6

When Vision Meets Touch: A Contemporary Review for Visuotactile Sensors From the Signal Processing Perspective DOI
Shoujie Li, Zihan Wang, Changsheng Wu

et al.

IEEE Journal of Selected Topics in Signal Processing, Journal Year: 2024, Volume and Issue: 18(3), P. 267 - 287

Published: April 1, 2024

Tactile sensors, which provide information about the physical properties of objects, are an essential component robotic systems.The visuotactile sensing technology with merits high resolution and low cost has facilitated development robotics from environment exploration to dexterous operation.Over years, several reviews on sensors for robots have been presented, but few them discussed significance signal processing methods sensors.Apart ingenious hardware design, full potential sensory system toward designated tasks can only be released appropriate methods.Therefore, this paper provides a comprehensive review perspective outlooks possible future research directions sensors.

Language: Английский

Citations

4

Application of Image Sensing System in Mineral/Rock Identification: Sensing Mode and Information Process DOI Creative Commons
Shixin Zhang, Yiyong Yang, Fuchun Sun

et al.

Advanced Intelligent Systems, Journal Year: 2023, Volume and Issue: 5(11)

Published: Aug. 17, 2023

Artificial intelligence promotes the development of mineral/rock automatic recognition technology to reduce labor costs and reliance on personal experience. With instrument digitization, image has played an increasingly important role. This review focuses sensing system applied identification redivides this into a module, imaging information processing module. The mechanisms are summarized via technologies sensing, imaging, display. is classified single‐modal or multimodal recognition. analyzes limitations discusses advantages significance challenges current concluded, such as public database, learning, portable system. interdisciplinary discussed, application feasibility explored. aims provide reference for research community in optimization.

Language: Английский

Citations

10

A braille detection system based on visuo-tactile sensing DOI
Shixin Zhang,

J. H. Du,

Yuhao Sun

et al.

Measurement, Journal Year: 2025, Volume and Issue: unknown, P. 116827 - 116827

Published: Jan. 1, 2025

Language: Английский

Citations

0

Tactile data generation and applications based on visuo-tactile sensors: A review DOI
Yuhao Sun, Ning Cheng, Shixin Zhang

et al.

Information Fusion, Journal Year: 2025, Volume and Issue: unknown, P. 103162 - 103162

Published: April 1, 2025

Language: Английский

Citations

0

MagicTac: A Novel High-Resolution 3D Multi-layer Grid-Based Tactile Sensor DOI
Wen Fan, Haoran Li,

Dandan Zhang

et al.

Published: May 13, 2024

Language: Английский

Citations

3

Skin-inspired multimodal tactile sensor aiming at smart space extravehicular multi-finger operations DOI

Kaixuan Kong,

Linfeng Wang, Haodong Wu

et al.

Chemical Engineering Journal, Journal Year: 2024, Volume and Issue: 498, P. 154870 - 154870

Published: Aug. 15, 2024

Language: Английский

Citations

3