Cargando…
Adaptive visual–tactile fusion recognition for robotic operation of multi-material system
The use of robots in various industries is evolving from mechanization to intelligence and precision. These systems often comprise parts made of different materials and thus require accurate and comprehensive target identification. While humans perceive the world through a highly diverse perceptual...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
Frontiers Media S.A.
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10318164/ https://www.ncbi.nlm.nih.gov/pubmed/37408585 http://dx.doi.org/10.3389/fnbot.2023.1181383 |
_version_ | 1785067978473078784 |
---|---|
author | Ding, Zihao Chen, Guodong Wang, Zhenhua Sun, Lining |
author_facet | Ding, Zihao Chen, Guodong Wang, Zhenhua Sun, Lining |
author_sort | Ding, Zihao |
collection | PubMed |
description | The use of robots in various industries is evolving from mechanization to intelligence and precision. These systems often comprise parts made of different materials and thus require accurate and comprehensive target identification. While humans perceive the world through a highly diverse perceptual system and can rapidly identify deformable objects through vision and touch to prevent slipping or excessive deformation during grasping, robot recognition technology mainly relies on visual sensors, which lack critical information such as object material, leading to incomplete cognition. Therefore, multimodal information fusion is believed to be key to the development of robot recognition. Firstly, a method of converting tactile sequences to images is proposed to deal with the obstacles of information exchange between different modalities for vision and touch, which overcomes the problems of the noise and instability of tactile data. Subsequently, a visual-tactile fusion network framework based on an adaptive dropout algorithm is constructed, together with an optimal joint mechanism between visual information and tactile information established, to solve the problem of mutual exclusion or unbalanced fusion in traditional fusion methods. Finally, experiments show that the proposed method effectively improves robot recognition ability, and the classification accuracy is as high as 99.3%. |
format | Online Article Text |
id | pubmed-10318164 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | Frontiers Media S.A. |
record_format | MEDLINE/PubMed |
spelling | pubmed-103181642023-07-05 Adaptive visual–tactile fusion recognition for robotic operation of multi-material system Ding, Zihao Chen, Guodong Wang, Zhenhua Sun, Lining Front Neurorobot Neuroscience The use of robots in various industries is evolving from mechanization to intelligence and precision. These systems often comprise parts made of different materials and thus require accurate and comprehensive target identification. While humans perceive the world through a highly diverse perceptual system and can rapidly identify deformable objects through vision and touch to prevent slipping or excessive deformation during grasping, robot recognition technology mainly relies on visual sensors, which lack critical information such as object material, leading to incomplete cognition. Therefore, multimodal information fusion is believed to be key to the development of robot recognition. Firstly, a method of converting tactile sequences to images is proposed to deal with the obstacles of information exchange between different modalities for vision and touch, which overcomes the problems of the noise and instability of tactile data. Subsequently, a visual-tactile fusion network framework based on an adaptive dropout algorithm is constructed, together with an optimal joint mechanism between visual information and tactile information established, to solve the problem of mutual exclusion or unbalanced fusion in traditional fusion methods. Finally, experiments show that the proposed method effectively improves robot recognition ability, and the classification accuracy is as high as 99.3%. Frontiers Media S.A. 2023-06-20 /pmc/articles/PMC10318164/ /pubmed/37408585 http://dx.doi.org/10.3389/fnbot.2023.1181383 Text en Copyright © 2023 Ding, Chen, Wang and Sun. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. |
spellingShingle | Neuroscience Ding, Zihao Chen, Guodong Wang, Zhenhua Sun, Lining Adaptive visual–tactile fusion recognition for robotic operation of multi-material system |
title | Adaptive visual–tactile fusion recognition for robotic operation of multi-material system |
title_full | Adaptive visual–tactile fusion recognition for robotic operation of multi-material system |
title_fullStr | Adaptive visual–tactile fusion recognition for robotic operation of multi-material system |
title_full_unstemmed | Adaptive visual–tactile fusion recognition for robotic operation of multi-material system |
title_short | Adaptive visual–tactile fusion recognition for robotic operation of multi-material system |
title_sort | adaptive visual–tactile fusion recognition for robotic operation of multi-material system |
topic | Neuroscience |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10318164/ https://www.ncbi.nlm.nih.gov/pubmed/37408585 http://dx.doi.org/10.3389/fnbot.2023.1181383 |
work_keys_str_mv | AT dingzihao adaptivevisualtactilefusionrecognitionforroboticoperationofmultimaterialsystem AT chenguodong adaptivevisualtactilefusionrecognitionforroboticoperationofmultimaterialsystem AT wangzhenhua adaptivevisualtactilefusionrecognitionforroboticoperationofmultimaterialsystem AT sunlining adaptivevisualtactilefusionrecognitionforroboticoperationofmultimaterialsystem |