Cargando…

Adaptive visual–tactile fusion recognition for robotic operation of multi-material system

The use of robots in various industries is evolving from mechanization to intelligence and precision. These systems often comprise parts made of different materials and thus require accurate and comprehensive target identification. While humans perceive the world through a highly diverse perceptual...

Descripción completa

Detalles Bibliográficos
Autores principales: Ding, Zihao, Chen, Guodong, Wang, Zhenhua, Sun, Lining
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10318164/
https://www.ncbi.nlm.nih.gov/pubmed/37408585
http://dx.doi.org/10.3389/fnbot.2023.1181383
_version_ 1785067978473078784
author Ding, Zihao
Chen, Guodong
Wang, Zhenhua
Sun, Lining
author_facet Ding, Zihao
Chen, Guodong
Wang, Zhenhua
Sun, Lining
author_sort Ding, Zihao
collection PubMed
description The use of robots in various industries is evolving from mechanization to intelligence and precision. These systems often comprise parts made of different materials and thus require accurate and comprehensive target identification. While humans perceive the world through a highly diverse perceptual system and can rapidly identify deformable objects through vision and touch to prevent slipping or excessive deformation during grasping, robot recognition technology mainly relies on visual sensors, which lack critical information such as object material, leading to incomplete cognition. Therefore, multimodal information fusion is believed to be key to the development of robot recognition. Firstly, a method of converting tactile sequences to images is proposed to deal with the obstacles of information exchange between different modalities for vision and touch, which overcomes the problems of the noise and instability of tactile data. Subsequently, a visual-tactile fusion network framework based on an adaptive dropout algorithm is constructed, together with an optimal joint mechanism between visual information and tactile information established, to solve the problem of mutual exclusion or unbalanced fusion in traditional fusion methods. Finally, experiments show that the proposed method effectively improves robot recognition ability, and the classification accuracy is as high as 99.3%.
format Online
Article
Text
id pubmed-10318164
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-103181642023-07-05 Adaptive visual–tactile fusion recognition for robotic operation of multi-material system Ding, Zihao Chen, Guodong Wang, Zhenhua Sun, Lining Front Neurorobot Neuroscience The use of robots in various industries is evolving from mechanization to intelligence and precision. These systems often comprise parts made of different materials and thus require accurate and comprehensive target identification. While humans perceive the world through a highly diverse perceptual system and can rapidly identify deformable objects through vision and touch to prevent slipping or excessive deformation during grasping, robot recognition technology mainly relies on visual sensors, which lack critical information such as object material, leading to incomplete cognition. Therefore, multimodal information fusion is believed to be key to the development of robot recognition. Firstly, a method of converting tactile sequences to images is proposed to deal with the obstacles of information exchange between different modalities for vision and touch, which overcomes the problems of the noise and instability of tactile data. Subsequently, a visual-tactile fusion network framework based on an adaptive dropout algorithm is constructed, together with an optimal joint mechanism between visual information and tactile information established, to solve the problem of mutual exclusion or unbalanced fusion in traditional fusion methods. Finally, experiments show that the proposed method effectively improves robot recognition ability, and the classification accuracy is as high as 99.3%. Frontiers Media S.A. 2023-06-20 /pmc/articles/PMC10318164/ /pubmed/37408585 http://dx.doi.org/10.3389/fnbot.2023.1181383 Text en Copyright © 2023 Ding, Chen, Wang and Sun. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Neuroscience
Ding, Zihao
Chen, Guodong
Wang, Zhenhua
Sun, Lining
Adaptive visual–tactile fusion recognition for robotic operation of multi-material system
title Adaptive visual–tactile fusion recognition for robotic operation of multi-material system
title_full Adaptive visual–tactile fusion recognition for robotic operation of multi-material system
title_fullStr Adaptive visual–tactile fusion recognition for robotic operation of multi-material system
title_full_unstemmed Adaptive visual–tactile fusion recognition for robotic operation of multi-material system
title_short Adaptive visual–tactile fusion recognition for robotic operation of multi-material system
title_sort adaptive visual–tactile fusion recognition for robotic operation of multi-material system
topic Neuroscience
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10318164/
https://www.ncbi.nlm.nih.gov/pubmed/37408585
http://dx.doi.org/10.3389/fnbot.2023.1181383
work_keys_str_mv AT dingzihao adaptivevisualtactilefusionrecognitionforroboticoperationofmultimaterialsystem
AT chenguodong adaptivevisualtactilefusionrecognitionforroboticoperationofmultimaterialsystem
AT wangzhenhua adaptivevisualtactilefusionrecognitionforroboticoperationofmultimaterialsystem
AT sunlining adaptivevisualtactilefusionrecognitionforroboticoperationofmultimaterialsystem