Cargando…

Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes

The human–robot collaboration (HRC) solutions presented so far have the disadvantage that the interaction between humans and robots is based on the human’s state or on specific gestures purposely performed by the human, thus increasing the time required to perform a task and slowing down the pace of...

Descripción completa

Detalles Bibliográficos
Autores principales: Garcia, Pedro P., Santos, Telmo G., Machado, Miguel A., Mendes, Nuno
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9823442/
https://www.ncbi.nlm.nih.gov/pubmed/36617153
http://dx.doi.org/10.3390/s23010553
_version_ 1784866160774217728
author Garcia, Pedro P.
Santos, Telmo G.
Machado, Miguel A.
Mendes, Nuno
author_facet Garcia, Pedro P.
Santos, Telmo G.
Machado, Miguel A.
Mendes, Nuno
author_sort Garcia, Pedro P.
collection PubMed
description The human–robot collaboration (HRC) solutions presented so far have the disadvantage that the interaction between humans and robots is based on the human’s state or on specific gestures purposely performed by the human, thus increasing the time required to perform a task and slowing down the pace of human labor, making such solutions uninteresting. In this study, a different concept of the HRC system is introduced, consisting of an HRC framework for managing assembly processes that are executed simultaneously or individually by humans and robots. This HRC framework based on deep learning models uses only one type of data, RGB camera data, to make predictions about the collaborative workspace and human action, and consequently manage the assembly process. To validate the HRC framework, an industrial HRC demonstrator was built to assemble a mechanical component. Four different HRC frameworks were created based on the convolutional neural network (CNN) model structures: Faster R-CNN ResNet-50 and ResNet-101, YOLOv2 and YOLOv3. The HRC framework with YOLOv3 structure showed the best performance, showing a mean average performance of 72.26% and allowed the HRC industrial demonstrator to successfully complete all assembly tasks within a desired time window. The HRC framework has proven effective for industrial assembly applications.
format Online
Article
Text
id pubmed-9823442
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-98234422023-01-08 Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes Garcia, Pedro P. Santos, Telmo G. Machado, Miguel A. Mendes, Nuno Sensors (Basel) Article The human–robot collaboration (HRC) solutions presented so far have the disadvantage that the interaction between humans and robots is based on the human’s state or on specific gestures purposely performed by the human, thus increasing the time required to perform a task and slowing down the pace of human labor, making such solutions uninteresting. In this study, a different concept of the HRC system is introduced, consisting of an HRC framework for managing assembly processes that are executed simultaneously or individually by humans and robots. This HRC framework based on deep learning models uses only one type of data, RGB camera data, to make predictions about the collaborative workspace and human action, and consequently manage the assembly process. To validate the HRC framework, an industrial HRC demonstrator was built to assemble a mechanical component. Four different HRC frameworks were created based on the convolutional neural network (CNN) model structures: Faster R-CNN ResNet-50 and ResNet-101, YOLOv2 and YOLOv3. The HRC framework with YOLOv3 structure showed the best performance, showing a mean average performance of 72.26% and allowed the HRC industrial demonstrator to successfully complete all assembly tasks within a desired time window. The HRC framework has proven effective for industrial assembly applications. MDPI 2023-01-03 /pmc/articles/PMC9823442/ /pubmed/36617153 http://dx.doi.org/10.3390/s23010553 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Garcia, Pedro P.
Santos, Telmo G.
Machado, Miguel A.
Mendes, Nuno
Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes
title Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes
title_full Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes
title_fullStr Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes
title_full_unstemmed Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes
title_short Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes
title_sort deep learning framework for controlling work sequence in collaborative human–robot assembly processes
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9823442/
https://www.ncbi.nlm.nih.gov/pubmed/36617153
http://dx.doi.org/10.3390/s23010553
work_keys_str_mv AT garciapedrop deeplearningframeworkforcontrollingworksequenceincollaborativehumanrobotassemblyprocesses
AT santostelmog deeplearningframeworkforcontrollingworksequenceincollaborativehumanrobotassemblyprocesses
AT machadomiguela deeplearningframeworkforcontrollingworksequenceincollaborativehumanrobotassemblyprocesses
AT mendesnuno deeplearningframeworkforcontrollingworksequenceincollaborativehumanrobotassemblyprocesses