Cargando…
Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes
The human–robot collaboration (HRC) solutions presented so far have the disadvantage that the interaction between humans and robots is based on the human’s state or on specific gestures purposely performed by the human, thus increasing the time required to perform a task and slowing down the pace of...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9823442/ https://www.ncbi.nlm.nih.gov/pubmed/36617153 http://dx.doi.org/10.3390/s23010553 |
_version_ | 1784866160774217728 |
---|---|
author | Garcia, Pedro P. Santos, Telmo G. Machado, Miguel A. Mendes, Nuno |
author_facet | Garcia, Pedro P. Santos, Telmo G. Machado, Miguel A. Mendes, Nuno |
author_sort | Garcia, Pedro P. |
collection | PubMed |
description | The human–robot collaboration (HRC) solutions presented so far have the disadvantage that the interaction between humans and robots is based on the human’s state or on specific gestures purposely performed by the human, thus increasing the time required to perform a task and slowing down the pace of human labor, making such solutions uninteresting. In this study, a different concept of the HRC system is introduced, consisting of an HRC framework for managing assembly processes that are executed simultaneously or individually by humans and robots. This HRC framework based on deep learning models uses only one type of data, RGB camera data, to make predictions about the collaborative workspace and human action, and consequently manage the assembly process. To validate the HRC framework, an industrial HRC demonstrator was built to assemble a mechanical component. Four different HRC frameworks were created based on the convolutional neural network (CNN) model structures: Faster R-CNN ResNet-50 and ResNet-101, YOLOv2 and YOLOv3. The HRC framework with YOLOv3 structure showed the best performance, showing a mean average performance of 72.26% and allowed the HRC industrial demonstrator to successfully complete all assembly tasks within a desired time window. The HRC framework has proven effective for industrial assembly applications. |
format | Online Article Text |
id | pubmed-9823442 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-98234422023-01-08 Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes Garcia, Pedro P. Santos, Telmo G. Machado, Miguel A. Mendes, Nuno Sensors (Basel) Article The human–robot collaboration (HRC) solutions presented so far have the disadvantage that the interaction between humans and robots is based on the human’s state or on specific gestures purposely performed by the human, thus increasing the time required to perform a task and slowing down the pace of human labor, making such solutions uninteresting. In this study, a different concept of the HRC system is introduced, consisting of an HRC framework for managing assembly processes that are executed simultaneously or individually by humans and robots. This HRC framework based on deep learning models uses only one type of data, RGB camera data, to make predictions about the collaborative workspace and human action, and consequently manage the assembly process. To validate the HRC framework, an industrial HRC demonstrator was built to assemble a mechanical component. Four different HRC frameworks were created based on the convolutional neural network (CNN) model structures: Faster R-CNN ResNet-50 and ResNet-101, YOLOv2 and YOLOv3. The HRC framework with YOLOv3 structure showed the best performance, showing a mean average performance of 72.26% and allowed the HRC industrial demonstrator to successfully complete all assembly tasks within a desired time window. The HRC framework has proven effective for industrial assembly applications. MDPI 2023-01-03 /pmc/articles/PMC9823442/ /pubmed/36617153 http://dx.doi.org/10.3390/s23010553 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Garcia, Pedro P. Santos, Telmo G. Machado, Miguel A. Mendes, Nuno Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes |
title | Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes |
title_full | Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes |
title_fullStr | Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes |
title_full_unstemmed | Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes |
title_short | Deep Learning Framework for Controlling Work Sequence in Collaborative Human–Robot Assembly Processes |
title_sort | deep learning framework for controlling work sequence in collaborative human–robot assembly processes |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9823442/ https://www.ncbi.nlm.nih.gov/pubmed/36617153 http://dx.doi.org/10.3390/s23010553 |
work_keys_str_mv | AT garciapedrop deeplearningframeworkforcontrollingworksequenceincollaborativehumanrobotassemblyprocesses AT santostelmog deeplearningframeworkforcontrollingworksequenceincollaborativehumanrobotassemblyprocesses AT machadomiguela deeplearningframeworkforcontrollingworksequenceincollaborativehumanrobotassemblyprocesses AT mendesnuno deeplearningframeworkforcontrollingworksequenceincollaborativehumanrobotassemblyprocesses |