Cargando…
Industrial Robot Control by Means of Gestures and Voice Commands in Off-Line and On-Line Mode
The paper presents the possibility of using the Kinect v2 module to control an industrial robot by means of gestures and voice commands. It describes the elements of creating software for off-line and on-line robot control. The application for the Kinect module was developed in the C# language in th...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7664672/ https://www.ncbi.nlm.nih.gov/pubmed/33171844 http://dx.doi.org/10.3390/s20216358 |
_version_ | 1783609881171853312 |
---|---|
author | Kaczmarek, Wojciech Panasiuk, Jarosław Borys, Szymon Banach, Patryk |
author_facet | Kaczmarek, Wojciech Panasiuk, Jarosław Borys, Szymon Banach, Patryk |
author_sort | Kaczmarek, Wojciech |
collection | PubMed |
description | The paper presents the possibility of using the Kinect v2 module to control an industrial robot by means of gestures and voice commands. It describes the elements of creating software for off-line and on-line robot control. The application for the Kinect module was developed in the C# language in the Visual Studio environment, while the industrial robot control program was developed in the RAPID language in the RobotStudio environment. The development of a two-threaded application in the RAPID language allowed separating two independent tasks for the IRB120 robot. The main task of the robot is performed in Thread No. 1 (responsible for movement). Simultaneously, Thread No. 2 ensures continuous communication with the Kinect system and provides information about the gesture and voice commands in real time without any interference in Thread No. 1. The applied solution allows the robot to work in industrial conditions without the negative impact of the communication task on the time of the robot’s work cycles. Thanks to the development of a digital twin of the real robot station, tests of proper application functioning in off-line mode (without using a real robot) were conducted. The obtained results were verified on-line (on the real test station). Tests of the correctness of gesture recognition were carried out, and the robot recognized all programmed gestures. Another test carried out was the recognition and execution of voice commands. A difference in the time of task completion between the actual and virtual station was noticed; the average difference was 0.67 s. The last test carried out was to examine the impact of interference on the recognition of voice commands. With a 10 dB difference between the command and noise, the recognition of voice commands was equal to 91.43%. The developed computer programs have a modular structure, which enables easy adaptation to process requirements. |
format | Online Article Text |
id | pubmed-7664672 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-76646722020-11-14 Industrial Robot Control by Means of Gestures and Voice Commands in Off-Line and On-Line Mode Kaczmarek, Wojciech Panasiuk, Jarosław Borys, Szymon Banach, Patryk Sensors (Basel) Article The paper presents the possibility of using the Kinect v2 module to control an industrial robot by means of gestures and voice commands. It describes the elements of creating software for off-line and on-line robot control. The application for the Kinect module was developed in the C# language in the Visual Studio environment, while the industrial robot control program was developed in the RAPID language in the RobotStudio environment. The development of a two-threaded application in the RAPID language allowed separating two independent tasks for the IRB120 robot. The main task of the robot is performed in Thread No. 1 (responsible for movement). Simultaneously, Thread No. 2 ensures continuous communication with the Kinect system and provides information about the gesture and voice commands in real time without any interference in Thread No. 1. The applied solution allows the robot to work in industrial conditions without the negative impact of the communication task on the time of the robot’s work cycles. Thanks to the development of a digital twin of the real robot station, tests of proper application functioning in off-line mode (without using a real robot) were conducted. The obtained results were verified on-line (on the real test station). Tests of the correctness of gesture recognition were carried out, and the robot recognized all programmed gestures. Another test carried out was the recognition and execution of voice commands. A difference in the time of task completion between the actual and virtual station was noticed; the average difference was 0.67 s. The last test carried out was to examine the impact of interference on the recognition of voice commands. With a 10 dB difference between the command and noise, the recognition of voice commands was equal to 91.43%. The developed computer programs have a modular structure, which enables easy adaptation to process requirements. MDPI 2020-11-07 /pmc/articles/PMC7664672/ /pubmed/33171844 http://dx.doi.org/10.3390/s20216358 Text en © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Kaczmarek, Wojciech Panasiuk, Jarosław Borys, Szymon Banach, Patryk Industrial Robot Control by Means of Gestures and Voice Commands in Off-Line and On-Line Mode |
title | Industrial Robot Control by Means of Gestures and Voice Commands in Off-Line and On-Line Mode |
title_full | Industrial Robot Control by Means of Gestures and Voice Commands in Off-Line and On-Line Mode |
title_fullStr | Industrial Robot Control by Means of Gestures and Voice Commands in Off-Line and On-Line Mode |
title_full_unstemmed | Industrial Robot Control by Means of Gestures and Voice Commands in Off-Line and On-Line Mode |
title_short | Industrial Robot Control by Means of Gestures and Voice Commands in Off-Line and On-Line Mode |
title_sort | industrial robot control by means of gestures and voice commands in off-line and on-line mode |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7664672/ https://www.ncbi.nlm.nih.gov/pubmed/33171844 http://dx.doi.org/10.3390/s20216358 |
work_keys_str_mv | AT kaczmarekwojciech industrialrobotcontrolbymeansofgesturesandvoicecommandsinofflineandonlinemode AT panasiukjarosław industrialrobotcontrolbymeansofgesturesandvoicecommandsinofflineandonlinemode AT borysszymon industrialrobotcontrolbymeansofgesturesandvoicecommandsinofflineandonlinemode AT banachpatryk industrialrobotcontrolbymeansofgesturesandvoicecommandsinofflineandonlinemode |