Cargando…

Reinforcement Learning and Control of a Lower Extremity Exoskeleton for Squat Assistance

A significant challenge for the control of a robotic lower extremity rehabilitation exoskeleton is to ensure stability and robustness during programmed tasks or motions, which is crucial for the safety of the mobility-impaired user. Due to various levels of the user’s disability, the human-exoskelet...

Descripción completa

Detalles Bibliográficos
Autores principales: Luo, Shuzhen, Androwis, Ghaith, Adamovich, Sergei, Su, Hao, Nunez, Erick, Zhou, Xianlian
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Frontiers Media S.A. 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8326457/
https://www.ncbi.nlm.nih.gov/pubmed/34350214
http://dx.doi.org/10.3389/frobt.2021.702845
_version_ 1783731824632004608
author Luo, Shuzhen
Androwis, Ghaith
Adamovich, Sergei
Su, Hao
Nunez, Erick
Zhou, Xianlian
author_facet Luo, Shuzhen
Androwis, Ghaith
Adamovich, Sergei
Su, Hao
Nunez, Erick
Zhou, Xianlian
author_sort Luo, Shuzhen
collection PubMed
description A significant challenge for the control of a robotic lower extremity rehabilitation exoskeleton is to ensure stability and robustness during programmed tasks or motions, which is crucial for the safety of the mobility-impaired user. Due to various levels of the user’s disability, the human-exoskeleton interaction forces and external perturbations are unpredictable and could vary substantially and cause conventional motion controllers to behave unreliably or the robot to fall down. In this work, we propose a new, reinforcement learning-based, motion controller for a lower extremity rehabilitation exoskeleton, aiming to perform collaborative squatting exercises with efficiency, stability, and strong robustness. Unlike most existing rehabilitation exoskeletons, our exoskeleton has ankle actuation on both sagittal and front planes and is equipped with multiple foot force sensors to estimate center of pressure (CoP), an important indicator of system balance. This proposed motion controller takes advantage of the CoP information by incorporating it in the state input of the control policy network and adding it to the reward during the learning to maintain a well balanced system state during motions. In addition, we use dynamics randomization and adversary force perturbations including large human interaction forces during the training to further improve control robustness. To evaluate the effectiveness of the learning controller, we conduct numerical experiments with different settings to demonstrate its remarkable ability on controlling the exoskeleton to repetitively perform well balanced and robust squatting motions under strong perturbations and realistic human interaction forces.
format Online
Article
Text
id pubmed-8326457
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher Frontiers Media S.A.
record_format MEDLINE/PubMed
spelling pubmed-83264572021-08-03 Reinforcement Learning and Control of a Lower Extremity Exoskeleton for Squat Assistance Luo, Shuzhen Androwis, Ghaith Adamovich, Sergei Su, Hao Nunez, Erick Zhou, Xianlian Front Robot AI Robotics and AI A significant challenge for the control of a robotic lower extremity rehabilitation exoskeleton is to ensure stability and robustness during programmed tasks or motions, which is crucial for the safety of the mobility-impaired user. Due to various levels of the user’s disability, the human-exoskeleton interaction forces and external perturbations are unpredictable and could vary substantially and cause conventional motion controllers to behave unreliably or the robot to fall down. In this work, we propose a new, reinforcement learning-based, motion controller for a lower extremity rehabilitation exoskeleton, aiming to perform collaborative squatting exercises with efficiency, stability, and strong robustness. Unlike most existing rehabilitation exoskeletons, our exoskeleton has ankle actuation on both sagittal and front planes and is equipped with multiple foot force sensors to estimate center of pressure (CoP), an important indicator of system balance. This proposed motion controller takes advantage of the CoP information by incorporating it in the state input of the control policy network and adding it to the reward during the learning to maintain a well balanced system state during motions. In addition, we use dynamics randomization and adversary force perturbations including large human interaction forces during the training to further improve control robustness. To evaluate the effectiveness of the learning controller, we conduct numerical experiments with different settings to demonstrate its remarkable ability on controlling the exoskeleton to repetitively perform well balanced and robust squatting motions under strong perturbations and realistic human interaction forces. Frontiers Media S.A. 2021-07-19 /pmc/articles/PMC8326457/ /pubmed/34350214 http://dx.doi.org/10.3389/frobt.2021.702845 Text en Copyright © 2021 Luo, Androwis, Adamovich, Su, Nunez and Zhou. https://creativecommons.org/licenses/by/4.0/This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
spellingShingle Robotics and AI
Luo, Shuzhen
Androwis, Ghaith
Adamovich, Sergei
Su, Hao
Nunez, Erick
Zhou, Xianlian
Reinforcement Learning and Control of a Lower Extremity Exoskeleton for Squat Assistance
title Reinforcement Learning and Control of a Lower Extremity Exoskeleton for Squat Assistance
title_full Reinforcement Learning and Control of a Lower Extremity Exoskeleton for Squat Assistance
title_fullStr Reinforcement Learning and Control of a Lower Extremity Exoskeleton for Squat Assistance
title_full_unstemmed Reinforcement Learning and Control of a Lower Extremity Exoskeleton for Squat Assistance
title_short Reinforcement Learning and Control of a Lower Extremity Exoskeleton for Squat Assistance
title_sort reinforcement learning and control of a lower extremity exoskeleton for squat assistance
topic Robotics and AI
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8326457/
https://www.ncbi.nlm.nih.gov/pubmed/34350214
http://dx.doi.org/10.3389/frobt.2021.702845
work_keys_str_mv AT luoshuzhen reinforcementlearningandcontrolofalowerextremityexoskeletonforsquatassistance
AT androwisghaith reinforcementlearningandcontrolofalowerextremityexoskeletonforsquatassistance
AT adamovichsergei reinforcementlearningandcontrolofalowerextremityexoskeletonforsquatassistance
AT suhao reinforcementlearningandcontrolofalowerextremityexoskeletonforsquatassistance
AT nunezerick reinforcementlearningandcontrolofalowerextremityexoskeletonforsquatassistance
AT zhouxianlian reinforcementlearningandcontrolofalowerextremityexoskeletonforsquatassistance