Cargando…
Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Human activity recognition (HAR) tasks have traditionally been solved using engineered features obtained by heuristic processes. Current research suggests that deep convolutional neural networks are suited to automate feature extraction from raw sensor inputs. However, human activities are made of c...
Autores principales: | , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2016
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4732148/ https://www.ncbi.nlm.nih.gov/pubmed/26797612 http://dx.doi.org/10.3390/s16010115 |
_version_ | 1782412664287264768 |
---|---|
author | Ordóñez, Francisco Javier Roggen, Daniel |
author_facet | Ordóñez, Francisco Javier Roggen, Daniel |
author_sort | Ordóñez, Francisco Javier |
collection | PubMed |
description | Human activity recognition (HAR) tasks have traditionally been solved using engineered features obtained by heuristic processes. Current research suggests that deep convolutional neural networks are suited to automate feature extraction from raw sensor inputs. However, human activities are made of complex sequences of motor movements, and capturing this temporal dynamics is fundamental for successful HAR. Based on the recent success of recurrent neural networks for time series domains, we propose a generic deep framework for activity recognition based on convolutional and LSTM recurrent units, which: (i) is suitable for multimodal wearable sensors; (ii) can perform sensor fusion naturally; (iii) does not require expert knowledge in designing features; and (iv) explicitly models the temporal dynamics of feature activations. We evaluate our framework on two datasets, one of which has been used in a public activity recognition challenge. Our results show that our framework outperforms competing deep non-recurrent networks on the challenge dataset by 4% on average; outperforming some of the previous reported results by up to 9%. Our results show that the framework can be applied to homogeneous sensor modalities, but can also fuse multimodal sensors to improve performance. We characterise key architectural hyperparameters’ influence on performance to provide insights about their optimisation. |
format | Online Article Text |
id | pubmed-4732148 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2016 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-47321482016-02-12 Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition Ordóñez, Francisco Javier Roggen, Daniel Sensors (Basel) Article Human activity recognition (HAR) tasks have traditionally been solved using engineered features obtained by heuristic processes. Current research suggests that deep convolutional neural networks are suited to automate feature extraction from raw sensor inputs. However, human activities are made of complex sequences of motor movements, and capturing this temporal dynamics is fundamental for successful HAR. Based on the recent success of recurrent neural networks for time series domains, we propose a generic deep framework for activity recognition based on convolutional and LSTM recurrent units, which: (i) is suitable for multimodal wearable sensors; (ii) can perform sensor fusion naturally; (iii) does not require expert knowledge in designing features; and (iv) explicitly models the temporal dynamics of feature activations. We evaluate our framework on two datasets, one of which has been used in a public activity recognition challenge. Our results show that our framework outperforms competing deep non-recurrent networks on the challenge dataset by 4% on average; outperforming some of the previous reported results by up to 9%. Our results show that the framework can be applied to homogeneous sensor modalities, but can also fuse multimodal sensors to improve performance. We characterise key architectural hyperparameters’ influence on performance to provide insights about their optimisation. MDPI 2016-01-18 /pmc/articles/PMC4732148/ /pubmed/26797612 http://dx.doi.org/10.3390/s16010115 Text en © 2016 by the authors; licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons by Attribution (CC-BY) license (http://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Ordóñez, Francisco Javier Roggen, Daniel Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition |
title | Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition |
title_full | Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition |
title_fullStr | Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition |
title_full_unstemmed | Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition |
title_short | Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition |
title_sort | deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4732148/ https://www.ncbi.nlm.nih.gov/pubmed/26797612 http://dx.doi.org/10.3390/s16010115 |
work_keys_str_mv | AT ordonezfranciscojavier deepconvolutionalandlstmrecurrentneuralnetworksformultimodalwearableactivityrecognition AT roggendaniel deepconvolutionalandlstmrecurrentneuralnetworksformultimodalwearableactivityrecognition |