Cargando…
Structure from Articulated Motion: Accurate and Stable Monocular 3D Reconstruction without Training Data
Recovery of articulated 3D structure from 2D observations is a challenging computer vision problem with many applications. Current learning-based approaches achieve state-of-the-art accuracy on public benchmarks but are restricted to specific types of objects and motions covered by the training data...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2019
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6833108/ https://www.ncbi.nlm.nih.gov/pubmed/31652665 http://dx.doi.org/10.3390/s19204603 |
_version_ | 1783466303902711808 |
---|---|
author | Kovalenko, Onorina Golyanik, Vladislav Malik, Jameel Elhayek, Ahmed Stricker, Didier |
author_facet | Kovalenko, Onorina Golyanik, Vladislav Malik, Jameel Elhayek, Ahmed Stricker, Didier |
author_sort | Kovalenko, Onorina |
collection | PubMed |
description | Recovery of articulated 3D structure from 2D observations is a challenging computer vision problem with many applications. Current learning-based approaches achieve state-of-the-art accuracy on public benchmarks but are restricted to specific types of objects and motions covered by the training datasets. Model-based approaches do not rely on training data but show lower accuracy on these datasets. In this paper, we introduce a model-based method called Structure from Articulated Motion (SfAM), which can recover multiple object and motion types without training on extensive data collections. At the same time, it performs on par with learning-based state-of-the-art approaches on public benchmarks and outperforms previous non-rigid structure from motion (NRSfM) methods. SfAM is built upon a general-purpose NRSfM technique while integrating a soft spatio-temporal constraint on the bone lengths. We use alternating optimization strategy to recover optimal geometry (i.e., bone proportions) together with 3D joint positions by enforcing the bone lengths consistency over a series of frames. SfAM is highly robust to noisy 2D annotations, generalizes to arbitrary objects and does not rely on training data, which is shown in extensive experiments on public benchmarks and real video sequences. We believe that it brings a new perspective on the domain of monocular 3D recovery of articulated structures, including human motion capture. |
format | Online Article Text |
id | pubmed-6833108 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2019 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-68331082019-11-25 Structure from Articulated Motion: Accurate and Stable Monocular 3D Reconstruction without Training Data Kovalenko, Onorina Golyanik, Vladislav Malik, Jameel Elhayek, Ahmed Stricker, Didier Sensors (Basel) Article Recovery of articulated 3D structure from 2D observations is a challenging computer vision problem with many applications. Current learning-based approaches achieve state-of-the-art accuracy on public benchmarks but are restricted to specific types of objects and motions covered by the training datasets. Model-based approaches do not rely on training data but show lower accuracy on these datasets. In this paper, we introduce a model-based method called Structure from Articulated Motion (SfAM), which can recover multiple object and motion types without training on extensive data collections. At the same time, it performs on par with learning-based state-of-the-art approaches on public benchmarks and outperforms previous non-rigid structure from motion (NRSfM) methods. SfAM is built upon a general-purpose NRSfM technique while integrating a soft spatio-temporal constraint on the bone lengths. We use alternating optimization strategy to recover optimal geometry (i.e., bone proportions) together with 3D joint positions by enforcing the bone lengths consistency over a series of frames. SfAM is highly robust to noisy 2D annotations, generalizes to arbitrary objects and does not rely on training data, which is shown in extensive experiments on public benchmarks and real video sequences. We believe that it brings a new perspective on the domain of monocular 3D recovery of articulated structures, including human motion capture. MDPI 2019-10-22 /pmc/articles/PMC6833108/ /pubmed/31652665 http://dx.doi.org/10.3390/s19204603 Text en © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Kovalenko, Onorina Golyanik, Vladislav Malik, Jameel Elhayek, Ahmed Stricker, Didier Structure from Articulated Motion: Accurate and Stable Monocular 3D Reconstruction without Training Data |
title | Structure from Articulated Motion: Accurate and Stable Monocular 3D Reconstruction without Training Data |
title_full | Structure from Articulated Motion: Accurate and Stable Monocular 3D Reconstruction without Training Data |
title_fullStr | Structure from Articulated Motion: Accurate and Stable Monocular 3D Reconstruction without Training Data |
title_full_unstemmed | Structure from Articulated Motion: Accurate and Stable Monocular 3D Reconstruction without Training Data |
title_short | Structure from Articulated Motion: Accurate and Stable Monocular 3D Reconstruction without Training Data |
title_sort | structure from articulated motion: accurate and stable monocular 3d reconstruction without training data |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6833108/ https://www.ncbi.nlm.nih.gov/pubmed/31652665 http://dx.doi.org/10.3390/s19204603 |
work_keys_str_mv | AT kovalenkoonorina structurefromarticulatedmotionaccurateandstablemonocular3dreconstructionwithouttrainingdata AT golyanikvladislav structurefromarticulatedmotionaccurateandstablemonocular3dreconstructionwithouttrainingdata AT malikjameel structurefromarticulatedmotionaccurateandstablemonocular3dreconstructionwithouttrainingdata AT elhayekahmed structurefromarticulatedmotionaccurateandstablemonocular3dreconstructionwithouttrainingdata AT strickerdidier structurefromarticulatedmotionaccurateandstablemonocular3dreconstructionwithouttrainingdata |