Cargando…
A Baseline for Cross-Database 3D Human Pose Estimation
Vision-based 3D human pose estimation approaches are typically evaluated on datasets that are limited in diversity regarding many factors, e.g., subjects, poses, cameras, and lighting. However, for real-life applications, it would be desirable to create systems that work under arbitrary conditions (...
Autores principales: | , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2021
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8198914/ https://www.ncbi.nlm.nih.gov/pubmed/34071704 http://dx.doi.org/10.3390/s21113769 |
_version_ | 1783707252395343872 |
---|---|
author | Rapczyński, Michał Werner, Philipp Handrich, Sebastian Al-Hamadi, Ayoub |
author_facet | Rapczyński, Michał Werner, Philipp Handrich, Sebastian Al-Hamadi, Ayoub |
author_sort | Rapczyński, Michał |
collection | PubMed |
description | Vision-based 3D human pose estimation approaches are typically evaluated on datasets that are limited in diversity regarding many factors, e.g., subjects, poses, cameras, and lighting. However, for real-life applications, it would be desirable to create systems that work under arbitrary conditions (“in-the-wild”). To advance towards this goal, we investigated the commonly used datasets HumanEva-I, Human3.6M, and Panoptic Studio, discussed their biases (that is, their limitations in diversity), and illustrated them in cross-database experiments (for which we used a surrogate for roughly estimating in-the-wild performance). For this purpose, we first harmonized the differing skeleton joint definitions of the datasets, reducing the biases and systematic test errors in cross-database experiments. We further proposed a scale normalization method that significantly improved generalization across camera viewpoints, subjects, and datasets. In additional experiments, we investigated the effect of using more or less cameras, training with multiple datasets, applying a proposed anatomy-based pose validation step, and using OpenPose as the basis for the 3D pose estimation. The experimental results showed the usefulness of the joint harmonization, of the scale normalization, and of augmenting virtual cameras to significantly improve cross-database and in-database generalization. At the same time, the experiments showed that there were dataset biases that could not be compensated and call for new datasets covering more diversity. We discussed our results and promising directions for future work. |
format | Online Article Text |
id | pubmed-8198914 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2021 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-81989142021-06-14 A Baseline for Cross-Database 3D Human Pose Estimation Rapczyński, Michał Werner, Philipp Handrich, Sebastian Al-Hamadi, Ayoub Sensors (Basel) Article Vision-based 3D human pose estimation approaches are typically evaluated on datasets that are limited in diversity regarding many factors, e.g., subjects, poses, cameras, and lighting. However, for real-life applications, it would be desirable to create systems that work under arbitrary conditions (“in-the-wild”). To advance towards this goal, we investigated the commonly used datasets HumanEva-I, Human3.6M, and Panoptic Studio, discussed their biases (that is, their limitations in diversity), and illustrated them in cross-database experiments (for which we used a surrogate for roughly estimating in-the-wild performance). For this purpose, we first harmonized the differing skeleton joint definitions of the datasets, reducing the biases and systematic test errors in cross-database experiments. We further proposed a scale normalization method that significantly improved generalization across camera viewpoints, subjects, and datasets. In additional experiments, we investigated the effect of using more or less cameras, training with multiple datasets, applying a proposed anatomy-based pose validation step, and using OpenPose as the basis for the 3D pose estimation. The experimental results showed the usefulness of the joint harmonization, of the scale normalization, and of augmenting virtual cameras to significantly improve cross-database and in-database generalization. At the same time, the experiments showed that there were dataset biases that could not be compensated and call for new datasets covering more diversity. We discussed our results and promising directions for future work. MDPI 2021-05-28 /pmc/articles/PMC8198914/ /pubmed/34071704 http://dx.doi.org/10.3390/s21113769 Text en © 2021 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Rapczyński, Michał Werner, Philipp Handrich, Sebastian Al-Hamadi, Ayoub A Baseline for Cross-Database 3D Human Pose Estimation |
title | A Baseline for Cross-Database 3D Human Pose Estimation |
title_full | A Baseline for Cross-Database 3D Human Pose Estimation |
title_fullStr | A Baseline for Cross-Database 3D Human Pose Estimation |
title_full_unstemmed | A Baseline for Cross-Database 3D Human Pose Estimation |
title_short | A Baseline for Cross-Database 3D Human Pose Estimation |
title_sort | baseline for cross-database 3d human pose estimation |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8198914/ https://www.ncbi.nlm.nih.gov/pubmed/34071704 http://dx.doi.org/10.3390/s21113769 |
work_keys_str_mv | AT rapczynskimichał abaselineforcrossdatabase3dhumanposeestimation AT wernerphilipp abaselineforcrossdatabase3dhumanposeestimation AT handrichsebastian abaselineforcrossdatabase3dhumanposeestimation AT alhamadiayoub abaselineforcrossdatabase3dhumanposeestimation AT rapczynskimichał baselineforcrossdatabase3dhumanposeestimation AT wernerphilipp baselineforcrossdatabase3dhumanposeestimation AT handrichsebastian baselineforcrossdatabase3dhumanposeestimation AT alhamadiayoub baselineforcrossdatabase3dhumanposeestimation |