Cargando…

HOTA: A Higher Order Metric for Evaluating Multi-object Tracking

Multi-object tracking (MOT) has been notoriously difficult to evaluate. Previous metrics overemphasize the importance of either detection or association. To address this, we present a novel MOT evaluation metric, higher order tracking accuracy (HOTA), which explicitly balances the effect of performi...

Descripción completa

Detalles Bibliográficos
Autores principales: Luiten, Jonathon, Os̆ep, Aljos̆a, Dendorfer, Patrick, Torr, Philip, Geiger, Andreas, Leal-Taixé, Laura, Leibe, Bastian
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Springer US 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7881978/
https://www.ncbi.nlm.nih.gov/pubmed/33642696
http://dx.doi.org/10.1007/s11263-020-01375-2
_version_ 1783650980737318912
author Luiten, Jonathon
Os̆ep, Aljos̆a
Dendorfer, Patrick
Torr, Philip
Geiger, Andreas
Leal-Taixé, Laura
Leibe, Bastian
author_facet Luiten, Jonathon
Os̆ep, Aljos̆a
Dendorfer, Patrick
Torr, Philip
Geiger, Andreas
Leal-Taixé, Laura
Leibe, Bastian
author_sort Luiten, Jonathon
collection PubMed
description Multi-object tracking (MOT) has been notoriously difficult to evaluate. Previous metrics overemphasize the importance of either detection or association. To address this, we present a novel MOT evaluation metric, higher order tracking accuracy (HOTA), which explicitly balances the effect of performing accurate detection, association and localization into a single unified metric for comparing trackers. HOTA decomposes into a family of sub-metrics which are able to evaluate each of five basic error types separately, which enables clear analysis of tracking performance. We evaluate the effectiveness of HOTA on the MOTChallenge benchmark, and show that it is able to capture important aspects of MOT performance not previously taken into account by established metrics. Furthermore, we show HOTA scores better align with human visual evaluation of tracking performance.
format Online
Article
Text
id pubmed-7881978
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher Springer US
record_format MEDLINE/PubMed
spelling pubmed-78819782021-02-25 HOTA: A Higher Order Metric for Evaluating Multi-object Tracking Luiten, Jonathon Os̆ep, Aljos̆a Dendorfer, Patrick Torr, Philip Geiger, Andreas Leal-Taixé, Laura Leibe, Bastian Int J Comput Vis Article Multi-object tracking (MOT) has been notoriously difficult to evaluate. Previous metrics overemphasize the importance of either detection or association. To address this, we present a novel MOT evaluation metric, higher order tracking accuracy (HOTA), which explicitly balances the effect of performing accurate detection, association and localization into a single unified metric for comparing trackers. HOTA decomposes into a family of sub-metrics which are able to evaluate each of five basic error types separately, which enables clear analysis of tracking performance. We evaluate the effectiveness of HOTA on the MOTChallenge benchmark, and show that it is able to capture important aspects of MOT performance not previously taken into account by established metrics. Furthermore, we show HOTA scores better align with human visual evaluation of tracking performance. Springer US 2020-10-08 2021 /pmc/articles/PMC7881978/ /pubmed/33642696 http://dx.doi.org/10.1007/s11263-020-01375-2 Text en © The Author(s) 2020 Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
spellingShingle Article
Luiten, Jonathon
Os̆ep, Aljos̆a
Dendorfer, Patrick
Torr, Philip
Geiger, Andreas
Leal-Taixé, Laura
Leibe, Bastian
HOTA: A Higher Order Metric for Evaluating Multi-object Tracking
title HOTA: A Higher Order Metric for Evaluating Multi-object Tracking
title_full HOTA: A Higher Order Metric for Evaluating Multi-object Tracking
title_fullStr HOTA: A Higher Order Metric for Evaluating Multi-object Tracking
title_full_unstemmed HOTA: A Higher Order Metric for Evaluating Multi-object Tracking
title_short HOTA: A Higher Order Metric for Evaluating Multi-object Tracking
title_sort hota: a higher order metric for evaluating multi-object tracking
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7881978/
https://www.ncbi.nlm.nih.gov/pubmed/33642696
http://dx.doi.org/10.1007/s11263-020-01375-2
work_keys_str_mv AT luitenjonathon hotaahigherordermetricforevaluatingmultiobjecttracking
AT osepaljosa hotaahigherordermetricforevaluatingmultiobjecttracking
AT dendorferpatrick hotaahigherordermetricforevaluatingmultiobjecttracking
AT torrphilip hotaahigherordermetricforevaluatingmultiobjecttracking
AT geigerandreas hotaahigherordermetricforevaluatingmultiobjecttracking
AT lealtaixelaura hotaahigherordermetricforevaluatingmultiobjecttracking
AT leibebastian hotaahigherordermetricforevaluatingmultiobjecttracking