Cargando…

MBT3D: Deep learning based multi-object tracker for bumblebee 3D flight path estimation

This work presents the Multi-Bees-Tracker (MBT3D) algorithm, a Python framework implementing a deep association tracker for Tracking-By-Detection, to address the challenging task of tracking flight paths of bumblebees in a social group. While tracking algorithms for bumblebees exist, they often come...

Descripción completa

Detalles Bibliográficos
Autores principales: Stiemer, Luc Nicolas, Thoma, Andreas, Braun, Carsten
Formato: Online Artículo Texto
Lenguaje:English
Publicado: Public Library of Science 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10516433/
https://www.ncbi.nlm.nih.gov/pubmed/37738269
http://dx.doi.org/10.1371/journal.pone.0291415
_version_ 1785109127186350080
author Stiemer, Luc Nicolas
Thoma, Andreas
Braun, Carsten
author_facet Stiemer, Luc Nicolas
Thoma, Andreas
Braun, Carsten
author_sort Stiemer, Luc Nicolas
collection PubMed
description This work presents the Multi-Bees-Tracker (MBT3D) algorithm, a Python framework implementing a deep association tracker for Tracking-By-Detection, to address the challenging task of tracking flight paths of bumblebees in a social group. While tracking algorithms for bumblebees exist, they often come with intensive restrictions, such as the need for sufficient lighting, high contrast between the animal and background, absence of occlusion, significant user input, etc. Tracking flight paths of bumblebees in a social group is challenging. They suddenly adjust movements and change their appearance during different wing beat states while exhibiting significant similarities in their individual appearance. The MBT3D tracker, developed in this research, is an adaptation of an existing ant tracking algorithm for bumblebee tracking. It incorporates an offline trained appearance descriptor along with a Kalman Filter for appearance and motion matching. Different detector architectures for upstream detections (You Only Look Once (YOLOv5), Faster Region Proposal Convolutional Neural Network (Faster R-CNN), and RetinaNet) are investigated in a comparative study to optimize performance. The detection models were trained on a dataset containing 11359 labeled bumblebee images. YOLOv5 reaches an Average Precision of AP = 53, 8%, Faster R-CNN achieves AP = 45, 3% and RetinaNet AP = 38, 4% on the bumblebee validation dataset, which consists of 1323 labeled bumblebee images. The tracker’s appearance model is trained on 144 samples. The tracker (with Faster R-CNN detections) reaches a Multiple Object Tracking Accuracy MOTA = 93, 5% and a Multiple Object Tracking Precision MOTP = 75, 6% on a validation dataset containing 2000 images, competing with state-of-the-art computer vision methods. The framework allows reliable tracking of different bumblebees in the same video stream with rarely occurring identity switches (IDS). MBT3D has much lower IDS than other commonly used algorithms, with one of the lowest false positive rates, competing with state-of-the-art animal tracking algorithms. The developed framework reconstructs the 3-dimensional (3D) flight paths of the bumblebees by triangulation. It also handles and compares two alternative stereo camera pairs if desired.
format Online
Article
Text
id pubmed-10516433
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher Public Library of Science
record_format MEDLINE/PubMed
spelling pubmed-105164332023-09-23 MBT3D: Deep learning based multi-object tracker for bumblebee 3D flight path estimation Stiemer, Luc Nicolas Thoma, Andreas Braun, Carsten PLoS One Research Article This work presents the Multi-Bees-Tracker (MBT3D) algorithm, a Python framework implementing a deep association tracker for Tracking-By-Detection, to address the challenging task of tracking flight paths of bumblebees in a social group. While tracking algorithms for bumblebees exist, they often come with intensive restrictions, such as the need for sufficient lighting, high contrast between the animal and background, absence of occlusion, significant user input, etc. Tracking flight paths of bumblebees in a social group is challenging. They suddenly adjust movements and change their appearance during different wing beat states while exhibiting significant similarities in their individual appearance. The MBT3D tracker, developed in this research, is an adaptation of an existing ant tracking algorithm for bumblebee tracking. It incorporates an offline trained appearance descriptor along with a Kalman Filter for appearance and motion matching. Different detector architectures for upstream detections (You Only Look Once (YOLOv5), Faster Region Proposal Convolutional Neural Network (Faster R-CNN), and RetinaNet) are investigated in a comparative study to optimize performance. The detection models were trained on a dataset containing 11359 labeled bumblebee images. YOLOv5 reaches an Average Precision of AP = 53, 8%, Faster R-CNN achieves AP = 45, 3% and RetinaNet AP = 38, 4% on the bumblebee validation dataset, which consists of 1323 labeled bumblebee images. The tracker’s appearance model is trained on 144 samples. The tracker (with Faster R-CNN detections) reaches a Multiple Object Tracking Accuracy MOTA = 93, 5% and a Multiple Object Tracking Precision MOTP = 75, 6% on a validation dataset containing 2000 images, competing with state-of-the-art computer vision methods. The framework allows reliable tracking of different bumblebees in the same video stream with rarely occurring identity switches (IDS). MBT3D has much lower IDS than other commonly used algorithms, with one of the lowest false positive rates, competing with state-of-the-art animal tracking algorithms. The developed framework reconstructs the 3-dimensional (3D) flight paths of the bumblebees by triangulation. It also handles and compares two alternative stereo camera pairs if desired. Public Library of Science 2023-09-22 /pmc/articles/PMC10516433/ /pubmed/37738269 http://dx.doi.org/10.1371/journal.pone.0291415 Text en © 2023 Stiemer et al https://creativecommons.org/licenses/by/4.0/This is an open access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/) , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
spellingShingle Research Article
Stiemer, Luc Nicolas
Thoma, Andreas
Braun, Carsten
MBT3D: Deep learning based multi-object tracker for bumblebee 3D flight path estimation
title MBT3D: Deep learning based multi-object tracker for bumblebee 3D flight path estimation
title_full MBT3D: Deep learning based multi-object tracker for bumblebee 3D flight path estimation
title_fullStr MBT3D: Deep learning based multi-object tracker for bumblebee 3D flight path estimation
title_full_unstemmed MBT3D: Deep learning based multi-object tracker for bumblebee 3D flight path estimation
title_short MBT3D: Deep learning based multi-object tracker for bumblebee 3D flight path estimation
title_sort mbt3d: deep learning based multi-object tracker for bumblebee 3d flight path estimation
topic Research Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10516433/
https://www.ncbi.nlm.nih.gov/pubmed/37738269
http://dx.doi.org/10.1371/journal.pone.0291415
work_keys_str_mv AT stiemerlucnicolas mbt3ddeeplearningbasedmultiobjecttrackerforbumblebee3dflightpathestimation
AT thomaandreas mbt3ddeeplearningbasedmultiobjecttrackerforbumblebee3dflightpathestimation
AT brauncarsten mbt3ddeeplearningbasedmultiobjecttrackerforbumblebee3dflightpathestimation