Cargando…

A Deep-Learning Based Pipeline for Estimating the Abundance and Size of Aquatic Organisms in an Unconstrained Underwater Environment from Continuously Captured Stereo Video

The utilization of stationary underwater cameras is a modern and well-adapted approach to provide a continuous and cost-effective long-term solution to monitor underwater habitats of particular interest. A common goal of such monitoring systems is to gain better insight into the dynamics and conditi...

Descripción completa

Detalles Bibliográficos
Autores principales: Böer, Gordon, Gröger, Joachim Paul, Badri-Höher, Sabah, Cisewski, Boris, Renkewitz, Helge, Mittermayer, Felix, Strickmann, Tobias, Schramm, Hauke
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10054324/
https://www.ncbi.nlm.nih.gov/pubmed/36992022
http://dx.doi.org/10.3390/s23063311
_version_ 1785015643330838528
author Böer, Gordon
Gröger, Joachim Paul
Badri-Höher, Sabah
Cisewski, Boris
Renkewitz, Helge
Mittermayer, Felix
Strickmann, Tobias
Schramm, Hauke
author_facet Böer, Gordon
Gröger, Joachim Paul
Badri-Höher, Sabah
Cisewski, Boris
Renkewitz, Helge
Mittermayer, Felix
Strickmann, Tobias
Schramm, Hauke
author_sort Böer, Gordon
collection PubMed
description The utilization of stationary underwater cameras is a modern and well-adapted approach to provide a continuous and cost-effective long-term solution to monitor underwater habitats of particular interest. A common goal of such monitoring systems is to gain better insight into the dynamics and condition of populations of various marine organisms, such as migratory or commercially relevant fish taxa. This paper describes a complete processing pipeline to automatically determine the abundance, type and estimate the size of biological taxa from stereoscopic video data captured by the stereo camera of a stationary Underwater Fish Observatory (UFO). A calibration of the recording system was carried out in situ and, afterward, validated using the synchronously recorded sonar data. The video data were recorded continuously for nearly one year in the Kiel Fjord, an inlet of the Baltic Sea in northern Germany. It shows underwater organisms in their natural behavior, as passive low-light cameras were used instead of active lighting to dampen attraction effects and allow for the least invasive recording possible. The recorded raw data are pre-filtered by an adaptive background estimation to extract sequences with activity, which are then processed by a deep detection network, i.e., Yolov5. This provides the location and type of organisms detected in each video frame of both cameras, which are used to calculate stereo correspondences following a basic matching scheme. In a subsequent step, the size and distance of the depicted organisms are approximated using the corner coordinates of the matched bounding boxes. The Yolov5 model employed in this study was trained on a novel dataset comprising 73,144 images and 92,899 bounding box annotations for 10 categories of marine animals. The model achieved a mean detection accuracy of 92.4%, a mean average precision (mAP) of 94.8% and an F1 score of 93%.
format Online
Article
Text
id pubmed-10054324
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-100543242023-03-30 A Deep-Learning Based Pipeline for Estimating the Abundance and Size of Aquatic Organisms in an Unconstrained Underwater Environment from Continuously Captured Stereo Video Böer, Gordon Gröger, Joachim Paul Badri-Höher, Sabah Cisewski, Boris Renkewitz, Helge Mittermayer, Felix Strickmann, Tobias Schramm, Hauke Sensors (Basel) Article The utilization of stationary underwater cameras is a modern and well-adapted approach to provide a continuous and cost-effective long-term solution to monitor underwater habitats of particular interest. A common goal of such monitoring systems is to gain better insight into the dynamics and condition of populations of various marine organisms, such as migratory or commercially relevant fish taxa. This paper describes a complete processing pipeline to automatically determine the abundance, type and estimate the size of biological taxa from stereoscopic video data captured by the stereo camera of a stationary Underwater Fish Observatory (UFO). A calibration of the recording system was carried out in situ and, afterward, validated using the synchronously recorded sonar data. The video data were recorded continuously for nearly one year in the Kiel Fjord, an inlet of the Baltic Sea in northern Germany. It shows underwater organisms in their natural behavior, as passive low-light cameras were used instead of active lighting to dampen attraction effects and allow for the least invasive recording possible. The recorded raw data are pre-filtered by an adaptive background estimation to extract sequences with activity, which are then processed by a deep detection network, i.e., Yolov5. This provides the location and type of organisms detected in each video frame of both cameras, which are used to calculate stereo correspondences following a basic matching scheme. In a subsequent step, the size and distance of the depicted organisms are approximated using the corner coordinates of the matched bounding boxes. The Yolov5 model employed in this study was trained on a novel dataset comprising 73,144 images and 92,899 bounding box annotations for 10 categories of marine animals. The model achieved a mean detection accuracy of 92.4%, a mean average precision (mAP) of 94.8% and an F1 score of 93%. MDPI 2023-03-21 /pmc/articles/PMC10054324/ /pubmed/36992022 http://dx.doi.org/10.3390/s23063311 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Böer, Gordon
Gröger, Joachim Paul
Badri-Höher, Sabah
Cisewski, Boris
Renkewitz, Helge
Mittermayer, Felix
Strickmann, Tobias
Schramm, Hauke
A Deep-Learning Based Pipeline for Estimating the Abundance and Size of Aquatic Organisms in an Unconstrained Underwater Environment from Continuously Captured Stereo Video
title A Deep-Learning Based Pipeline for Estimating the Abundance and Size of Aquatic Organisms in an Unconstrained Underwater Environment from Continuously Captured Stereo Video
title_full A Deep-Learning Based Pipeline for Estimating the Abundance and Size of Aquatic Organisms in an Unconstrained Underwater Environment from Continuously Captured Stereo Video
title_fullStr A Deep-Learning Based Pipeline for Estimating the Abundance and Size of Aquatic Organisms in an Unconstrained Underwater Environment from Continuously Captured Stereo Video
title_full_unstemmed A Deep-Learning Based Pipeline for Estimating the Abundance and Size of Aquatic Organisms in an Unconstrained Underwater Environment from Continuously Captured Stereo Video
title_short A Deep-Learning Based Pipeline for Estimating the Abundance and Size of Aquatic Organisms in an Unconstrained Underwater Environment from Continuously Captured Stereo Video
title_sort deep-learning based pipeline for estimating the abundance and size of aquatic organisms in an unconstrained underwater environment from continuously captured stereo video
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10054324/
https://www.ncbi.nlm.nih.gov/pubmed/36992022
http://dx.doi.org/10.3390/s23063311
work_keys_str_mv AT boergordon adeeplearningbasedpipelineforestimatingtheabundanceandsizeofaquaticorganismsinanunconstrainedunderwaterenvironmentfromcontinuouslycapturedstereovideo
AT grogerjoachimpaul adeeplearningbasedpipelineforestimatingtheabundanceandsizeofaquaticorganismsinanunconstrainedunderwaterenvironmentfromcontinuouslycapturedstereovideo
AT badrihohersabah adeeplearningbasedpipelineforestimatingtheabundanceandsizeofaquaticorganismsinanunconstrainedunderwaterenvironmentfromcontinuouslycapturedstereovideo
AT cisewskiboris adeeplearningbasedpipelineforestimatingtheabundanceandsizeofaquaticorganismsinanunconstrainedunderwaterenvironmentfromcontinuouslycapturedstereovideo
AT renkewitzhelge adeeplearningbasedpipelineforestimatingtheabundanceandsizeofaquaticorganismsinanunconstrainedunderwaterenvironmentfromcontinuouslycapturedstereovideo
AT mittermayerfelix adeeplearningbasedpipelineforestimatingtheabundanceandsizeofaquaticorganismsinanunconstrainedunderwaterenvironmentfromcontinuouslycapturedstereovideo
AT strickmanntobias adeeplearningbasedpipelineforestimatingtheabundanceandsizeofaquaticorganismsinanunconstrainedunderwaterenvironmentfromcontinuouslycapturedstereovideo
AT schrammhauke adeeplearningbasedpipelineforestimatingtheabundanceandsizeofaquaticorganismsinanunconstrainedunderwaterenvironmentfromcontinuouslycapturedstereovideo
AT boergordon deeplearningbasedpipelineforestimatingtheabundanceandsizeofaquaticorganismsinanunconstrainedunderwaterenvironmentfromcontinuouslycapturedstereovideo
AT grogerjoachimpaul deeplearningbasedpipelineforestimatingtheabundanceandsizeofaquaticorganismsinanunconstrainedunderwaterenvironmentfromcontinuouslycapturedstereovideo
AT badrihohersabah deeplearningbasedpipelineforestimatingtheabundanceandsizeofaquaticorganismsinanunconstrainedunderwaterenvironmentfromcontinuouslycapturedstereovideo
AT cisewskiboris deeplearningbasedpipelineforestimatingtheabundanceandsizeofaquaticorganismsinanunconstrainedunderwaterenvironmentfromcontinuouslycapturedstereovideo
AT renkewitzhelge deeplearningbasedpipelineforestimatingtheabundanceandsizeofaquaticorganismsinanunconstrainedunderwaterenvironmentfromcontinuouslycapturedstereovideo
AT mittermayerfelix deeplearningbasedpipelineforestimatingtheabundanceandsizeofaquaticorganismsinanunconstrainedunderwaterenvironmentfromcontinuouslycapturedstereovideo
AT strickmanntobias deeplearningbasedpipelineforestimatingtheabundanceandsizeofaquaticorganismsinanunconstrainedunderwaterenvironmentfromcontinuouslycapturedstereovideo
AT schrammhauke deeplearningbasedpipelineforestimatingtheabundanceandsizeofaquaticorganismsinanunconstrainedunderwaterenvironmentfromcontinuouslycapturedstereovideo