Cargando…

Real-Time Ship Segmentation in Maritime Surveillance Videos Using Automatically Annotated Synthetic Datasets

This work proposes a new system capable of real-time ship instance segmentation during maritime surveillance missions by unmanned aerial vehicles using an onboard standard RGB camera. The implementation requires two stages: an instance segmentation network able to produce fast and reliable prelimina...

Descripción completa

Detalles Bibliográficos
Autores principales: Ribeiro, Miguel, Damas, Bruno, Bernardino, Alexandre
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9656424/
https://www.ncbi.nlm.nih.gov/pubmed/36365787
http://dx.doi.org/10.3390/s22218090
_version_ 1784829430880796672
author Ribeiro, Miguel
Damas, Bruno
Bernardino, Alexandre
author_facet Ribeiro, Miguel
Damas, Bruno
Bernardino, Alexandre
author_sort Ribeiro, Miguel
collection PubMed
description This work proposes a new system capable of real-time ship instance segmentation during maritime surveillance missions by unmanned aerial vehicles using an onboard standard RGB camera. The implementation requires two stages: an instance segmentation network able to produce fast and reliable preliminary segmentation results and a post-processing 3D fully connected Conditional Random Field, which significantly improves segmentation results by exploring temporal correlations between nearby frames in video sequences. Moreover, due to the absence of maritime datasets consisting of properly labeled video sequences, we create a new dataset comprising synthetic video sequences of maritime surveillance scenarios (MarSyn). The main advantages of this approach are the possibility of generating a vast set of images and videos, being able to represent real-world scenarios without the necessity of deploying the real vehicle, and automatic labels, which eliminate human labeling errors. We train the system with the MarSyn dataset and with aerial footage from publicly available annotated maritime datasets to validate the proposed approach. We present some experimental results and compare them to other approaches, and we also illustrate the temporal stability provided by the second stage in missing frames and wrong segmentation scenarios.
format Online
Article
Text
id pubmed-9656424
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-96564242022-11-15 Real-Time Ship Segmentation in Maritime Surveillance Videos Using Automatically Annotated Synthetic Datasets Ribeiro, Miguel Damas, Bruno Bernardino, Alexandre Sensors (Basel) Article This work proposes a new system capable of real-time ship instance segmentation during maritime surveillance missions by unmanned aerial vehicles using an onboard standard RGB camera. The implementation requires two stages: an instance segmentation network able to produce fast and reliable preliminary segmentation results and a post-processing 3D fully connected Conditional Random Field, which significantly improves segmentation results by exploring temporal correlations between nearby frames in video sequences. Moreover, due to the absence of maritime datasets consisting of properly labeled video sequences, we create a new dataset comprising synthetic video sequences of maritime surveillance scenarios (MarSyn). The main advantages of this approach are the possibility of generating a vast set of images and videos, being able to represent real-world scenarios without the necessity of deploying the real vehicle, and automatic labels, which eliminate human labeling errors. We train the system with the MarSyn dataset and with aerial footage from publicly available annotated maritime datasets to validate the proposed approach. We present some experimental results and compare them to other approaches, and we also illustrate the temporal stability provided by the second stage in missing frames and wrong segmentation scenarios. MDPI 2022-10-22 /pmc/articles/PMC9656424/ /pubmed/36365787 http://dx.doi.org/10.3390/s22218090 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Ribeiro, Miguel
Damas, Bruno
Bernardino, Alexandre
Real-Time Ship Segmentation in Maritime Surveillance Videos Using Automatically Annotated Synthetic Datasets
title Real-Time Ship Segmentation in Maritime Surveillance Videos Using Automatically Annotated Synthetic Datasets
title_full Real-Time Ship Segmentation in Maritime Surveillance Videos Using Automatically Annotated Synthetic Datasets
title_fullStr Real-Time Ship Segmentation in Maritime Surveillance Videos Using Automatically Annotated Synthetic Datasets
title_full_unstemmed Real-Time Ship Segmentation in Maritime Surveillance Videos Using Automatically Annotated Synthetic Datasets
title_short Real-Time Ship Segmentation in Maritime Surveillance Videos Using Automatically Annotated Synthetic Datasets
title_sort real-time ship segmentation in maritime surveillance videos using automatically annotated synthetic datasets
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9656424/
https://www.ncbi.nlm.nih.gov/pubmed/36365787
http://dx.doi.org/10.3390/s22218090
work_keys_str_mv AT ribeiromiguel realtimeshipsegmentationinmaritimesurveillancevideosusingautomaticallyannotatedsyntheticdatasets
AT damasbruno realtimeshipsegmentationinmaritimesurveillancevideosusingautomaticallyannotatedsyntheticdatasets
AT bernardinoalexandre realtimeshipsegmentationinmaritimesurveillancevideosusingautomaticallyannotatedsyntheticdatasets