Cargando…

Saliency Detection with Moving Camera via Background Model Completion

Detecting saliency in videos is a fundamental step in many computer vision systems. Saliency is the significant target(s) in the video. The object of interest is further analyzed for high-level applications. The segregation of saliency and the background can be made if they exhibit different visual...

Descripción completa

Detalles Bibliográficos
Autores principales: Zhang, Yu-Pei, Chan, Kwok-Leung
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2021
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8707474/
https://www.ncbi.nlm.nih.gov/pubmed/34960461
http://dx.doi.org/10.3390/s21248374
_version_ 1784622445683015680
author Zhang, Yu-Pei
Chan, Kwok-Leung
author_facet Zhang, Yu-Pei
Chan, Kwok-Leung
author_sort Zhang, Yu-Pei
collection PubMed
description Detecting saliency in videos is a fundamental step in many computer vision systems. Saliency is the significant target(s) in the video. The object of interest is further analyzed for high-level applications. The segregation of saliency and the background can be made if they exhibit different visual cues. Therefore, saliency detection is often formulated as background subtraction. However, saliency detection is challenging. For instance, dynamic background can result in false positive errors. In another scenario, camouflage will result in false negative errors. With moving cameras, the captured scenes are even more complicated to handle. We propose a new framework, called saliency detection via background model completion (SD-BMC), that comprises a background modeler and a deep learning background/foreground segmentation network. The background modeler generates an initial clean background image from a short image sequence. Based on the idea of video completion, a good background frame can be synthesized with the co-existence of changing background and moving objects. We adopt the background/foreground segmenter, which was pre-trained with a specific video dataset. It can also detect saliency in unseen videos. The background modeler can adjust the background image dynamically when the background/foreground segmenter output deteriorates during processing a long video. To the best of our knowledge, our framework is the first one to adopt video completion for background modeling and saliency detection in videos captured by moving cameras. The F-measure results, obtained from the pan-tilt-zoom (PTZ) videos, show that our proposed framework outperforms some deep learning-based background subtraction models by 11% or more. With more challenging videos, our framework also outperforms many high-ranking background subtraction methods by more than 3%.
format Online
Article
Text
id pubmed-8707474
institution National Center for Biotechnology Information
language English
publishDate 2021
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-87074742021-12-25 Saliency Detection with Moving Camera via Background Model Completion Zhang, Yu-Pei Chan, Kwok-Leung Sensors (Basel) Article Detecting saliency in videos is a fundamental step in many computer vision systems. Saliency is the significant target(s) in the video. The object of interest is further analyzed for high-level applications. The segregation of saliency and the background can be made if they exhibit different visual cues. Therefore, saliency detection is often formulated as background subtraction. However, saliency detection is challenging. For instance, dynamic background can result in false positive errors. In another scenario, camouflage will result in false negative errors. With moving cameras, the captured scenes are even more complicated to handle. We propose a new framework, called saliency detection via background model completion (SD-BMC), that comprises a background modeler and a deep learning background/foreground segmentation network. The background modeler generates an initial clean background image from a short image sequence. Based on the idea of video completion, a good background frame can be synthesized with the co-existence of changing background and moving objects. We adopt the background/foreground segmenter, which was pre-trained with a specific video dataset. It can also detect saliency in unseen videos. The background modeler can adjust the background image dynamically when the background/foreground segmenter output deteriorates during processing a long video. To the best of our knowledge, our framework is the first one to adopt video completion for background modeling and saliency detection in videos captured by moving cameras. The F-measure results, obtained from the pan-tilt-zoom (PTZ) videos, show that our proposed framework outperforms some deep learning-based background subtraction models by 11% or more. With more challenging videos, our framework also outperforms many high-ranking background subtraction methods by more than 3%. MDPI 2021-12-15 /pmc/articles/PMC8707474/ /pubmed/34960461 http://dx.doi.org/10.3390/s21248374 Text en © 2021 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Zhang, Yu-Pei
Chan, Kwok-Leung
Saliency Detection with Moving Camera via Background Model Completion
title Saliency Detection with Moving Camera via Background Model Completion
title_full Saliency Detection with Moving Camera via Background Model Completion
title_fullStr Saliency Detection with Moving Camera via Background Model Completion
title_full_unstemmed Saliency Detection with Moving Camera via Background Model Completion
title_short Saliency Detection with Moving Camera via Background Model Completion
title_sort saliency detection with moving camera via background model completion
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8707474/
https://www.ncbi.nlm.nih.gov/pubmed/34960461
http://dx.doi.org/10.3390/s21248374
work_keys_str_mv AT zhangyupei saliencydetectionwithmovingcameraviabackgroundmodelcompletion
AT chankwokleung saliencydetectionwithmovingcameraviabackgroundmodelcompletion