Cargando…

Content Swapping: A New Image Synthesis for Construction Sign Detection in Autonomous Vehicles

Construction signs alert drivers to the dangers of abnormally blocked roads. In the case of autonomous vehicles, construction signs should be detected automatically to prevent accidents. One might think that we can accomplish the goal easily using the popular deep-learning-based detectors, but it is...

Descripción completa

Detalles Bibliográficos
Autores principales: Seong, Hongje, Baik, Seunghyun, Lee, Youngjo, Lee, Suhyeon, Kim, Euntai
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2022
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9105141/
https://www.ncbi.nlm.nih.gov/pubmed/35591185
http://dx.doi.org/10.3390/s22093494
_version_ 1784707967662161920
author Seong, Hongje
Baik, Seunghyun
Lee, Youngjo
Lee, Suhyeon
Kim, Euntai
author_facet Seong, Hongje
Baik, Seunghyun
Lee, Youngjo
Lee, Suhyeon
Kim, Euntai
author_sort Seong, Hongje
collection PubMed
description Construction signs alert drivers to the dangers of abnormally blocked roads. In the case of autonomous vehicles, construction signs should be detected automatically to prevent accidents. One might think that we can accomplish the goal easily using the popular deep-learning-based detectors, but it is not the case. To train the deep learning detectors to detect construction signs, we need a large amount of training images which contain construction signs. However, collecting training images including construction signs is very difficult in the real world because construction events do not occur frequently. To make matters worse, the construction signs might have dozens of different construction signs (i.e., contents). To address this problem, we propose a new method named content swapping. Our content swapping divides a construction sign into two parts: the board and the frame. Content swapping generates numerous synthetic construction signs by combining the board images (i.e., contents) taken from the in-domain images and the frames (i.e., geometric shapes) taken from the out-domain images. The generated synthetic construction signs are then added to the background road images via the cut-and-paste mechanism, increasing the number of training images. Furthermore, three fine-tuning methods regarding the region, size, and color of the construction signs are developed to make the generated training images look more realistic. To validate our approach, we applied our method to real-world images captured in South Korea. Finally, we achieve an average precision (AP(50)) score of 84.98%, which surpasses that of the off-the-shelf method by 9.15%. Full experimental results are available online as a supplemental video. The images used in the experiments are also released as a new dataset CSS138 for the benefit of the autonomous driving community.
format Online
Article
Text
id pubmed-9105141
institution National Center for Biotechnology Information
language English
publishDate 2022
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-91051412022-05-14 Content Swapping: A New Image Synthesis for Construction Sign Detection in Autonomous Vehicles Seong, Hongje Baik, Seunghyun Lee, Youngjo Lee, Suhyeon Kim, Euntai Sensors (Basel) Article Construction signs alert drivers to the dangers of abnormally blocked roads. In the case of autonomous vehicles, construction signs should be detected automatically to prevent accidents. One might think that we can accomplish the goal easily using the popular deep-learning-based detectors, but it is not the case. To train the deep learning detectors to detect construction signs, we need a large amount of training images which contain construction signs. However, collecting training images including construction signs is very difficult in the real world because construction events do not occur frequently. To make matters worse, the construction signs might have dozens of different construction signs (i.e., contents). To address this problem, we propose a new method named content swapping. Our content swapping divides a construction sign into two parts: the board and the frame. Content swapping generates numerous synthetic construction signs by combining the board images (i.e., contents) taken from the in-domain images and the frames (i.e., geometric shapes) taken from the out-domain images. The generated synthetic construction signs are then added to the background road images via the cut-and-paste mechanism, increasing the number of training images. Furthermore, three fine-tuning methods regarding the region, size, and color of the construction signs are developed to make the generated training images look more realistic. To validate our approach, we applied our method to real-world images captured in South Korea. Finally, we achieve an average precision (AP(50)) score of 84.98%, which surpasses that of the off-the-shelf method by 9.15%. Full experimental results are available online as a supplemental video. The images used in the experiments are also released as a new dataset CSS138 for the benefit of the autonomous driving community. MDPI 2022-05-04 /pmc/articles/PMC9105141/ /pubmed/35591185 http://dx.doi.org/10.3390/s22093494 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Seong, Hongje
Baik, Seunghyun
Lee, Youngjo
Lee, Suhyeon
Kim, Euntai
Content Swapping: A New Image Synthesis for Construction Sign Detection in Autonomous Vehicles
title Content Swapping: A New Image Synthesis for Construction Sign Detection in Autonomous Vehicles
title_full Content Swapping: A New Image Synthesis for Construction Sign Detection in Autonomous Vehicles
title_fullStr Content Swapping: A New Image Synthesis for Construction Sign Detection in Autonomous Vehicles
title_full_unstemmed Content Swapping: A New Image Synthesis for Construction Sign Detection in Autonomous Vehicles
title_short Content Swapping: A New Image Synthesis for Construction Sign Detection in Autonomous Vehicles
title_sort content swapping: a new image synthesis for construction sign detection in autonomous vehicles
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9105141/
https://www.ncbi.nlm.nih.gov/pubmed/35591185
http://dx.doi.org/10.3390/s22093494
work_keys_str_mv AT seonghongje contentswappinganewimagesynthesisforconstructionsigndetectioninautonomousvehicles
AT baikseunghyun contentswappinganewimagesynthesisforconstructionsigndetectioninautonomousvehicles
AT leeyoungjo contentswappinganewimagesynthesisforconstructionsigndetectioninautonomousvehicles
AT leesuhyeon contentswappinganewimagesynthesisforconstructionsigndetectioninautonomousvehicles
AT kimeuntai contentswappinganewimagesynthesisforconstructionsigndetectioninautonomousvehicles