Cargando…
Content Swapping: A New Image Synthesis for Construction Sign Detection in Autonomous Vehicles
Construction signs alert drivers to the dangers of abnormally blocked roads. In the case of autonomous vehicles, construction signs should be detected automatically to prevent accidents. One might think that we can accomplish the goal easily using the popular deep-learning-based detectors, but it is...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2022
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9105141/ https://www.ncbi.nlm.nih.gov/pubmed/35591185 http://dx.doi.org/10.3390/s22093494 |
_version_ | 1784707967662161920 |
---|---|
author | Seong, Hongje Baik, Seunghyun Lee, Youngjo Lee, Suhyeon Kim, Euntai |
author_facet | Seong, Hongje Baik, Seunghyun Lee, Youngjo Lee, Suhyeon Kim, Euntai |
author_sort | Seong, Hongje |
collection | PubMed |
description | Construction signs alert drivers to the dangers of abnormally blocked roads. In the case of autonomous vehicles, construction signs should be detected automatically to prevent accidents. One might think that we can accomplish the goal easily using the popular deep-learning-based detectors, but it is not the case. To train the deep learning detectors to detect construction signs, we need a large amount of training images which contain construction signs. However, collecting training images including construction signs is very difficult in the real world because construction events do not occur frequently. To make matters worse, the construction signs might have dozens of different construction signs (i.e., contents). To address this problem, we propose a new method named content swapping. Our content swapping divides a construction sign into two parts: the board and the frame. Content swapping generates numerous synthetic construction signs by combining the board images (i.e., contents) taken from the in-domain images and the frames (i.e., geometric shapes) taken from the out-domain images. The generated synthetic construction signs are then added to the background road images via the cut-and-paste mechanism, increasing the number of training images. Furthermore, three fine-tuning methods regarding the region, size, and color of the construction signs are developed to make the generated training images look more realistic. To validate our approach, we applied our method to real-world images captured in South Korea. Finally, we achieve an average precision (AP(50)) score of 84.98%, which surpasses that of the off-the-shelf method by 9.15%. Full experimental results are available online as a supplemental video. The images used in the experiments are also released as a new dataset CSS138 for the benefit of the autonomous driving community. |
format | Online Article Text |
id | pubmed-9105141 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2022 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-91051412022-05-14 Content Swapping: A New Image Synthesis for Construction Sign Detection in Autonomous Vehicles Seong, Hongje Baik, Seunghyun Lee, Youngjo Lee, Suhyeon Kim, Euntai Sensors (Basel) Article Construction signs alert drivers to the dangers of abnormally blocked roads. In the case of autonomous vehicles, construction signs should be detected automatically to prevent accidents. One might think that we can accomplish the goal easily using the popular deep-learning-based detectors, but it is not the case. To train the deep learning detectors to detect construction signs, we need a large amount of training images which contain construction signs. However, collecting training images including construction signs is very difficult in the real world because construction events do not occur frequently. To make matters worse, the construction signs might have dozens of different construction signs (i.e., contents). To address this problem, we propose a new method named content swapping. Our content swapping divides a construction sign into two parts: the board and the frame. Content swapping generates numerous synthetic construction signs by combining the board images (i.e., contents) taken from the in-domain images and the frames (i.e., geometric shapes) taken from the out-domain images. The generated synthetic construction signs are then added to the background road images via the cut-and-paste mechanism, increasing the number of training images. Furthermore, three fine-tuning methods regarding the region, size, and color of the construction signs are developed to make the generated training images look more realistic. To validate our approach, we applied our method to real-world images captured in South Korea. Finally, we achieve an average precision (AP(50)) score of 84.98%, which surpasses that of the off-the-shelf method by 9.15%. Full experimental results are available online as a supplemental video. The images used in the experiments are also released as a new dataset CSS138 for the benefit of the autonomous driving community. MDPI 2022-05-04 /pmc/articles/PMC9105141/ /pubmed/35591185 http://dx.doi.org/10.3390/s22093494 Text en © 2022 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Seong, Hongje Baik, Seunghyun Lee, Youngjo Lee, Suhyeon Kim, Euntai Content Swapping: A New Image Synthesis for Construction Sign Detection in Autonomous Vehicles |
title | Content Swapping: A New Image Synthesis for Construction Sign Detection in Autonomous Vehicles |
title_full | Content Swapping: A New Image Synthesis for Construction Sign Detection in Autonomous Vehicles |
title_fullStr | Content Swapping: A New Image Synthesis for Construction Sign Detection in Autonomous Vehicles |
title_full_unstemmed | Content Swapping: A New Image Synthesis for Construction Sign Detection in Autonomous Vehicles |
title_short | Content Swapping: A New Image Synthesis for Construction Sign Detection in Autonomous Vehicles |
title_sort | content swapping: a new image synthesis for construction sign detection in autonomous vehicles |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9105141/ https://www.ncbi.nlm.nih.gov/pubmed/35591185 http://dx.doi.org/10.3390/s22093494 |
work_keys_str_mv | AT seonghongje contentswappinganewimagesynthesisforconstructionsigndetectioninautonomousvehicles AT baikseunghyun contentswappinganewimagesynthesisforconstructionsigndetectioninautonomousvehicles AT leeyoungjo contentswappinganewimagesynthesisforconstructionsigndetectioninautonomousvehicles AT leesuhyeon contentswappinganewimagesynthesisforconstructionsigndetectioninautonomousvehicles AT kimeuntai contentswappinganewimagesynthesisforconstructionsigndetectioninautonomousvehicles |