Cargando…
Intelligent Complementary Multi-Modal Fusion for Anomaly Surveillance and Security System
Recently, security monitoring facilities have mainly adopted artificial intelligence (AI) technology to provide both increased security and improved performance. However, there are technical challenges in the pursuit of elevating system performance, automation, and security efficiency. In this paper...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10675370/ https://www.ncbi.nlm.nih.gov/pubmed/38005599 http://dx.doi.org/10.3390/s23229214 |
_version_ | 1785149811816660992 |
---|---|
author | Jeong, Jae-hyeok Jung, Hwan-hee Choi, Yong-hoon Park, Seong-hee Kim, Min-suk |
author_facet | Jeong, Jae-hyeok Jung, Hwan-hee Choi, Yong-hoon Park, Seong-hee Kim, Min-suk |
author_sort | Jeong, Jae-hyeok |
collection | PubMed |
description | Recently, security monitoring facilities have mainly adopted artificial intelligence (AI) technology to provide both increased security and improved performance. However, there are technical challenges in the pursuit of elevating system performance, automation, and security efficiency. In this paper, we proposed intelligent anomaly detection and classification based on deep learning (DL) using multi-modal fusion. To verify the method, we combined two DL-based schemes, such as (i) the 3D Convolutional AutoEncoder (3D-AE) for anomaly detection and (ii) the SlowFast neural network for anomaly classification. The 3D-AE can detect occurrence points of abnormal events and generate regions of interest (ROI) by the points. The SlowFast model can classify abnormal events using the ROI. These multi-modal approaches can complement weaknesses and leverage strengths in the existing security system. To enhance anomaly learning effectiveness, we also attempted to create a new dataset using the virtual environment in Grand Theft Auto 5 (GTA5). The dataset consists of 400 abnormal-state data and 78 normal-state data with clip sizes in the 8–20 s range. Virtual data collection can also supplement the original dataset, as replicating abnormal states in the real world is challenging. Consequently, the proposed method can achieve a classification accuracy of 85%, which is higher compared to the 77.5% accuracy achieved when only employing the single classification model. Furthermore, we validated the trained model with the GTA dataset by using a real-world assault class dataset, consisting of 1300 instances that we reproduced. As a result, 1100 data as the assault were classified and achieved 83.5% accuracy. This also shows that the proposed method can provide high performance in real-world environments. |
format | Online Article Text |
id | pubmed-10675370 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-106753702023-11-16 Intelligent Complementary Multi-Modal Fusion for Anomaly Surveillance and Security System Jeong, Jae-hyeok Jung, Hwan-hee Choi, Yong-hoon Park, Seong-hee Kim, Min-suk Sensors (Basel) Article Recently, security monitoring facilities have mainly adopted artificial intelligence (AI) technology to provide both increased security and improved performance. However, there are technical challenges in the pursuit of elevating system performance, automation, and security efficiency. In this paper, we proposed intelligent anomaly detection and classification based on deep learning (DL) using multi-modal fusion. To verify the method, we combined two DL-based schemes, such as (i) the 3D Convolutional AutoEncoder (3D-AE) for anomaly detection and (ii) the SlowFast neural network for anomaly classification. The 3D-AE can detect occurrence points of abnormal events and generate regions of interest (ROI) by the points. The SlowFast model can classify abnormal events using the ROI. These multi-modal approaches can complement weaknesses and leverage strengths in the existing security system. To enhance anomaly learning effectiveness, we also attempted to create a new dataset using the virtual environment in Grand Theft Auto 5 (GTA5). The dataset consists of 400 abnormal-state data and 78 normal-state data with clip sizes in the 8–20 s range. Virtual data collection can also supplement the original dataset, as replicating abnormal states in the real world is challenging. Consequently, the proposed method can achieve a classification accuracy of 85%, which is higher compared to the 77.5% accuracy achieved when only employing the single classification model. Furthermore, we validated the trained model with the GTA dataset by using a real-world assault class dataset, consisting of 1300 instances that we reproduced. As a result, 1100 data as the assault were classified and achieved 83.5% accuracy. This also shows that the proposed method can provide high performance in real-world environments. MDPI 2023-11-16 /pmc/articles/PMC10675370/ /pubmed/38005599 http://dx.doi.org/10.3390/s23229214 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Jeong, Jae-hyeok Jung, Hwan-hee Choi, Yong-hoon Park, Seong-hee Kim, Min-suk Intelligent Complementary Multi-Modal Fusion for Anomaly Surveillance and Security System |
title | Intelligent Complementary Multi-Modal Fusion for Anomaly Surveillance and Security System |
title_full | Intelligent Complementary Multi-Modal Fusion for Anomaly Surveillance and Security System |
title_fullStr | Intelligent Complementary Multi-Modal Fusion for Anomaly Surveillance and Security System |
title_full_unstemmed | Intelligent Complementary Multi-Modal Fusion for Anomaly Surveillance and Security System |
title_short | Intelligent Complementary Multi-Modal Fusion for Anomaly Surveillance and Security System |
title_sort | intelligent complementary multi-modal fusion for anomaly surveillance and security system |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10675370/ https://www.ncbi.nlm.nih.gov/pubmed/38005599 http://dx.doi.org/10.3390/s23229214 |
work_keys_str_mv | AT jeongjaehyeok intelligentcomplementarymultimodalfusionforanomalysurveillanceandsecuritysystem AT junghwanhee intelligentcomplementarymultimodalfusionforanomalysurveillanceandsecuritysystem AT choiyonghoon intelligentcomplementarymultimodalfusionforanomalysurveillanceandsecuritysystem AT parkseonghee intelligentcomplementarymultimodalfusionforanomalysurveillanceandsecuritysystem AT kimminsuk intelligentcomplementarymultimodalfusionforanomalysurveillanceandsecuritysystem |