Cargando…

Intelligent Complementary Multi-Modal Fusion for Anomaly Surveillance and Security System

Recently, security monitoring facilities have mainly adopted artificial intelligence (AI) technology to provide both increased security and improved performance. However, there are technical challenges in the pursuit of elevating system performance, automation, and security efficiency. In this paper...

Descripción completa

Detalles Bibliográficos
Autores principales: Jeong, Jae-hyeok, Jung, Hwan-hee, Choi, Yong-hoon, Park, Seong-hee, Kim, Min-suk
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10675370/
https://www.ncbi.nlm.nih.gov/pubmed/38005599
http://dx.doi.org/10.3390/s23229214
_version_ 1785149811816660992
author Jeong, Jae-hyeok
Jung, Hwan-hee
Choi, Yong-hoon
Park, Seong-hee
Kim, Min-suk
author_facet Jeong, Jae-hyeok
Jung, Hwan-hee
Choi, Yong-hoon
Park, Seong-hee
Kim, Min-suk
author_sort Jeong, Jae-hyeok
collection PubMed
description Recently, security monitoring facilities have mainly adopted artificial intelligence (AI) technology to provide both increased security and improved performance. However, there are technical challenges in the pursuit of elevating system performance, automation, and security efficiency. In this paper, we proposed intelligent anomaly detection and classification based on deep learning (DL) using multi-modal fusion. To verify the method, we combined two DL-based schemes, such as (i) the 3D Convolutional AutoEncoder (3D-AE) for anomaly detection and (ii) the SlowFast neural network for anomaly classification. The 3D-AE can detect occurrence points of abnormal events and generate regions of interest (ROI) by the points. The SlowFast model can classify abnormal events using the ROI. These multi-modal approaches can complement weaknesses and leverage strengths in the existing security system. To enhance anomaly learning effectiveness, we also attempted to create a new dataset using the virtual environment in Grand Theft Auto 5 (GTA5). The dataset consists of 400 abnormal-state data and 78 normal-state data with clip sizes in the 8–20 s range. Virtual data collection can also supplement the original dataset, as replicating abnormal states in the real world is challenging. Consequently, the proposed method can achieve a classification accuracy of 85%, which is higher compared to the 77.5% accuracy achieved when only employing the single classification model. Furthermore, we validated the trained model with the GTA dataset by using a real-world assault class dataset, consisting of 1300 instances that we reproduced. As a result, 1100 data as the assault were classified and achieved 83.5% accuracy. This also shows that the proposed method can provide high performance in real-world environments.
format Online
Article
Text
id pubmed-10675370
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-106753702023-11-16 Intelligent Complementary Multi-Modal Fusion for Anomaly Surveillance and Security System Jeong, Jae-hyeok Jung, Hwan-hee Choi, Yong-hoon Park, Seong-hee Kim, Min-suk Sensors (Basel) Article Recently, security monitoring facilities have mainly adopted artificial intelligence (AI) technology to provide both increased security and improved performance. However, there are technical challenges in the pursuit of elevating system performance, automation, and security efficiency. In this paper, we proposed intelligent anomaly detection and classification based on deep learning (DL) using multi-modal fusion. To verify the method, we combined two DL-based schemes, such as (i) the 3D Convolutional AutoEncoder (3D-AE) for anomaly detection and (ii) the SlowFast neural network for anomaly classification. The 3D-AE can detect occurrence points of abnormal events and generate regions of interest (ROI) by the points. The SlowFast model can classify abnormal events using the ROI. These multi-modal approaches can complement weaknesses and leverage strengths in the existing security system. To enhance anomaly learning effectiveness, we also attempted to create a new dataset using the virtual environment in Grand Theft Auto 5 (GTA5). The dataset consists of 400 abnormal-state data and 78 normal-state data with clip sizes in the 8–20 s range. Virtual data collection can also supplement the original dataset, as replicating abnormal states in the real world is challenging. Consequently, the proposed method can achieve a classification accuracy of 85%, which is higher compared to the 77.5% accuracy achieved when only employing the single classification model. Furthermore, we validated the trained model with the GTA dataset by using a real-world assault class dataset, consisting of 1300 instances that we reproduced. As a result, 1100 data as the assault were classified and achieved 83.5% accuracy. This also shows that the proposed method can provide high performance in real-world environments. MDPI 2023-11-16 /pmc/articles/PMC10675370/ /pubmed/38005599 http://dx.doi.org/10.3390/s23229214 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Jeong, Jae-hyeok
Jung, Hwan-hee
Choi, Yong-hoon
Park, Seong-hee
Kim, Min-suk
Intelligent Complementary Multi-Modal Fusion for Anomaly Surveillance and Security System
title Intelligent Complementary Multi-Modal Fusion for Anomaly Surveillance and Security System
title_full Intelligent Complementary Multi-Modal Fusion for Anomaly Surveillance and Security System
title_fullStr Intelligent Complementary Multi-Modal Fusion for Anomaly Surveillance and Security System
title_full_unstemmed Intelligent Complementary Multi-Modal Fusion for Anomaly Surveillance and Security System
title_short Intelligent Complementary Multi-Modal Fusion for Anomaly Surveillance and Security System
title_sort intelligent complementary multi-modal fusion for anomaly surveillance and security system
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10675370/
https://www.ncbi.nlm.nih.gov/pubmed/38005599
http://dx.doi.org/10.3390/s23229214
work_keys_str_mv AT jeongjaehyeok intelligentcomplementarymultimodalfusionforanomalysurveillanceandsecuritysystem
AT junghwanhee intelligentcomplementarymultimodalfusionforanomalysurveillanceandsecuritysystem
AT choiyonghoon intelligentcomplementarymultimodalfusionforanomalysurveillanceandsecuritysystem
AT parkseonghee intelligentcomplementarymultimodalfusionforanomalysurveillanceandsecuritysystem
AT kimminsuk intelligentcomplementarymultimodalfusionforanomalysurveillanceandsecuritysystem