Cargando…
A Multitask Cascading CNN with MultiScale Infrared Optical Flow Feature Fusion-Based Abnormal Crowd Behavior Monitoring UAV †
Visual-based object detection and understanding is an important problem in computer vision and signal processing. Due to their advantages of high mobility and easy deployment, unmanned aerial vehicles (UAV) have become a flexible monitoring platform in recent years. However, visible-light-based meth...
Autores principales: | , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7582990/ https://www.ncbi.nlm.nih.gov/pubmed/32998316 http://dx.doi.org/10.3390/s20195550 |
_version_ | 1783599318256582656 |
---|---|
author | Shao, Yanhua Li, Wenfeng Chu, Hongyu Chang, Zhiyuan Zhang, Xiaoqiang Zhan, Huayi |
author_facet | Shao, Yanhua Li, Wenfeng Chu, Hongyu Chang, Zhiyuan Zhang, Xiaoqiang Zhan, Huayi |
author_sort | Shao, Yanhua |
collection | PubMed |
description | Visual-based object detection and understanding is an important problem in computer vision and signal processing. Due to their advantages of high mobility and easy deployment, unmanned aerial vehicles (UAV) have become a flexible monitoring platform in recent years. However, visible-light-based methods are often greatly influenced by the environment. As a result, a single type of feature derived from aerial monitoring videos is often insufficient to characterize variations among different abnormal crowd behaviors. To address this, we propose combining two types of features to better represent behavior, namely, multitask cascading CNN (MC-CNN) and multiscale infrared optical flow (MIR-OF), capturing both crowd density and average speed and the appearances of the crowd behaviors, respectively. First, an infrared (IR) camera and Nvidia Jetson TX1 were chosen as an infrared vision system. Since there are no published infrared-based aerial abnormal-behavior datasets, we provide a new infrared aerial dataset named the IR-flying dataset, which includes sample pictures and videos in different scenes of public areas. Second, MC-CNN was used to estimate the crowd density. Third, MIR-OF was designed to characterize the average speed of crowd. Finally, considering two typical abnormal crowd behaviors of crowd aggregating and crowd escaping, the experimental results show that the monitoring UAV system can detect abnormal crowd behaviors in public areas effectively. |
format | Online Article Text |
id | pubmed-7582990 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-75829902020-10-28 A Multitask Cascading CNN with MultiScale Infrared Optical Flow Feature Fusion-Based Abnormal Crowd Behavior Monitoring UAV † Shao, Yanhua Li, Wenfeng Chu, Hongyu Chang, Zhiyuan Zhang, Xiaoqiang Zhan, Huayi Sensors (Basel) Article Visual-based object detection and understanding is an important problem in computer vision and signal processing. Due to their advantages of high mobility and easy deployment, unmanned aerial vehicles (UAV) have become a flexible monitoring platform in recent years. However, visible-light-based methods are often greatly influenced by the environment. As a result, a single type of feature derived from aerial monitoring videos is often insufficient to characterize variations among different abnormal crowd behaviors. To address this, we propose combining two types of features to better represent behavior, namely, multitask cascading CNN (MC-CNN) and multiscale infrared optical flow (MIR-OF), capturing both crowd density and average speed and the appearances of the crowd behaviors, respectively. First, an infrared (IR) camera and Nvidia Jetson TX1 were chosen as an infrared vision system. Since there are no published infrared-based aerial abnormal-behavior datasets, we provide a new infrared aerial dataset named the IR-flying dataset, which includes sample pictures and videos in different scenes of public areas. Second, MC-CNN was used to estimate the crowd density. Third, MIR-OF was designed to characterize the average speed of crowd. Finally, considering two typical abnormal crowd behaviors of crowd aggregating and crowd escaping, the experimental results show that the monitoring UAV system can detect abnormal crowd behaviors in public areas effectively. MDPI 2020-09-28 /pmc/articles/PMC7582990/ /pubmed/32998316 http://dx.doi.org/10.3390/s20195550 Text en © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Shao, Yanhua Li, Wenfeng Chu, Hongyu Chang, Zhiyuan Zhang, Xiaoqiang Zhan, Huayi A Multitask Cascading CNN with MultiScale Infrared Optical Flow Feature Fusion-Based Abnormal Crowd Behavior Monitoring UAV † |
title | A Multitask Cascading CNN with MultiScale Infrared Optical Flow Feature Fusion-Based Abnormal Crowd Behavior Monitoring UAV † |
title_full | A Multitask Cascading CNN with MultiScale Infrared Optical Flow Feature Fusion-Based Abnormal Crowd Behavior Monitoring UAV † |
title_fullStr | A Multitask Cascading CNN with MultiScale Infrared Optical Flow Feature Fusion-Based Abnormal Crowd Behavior Monitoring UAV † |
title_full_unstemmed | A Multitask Cascading CNN with MultiScale Infrared Optical Flow Feature Fusion-Based Abnormal Crowd Behavior Monitoring UAV † |
title_short | A Multitask Cascading CNN with MultiScale Infrared Optical Flow Feature Fusion-Based Abnormal Crowd Behavior Monitoring UAV † |
title_sort | multitask cascading cnn with multiscale infrared optical flow feature fusion-based abnormal crowd behavior monitoring uav † |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7582990/ https://www.ncbi.nlm.nih.gov/pubmed/32998316 http://dx.doi.org/10.3390/s20195550 |
work_keys_str_mv | AT shaoyanhua amultitaskcascadingcnnwithmultiscaleinfraredopticalflowfeaturefusionbasedabnormalcrowdbehaviormonitoringuav AT liwenfeng amultitaskcascadingcnnwithmultiscaleinfraredopticalflowfeaturefusionbasedabnormalcrowdbehaviormonitoringuav AT chuhongyu amultitaskcascadingcnnwithmultiscaleinfraredopticalflowfeaturefusionbasedabnormalcrowdbehaviormonitoringuav AT changzhiyuan amultitaskcascadingcnnwithmultiscaleinfraredopticalflowfeaturefusionbasedabnormalcrowdbehaviormonitoringuav AT zhangxiaoqiang amultitaskcascadingcnnwithmultiscaleinfraredopticalflowfeaturefusionbasedabnormalcrowdbehaviormonitoringuav AT zhanhuayi amultitaskcascadingcnnwithmultiscaleinfraredopticalflowfeaturefusionbasedabnormalcrowdbehaviormonitoringuav AT shaoyanhua multitaskcascadingcnnwithmultiscaleinfraredopticalflowfeaturefusionbasedabnormalcrowdbehaviormonitoringuav AT liwenfeng multitaskcascadingcnnwithmultiscaleinfraredopticalflowfeaturefusionbasedabnormalcrowdbehaviormonitoringuav AT chuhongyu multitaskcascadingcnnwithmultiscaleinfraredopticalflowfeaturefusionbasedabnormalcrowdbehaviormonitoringuav AT changzhiyuan multitaskcascadingcnnwithmultiscaleinfraredopticalflowfeaturefusionbasedabnormalcrowdbehaviormonitoringuav AT zhangxiaoqiang multitaskcascadingcnnwithmultiscaleinfraredopticalflowfeaturefusionbasedabnormalcrowdbehaviormonitoringuav AT zhanhuayi multitaskcascadingcnnwithmultiscaleinfraredopticalflowfeaturefusionbasedabnormalcrowdbehaviormonitoringuav |