Cargando…

DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational Assistance

In this paper, we introduce the so-called DEEP-SEE framework that jointly exploits computer vision algorithms and deep convolutional neural networks (CNNs) to detect, track and recognize in real time objects encountered during navigation in the outdoor environment. A first feature concerns an object...

Descripción completa

Detalles Bibliográficos
Autores principales: Tapu, Ruxandra, Mocanu, Bogdan, Zaharia, Titus
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2017
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5713031/
https://www.ncbi.nlm.nih.gov/pubmed/29143795
http://dx.doi.org/10.3390/s17112473
_version_ 1783283330465136640
author Tapu, Ruxandra
Mocanu, Bogdan
Zaharia, Titus
author_facet Tapu, Ruxandra
Mocanu, Bogdan
Zaharia, Titus
author_sort Tapu, Ruxandra
collection PubMed
description In this paper, we introduce the so-called DEEP-SEE framework that jointly exploits computer vision algorithms and deep convolutional neural networks (CNNs) to detect, track and recognize in real time objects encountered during navigation in the outdoor environment. A first feature concerns an object detection technique designed to localize both static and dynamic objects without any a priori knowledge about their position, type or shape. The methodological core of the proposed approach relies on a novel object tracking method based on two convolutional neural networks trained offline. The key principle consists of alternating between tracking using motion information and predicting the object location in time based on visual similarity. The validation of the tracking technique is performed on standard benchmark VOT datasets, and shows that the proposed approach returns state-of-the-art results while minimizing the computational complexity. Then, the DEEP-SEE framework is integrated into a novel assistive device, designed to improve cognition of VI people and to increase their safety when navigating in crowded urban scenes. The validation of our assistive device is performed on a video dataset with 30 elements acquired with the help of VI users. The proposed system shows high accuracy (>90%) and robustness (>90%) scores regardless on the scene dynamics.
format Online
Article
Text
id pubmed-5713031
institution National Center for Biotechnology Information
language English
publishDate 2017
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-57130312017-12-07 DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational Assistance Tapu, Ruxandra Mocanu, Bogdan Zaharia, Titus Sensors (Basel) Article In this paper, we introduce the so-called DEEP-SEE framework that jointly exploits computer vision algorithms and deep convolutional neural networks (CNNs) to detect, track and recognize in real time objects encountered during navigation in the outdoor environment. A first feature concerns an object detection technique designed to localize both static and dynamic objects without any a priori knowledge about their position, type or shape. The methodological core of the proposed approach relies on a novel object tracking method based on two convolutional neural networks trained offline. The key principle consists of alternating between tracking using motion information and predicting the object location in time based on visual similarity. The validation of the tracking technique is performed on standard benchmark VOT datasets, and shows that the proposed approach returns state-of-the-art results while minimizing the computational complexity. Then, the DEEP-SEE framework is integrated into a novel assistive device, designed to improve cognition of VI people and to increase their safety when navigating in crowded urban scenes. The validation of our assistive device is performed on a video dataset with 30 elements acquired with the help of VI users. The proposed system shows high accuracy (>90%) and robustness (>90%) scores regardless on the scene dynamics. MDPI 2017-10-28 /pmc/articles/PMC5713031/ /pubmed/29143795 http://dx.doi.org/10.3390/s17112473 Text en © 2017 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Tapu, Ruxandra
Mocanu, Bogdan
Zaharia, Titus
DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational Assistance
title DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational Assistance
title_full DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational Assistance
title_fullStr DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational Assistance
title_full_unstemmed DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational Assistance
title_short DEEP-SEE: Joint Object Detection, Tracking and Recognition with Application to Visually Impaired Navigational Assistance
title_sort deep-see: joint object detection, tracking and recognition with application to visually impaired navigational assistance
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5713031/
https://www.ncbi.nlm.nih.gov/pubmed/29143795
http://dx.doi.org/10.3390/s17112473
work_keys_str_mv AT tapuruxandra deepseejointobjectdetectiontrackingandrecognitionwithapplicationtovisuallyimpairednavigationalassistance
AT mocanubogdan deepseejointobjectdetectiontrackingandrecognitionwithapplicationtovisuallyimpairednavigationalassistance
AT zahariatitus deepseejointobjectdetectiontrackingandrecognitionwithapplicationtovisuallyimpairednavigationalassistance