Cargando…

Improving Temporal Stability and Accuracy for Endoscopic Video Tissue Classification Using Recurrent Neural Networks

Early Barrett’s neoplasia are often missed due to subtle visual features and inexperience of the non-expert endoscopist with such lesions. While promising results have been reported on the automated detection of this type of early cancer in still endoscopic images, video-based detection using the te...

Descripción completa

Detalles Bibliográficos
Autores principales: Boers, Tim, van der Putten, Joost, Struyvenberg, Maarten, Fockens, Kiki, Jukema, Jelmer, Schoon, Erik, van der Sommen, Fons, Bergman, Jacques, de With, Peter
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2020
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7436238/
https://www.ncbi.nlm.nih.gov/pubmed/32722344
http://dx.doi.org/10.3390/s20154133
_version_ 1783572501379416064
author Boers, Tim
van der Putten, Joost
Struyvenberg, Maarten
Fockens, Kiki
Jukema, Jelmer
Schoon, Erik
van der Sommen, Fons
Bergman, Jacques
de With, Peter
author_facet Boers, Tim
van der Putten, Joost
Struyvenberg, Maarten
Fockens, Kiki
Jukema, Jelmer
Schoon, Erik
van der Sommen, Fons
Bergman, Jacques
de With, Peter
author_sort Boers, Tim
collection PubMed
description Early Barrett’s neoplasia are often missed due to subtle visual features and inexperience of the non-expert endoscopist with such lesions. While promising results have been reported on the automated detection of this type of early cancer in still endoscopic images, video-based detection using the temporal domain is still open. The temporally stable nature of video data in endoscopic examinations enables to develop a framework that can diagnose the imaged tissue class over time, thereby yielding a more robust and improved model for spatial predictions. We show that the introduction of Recurrent Neural Network nodes offers a more stable and accurate model for tissue classification, compared to classification on individual images. We have developed a customized Resnet18 feature extractor with four types of classifiers: Fully Connected (FC), Fully Connected with an averaging filter (FC Avg (n = 5)), Long Short Term Memory (LSTM) and a Gated Recurrent Unit (GRU). Experimental results are based on 82 pullback videos of the esophagus with 46 high-grade dysplasia patients. Our results demonstrate that the LSTM classifier outperforms the FC, FC Avg (n = 5) and GRU classifier with an average accuracy of 85.9% compared to 82.2%, 83.0% and 85.6%, respectively. The benefit of our novel implementation for endoscopic tissue classification is the inclusion of spatio-temporal information for improved and robust decision making, and it is the first step towards full temporal learning of esophageal cancer detection in endoscopic video.
format Online
Article
Text
id pubmed-7436238
institution National Center for Biotechnology Information
language English
publishDate 2020
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-74362382020-08-24 Improving Temporal Stability and Accuracy for Endoscopic Video Tissue Classification Using Recurrent Neural Networks Boers, Tim van der Putten, Joost Struyvenberg, Maarten Fockens, Kiki Jukema, Jelmer Schoon, Erik van der Sommen, Fons Bergman, Jacques de With, Peter Sensors (Basel) Letter Early Barrett’s neoplasia are often missed due to subtle visual features and inexperience of the non-expert endoscopist with such lesions. While promising results have been reported on the automated detection of this type of early cancer in still endoscopic images, video-based detection using the temporal domain is still open. The temporally stable nature of video data in endoscopic examinations enables to develop a framework that can diagnose the imaged tissue class over time, thereby yielding a more robust and improved model for spatial predictions. We show that the introduction of Recurrent Neural Network nodes offers a more stable and accurate model for tissue classification, compared to classification on individual images. We have developed a customized Resnet18 feature extractor with four types of classifiers: Fully Connected (FC), Fully Connected with an averaging filter (FC Avg (n = 5)), Long Short Term Memory (LSTM) and a Gated Recurrent Unit (GRU). Experimental results are based on 82 pullback videos of the esophagus with 46 high-grade dysplasia patients. Our results demonstrate that the LSTM classifier outperforms the FC, FC Avg (n = 5) and GRU classifier with an average accuracy of 85.9% compared to 82.2%, 83.0% and 85.6%, respectively. The benefit of our novel implementation for endoscopic tissue classification is the inclusion of spatio-temporal information for improved and robust decision making, and it is the first step towards full temporal learning of esophageal cancer detection in endoscopic video. MDPI 2020-07-24 /pmc/articles/PMC7436238/ /pubmed/32722344 http://dx.doi.org/10.3390/s20154133 Text en © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
spellingShingle Letter
Boers, Tim
van der Putten, Joost
Struyvenberg, Maarten
Fockens, Kiki
Jukema, Jelmer
Schoon, Erik
van der Sommen, Fons
Bergman, Jacques
de With, Peter
Improving Temporal Stability and Accuracy for Endoscopic Video Tissue Classification Using Recurrent Neural Networks
title Improving Temporal Stability and Accuracy for Endoscopic Video Tissue Classification Using Recurrent Neural Networks
title_full Improving Temporal Stability and Accuracy for Endoscopic Video Tissue Classification Using Recurrent Neural Networks
title_fullStr Improving Temporal Stability and Accuracy for Endoscopic Video Tissue Classification Using Recurrent Neural Networks
title_full_unstemmed Improving Temporal Stability and Accuracy for Endoscopic Video Tissue Classification Using Recurrent Neural Networks
title_short Improving Temporal Stability and Accuracy for Endoscopic Video Tissue Classification Using Recurrent Neural Networks
title_sort improving temporal stability and accuracy for endoscopic video tissue classification using recurrent neural networks
topic Letter
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7436238/
https://www.ncbi.nlm.nih.gov/pubmed/32722344
http://dx.doi.org/10.3390/s20154133
work_keys_str_mv AT boerstim improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks
AT vanderputtenjoost improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks
AT struyvenbergmaarten improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks
AT fockenskiki improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks
AT jukemajelmer improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks
AT schoonerik improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks
AT vandersommenfons improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks
AT bergmanjacques improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks
AT dewithpeter improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks