Cargando…
Improving Temporal Stability and Accuracy for Endoscopic Video Tissue Classification Using Recurrent Neural Networks
Early Barrett’s neoplasia are often missed due to subtle visual features and inexperience of the non-expert endoscopist with such lesions. While promising results have been reported on the automated detection of this type of early cancer in still endoscopic images, video-based detection using the te...
Autores principales: | , , , , , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2020
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7436238/ https://www.ncbi.nlm.nih.gov/pubmed/32722344 http://dx.doi.org/10.3390/s20154133 |
_version_ | 1783572501379416064 |
---|---|
author | Boers, Tim van der Putten, Joost Struyvenberg, Maarten Fockens, Kiki Jukema, Jelmer Schoon, Erik van der Sommen, Fons Bergman, Jacques de With, Peter |
author_facet | Boers, Tim van der Putten, Joost Struyvenberg, Maarten Fockens, Kiki Jukema, Jelmer Schoon, Erik van der Sommen, Fons Bergman, Jacques de With, Peter |
author_sort | Boers, Tim |
collection | PubMed |
description | Early Barrett’s neoplasia are often missed due to subtle visual features and inexperience of the non-expert endoscopist with such lesions. While promising results have been reported on the automated detection of this type of early cancer in still endoscopic images, video-based detection using the temporal domain is still open. The temporally stable nature of video data in endoscopic examinations enables to develop a framework that can diagnose the imaged tissue class over time, thereby yielding a more robust and improved model for spatial predictions. We show that the introduction of Recurrent Neural Network nodes offers a more stable and accurate model for tissue classification, compared to classification on individual images. We have developed a customized Resnet18 feature extractor with four types of classifiers: Fully Connected (FC), Fully Connected with an averaging filter (FC Avg (n = 5)), Long Short Term Memory (LSTM) and a Gated Recurrent Unit (GRU). Experimental results are based on 82 pullback videos of the esophagus with 46 high-grade dysplasia patients. Our results demonstrate that the LSTM classifier outperforms the FC, FC Avg (n = 5) and GRU classifier with an average accuracy of 85.9% compared to 82.2%, 83.0% and 85.6%, respectively. The benefit of our novel implementation for endoscopic tissue classification is the inclusion of spatio-temporal information for improved and robust decision making, and it is the first step towards full temporal learning of esophageal cancer detection in endoscopic video. |
format | Online Article Text |
id | pubmed-7436238 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2020 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-74362382020-08-24 Improving Temporal Stability and Accuracy for Endoscopic Video Tissue Classification Using Recurrent Neural Networks Boers, Tim van der Putten, Joost Struyvenberg, Maarten Fockens, Kiki Jukema, Jelmer Schoon, Erik van der Sommen, Fons Bergman, Jacques de With, Peter Sensors (Basel) Letter Early Barrett’s neoplasia are often missed due to subtle visual features and inexperience of the non-expert endoscopist with such lesions. While promising results have been reported on the automated detection of this type of early cancer in still endoscopic images, video-based detection using the temporal domain is still open. The temporally stable nature of video data in endoscopic examinations enables to develop a framework that can diagnose the imaged tissue class over time, thereby yielding a more robust and improved model for spatial predictions. We show that the introduction of Recurrent Neural Network nodes offers a more stable and accurate model for tissue classification, compared to classification on individual images. We have developed a customized Resnet18 feature extractor with four types of classifiers: Fully Connected (FC), Fully Connected with an averaging filter (FC Avg (n = 5)), Long Short Term Memory (LSTM) and a Gated Recurrent Unit (GRU). Experimental results are based on 82 pullback videos of the esophagus with 46 high-grade dysplasia patients. Our results demonstrate that the LSTM classifier outperforms the FC, FC Avg (n = 5) and GRU classifier with an average accuracy of 85.9% compared to 82.2%, 83.0% and 85.6%, respectively. The benefit of our novel implementation for endoscopic tissue classification is the inclusion of spatio-temporal information for improved and robust decision making, and it is the first step towards full temporal learning of esophageal cancer detection in endoscopic video. MDPI 2020-07-24 /pmc/articles/PMC7436238/ /pubmed/32722344 http://dx.doi.org/10.3390/s20154133 Text en © 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Letter Boers, Tim van der Putten, Joost Struyvenberg, Maarten Fockens, Kiki Jukema, Jelmer Schoon, Erik van der Sommen, Fons Bergman, Jacques de With, Peter Improving Temporal Stability and Accuracy for Endoscopic Video Tissue Classification Using Recurrent Neural Networks |
title | Improving Temporal Stability and Accuracy for Endoscopic Video Tissue Classification Using Recurrent Neural Networks |
title_full | Improving Temporal Stability and Accuracy for Endoscopic Video Tissue Classification Using Recurrent Neural Networks |
title_fullStr | Improving Temporal Stability and Accuracy for Endoscopic Video Tissue Classification Using Recurrent Neural Networks |
title_full_unstemmed | Improving Temporal Stability and Accuracy for Endoscopic Video Tissue Classification Using Recurrent Neural Networks |
title_short | Improving Temporal Stability and Accuracy for Endoscopic Video Tissue Classification Using Recurrent Neural Networks |
title_sort | improving temporal stability and accuracy for endoscopic video tissue classification using recurrent neural networks |
topic | Letter |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7436238/ https://www.ncbi.nlm.nih.gov/pubmed/32722344 http://dx.doi.org/10.3390/s20154133 |
work_keys_str_mv | AT boerstim improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks AT vanderputtenjoost improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks AT struyvenbergmaarten improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks AT fockenskiki improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks AT jukemajelmer improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks AT schoonerik improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks AT vandersommenfons improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks AT bergmanjacques improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks AT dewithpeter improvingtemporalstabilityandaccuracyforendoscopicvideotissueclassificationusingrecurrentneuralnetworks |