Cargando…
A Multi-Task Convolutional Neural Network for Semantic Segmentation and Event Detection in Laparoscopic Surgery
The current study presents a multi-task end-to-end deep learning model for real-time blood accumulation detection and tools semantic segmentation from a laparoscopic surgery video. Intraoperative bleeding is one of the most problematic aspects of laparoscopic surgery. It is challenging to control an...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10054284/ https://www.ncbi.nlm.nih.gov/pubmed/36983595 http://dx.doi.org/10.3390/jpm13030413 |
_version_ | 1785015634372853760 |
---|---|
author | Marullo, Giorgia Tanzi, Leonardo Ulrich, Luca Porpiglia, Francesco Vezzetti, Enrico |
author_facet | Marullo, Giorgia Tanzi, Leonardo Ulrich, Luca Porpiglia, Francesco Vezzetti, Enrico |
author_sort | Marullo, Giorgia |
collection | PubMed |
description | The current study presents a multi-task end-to-end deep learning model for real-time blood accumulation detection and tools semantic segmentation from a laparoscopic surgery video. Intraoperative bleeding is one of the most problematic aspects of laparoscopic surgery. It is challenging to control and limits the visibility of the surgical site. Consequently, prompt treatment is required to avoid undesirable outcomes. This system exploits a shared backbone based on the encoder of the U-Net architecture and two separate branches to classify the blood accumulation event and output the segmentation map, respectively. Our main contribution is an efficient multi-task approach that achieved satisfactory results during the test on surgical videos, although trained with only RGB images and no other additional information. The proposed multi-tasking convolutional neural network did not employ any pre- or postprocessing step. It achieved a Dice Score equal to 81.89% for the semantic segmentation task and an accuracy of 90.63% for the event detection task. The results demonstrated that the concurrent tasks were properly combined since the common backbone extracted features proved beneficial for tool segmentation and event detection. Indeed, active bleeding usually happens when one of the instruments closes or interacts with anatomical tissues, and it decreases when the aspirator begins to remove the accumulated blood. Even if different aspects of the presented methodology could be improved, this work represents a preliminary attempt toward an end-to-end multi-task deep learning model for real-time video understanding. |
format | Online Article Text |
id | pubmed-10054284 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-100542842023-03-30 A Multi-Task Convolutional Neural Network for Semantic Segmentation and Event Detection in Laparoscopic Surgery Marullo, Giorgia Tanzi, Leonardo Ulrich, Luca Porpiglia, Francesco Vezzetti, Enrico J Pers Med Article The current study presents a multi-task end-to-end deep learning model for real-time blood accumulation detection and tools semantic segmentation from a laparoscopic surgery video. Intraoperative bleeding is one of the most problematic aspects of laparoscopic surgery. It is challenging to control and limits the visibility of the surgical site. Consequently, prompt treatment is required to avoid undesirable outcomes. This system exploits a shared backbone based on the encoder of the U-Net architecture and two separate branches to classify the blood accumulation event and output the segmentation map, respectively. Our main contribution is an efficient multi-task approach that achieved satisfactory results during the test on surgical videos, although trained with only RGB images and no other additional information. The proposed multi-tasking convolutional neural network did not employ any pre- or postprocessing step. It achieved a Dice Score equal to 81.89% for the semantic segmentation task and an accuracy of 90.63% for the event detection task. The results demonstrated that the concurrent tasks were properly combined since the common backbone extracted features proved beneficial for tool segmentation and event detection. Indeed, active bleeding usually happens when one of the instruments closes or interacts with anatomical tissues, and it decreases when the aspirator begins to remove the accumulated blood. Even if different aspects of the presented methodology could be improved, this work represents a preliminary attempt toward an end-to-end multi-task deep learning model for real-time video understanding. MDPI 2023-02-25 /pmc/articles/PMC10054284/ /pubmed/36983595 http://dx.doi.org/10.3390/jpm13030413 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Marullo, Giorgia Tanzi, Leonardo Ulrich, Luca Porpiglia, Francesco Vezzetti, Enrico A Multi-Task Convolutional Neural Network for Semantic Segmentation and Event Detection in Laparoscopic Surgery |
title | A Multi-Task Convolutional Neural Network for Semantic Segmentation and Event Detection in Laparoscopic Surgery |
title_full | A Multi-Task Convolutional Neural Network for Semantic Segmentation and Event Detection in Laparoscopic Surgery |
title_fullStr | A Multi-Task Convolutional Neural Network for Semantic Segmentation and Event Detection in Laparoscopic Surgery |
title_full_unstemmed | A Multi-Task Convolutional Neural Network for Semantic Segmentation and Event Detection in Laparoscopic Surgery |
title_short | A Multi-Task Convolutional Neural Network for Semantic Segmentation and Event Detection in Laparoscopic Surgery |
title_sort | multi-task convolutional neural network for semantic segmentation and event detection in laparoscopic surgery |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10054284/ https://www.ncbi.nlm.nih.gov/pubmed/36983595 http://dx.doi.org/10.3390/jpm13030413 |
work_keys_str_mv | AT marullogiorgia amultitaskconvolutionalneuralnetworkforsemanticsegmentationandeventdetectioninlaparoscopicsurgery AT tanzileonardo amultitaskconvolutionalneuralnetworkforsemanticsegmentationandeventdetectioninlaparoscopicsurgery AT ulrichluca amultitaskconvolutionalneuralnetworkforsemanticsegmentationandeventdetectioninlaparoscopicsurgery AT porpigliafrancesco amultitaskconvolutionalneuralnetworkforsemanticsegmentationandeventdetectioninlaparoscopicsurgery AT vezzettienrico amultitaskconvolutionalneuralnetworkforsemanticsegmentationandeventdetectioninlaparoscopicsurgery AT marullogiorgia multitaskconvolutionalneuralnetworkforsemanticsegmentationandeventdetectioninlaparoscopicsurgery AT tanzileonardo multitaskconvolutionalneuralnetworkforsemanticsegmentationandeventdetectioninlaparoscopicsurgery AT ulrichluca multitaskconvolutionalneuralnetworkforsemanticsegmentationandeventdetectioninlaparoscopicsurgery AT porpigliafrancesco multitaskconvolutionalneuralnetworkforsemanticsegmentationandeventdetectioninlaparoscopicsurgery AT vezzettienrico multitaskconvolutionalneuralnetworkforsemanticsegmentationandeventdetectioninlaparoscopicsurgery |