Cargando…

Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM

Binocular endoscopy is gradually becoming the future of minimally invasive surgery (MIS) thanks to the development of stereo vision. However, some problems still exist, such as the low reconstruction accuracy, small surgical field, and low computational efficiency. To solve these problems, we design...

Descripción completa

Detalles Bibliográficos
Autores principales: Huo, Jiayi, Zhou, Changjiang, Yuan, Bo, Yang, Qing, Wang, Liqiang
Formato: Online Artículo Texto
Lenguaje:English
Publicado: MDPI 2023
Materias:
Acceso en línea:https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9959903/
https://www.ncbi.nlm.nih.gov/pubmed/36850671
http://dx.doi.org/10.3390/s23042074
_version_ 1784895393869332480
author Huo, Jiayi
Zhou, Changjiang
Yuan, Bo
Yang, Qing
Wang, Liqiang
author_facet Huo, Jiayi
Zhou, Changjiang
Yuan, Bo
Yang, Qing
Wang, Liqiang
author_sort Huo, Jiayi
collection PubMed
description Binocular endoscopy is gradually becoming the future of minimally invasive surgery (MIS) thanks to the development of stereo vision. However, some problems still exist, such as the low reconstruction accuracy, small surgical field, and low computational efficiency. To solve these problems, we designed a framework for real-time dense reconstruction in binocular endoscopy scenes. First, we obtained the initial disparity map using an SGBM algorithm and proposed the disparity confidence map as a dataset to provide StereoNet training. Then, based on the depth map predicted by StereoNet, the corresponding left image of each depth map was input into the Oriented Fast and Brief-Simultaneous Localization and Mapping (ORB-SLAM) framework using an RGB-D camera to realize the real-time dense reconstruction of the binocular endoscopy scene. The proposed algorithm was verified in the stomach phantom and a real pig stomach. Compared with the ground truth, the proposed algorithm’s RMSE is 1.620 mm, and the number of effective points in the point cloud is 834,650, which is a significant improvement in the mapping ability compared with binocular SLAM and ensures the real-time performance of the algorithm while performing dense reconstruction. The effectiveness of the proposed algorithm is verified.
format Online
Article
Text
id pubmed-9959903
institution National Center for Biotechnology Information
language English
publishDate 2023
publisher MDPI
record_format MEDLINE/PubMed
spelling pubmed-99599032023-02-26 Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM Huo, Jiayi Zhou, Changjiang Yuan, Bo Yang, Qing Wang, Liqiang Sensors (Basel) Article Binocular endoscopy is gradually becoming the future of minimally invasive surgery (MIS) thanks to the development of stereo vision. However, some problems still exist, such as the low reconstruction accuracy, small surgical field, and low computational efficiency. To solve these problems, we designed a framework for real-time dense reconstruction in binocular endoscopy scenes. First, we obtained the initial disparity map using an SGBM algorithm and proposed the disparity confidence map as a dataset to provide StereoNet training. Then, based on the depth map predicted by StereoNet, the corresponding left image of each depth map was input into the Oriented Fast and Brief-Simultaneous Localization and Mapping (ORB-SLAM) framework using an RGB-D camera to realize the real-time dense reconstruction of the binocular endoscopy scene. The proposed algorithm was verified in the stomach phantom and a real pig stomach. Compared with the ground truth, the proposed algorithm’s RMSE is 1.620 mm, and the number of effective points in the point cloud is 834,650, which is a significant improvement in the mapping ability compared with binocular SLAM and ensures the real-time performance of the algorithm while performing dense reconstruction. The effectiveness of the proposed algorithm is verified. MDPI 2023-02-12 /pmc/articles/PMC9959903/ /pubmed/36850671 http://dx.doi.org/10.3390/s23042074 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
spellingShingle Article
Huo, Jiayi
Zhou, Changjiang
Yuan, Bo
Yang, Qing
Wang, Liqiang
Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM
title Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM
title_full Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM
title_fullStr Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM
title_full_unstemmed Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM
title_short Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM
title_sort real-time dense reconstruction with binocular endoscopy based on stereonet and orb-slam
topic Article
url https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9959903/
https://www.ncbi.nlm.nih.gov/pubmed/36850671
http://dx.doi.org/10.3390/s23042074
work_keys_str_mv AT huojiayi realtimedensereconstructionwithbinocularendoscopybasedonstereonetandorbslam
AT zhouchangjiang realtimedensereconstructionwithbinocularendoscopybasedonstereonetandorbslam
AT yuanbo realtimedensereconstructionwithbinocularendoscopybasedonstereonetandorbslam
AT yangqing realtimedensereconstructionwithbinocularendoscopybasedonstereonetandorbslam
AT wangliqiang realtimedensereconstructionwithbinocularendoscopybasedonstereonetandorbslam