Cargando…
Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM
Binocular endoscopy is gradually becoming the future of minimally invasive surgery (MIS) thanks to the development of stereo vision. However, some problems still exist, such as the low reconstruction accuracy, small surgical field, and low computational efficiency. To solve these problems, we design...
Autores principales: | , , , , |
---|---|
Formato: | Online Artículo Texto |
Lenguaje: | English |
Publicado: |
MDPI
2023
|
Materias: | |
Acceso en línea: | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9959903/ https://www.ncbi.nlm.nih.gov/pubmed/36850671 http://dx.doi.org/10.3390/s23042074 |
_version_ | 1784895393869332480 |
---|---|
author | Huo, Jiayi Zhou, Changjiang Yuan, Bo Yang, Qing Wang, Liqiang |
author_facet | Huo, Jiayi Zhou, Changjiang Yuan, Bo Yang, Qing Wang, Liqiang |
author_sort | Huo, Jiayi |
collection | PubMed |
description | Binocular endoscopy is gradually becoming the future of minimally invasive surgery (MIS) thanks to the development of stereo vision. However, some problems still exist, such as the low reconstruction accuracy, small surgical field, and low computational efficiency. To solve these problems, we designed a framework for real-time dense reconstruction in binocular endoscopy scenes. First, we obtained the initial disparity map using an SGBM algorithm and proposed the disparity confidence map as a dataset to provide StereoNet training. Then, based on the depth map predicted by StereoNet, the corresponding left image of each depth map was input into the Oriented Fast and Brief-Simultaneous Localization and Mapping (ORB-SLAM) framework using an RGB-D camera to realize the real-time dense reconstruction of the binocular endoscopy scene. The proposed algorithm was verified in the stomach phantom and a real pig stomach. Compared with the ground truth, the proposed algorithm’s RMSE is 1.620 mm, and the number of effective points in the point cloud is 834,650, which is a significant improvement in the mapping ability compared with binocular SLAM and ensures the real-time performance of the algorithm while performing dense reconstruction. The effectiveness of the proposed algorithm is verified. |
format | Online Article Text |
id | pubmed-9959903 |
institution | National Center for Biotechnology Information |
language | English |
publishDate | 2023 |
publisher | MDPI |
record_format | MEDLINE/PubMed |
spelling | pubmed-99599032023-02-26 Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM Huo, Jiayi Zhou, Changjiang Yuan, Bo Yang, Qing Wang, Liqiang Sensors (Basel) Article Binocular endoscopy is gradually becoming the future of minimally invasive surgery (MIS) thanks to the development of stereo vision. However, some problems still exist, such as the low reconstruction accuracy, small surgical field, and low computational efficiency. To solve these problems, we designed a framework for real-time dense reconstruction in binocular endoscopy scenes. First, we obtained the initial disparity map using an SGBM algorithm and proposed the disparity confidence map as a dataset to provide StereoNet training. Then, based on the depth map predicted by StereoNet, the corresponding left image of each depth map was input into the Oriented Fast and Brief-Simultaneous Localization and Mapping (ORB-SLAM) framework using an RGB-D camera to realize the real-time dense reconstruction of the binocular endoscopy scene. The proposed algorithm was verified in the stomach phantom and a real pig stomach. Compared with the ground truth, the proposed algorithm’s RMSE is 1.620 mm, and the number of effective points in the point cloud is 834,650, which is a significant improvement in the mapping ability compared with binocular SLAM and ensures the real-time performance of the algorithm while performing dense reconstruction. The effectiveness of the proposed algorithm is verified. MDPI 2023-02-12 /pmc/articles/PMC9959903/ /pubmed/36850671 http://dx.doi.org/10.3390/s23042074 Text en © 2023 by the authors. https://creativecommons.org/licenses/by/4.0/Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). |
spellingShingle | Article Huo, Jiayi Zhou, Changjiang Yuan, Bo Yang, Qing Wang, Liqiang Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM |
title | Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM |
title_full | Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM |
title_fullStr | Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM |
title_full_unstemmed | Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM |
title_short | Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM |
title_sort | real-time dense reconstruction with binocular endoscopy based on stereonet and orb-slam |
topic | Article |
url | https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9959903/ https://www.ncbi.nlm.nih.gov/pubmed/36850671 http://dx.doi.org/10.3390/s23042074 |
work_keys_str_mv | AT huojiayi realtimedensereconstructionwithbinocularendoscopybasedonstereonetandorbslam AT zhouchangjiang realtimedensereconstructionwithbinocularendoscopybasedonstereonetandorbslam AT yuanbo realtimedensereconstructionwithbinocularendoscopybasedonstereonetandorbslam AT yangqing realtimedensereconstructionwithbinocularendoscopybasedonstereonetandorbslam AT wangliqiang realtimedensereconstructionwithbinocularendoscopybasedonstereonetandorbslam |