Artificial Intelligence-Based Hazard Detection in Robotic-Assisted Single-Incision Oncologic Surgery

SIMPLE SUMMARY: SILS and U-VATS procedures present many advantages, but also a series of limitations, such as the narrow operating space and poor surgeon ergonomics. To enhance the surgeon’s performance, this study aims to develop an AI and AR-based hazard detection system. The focus is on detecting and locating hemorrhages, a common occurrence especially in cancer patients. The system utilizes the YOLO v5 algorithm, trained to accurately identify bleeding in real-time. When bleeding is detected, it is displayed on a Hololens 2 device, alerting the doctor to its occurrence and origin. The system demonstrated its capability to distinguish between instances of bleeding and intraoperative irrigation, reducing the risk of false-negative and false-positive results. The current advancements in AI and AR technologies present potential of real-time hazard detection systems, which can serve as effective tools to assist surgeons, particularly in high-risk surgeries. ABSTRACT: The problem: Single-incision surgery is a complex procedure in which any additional information automatically collected from the operating field can be of significance. While the use of robotic devices has greatly improved surgical outcomes, there are still many unresolved issues. One of the major surgical complications, with higher occurrence in cancer patients, is intraoperative hemorrhages, which if detected early, can be more efficiently controlled. Aim: This paper proposes a hazard detection system which incorporates the advantages of both Artificial Intelligence (AI) and Augmented Reality (AR) agents, capable of identifying, in real-time, intraoperative bleedings, which are subsequently displayed on a Hololens 2 device. Methods: The authors explored the different techniques for real-time processing and determined, based on a critical analysis, that YOLOv5 is one of the most promising solutions. An innovative, real-time, bleeding detection system, developed using the YOLOv5 algorithm and the Hololens 2 device, was evaluated on different surgical procedures and tested in multiple configurations to obtain the optimal prediction time and accuracy. Results: The detection system was able to identify the bleeding occurrence in multiple surgical procedures with a high rate of accuracy. Once detected, the area of interest was marked with a bounding box and displayed on the Hololens 2 device. During the tests, the system was able to differentiate between bleeding occurrence and intraoperative irrigation; thus, reducing the risk of false-negative and false-positive results. Conclusion: The current level of AI and AR technologies enables the development of real-time hazard detection systems as efficient assistance tools for surgeons, especially in high-risk interventions.