Augmented Reality–Assisted Robotic Segmentectomy Using Open-Source Tools: A Practical and Low-Cost Solution

This video is the second-place thoracic winner from the 2025 CTSNet Instructional Video Competition. Watch all entries from the competition, including the other winning videos.  

The patient was a 69-year-old woman, a former smoker, with bilateral subsolid pulmonary lesions smaller than 2 cm and impaired pulmonary function (forced expiratory volume in 1 second (FEV1) 68 percent, forced vital capacity (FVC) 70 percent, diffusing capacity of the lungs for carbon monoxide (DLCO) 65 percent). Due to early-stage bilateral disease and limited respiratory reserve, sequential segmentectomies were planned: left S2 followed by right S10. This case study focused on the first procedure, a left S2 segmentectomy performed with the da Vinci Xi system. 

The augmented reality (AR) process began with a preoperative computed tomography (CT) scan to construct a 3D anatomical model using 3D-Slicer, an open-source software. Segmentation was performed semiautomatically by the surgeon, with manual delineation of vessels and bronchopulmonary segments. Intraoperatively, AR visualization was achieved by overlaying the 3D model onto the surgical field. This was accomplished using a standard USB video capture card (approximate cost: US $10), an HDMI-DVI cable for image capture, and the open-source software for image overlay. The composite image was displayed on the surgeon's console, using a second HDMI-DVI cable. 

Following standard lymphadenectomy, arterial dissection was performed, with AR-assisted navigation facilitating the identification of arteries A2 and B2. The intersegmental plane was defined using indocyanine green and divided with a surgical stapler. After resection, satisfactory re-expansion of the remaining lung segments was confirmed, and the final shape of the lung was compared to the preoperative 3D model using AR. 

References

1. Peek, J. J., Zhang, X., Hildebrandt, K., Max, S. A., Sadeghi, A. H., Bogers, A. J. J. C., & Mahtab, E. A. F. (2024). A novel 3D image registration technique for augmented reality vision in minimally invasive thoracoscopic pulmonary segmentectomy. International journal of computer assisted radiology and surgery, 10.1007/s11548-024-03308-7. Advance online publication. https://doi.org/10.1007/s11548-024-03308-7

Disclaimer

The information and views presented on CTSNet.org represent the views of the authors and contributors of the material and not of CTSNet. Please review our full disclaimer page here.