Virtual Reality planning for chest wall resection

Oncologic chest wall resection and reconstruction are complex procedures due to anatomical and physiological challenges. For this reason, extended reality and virtual reality are increasingly explored in surgery for their ability to provide immersive anatomical insights, support preoperative planning, and improve intraoperative guidance.

The study conducted at University Hospital Bonn aims to evaluate the potential of VR and AI in the complex surgical domain by retrospectively analyzing interdisciplinary, collaborative VR planning enhanced by automated AI-based segmentation, and comparing it with actual surgical outcomes.

Challenges of chest resection and reconstruction

Chest wall resection and reconstruction are challenging due to the complex interplay of anatomical and physiological factors. Achieving adequate oncologic margins while preserving structural integrity and function is essential for optimal outcomes.

Large tumors, particularly sarcomas involving soft tissue and bone, require a multidisciplinary approach for both resection and reconstruction. These procedures often involve not only an experienced thoracic surgeon but also a plastic surgeon when extensive skin coverage or muscle flaps are needed.

Sarcomas and metastatic disease are the most common chest wall malignancies, and treatment strategies for sarcomas should be discussed in a multidisciplinary tumor board at a certified sarcoma center. When adjacent structures must be resected, an onco-orthopedic surgeon should be involved in preoperative planning to ensure comprehensive management.

Traditional two-dimensional imaging are often insufficient for such surgical planning, as it does not fully capture complex three-dimensional anatomical relationships.

VR planning workflow

Virtual reality planning was performed using Medical Imaging XR, which enabled collaborative, multi-user sessions for real-time discussion and shared visualization of 3D-reconstructed anatomy. Automated segmentation of anatomical structures was conducted using the open-source TotalSegmentator model, providing delineation of major structures such as bones, vessels, and organs.

Tumor segmentation was not provided and was therefore manually performed by the surgical team within the VR environment. Segmentation masks were integrated into the VR volume-rendering system to enhance visualization and interaction.

Each case was reviewed by an interdisciplinary team including a thoracic surgeon, an oncological orthopedic surgeon, and plastic/reconstructive surgeon. Using the Medical Imaging XR toolset, the team performed tumor and anatomical landmark identification, resection margin planning, defect size estimation, reconstruction strategy planning, and documentation of the planned surgical approach.

The following parameters were quantitatively compared between the VR-planned approach and the actual surgical outcomes:

Assessment of VR planning accuracy

In 3 of 8 cases (37.5%), intraoperative resection was more extensive than predicted by VR planning, mainly due to underestimated tumor infiltration into adjacent tissues. This highlights a key limitation of preoperative imaging, which persists despite advanced visualization techniques, as intraoperative assessment remains superior.

In contrast, moderate overestimation of resection extent (up to 24%) in 4 of 8 cases was considered acceptable. One case showed large overestimation, likely due to differing oncological scenario and therefore overestimated the surgical margins.

VR simulation supports surgical planning by improving spatial understanding but may underestimate tumor spread due to imaging limitations, particularly in detecting subtle or microscopic disease.

Thus, it should be regarded as a complementary tool rather than a definitive predictor, and intraoperative adaptability remains essential.

Participants generally reported a positive user experience with the VR planning system, describing the environment as engaging, innovative, and enjoyable to use. The User Experience Questionnaire (UEQ) indicated that surgeons found the VR platform stimulating and novel, with overall usability perceived as favorable and the immersive workspace well‑received.

Regarding tolerability, symptoms related to cybersickness were minimal. Users experienced only slight discomfort, and these effects did not disrupt the planning sessions or require anyone to stop early. Overall, the VR system was perceived as comfortable and acceptable.