Intraoperative use of mixed reality (MR) in humans across surgical disciplines: a major review. 3D Print Med 11, 59
https://doi.org/10.1186/s41205-025-00305-7XR technologies
Extended reality (XR) includes virtual reality (VR), augmented reality (AR), and mixed reality (MR). While VR creates fully immersive, digital environments, and AR overlays virtual objects onto the real world, MR combines both approaches. This enables real-time interaction with virtual data within the physical environment.
Example of use of MR Image Reference System
Surgical navigation
Surgical navigation systems track surgical instruments and overlay their position onto 3D preoperative medical imaging. Intraoperatively, such models get aligned with the patient’s anatomy, allowing the surgeon to follow the plan precisely. Traditional surgical navigation systems require an additional flat screen which displays instruments’ positioning. The application of MR enables a direct projection of the model onto the operative view through a head-mounted display, reducing the surgeon’s attention shifts.
Spinal surgery
MR has been applied to navigation systems for pedicle screw placement in over fifty cases, such as posterior vertebrectomy for lumbar fractures. The real-time anatomical guidance improved placement accuracy compared to traditional methods, resulting in reduced intraoperative bleeding, shorter operation time, and faster postoperative recovery.
MR has been applied to navigation systems for pedicle screw placement in over fifty cases, such as posterior vertebrectomy for lumbar fractures. The real-time anatomical guidance improved placement accuracy compared to traditional methods, resulting in reduced intraoperative bleeding, shorter operation time, and faster postoperative recovery.
A live intraoperative view through the HMD. The navigational data are overlaid directly on the operative field.
Neurosurgery
MR supports a range of neurosurgical procedures, including external ventricular drain insertion, where it provides guidance to align the catheter with the planned trajectory. In patients with hypertensive intracerebral hemorrhage, it allows for minimally invasive puncture and drainage of hematomas, leading to uneventful recoveries. Additionally, MR assists in intracranial lesion and tumor surgeries, enabling precise incisions and real-time monitored needle placement.
MR supports a range of neurosurgical procedures, including external ventricular drain insertion, where it provides guidance to align the catheter with the planned trajectory. In patients with hypertensive intracerebral hemorrhage, it allows for minimally invasive puncture and drainage of hematomas, leading to uneventful recoveries. Additionally, MR assists in intracranial lesion and tumor surgeries, enabling precise incisions and real-time monitored needle placement.
Maxillofacial surgery
3D holographic images were used in oral and maxillofacial tumor resections, enabling the surgeon to accurately verify and execute planned osteotomy lines. The use of MR-based navigation system resulted in negative resection margins in all cases, with no reported postoperative tumor recurrence or complications.
3D holographic images were used in oral and maxillofacial tumor resections, enabling the surgeon to accurately verify and execute planned osteotomy lines. The use of MR-based navigation system resulted in negative resection margins in all cases, with no reported postoperative tumor recurrence or complications.
From the view of spectator, the image is located above the surgical area, and the three-dimensional reconstructed image and the actual surgical area are both in the surgeon’s field of view (A) the osteotomy line is determined with the help of the mixed reality image (B).
Thoracic surgery
The density and small size of ground glass opacity-dominant pulmonary nodules often makes their intraoperative localization challenging. A surgical navigation system combining 3D-printing navigational templates with MR was used to accurately localize these nodules. This approach allowed to perform video-assisted thoracoscopic surgery for precise nodule resection.
The density and small size of ground glass opacity-dominant pulmonary nodules often makes their intraoperative localization challenging. A surgical navigation system combining 3D-printing navigational templates with MR was used to accurately localize these nodules. This approach allowed to perform video-assisted thoracoscopic surgery for precise nodule resection.
Image guidance
MR has also been used in image guidance systems. Their purpose is to superimpose 3D medical imaging onto the patient’s anatomy and align it with anatomical landmarks. Unlike surgical navigation, image guidance doesn’t support instrument tracking.
Breast cancer surgery
Conventional sentinel lymph node biopsy relies on methylene blue injection to visually identify lymphatic drainage and node locations. To improve localization speed and accuracy, surgeons used an MR-based imaging guidance system that overlaid 3D models directly onto the patient’s anatomy. This approach was associated with fewer complications and increased overall patient satisfaction.
Conventional sentinel lymph node biopsy relies on methylene blue injection to visually identify lymphatic drainage and node locations. To improve localization speed and accuracy, surgeons used an MR-based imaging guidance system that overlaid 3D models directly onto the patient’s anatomy. This approach was associated with fewer complications and increased overall patient satisfaction.
Urology
A total of one hundred patients undergoing laparoscopic nephrectomy were divided into MR-assisted and non-MR-assisted groups. The MR-assisted group demonstrated improved operative planning and intraoperative navigation, leading to a higher rate of successful laparoscopic procedures. Additionally, MR guidance reduced operative and warm ischemia times, as well as intraoperative blood loss.
A total of one hundred patients undergoing laparoscopic nephrectomy were divided into MR-assisted and non-MR-assisted groups. The MR-assisted group demonstrated improved operative planning and intraoperative navigation, leading to a higher rate of successful laparoscopic procedures. Additionally, MR guidance reduced operative and warm ischemia times, as well as intraoperative blood loss.
Application scenarios of MRASN in surgery
Plastic and reconstructive surgery
MR has been used to guide reconstructive procedures by enabling precise identification and dissection of vascular pedunculated flaps. Preoperative CT-angiography scans were converted into segmented 3D models and overlaid onto the surgical field, assisting intraoperative flap planning and execution. An image guidance system was also successfully applied in ear reconstruction surgery, using 3D photographic models to support anatomically accurate procedure.
MR has been used to guide reconstructive procedures by enabling precise identification and dissection of vascular pedunculated flaps. Preoperative CT-angiography scans were converted into segmented 3D models and overlaid onto the surgical field, assisting intraoperative flap planning and execution. An image guidance system was also successfully applied in ear reconstruction surgery, using 3D photographic models to support anatomically accurate procedure.
Image reference
Image reference systems are the least complex of the described approaches. Using a head-mounted display, the surgeon can apply MR to visualize 3D patient data in a space adjacent to the surgical field. Since the model isn’t aligned with the anatomy, it serves as a visual reference.
Ophthalmic surgery
An MR-based image reference system was applied in endoscopic dacryocystorhinostomy, a procedure for treating lacrimal drainage obstructions. Surgeons could access both - prearranged GLB models, as well as real-time data, requiring more computing power obtained via Medicalholodeck software, in the operating room through a head-mounted display. The system allowed interactive manipulation of the models, such as slicing or rotating, supporting more informed surgical decisions.
An MR-based image reference system was applied in endoscopic dacryocystorhinostomy, a procedure for treating lacrimal drainage obstructions. Surgeons could access both - prearranged GLB models, as well as real-time data, requiring more computing power obtained via Medicalholodeck software, in the operating room through a head-mounted display. The system allowed interactive manipulation of the models, such as slicing or rotating, supporting more informed surgical decisions.
Intraoperative MR and Apert syndrome: Clinical external image demonstrating the senior author (MJA) actively interacting with MR models in the operating room (A). Intraoperative photograph showing the surgeon's view of the patient's MR model (B), which is being sliced to ascertain details of the anomalies (C). Intraoperative photograph demonstrating the senior author (MJA) actively operating. Note the several models and screens hanging virtually in the OR while he is performing endoscopic DCR (D).
Otolaryngology
Intraoperative access to segmented 3D data increases surgical safety by strengthening spatial understanding of patient-specific anatomy and its relationship to critical structures. This approach was particularly valuable in otolaryngological tumor resections and cholesteatoma management involving temporal bone surgery, where accurate differentiation between pathological and healthy tissue was essential.
Intraoperative access to segmented 3D data increases surgical safety by strengthening spatial understanding of patient-specific anatomy and its relationship to critical structures. This approach was particularly valuable in otolaryngological tumor resections and cholesteatoma management involving temporal bone surgery, where accurate differentiation between pathological and healthy tissue was essential.
A case-specific three-dimensional hologram on head mount displays was used during the preoperative meeting, A, and for intraoperative image reference, B. For a postoperative recurrence of left maxillary carcinoma, a partial maxillectomy was performed with a Weber-Ferguson skin incision. The coronoid process of the mandible was removed to secure a field of view from the lateral side. Most of the pterygoid process was resected to ensure a sufficient margin. The inferior orbital wall was preserved. Arrows: coronoid process of mandible. Arrow heads: inferior orbital wall. Asterisk: pterygoid process
Visceral surgery
Image reference systems were successfully applied in a range of abdominal procedures, including rectal cancer surgery, laparoscopic cholecystectomy, liver surgery, and open hepatic resection. Intraoperative holographic visualization was associated with more efficient task execution with reduced intraoperative blood loss and complication rates.
Image reference systems were successfully applied in a range of abdominal procedures, including rectal cancer surgery, laparoscopic cholecystectomy, liver surgery, and open hepatic resection. Intraoperative holographic visualization was associated with more efficient task execution with reduced intraoperative blood loss and complication rates.
How to start with Medicalholodeck
Medical Imaging XR allows surgeons to turn 2D patient-specific data into 3D models. It enables real-time data interaction, improving spatial understanding and intraoperative confidence.
Medicalholodeck connects securely to hospital systems, offering PACS integration, HIPAA-compliant data handling, and full patient data protection. Available on VR headsets, PCs, iPads, and iPhones, it is accessible for hospitals, classrooms, and training centers.
Advanced surgical planning functions are exclusive to Medical Imaging XR PRO FDA.
Currently, Medicalholodeck is available only for educational use. The platform is undergoing FDA and CE certification, and we expect Medical Imaging XR PRO to be available soon in the U.S. and EU markets.
For more information, contact info@medicalholodeck.com February 2026
