Rethinking Medical Imaging:
From Images to Spatial Understanding

The spatial information is already there

CT and MRI already produce volumetric data, and radiologists routinely use multiplanar reconstructions, maximum-intensity projections and volume rendering. Three-dimensional imaging has been part of medicine for decades. What deserves renewed attention is the effort required to understand the anatomy those images describe.

A cross-section can show a vessel with great clarity. Following its course through a volume requires a different kind of work: recognizing it across successive images, keeping track of its branches and relating it to neighbouring structures. Experienced readers develop sophisticated ways of doing this. Their skill is one reason conventional imaging works so well. It is also a reason to take the interface seriously. Some of the understanding comes from the display; some comes from what the reader brings to it.

Spatial medicine places this relationship between information and interpretation at the centre of the discussion. The term covers ways of examining and working with medical data spatially, from interactive models on a monitor to stereoscopic displays and immersive environments. Its value depends on whether those representations help answer a medical or educational question more clearly. The appearance of depth is only the beginning.

What a different view makes visible

Consider what changes when an anatomical reconstruction is rotated. A branch that was hidden becomes visible; two overlapping structures separate; the course of a vessel can be examined from another angle. These are ordinary features of 3D visualization, yet they can change the amount of mental reconstruction a task requires. A flat screen conveys depth through perspective, shading and movement. Stereoscopy adds binocular disparity, and head tracking allows the view to respond to the observer’s position. Each offers a different way to inspect the same geometry.

The clarity of that geometry depends on the reconstruction. A segmented vessel tree can look complete even when a small branch has been missed. Threshold choices can shift a boundary, and neighbouring structures can be joined by mistake. The ability to return to the relevant source slices is therefore part of a useful spatial interface. It lets the reader examine what the model represents and where its certainty ends.

The work involved in seeing anatomy

Roger Shepard and Jacqueline Metzler’s “Mental Rotation of Three-Dimensional Objects”, published in Science in 1971, gave spatial thinking an unusually concrete experimental form. Participants compared objects shown in different orientations. The greater the angular difference, the longer recognition generally took. Spatial transformation had a measurable cost, even with simple geometric objects.

Anatomical interpretation involves more than rotation. Readers must identify structures, follow continuity and combine observations from several views while judging their medical significance. A spatial display may make some of those relationships easier to inspect directly. It can also create problems of its own: an opaque surface may hide an important structure, or awkward controls may interrupt the examination. The cognitive argument for spatial visualization is strongest when it identifies a particular difficulty and explains how the display might reduce it. Shepard and Metzler’s experiment supplies a foundation for asking those questions, rather than clinical evidence for a particular device.

The relationship between spatial ability and anatomical knowledge is similarly specific. In their 2017 systematic review, Jean Langlois and colleagues found significant relationships for practical examinations, cross-sections and the synthesis of 3D anatomy from 2D views. Essays and non-spatial multiple-choice questions showed no significant relationship. Knowing anatomical facts and understanding anatomical space are related competencies, but an assessment can place very different demands on each.

When a clearer view changes a decision

The clinical literature becomes easier to interpret when its outcomes are kept distinct. Queisner and Eisenträger’s 2024 systematic review of surgical planning in VR reported improvements in anatomical understanding and changes in surgical plans after examination of patient-specific imaging. The review also found uneven methods and reporting, with some conclusions resting on subjective or case-specific observations. A changed plan tells us that the additional view influenced a decision. Whether that decision led to a better operation is a further question.

Thoracoscopic segmentectomy offers a more concrete test because individual bronchovascular anatomy bears directly on the procedure. Hongbo He and colleagues’ 2025 systematic review and meta-analysis included ten studies and 1,424 patients. Planning with patient-specific 3D reconstruction and virtual simulation was associated with lower odds of inadequate surgical margins and postoperative complications. In the subgroup of complex segmentectomies, operative time and blood loss also favoured 3D planning. Several other endpoints showed no significant difference.

These findings merit attention precisely because they concern surgical outcomes. The predominantly observational evidence still leaves room for differences in surgeon experience, case selection and reconstruction methods to explain part of the association. It also evaluates a planning workflow, not an isolated display feature. The result is a useful direction for prospective research: establish where spatial reconstruction changes care, and which parts of the process account for the benefit.

Learning anatomy across representations

Anatomy teaching presents a different question: what does spatial exploration help students learn? García-Robles and colleagues’ 2024 systematic review and meta-analysis of immersive VR and augmented reality in anatomy education analysed 27 experimental studies involving 2,199 health sciences students. The pooled results favoured XR for anatomy knowledge gains, including when it supplemented other teaching resources. The studies combined different technologies, teaching methods and comparison groups, so the finding is best read as support for an educational approach rather than a verdict on a particular platform.

The timing of assessment matters. Most interventions lasted one day, and knowledge was measured immediately afterwards. Retention and transfer to clinical work remain open questions. For educators, a useful next step is to connect the representations: let students examine a structure in 3D, locate it in cross-sectional images and explain its relationships from an unfamiliar view. This gives spatial exploration a defined purpose within the curriculum and keeps image interpretation part of the learning task.

Choosing the view that serves the task

The choice of display follows the work being done. Detailed review of image series may suit a familiar workstation. A teaching group needs a shared view and a clear way to direct attention. An individual exploring complicated anatomy may benefit from controlling the viewpoint independently. Touchscreens, stereoscopic monitors and VR offer different arrangements for these activities. Mixed and augmented reality add the challenge of relating virtual content to the physical environment.

That choice is only one part of the system. Importing the data, checking a segmentation, saving an annotation and returning to a particular slice all affect how useful the spatial view becomes. Medicalholodeck’s Spatial OS brings medical content across interfaces within this broader approach. The research discussed here covers a range of technologies and workflows; it does not validate a particular Medicalholodeck configuration for clinical use. The site’s clinical-use notice applies.

Readers interested in how these representations are built can explore 3D Slicer, an independent, open-source platform for medical-image visualization and analysis. Its segmentation guide is a useful companion to this discussion: the visible model is the result of processing choices that need to be understood and reviewed.

What deserves to be measured

As reconstruction and automated segmentation become more accessible, producing a spatial model is only part of the challenge. The harder judgment is whether it is useful enough to justify the time spent preparing, checking and examining it. A fair comparison with conventional methods should include that work, alongside interpretation accuracy, errors and the demands placed on the user.

Medical imaging has given clinicians increasingly rich information. Display design deserves the same scrutiny as the methods used to acquire and process it. Spatial medicine is worth pursuing wherever the relationships between structures are central to understanding them. Its progress will be measured by how reliably people can interpret, teach and act on those relationships.

Explore Medicalholodeck Spatial OS →
Explore Medical Imaging →

Part 1 of the “Why Spatial Medicine?” editorial series. Research discussed here includes technologies and workflows beyond Medicalholodeck's own products.