Augmented Reality Can Improve Real-World Brachytherapy

Brachytherapy is among the most procedurally demanding skills a radiation oncology resident can develop. Unlike external beam radiation therapy, where planning and delivery are largely separated in time, brachytherapy demands that the physician think spatially, act decisively, and coordinate a complex team in real time. The learning curve is steep, and the consequences of technical error are immediate.

With case volumes varying considerably across institutions, residents and educators are increasingly looking for new tools that can support the development of procedural expertise in brachytherapy. For these reasons, our specialty stands to benefit meaningfully from augmented reality (AR).

AR refers to the overlay of digital content, such as images, annotations, or 3D renderings, onto a user’s real-world field of view through a wearable headset. Unlike virtual reality, which replaces the physical environment, AR keeps the operator present and adds information that would otherwise require looking away from the operative field.

Other procedural specialties have already begun to capitalize on this technology. Surgeons at UC Davis Health perform complex otolaryngologic and orthopedic operations using AR goggles that project 3D CT and MRI reconstructions into the operative field, allowing visualization of vascular and bony structures that would otherwise remain hidden.¹ Similarly, in interventional radiology, a growing body of literature suggests that AR-based tools improve procedural accuracy, shorten training duration, and increase trainee confidence, with intraprocedural guidance demonstrating accuracy within 5 mm in live patients.^2,3 These specialties share the challenge of having to guide instruments through anatomy that cannot always be seen, which is the case with brachytherapy.

Over the past 2 years, our institution has been exploring various ways to use AR to support brachytherapy training and performance. As the procedure requires physicians to consistently shift their attention among multiple modalities—US, diagnostic imaging, and other monitors—our first goal was to use the Apple Vision Pro headset to bring all of these screens into a single, customizable field of view. This allows the physician to work with fewer interruptions, review imaging, and reference patient details mid-procedure without breaking sterility. While using the headset, we were struck by how intuitive the gestures felt and how naturally a virtual screen could be placed within our fields of view.

In brachytherapy, procedural steps are numerous and team composition varies from case to case. We created the checklist as a web-based application that loads directly onto the headset and appears as a floating screen within the trainee’s field of view. The trainee can move it in and out of sight with simple hand gestures while performing the procedure, allowing real-time progress tracking and timely communication with nursing and physics teams at the appropriate steps. A structured, visible checklist reduces cognitive load, frees mental bandwidth for skill acquisition rather than step recall, and provides an additional layer of safety and continuity across disciplines. Time-stamped tracking of each step also proved useful in highlighting inefficiencies worth examining in subsequent cases.

Translating 2D axial imaging into 3D spatial understanding under operative time pressure, as required with brachytherapy, is genuinely difficult. Using the AR headset to render DICOM data as interactive 3D models offers physicians a more direct way to engage with patient anatomy. The physician can assess applicator geometry, anticipate needle trajectory, and appreciate proximity to organs at risk in ways that scrolling through slices cannot replicate. The same platform places applicator schematics, manufacturer reference images, and other procedural materials within virtual reach, allowing the physician to consult them without stepping away from the operative field.

Perhaps the most consequential feature of AR-facilitated brachytherapy is real-time remote collaboration. The headset’s outward-facing cameras allow a remote expert to see precisely what the operator sees and provide live guidance from a distance.⁴ This capability could broaden access to subspecialty mentorship, supporting physicians as they expand or update brachytherapy programs. For trainees, it could provide additional opportunities to learn from experienced colleagues across institutions.

As with any emerging technology, AR is still maturing. The headset is heavier than we anticipated, and on days with multiple cases, the cumulative weight on the face would likely become a limitation. Lighter, more ergonomic iterations of the hardware will be important if AR is to be integrated into routine clinical practice. Beyond ergonomics, users with significant myopia require custom optical inserts, and connectivity within shielded brachytherapy suites can be inconsistent. The clinical evidence base in radiation oncology is also in its early stages. These challenges are being actively addressed, and successive iterations of the hardware and software continue to improve.

The trajectory is encouraging. As residents, we have an opportunity to engage with these tools early, contribute to their refinement, and help shape how they integrate into our specialty. By offering our perspectives now, we can ensure that AR technology continues to improve precision-driven procedures like brachytherapy and other areas of radiation oncology.

Note: This is the third in a series of Resident Voice columns addressing options for enhancing brachytherapy training in residency. See also Procedural Competency in Brachytherapy: Stepping Beyond Case Minimums and Closing the Global Brachytherapy Training Gap with Virtual Reality.