Peer Review in Radiation Oncology: Present and Future Directions

Peer review identified errors that resulted in alterations to treatment plans in 3.3% to 12.2% of the time.^5-9 “Near misses” were seen and rectified in 9% of reviewed cases in some series.¹⁰ The treatment sites most frequently associated with peer review-recommended plan changes were head and neck (1%-66%), gynecologic (6%-18%), and lung (13%-27%).^5,10-12 Changes in treatment plans that resulted from peer review centered on target volume, dose, organs at risk, and contouring, with target volume being one of the most common reasons for changes.^5,10,13 Most institutions that participated in peer review reviewed >50% of curative cases, and peer review was seen broadly to apply in some fashion to all types of radiation courses; however, palliative cases had lower rates of audit.^5,12

The complexity of the target site and the plan was correlated with the rate of changes recommended by peer review. For example, breast plans that included regional lymph nodes had an odds ratio of 2.12 to be classified requiring changes vs breast only plans.¹³ Complexity related to novel technology was attributed as a factor that resulted in increased rates of changes from peer review. For example, head and neck plans often involving advanced modalities like intensity-modulated radiation therapy, volumetric modulated arc thearpy (VMAT), and proton therapy can have higher rates of recommended changes during peer review (44.9%), compared to lung (24.5%), stereotactic body radiotherapy (SBRT) (17.4%), and other sites.^12,14 Heterogeneity in patient populations studied and peer review methods make comparison between reported outcomes difficult. It was seen that as a department overcame the hurdles with learning new treatment modalities, their rate of changes from peer review decreased.¹¹ Rates of change generally decreased over time after the initiation of peer review, which is thought to be from the standardization that results from frequent review, along with the education to all those involved.¹² Clear and established department guidelines may serve to reduce the variation in treatment plans and therefore reduce the needed changes from peer review, as was seen in one series.¹⁰

Peer review protocols differed between programs (Table 1). A typical process was prospective and involved a multidisciplinary team that met to examine cases (Figure 1). Decisions would be made during the review and plans approved, or changes recommended. Changes would be made and plans enacted or presented for a second review, depending on the nature of the proposed changes.^4,5,15,16 The selection process for cases included for peer review varies significantly among programs. Some locations attempted to audit all new patients, while others would choose cases based on the complexity, site, dosage, and modality.^1,15 Whether a patient was decided for review through random selection or because the treatment intention was curative, the decision-making behind which patients will be reviewed appears to be highly specific to the department and faculty.⁴ Peer review meetings tend to occur on a weekly basis,⁴ with some institutions holding sessions more frequently, and some even having impromptu meetings, if necessary, to reduce treatment delays.¹ Most of the time, meetings would be held in-person, though technology was leveraged to allow for meetings when scheduling issues came up, for example, through video-based meetings or individual review.^1,4,15

Peer review in radiation oncology departments has been documented through structured recording systems that track case presentations, recommendations, and outcomes, though practices vary significantly across institutions. The American Society for Radiation Oncology recommends that practices create well-developed peer review strategies that include review processes for the entire practice and for individual clinical care.¹⁷ However, documentation of peer review remains inconsistent. In a survey of German radiation oncology departments, it was found that attendance tracking and implementation of recommended changes were inconsistent.¹⁸ On a similar note, in a Canadian provincial survey, only 36% of centers reported recording peer review outcomes on the medical record.¹⁹

An effective documentation system vastly improves patient outcomes in medicine. Improvements in many aspects to the documentation system include conscious effort on the following: recording of attendance by multidisciplinary team members (physicians, medical physicists, dosimetrists, radiation therapists), documentation of case-specific recommendations and their classification (e.g., no change, minor change, major change), tracking of whether recommended changes were implemented, and use of standardized peer review audit tools (PRATs), such as those developed by the Royal Australian and New Zealand College of Radiologists.^{1,3,10,18,20-22} The timing and format of documentation varies by institution. Some departments conduct prospective documentation during treatment planning conferences, while others use electronic peer review systems or collaborative methods.^1,17,23 The mean time for reviewing each plan ranges from approximately 2 to 8 minutes per case.^11,20,24 Despite recommendations for systematic documentation, surveys reveal that many institutions lack standardized approaches. A systematic review found significant variation in documentation and reporting practices internationally, with no consensus on grading systems for peer review outcomes.⁴ This has led to calls for standardized frameworks to facilitate meaningful evaluation of peer review’s clinical impact.^4,12

The incorporation of video-based meeting programs allows for greater flexibility, and the involvement of multiple departments within an institution. This greater logistical freedom improves adherence to peer review sessions.^1,4 Others take a different approach, meeting for peer review sessions with the patient to allow for the incorporation of in-meeting physical examination—for example, in a head and neck oncology program.^25,26 Peer review sessions tended to be between 30 and 60 minutes at most institutions.^4,5,12 Planning sessions most commonly took less than 30 minutes, and rarely would take longer than 1 hour.⁵ The time needed per case could be as low as 2.7 minutes, up to 12 to 20 minutes.^3,5,26-28 Longer time spent per patient was seen with those centers that would include physical examination and was thought to be related to the complexity of the cases.^12,26 The more time spent per patient also had a correlation with the rate of change per case, reaching as high as 22.6% to 66% of cases in those that averaged near 20 minutes per patient.¹²

Effective peer review, by definition, typically requires that the physician attending the case as well as a nontreating “peer” with experience in the relevant tumor site participated at minimum in each session.⁴ The radiation oncologist was almost always present, as well as a reviewing physician, and teams were often multidisciplinary with attendance from the medical physicists, dosimetrist, radiation therapist, residents, and trainees.^4,5,16 The involvement of radiologists in the peer review process was not a common occurrence; however, when it was performed, there was a significant increase in the number of meaningful changes made, emphasizing the value of an interdisciplinary team.¹⁴ Larger institutions with multiple departments are able to leverage interdepartmental peer review with faculty from different sites.¹

The structure of the peer review proceedings was unique to institutions. Various methods have been devised, with some as simple as recommending changes as a particular case is discussed.¹⁵ Formal case review methodologies provided a structured approach to feedback, with a systematic evaluation of various aspects of a treatment plan for each case. One institution devised a form that evaluated 8 aspects of the treatment (workup and staging, treatment intent and prescription, position/immobilization/simulation, motion assessment/management, target contours, normal tissue contours, target dosimetry, and normal tissue dosimetry).¹⁶ Grading of changes was another area of variation among centers. Commonly, they would be graded as “no change,” “minor change,” and “major change.”^4,5,16 Lymberiou et al. developed an ABC categorization of changes, with “A” referring to Adequate, “B” as treatment may proceed with potential changes in the future, and “C” as unsatisfactory and requiring changes before the next set of treatment.¹³

For each of the various categories of a case, the reviewing physician had a chance to approve or recommend a different course of action. The feedback that was provided during review was most often documented into the electronic medical record, in a separate secure electronic database, or the changes were made in real time through face-to-face discussion.^4,5 The response from the attending physician to the proposed changes was infrequently recorded and changes were inconsistently brought back for further peer review.^4,5

Peer review was typically performed before treatment began. Intuitively, this allowed for changes and potential overhaul to occur freely without the burdens of re-optimizing a plan that is already in place.^4,5 When peer review occurred after the onset of treatment, changes were often met with reluctance by the attending physician due to the need to replan.³ Particularly for technically complicated cases, peer review is best done before the medical physics or dosimetry planning steps to avoid re-planning.⁴ Retrospective peer review had its benefits as an educational tool.⁴ In some cases, it is a necessity to review after the treatment began if radiation was administered urgently.¹⁵ In one study from Canada, approximately half of the surveyed programs were able to review >80% of curative treatment plans with limitations from time constraints and oncologist availability.⁵ The protection of time specifically for peer review was highly successful in attaining regularity of attendance, especially when oncologist schedules were specifically planned to maintain time for the review session. Set time blocks allowed for planners and physicians to prepare appropriately.^1,12 Maintaining flexibility with the format of peer review facilitates meetings when obstacles occur. Technology and video conferencing should be leveraged to allow for members to join off-site and from other campuses.^1,4,5

Advancements in automation should be leveraged to identify cases where peer review may have the most impact. Automated technologies may assess large numbers of patients across broad disease and treatment intent categories with an ability to flag critical issues for potential intervention. Deep learning and artificial intelligence (AI) are developing technologies that may make improvements in the quality of peer review while decreasing the clinician and team workload. For example, models have been successfully trained to rapidly predict dose distributions and aid in identifying which plans needed adjustment in the treatment of nasopharyngeal carcinoma, with applied results comparable to what was planned by the treating teams. It is important to keep in mind that AI models tend to address specific technical concerns, for example, dose distribution optimization, whereas multidisciplinary peer review engages in broader clinical decision-making.^29,30

Algorithms can act as a safeguard against erroneous radiation prescriptions, with one trained on thoracic cancer patients achieving a lower error rate than manual peer review.³¹ These models are trained on high-quality cases and end up personalized for the institution, anatomy, and the metrics that the team selects for. This allows AI to complement the process with the addition of more tangible data to the already existing multidisciplinary clinical expertise. Limitations with these models existed on complex patients and those with anomalies; however, this was thought to be because of a lack of sufficient training data, which is an area that may improve as wider incorporation leads to exposure to these rare cases.^29,31

Common clinical tasks can be automated, such as the review of relevant literature that informs radiation planning and scouring of the electronic health record, with nearly 95% accuracy/precision.³⁰ In the realm of quality improvement, the production of incident reports can be assisted with natural language processing and machine learning to aid in the classification and severity determination of incidents.^32,33 Ambient AI and other tools are being quickly adopted by institutions as a time-saving means in documentation and may find a place in the radiation oncologist’s clinic.³⁴ The time saved from the implementation of automation in various aspects of clinical and administrative realms of practice can be employed towards high-quality peer review for manual and automated processes.

Wherever possible, aspects of the peer review process should be standardized. Protocols and local institutional guidelines should serve as a standard that aids the audit.⁴ Forms should be made to ensure that all relevant aspects are addressed independently and to improve the efficiency of the review process.^4,16 Attendance, data collection, recommendation adherence, response from the attending physician, and other “process outcomes” should be tracked and documented to continually assess the efficacy of the peer review process.⁵ Tools such as a color-coded automated dose volume histogram (DVH) analysis can assess DVH adherence to set department criteria.¹ The RANZCR’s PRAT provides a framework to fulfill such needs.² Automated systems may aid in the processes of peer review documentation, for example, with the use of AI summarization and contextual enrichment to relay recommendations made during peer review sessions. With structured data gathering, automated analysis can highlight workflow steps that require intervention.

This study faces several limitations. The nonsystematic nature of this review inclines selection bias in terms of which papers were chosen, the potential papers that were missed in the search process, and the use of only one database to gather articles. This study is also limited in the heterogeneity of the included studies’ methodologies and populations, which limits direct comparisons of results, and as such listed ranges should be interpreted with caution. Despite these limitations, we aim to provide a broad overview of the current state of peer review in radiation oncology.