Computer-aided detection and second reading utility and implementation in a high-volume breast clinic

Breast cancer is among the top three cancers in American women. In the United States, the American Cancer Society estimates that in 2004, 215,990 new cases of breast carcinoma will be diagnosed. It is the leading cause of death due to cancer in women under the age of 65. Incidence rates have continued to increase, although at a slower pace than in previous years, in part due to increased screening using mammography.¹ Early detection is the primary method of reducing mortality from breast cancer since there is no way to prevent it at this time. Screening mammography remains the gold standard for early detection. However, we are all aware of the shortcomings of mammography.

Detecting early cancer on a mammogram is a difficult task. In women who started screening at age 40, it has been estimated that a radiologist will diagnose 6 cancers per 1000 women screened. The following years of screening will detect even less in this same group of screened women.² The challenge of identifying subtle changes, whether masses, microcalcifications, or architectural distortion from a four-view mammogram can be frustrating.

The experience and interpretation skill of the radiologist impacts the false-negative rate (miss rate), but even the most experienced mammographer misses cancers that are visible in retrospect. This can occur because the radiologist failed to perceive a worrisome lesion, subtle or not. Perception plays a large role in increasing the sensitivity of the mammographic interpretation. Tackling obstacles of perception can allow for a higher detection rate.

It is generally believed that computer-aided detection (CAD) can provide a valuable second look and improve the accuracy of breast cancer detection at an earlier stage. As the number of breast imaging fellowship programs remain stable or dwindle, the use of CAD in breast imaging to increase the sensitivity of mammography is gaining in popularity.³

A recognized means of reducing false negatives in screening mammography is the double reading of mammograms. Many methods (consensus versus blinded double reads) have been described in the literature with reported increases in cancer detection by as much as 15%. The cost effectiveness and the practicality of double reading are clearly in question, however, as mammographers become scarcer, financial rewards for mammographers decline, and work-load increases in this highly visible, litigation-happy field. A recent review of single- and double-reading effectiveness and cost-analysis revealed an 11% increase in cancer detection by double reading. The added increase in cost per cancer found by double reading was an incremental 39%.⁴

It was found that the double reading had a higher carcinoma in situ detection rate,⁴ causing some concerns regarding over-diagnosis of minimal disease versus earlier treatment. Those physicians who consider it their priority to diagnose cancer as early as possible may consider that the incremental cost of double reading may be well worth it. Most mammographers spend their career believing this is true.

Retrospective studies clearly have illustrated the potential of CAD to reduce the false-negative rate.⁵ Prospective studies have also indicated that the increased sensitivity of CAD is not at the expense of increased biopsy ratio or increased recall rate.⁶

The typical CAD system consists of two freestanding units: a processing unit that digitizes and analyzes the film images, and a display unit consisting of a dedicated mammography viewer equipped with monitors that display low spatial resolution digital images of the examination. The digital images are linked by a barcode to the panels where the films are mounted and are displayed by pressing a button on the auto-view-er control panel (Figures 1 and 4D).

Each digital image may contain zero to several marks indicating areas where the detection algorithm recognizes a pattern that warrants evaluation by the radiologist. Two different types of marks are typically used—asterisks [] indicating masses or architectural distortions or triangles [] indicating microcalcifications (these marks will be different for the various computer-aided detection systems; Figure 2).

The Freer and Ulissey study,⁶ a prospective study involving 12,860 screening mammograms interpreted with the assistance of a CAD system, indicated a 20% increase in sensitivity in detecting malignancies. Without CAD, the radiologists found 41 cancers (3.2 cancers in 1000 women); however, with the aid of CAD, the detection rate increased to 49 found cancers (3.8 cancers in 1000 women). They also found that CAD detected cancers that were Stage 0 (42%) and Stage 1 (17%).

In this study, the radiologist alone found 84% (41 of 49) of total malignancies, 96% (26 of 27) of masses, and 68% (15 of 22) of malignant calcifications. The radiologist missed 8 lesions, 7 of which were calcifications and 1 of which was a mass. The CAD alone marked 82% (40 of 49) of total malignancies, 67% (18 of 27) of masses, and 100% (22 of 22) of calcifications. All of the 9 malignancies that CAD missed were masses. The increase in recall rate was a minimal 1% to 2%.

The criticism of the Freer and Ulissey study⁶ is that the increased detection for cancer was for Stage 0 cancers, as CAD increased detection of Stage 0 lesions by 42% and Stage 1 lesions by 17%.

The authors Freer and Ulissey⁶ noted that high sensitivity and low specificity of the CAD system minimize undue infiuence on the radiologist and focus the radiologist’s attention on subtle, easily missed findings. Microcalcifications can be so tiny, especially in dense tissue, that one can understand why many published studies concerning missed cancers have commented that calcifications compose 19% to 31% of lesions missed at screening.⁵ In the Freer and Ulissey study, the radiologist missed cancers that were predominantly microcalcifications.⁶

Our practice, The Elizabeth Wende Breast Clinic (EWBC), is a freestanding, private clinic established in 1976 by Wende Logan-Young in Rochester, NY. Dr. Young worked alone at the clinic until 1990, when more doctors arrived to help with the workload. We now have a total of 7 physicians working multiple combinations of part-time versus full-time. In addition to the author, Dr. Destounis, the other physicians currently working with Dr. Wende Logan-Young are Ermelinda Bonaccio, MD; Lari Scorza, MD; Posy Seifert, DO; Patricia Somerville, MD; and Margarita Zuley, MD. Most physicians of the EWBC are fellowship-trained, and they offer a combined total of 67 years of breast imaging experience.

Since 1995, we have instituted double reading of all screening mammograms and, soon thereafter, of all diagnostic mammograms (Figure 3). In 2002, ap-proximately 67,670 patients were seen at the clinic (48,755 screening mammograms, 15,309 diagnostic appointments, and 3876 other types of appointments). In 2003, we saw more than 70,000 patients. We provide every woman with online results of screening and diagnostic studies. Typically, while the patient waits, we work up abnormalities found on the mam-mogram online, including extra views to evaluate an area seen, ultrasound, and/or needle biopsy. In 2002, a total of 12,931 ultrasounds, 1771 stereotactic core biopsies, and 2857 ultrasound-guided core biopsies were performed.

We have used CAD as a research tool since 1999. We performed initial retrospective work by evaluating prior screening mammograms of patients who had a cancer detected at a subsequent screening mammogram. These mammograms were digitized and analyzed with CAD. We found that although we double read, there was room for improvement in cancer detection by implementing CAD.

Burhenne et al⁵ revealed their false-negative rate (miss rate) in screening mammography to be 21%. At the EWBC, our own internal audits of cancers detected from 1996 through 2000 revealed a decrease in our false-negative rate of 9% due to double reading. There are 3 main barriers to tracking false negatives: 1) inherent bias to retrospective review; 2) medicolegal concerns; and 3) patients who may not return to the same institution for yearly exams.

The most probable reasons for false-negative mammograms are: oversight; technical limitations; ever-increasing workload and demands; breast density; and human factors such as fatigue and distraction. Overcoming the human factors is an area in which the CAD shows its greatest potential.

Advertisement

Advertisement

EWBC clinical experience with CAD

At the EWBC, we “consensus double read”—two radiologists independently interpret studies online. The second reader usually knows the results of the first reader. We typically discuss difficult cases and further workup; however, we try not to talk each other out of further evaluating a questionable area.

The EWBC employs CAD on all screening and diagnostic patients. It became our standard of care approximately 2 years ago. The workfiow involves one added office staff member sorting the mammograms, scanning and re-sorting the films, and at-taching them to patients’ charts (Figure 4).

For any radiologic procedure, reimbursement issues are, of course, important. There are existing CPT codes for CAD, and third-party payers usually re-imburse for the procedure.

When CAD is first introduced into practice, there is a learning curve for the radiologist, as interpretation of the mammogram with CAD is initially slower than it was without the CAD review.

Most screening studies average 2 marks/case (Figure 5). However, not all of these marks indicate cancer. CAD will usually mark areas of normal asymmetric breast tissue or benign microcalcifications, such as vascular calcifications.

Advertisement

Advertisement

A summary of the EWBC retrospective and prospective CAD studies

The results from our retrospective study of false negatives were reported in 2001 at the annual meeting of the Radiological Society of North America (RSNA) and have recently been published.⁷

Our retrospective review found a total of 519 cancers detected in the year 2000. A total of 318 cancers were included in the study since they had prior mammograms available for review. On retrospective re-view, 52 cancers were determined to have been missed by the clinic on the prior mammograms; all 52 were analyzed with CAD.

The CAD review marked 71% of the cancers on mammograms taken ≥1 year prior to the radiologist finding the cancer. Of a total of 218 marks on the digitized images by CAD, 75 (34.4%) marked cancer. There was an average of 4.5 marks per case in the cancer cohort.

In November 2000, the center instituted our independent double-blinded prospective screening trial. A total of 19,586 patients enrolled through July 2002, and 91 cancers were detected. Preliminary results were reported at RSNA 2002.⁸ Through double reading, 93% of the cancers were detected by one or both radiologists. In addition, the CAD review marked 77% of the cancers and prompted the detection of an additional 6 (7%) cancers.

For individual radiologists, our recall rate has traditionally been 6% to 19%. Recalls increased with the use of CAD by 2%. Of the 91 cancers originally included in the study, CAD missed 21 (23%): 20 invasive cancers and 1 ductal carcinoma in situ. The mammographic findings on these cases included 13 masses, 4 architectural distortions, 3 calcifications, and 1 mass with calcification (Figure 5).

A total of 6 radiologists participated in the study. One of the two readers failed to detect 42 (47%) of the 91 cancers. Yet, 67% of these cancers missed by one of the radiologists were marked by the CAD review. This occurs because the radiologist does not act on all CAD marks (Figures 7 and 8).

This finding raises a very important point regarding the focus on the “potential” of CAD. Certainly, CAD will prompt the detection of otherwise missed cancers, especially in the setting of the single read. In reality, however, no matter how many cancers the CAD marks, action on the marks by the radiologist is the sole point of detection. While CAD companies make claims about CAD sensitivities and false marks, it is more important to focus on the science behind the claims and how the radiologist intends to use CAD. For CAD development, each company must have a training set of cases to develop algorithms and a separate set of cases on which to test the algorithms. The validity of the algorithm is dependent on the sci-entist’s acumen and on the characteristics and quality of the cases. For example, the number of cancers that the algorithm is trained and tested on and the size and morphological features of the training set affect its validity. Just as radiologists diagnose based on their own collective experience, the CAD can only possibly mark based on its collective “experience.” The confidence to act upon CAD marks, without unduly raising recall rates, will depend on the confidence the radiologist has in the science behind the algorithm.