Applied Radiology

Dr. Shile is Medical Director and Dr. Guingrich is a Breast Imager at the Susan G. Komen Breast Center, Peoria, IL.

Breast cancer is among the most common forms of cancer in American women. Second only to skin cancer, it accounts for one of every three cancer diagnoses. According to data from the American Cancer Society, the year 2001 witnessed approximately 192,200 new cases of invasive breast cancer, nearly 47,100 additional cases of in situ breast cancer, and some 40,200 breast cancer deaths. ¹ Only lung cancer accounts for more cancer deaths in women.

Early detection is critical to reducing breast cancer mortality and improving patient outcomes. Screening mammography is the best method available for detecting breast cancer early, when treatment is most effective. With a reported sensitivity of 70% to 90%, ^1-9 mammography is highly effective. It is, nonetheless, imperfect. Intensive research has focused on ways to improve detection of subtle mammographic abnormalities that may signal the earliest stages of tumor development, but may be overlooked during visual examination.

Evaluation of mammograms by more than one radiologist--so-called double reading--has been shown to improve sensitivity substantially, by as much as 15%, ^10-20 but at considerable time and expense. Computer-aided detection (CAD) offers an alternative method of double reading that may be both useful ^21-23 and efficient ^22,24 in reducing the rate of false-negative mammographic interpretations.

Computer-aided detection systems use mathematical algorithms to analyze mammograms for possible abnormalities, including masses, microcalcifications, distortions in breast architecture, and asymmetric densities. After an initial examination of the mammographic images, the radiologist reviews potentially suspicious areas highlighted by the CAD software and determines whether they warrant further work-up. In this way, the CAD system functions as an automated second reader that confirms the presence of suspicious areas of breast tissue or identifies possible abnormalities that might otherwise be missed.

Although commercially available CAD systems operate on similar principles, there are differences in their design and performance. How these differences affect their relative performance in the detection of biopsy-proven breast cancer has been compared for two CAD systems in a study that used the mammograms that led to the diagnosis of each case. ²⁵ The study reported here compared the same two CAD systems, evaluating their ability to detect breast cancers that had been missed during conventional mammographic interpretation.

Methods and materials

Consecutive cases of breast cancer were identified from five private-practice mammography facilities in central Illinois between April 1998 and June 2000. Patient volume at the mammography centers covered a wide range, from low to high volume. All facilities and radiologists were certified under the Mammography Quality Standards Act (MQSA).

Clinical records from each case were examined to determine whether the patients had had a negative screening mammogram within 2 years prior to the diagnosis of cancer. A total of 93 such cases were identified for which a prior mammogram was available.

Two MQSA-certified radiologists at the Susan G. Komen Breast Center, Peoria, IL, examined the 93 prior mammograms to identify findings that warranted any type of follow-up, including comparison with prior films, acquisition of additional mammographic views, and biopsy. The radiologists, who worked independently of one another, were not informed about the specific clinical history associated with the prior mammograms, but were aware that, in each case, cancer had eventually been detected.

The mammographic films were also processed by two CAD systems: Second Look ^® , version 4.0 (CADx Medical Systems, Quebec, Canada) and ImageChecker ^® , version 3.1 (R2 Technology, Los Altos, CA). At the time of the study, version 4.0 of Second Look had just been approved by the Food and Drug Administration (FDA) for sale in the United States. Version 3.1 of ImageChecker had not yet been approved by the FDA, but was the latest version of the software in use in Canada.

One mammographer later reviewed the exams, along with imaging and clinical information from the time of cancer diagnosis, in order to identify whether there was evidence of breast cancer on prior mammography. The CAD output was also examined to identify cases in which CAD had accurately highlighted the site of subsequent cancer. The CAD output and the radiologists' interpretations of prior exams were correlated. These correlations were used to assess CAD-identified imaging features that would lead to further patient work-up.

Results

There was retrospective evidence of cancer in 56% (52 of 93) of the prior mammograms. Table 1 outlines the characteristics of cancers that could be detected on retrospective review (missed cancers) and those that could not (true negatives).

Cases were grouped according to the number of radiologists who indicated a need for further work-up after reviewing each mammogram. The degree of agreement between the two radiologists was considered an indication of how a single radiologist might respond to the identification by CAD of a suspicious mammographic finding.

When abnormalities were identified by both radiologists, the likelihood was considered to be high that identification of these same findings by CAD would prompt a single radiologist to recall the patient for additional follow-up.

Findings identified by only one radiologist signaled a medium-to-low likelihood that a similar finding on CAD would prompt recall of the patient for further work-up. Findings identified by neither radiologist were considered unlikely to prompt patient recall, regardless of the CAD results.

Overall, Second Look (SL) correctly identified 60% (31 of 52) of the missed cancers, and ImageChecker (IC) identified 52% (27 of 52) (Table 2). Of the high-likelihood findings, SL identified 12 of 12 calcifications, 4 of 11 masses, and 1 of 1 distortion. By comparison, IC identified 9 of 12 calcifications, 8 of 11 masses, and 1 of 1 distortion. Of the medium-to-low likelihood findings, SL identified 4 of 4 calcifications, 4 of 8 masses, and 1 of 2 distortions, whereas IC identified 1 of 4 calcifications, 4 of 8 masses, and 0 of 2 distortions.

Discussion

Particularly in the earliest stages of breast cancer, abnormalities on screening mammography can be very subtle. Mammographic interpretation by more than one radiologist has been shown to improve sensitivity, but at a substantially increased cost and investment of radiologist time. ^10-20

Computer-aided detection systems represent an alternative approach to improving the diagnostic accuracy and consistency of mammographic interpretation. In essence, the CAD system offers an automated "second opinion" that the radiologist can consider when reviewing the mammogram and arriving at a diagnosis.

Recent research suggests that CAD systems offer information that can influence the interpretation of mammograms in clinically important ways. A study by Freer and Ulissey ²¹ compared the interpretation of nearly 13,000 screening mammograms by radiologists alone and with the assistance of CAD. They found that nearly 20% more cancers were detected with the assistance of CAD and that a larger percentage were early-stage malignancies (78%, as compared with 73% of those identified by radiologists alone). ²¹

A study by Burhenne et al ²² examined whether the use of CAD could reduce the false-negative rate in screening mammography. Investigators from 13 mammographic facilities identified a total of 427 patients whose screening mammograms had led to the detection of biopsy-proven cancer, and who had a prior mammogram on file. Blinded retrospective review showed that mammographic interpretation by the original radiologists had a false-negative rate of 21%, and that CAD prompting had the potential to reduce that rate by 77%. ²²

In a related study, Birdwell et al ²³ found that of the 115 cancers missed on screening mammography, 30% were calcifications, of which 49% were clustered or pleomorphic. The remaining 70% were mass lesions, of which 40% were spiculated or irregular. The CAD software marked 86% of the missed calcifications and 73% of the missed masses.

The study reported here compares the performance of two different CAD systems for the detection of breast cancers missed by conventional evaluation of mammographic films. Because commercially available CAD systems use proprietary mathematical algorithms to distinguish normal from abnormal breast tissue, it is possible they differ in their abilities to detect breast cancer.

The study's findings confirm previous evidence that CAD markedly improves the detection of suspicious breast lesions that warrant recall of patients for further work-up. Computer-aided detection was particularly effective in highlighting calcifications, which are often the earliest signs of breast cancer. The study's findings also suggest, however, that there may be clinically meaningful differences in the accuracy with which CAD systems detect both calcifications and masses.

Overall, the performance of Second Look was superior to that of ImageChecker. Second Look correctly identified 60% of the missed cancers, and ImageChecker identified 52%. Second Look was also better at detecting calcifications in "high-likelihood" cases, where such information would be most likely to prompt further work-up of patients. ImageChecker detected more masses in this high-likelihood group, and the two systems detected the same number of distortions in breast architecture. In the medium-to-low likelihood cases, Second Look identified more distortions and calcifications than ImageChecker. The two CAD systems were equally effective in highlighting masses.

In the case of high-likelihood lesions, CAD prompting could be expected to result in the recall of all patients for further work-up. Only a portion of patients with medium-to-low likelihood findings would be recalled, however. Thus, of the 52 "missed" cancers, prompting by Second Look would have contributed to the recall of 33% (17 of 52) to 50% (26 of 52) of patients for further work-up. By comparison, ImageChecker would have contributed to the recall of between 35% (18 of 52) to 44% (23 of 52) of patients.

Conclusion

Computer-aided detection systems improve the accuracy and consistency of mammographic interpretation. In a side-by-side comparison of two such systems, Second Look's overall performance was better than that of ImageChecker. Additional research could further improve the mathematical algorithms that CAD systems use to distinguish normal from abnormal breast tissue.