Dr. Lechner is the Medical Director and Dr. Nelson and Dr. Elvecrog are Staff Radiologists specializing in mammography at the Jane Brattain Breast Center, Park Nicollet Clinic, Minneapolis, MN.

Breast cancer is one of the leading causes of cancer death in women. ¹ Early diagnosis of breast cancer plays an essential role in reducing mortality rates and improving patient prognosis. ¹ Currently, mammography is the gold standard technique for screening detection of nonpalpable malignancies. ^1-4 However, this method has some limitations, as some studies indicate that up to 25% of lesions are not detected by mammography. ^4-6 Mammographic interpretation can be improved when a second radiologist reads the mammogram (known as double reading), increasing breast cancer detection by up to 15%. ^7,8 However, double reading is often not achievable and is not currently advocated as a standard of care. ⁹ Therefore, further improvement in the sensitivity of screening mammography is needed.

Such improvements have been made possible with the development of computer-aided detection (CAD). ^10,11 This emerging technology can help to improve the sensitivity of breast cancer diagnosis and the consistency of radiologist interpretation of mammograms. ^10,11 Clinical studies have shown that CAD increases radiologist sensitivity for the detection of breast cancer by up to 20%, without increasing the work-up rate or the radiologist reading time (hence, radiologist workload). ^12-15 This indicates that CAD may help to reduce the rate of missed cancers in the context of screening mammography and, potentially, may provide clinical benefits.

Three CAD systems are now commercially available to the radiologist: ImageChecker M1000 (R2 Technology, Los Altos, CA), Mammoreader (Intelligent Systems Software, Clearwater, FL), and Second Look (CADx Medical Systems, Montreal, Quebec, Canada). In practice, the radiologist uses CAD after the initial review of the mammograms. CAD systems are designed to provide the radiologist with visual prompts in specific regions on the image. ^12-15 The films are processed by the CAD system, which uses proprietary algorithms to detect potential areas of concern and highlight potentially suspicious areas (such as masses, microcalcifications, architectural distortions, and asymmetric densities). With the information provided by the CAD system, the radiologist decides whether the highlighted regions of concern warrant work-up and retains the ability to make the final diagnosis. These three CAD systems therefore represent second opinion readers that can assist the radiologist by identifying areas that may have otherwise been missed.

This study was undertaken in October 2001 to compare the performance of ImageChecker M1000 and Second Look in the detection of biopsy-proven breast cancer. The sensitivity of these two CAD systems for the detection of the breast cancer lesions was determined.

Materials and methods

Study design

The mammograms used in this study were obtained from the Jane Brattain Breast Center, Park Nicollet Clinic (Minneapolis, MN). Three experienced mammographers participated in the study. They randomly selected 120 biopsy-proven breast cancer cases. The mammograms that led to the diagnosis of each case were reviewed by both CAD systems. There were 126 lesions identified on the 120 mammograms, which consisted of two types of projection images: craniocaudal (CC) and mediolateral oblique (MLO) views. The lesions found on the mammograms included 69 masses, 47 microcalcifications, and 10 mixed mass/microcalcification lesions. Of the 120 cases of breast cancer found, in 67 cases the lesion was predominantly a mass, 43 had microcalcifications predominantly; 10 cases had mixed mass/microcalcification lesions. The cancers were of various types and grades and included ductal carcinoma in situ (DCIS) and invasive carcinoma (ductal and lobular). The sensitivity of the systems was assessed from the percentage of cancer cases identified correctly. A case was considered true positive if cancer was identified by CAD in at least one view. The number of marks per film made by the CAD systems was also determined.

Image Checker system

The ImageChecker M1000 system consists of a laser digitizer, a computer using proprietary signal-processing algorithms, and a customized motorized viewer with video display monitors. ^12,13 ImageChecker M1000 digitizes the mammograms at high resolution (50 µm pixels) and 12 bits of gray scale (4096 shades of gray). The processing algorithm searches for areas of microcalcifications and masses or architectural distortions. ImageChecker M1000 marks microcalcifications with solid triangles and highlights masses or architectural distortions with asterisks.

Second Look system

The Second Look CAD system is a self-contained unit that includes a high-resolution digitizer, a computer running proprietary algorithms, a printer, and a touch-screen monitor. The system digitizes the mammograms at high resolution (43.5 µm) and 12 bits of gray scale (4096 shades of gray). The proprietary algorithms search for areas of microcalcifications and masses (spiculated and nonspiculated, asymmetric densities, and architectural distortions) prior to the generation of a paper report, known as the Mammagraph. Second Look marks microcalcifications with rectangles and suspicious masses with ovals.

The radiologist reviews the CAD results and takes a second look at the films in the locations of the potential regions of concern detected by CAD, to determine whether these suspicious areas warrant work-up.

Comparison of the CAD systems

The performance of each system was compared with the other by assessing the sensitivity and specificity of breast cancer detection for the ImageChecker and Second Look systems.

Results

Lesion-based performance

Analysis of the performance of both CAD systems, based on the total number of lesions, revealed high sensitivity for the detection of breast cancer (Table 1). ImageChecker M1000 correctly marked 114 (90.5%, 95% CI: 84 to 96) and Second Look correctly marked 111 (88.1%, 95% CI: 81 to 93) of the126 lesions (Table 1).

Both CAD systems showed a greater sensitivity for the detection of microcalcifications than for masses. ImageChecker M1000 correctly marked 100% of microcalcifications, while Second Look correctly identified 93.6%. Second Look correctly marked 82.6% of the mass lesions and identified 100% of the mixed mass/microcalcification lesions. ImageChecker M1000 highlighted 84.1% of the mass lesions and correctly marked 90% of the mixed mass/microcalcification lesions.

Case-based performance

Both CAD systems demonstrated a high sensitivity for the detection of breast cancer cases (Table 2). ImageChecker M1000 correctly marked 108 (90%, 95% CI: 83 to 95) of the 120 cases of breast cancer, while Second Look correctly identified 107 (89.2%, 95% CI: 82 to 94) of them (figure 1). Second Look detected 97.7% of the cases of microcalcifications and ImageChecker M1000 marked 100%. ImageChecker M1000 highlighted 83.6% of the cases including masses and Second Look identified 82.1%. While ImageChecker M1000 correctly marked 90% of the mixed mass/microcalcification lesions, Second Look identified 100%.

Characteristics of cancers missed by CAD

A subgroup of 6 breast cancer cases was not identified by either of the CAD systems (Table 3). There were 9 cases that were identified correctly by ImageChecker M1000, but not by Second Look and 6 cases marked correctly by Second Look, but not by ImageChecker M1000. These subgroups of missed cases consisted of 19 invasive ductal and lobular cancers and 2 cases of DCIS with various grades.

Distribution of marks per case

The distribution of marks made by the CAD systems is illustrated in Figure 2. Four films were used for each case. The number of Second Look marks per case ranged from 0 to 7, with an average of 1.33 marks/film. For ImageChecker M1000, the range of marks per case was 0 to 12, with an average of 1.23 marks per film.

Statistical analysis of the CAD systems' performance

The sensitivities of the CAD systems were compared with a hypothesis test. Null hypothesis : Second Look and ImageChecker M1000 have the same sensitivity in detecting breast cancer in at least one view. Alternate hypothesis : The sensitivity of these CAD systems are different in their ability to detect breast cancer in at least one view.

To reject the null hypothesis with a 90% confidence interval requires a z score of 1.645. Comparisons of the case and lesion sensitivities of ImageChecker M1000 and Second Look give: z case = 0.2113; z lesion = 0.616. Therefore, this result supports the statement that ImageChecker M1000 and Second Look have equivalent sensitivity in detecting screen-detected breast cancers in at least one view.

Discussion

This study is the first head-to-head comparison of the performance of two commercially available CAD systems, ImageChecker M1000 and Second Look, on the same breast cancer cases and at the same time. Second Look and ImageChecker M1000 showed a similar high sensitivity for the detection of breast cancer in at least one view. The results obtained in the present study are consistent with those of previous clinical studies separately assessing the sensitivity of ImageChecker M1000 ^12,16 and of Second Look. ^13-15

It has been documented that mammography can reduce mortality by approximately 30%. ^2,3 But, despite the recognized usefulness of mammography, the chance that breast cancer can be missed with this method can be as high as 25%. It has been suggested that up to two-thirds of the cancers missed by mammography reading are potentially detectable. ¹⁷ Therefore, it is evident that an improvement in mammographic sensitivity is needed.

Some studies have shown that the detection of breast cancer may increase by up to 15% when a second radiologist interprets the mammograms. ^7,8 However, the practice of double reading has not been widely instituted and is not currently advocated as a standard of care. ⁹

Advances in computer technology have led to the development of CAD systems that can serve as second readers to assist radiologists in the clinical setting. ¹⁰ With CAD systems, radiologists have a means to overcome the known limitations of human interpretation of mammograms. Studies have shown that CAD significantly increases radiologist sensitivity for the detection of breast cancer, without increasing the radiologist's workload. ^12-16 The present study demonstrated that ImageChecker M1000 and Second Look have a high detection rate for features of biopsy-proven breast cancer on mammograms.

This study was designed to consider one-view detection a true positive. It should be stated, however, that two-view detection would reflect the sensitivity of a CAD system more accurately. Radiologists must be aware that true abnormalities, including cancers, may be marked by a CAD system on only one view. There were 14 cancer cases identified correctly by ImageChecker M1000 in two views and by Second Look in one view. On the other hand, 6 cases were identified correctly by Second Look in two views and by ImageChecker M1000 in one view.

Conclusion

The results of this study indicate that CAD systems may help to reduce the rate of missed breast cancers. The two CAD systems evaluated in this study, ImageChecker M1000 and Second Look, have demonstrated an equivalent performance in their sensitivity of detection of breast cancers when using one mammographic view as the criterion for a positive finding. AR