Applied Radiology

Dr. Taft is the Director of and Dr. Taylor is a Radiologist at Maroondah BreastScreen, Victoria, Australia. Neither the authors nor their colleagues have a direct financial or corporate association with the equipment and vendor discussed in this article.

Breast cancer is one of the most common types of cancer and is the second leading cause of cancer death in women, exceeded only by lung cancer. ¹ The American Cancer Society estimates that approximately 192,200 new breast cancer cases will be diagnosed and more than 40,000 deaths will occur in 2001. ¹ Early and accurate detection is therefore important for better prognosis and therapy of breast cancer.

Mammography is considered the gold standard for the early detection of breast cancer and is effective in reducing breast-cancer mortality by approximately 30%. ^2,3 However, there appears to be a wide variability in the interpretation of mammograms, mainly because of the complex structure of the breast (particularly dense breast tissue), the subtle nature of the early mammographic features of breast cancer, and radiologists' varied experience, eye fatigue, or inattention. ^4,5 Some studies indicate that up to 25% of malignancies are not detected by mammography. ^6-8 It has been shown that the accuracy of mammographic interpretation can be increased when a second radiologist reads the mammogram; such double reading improves breast cancer detection by up to 15%. ^8,9 It is clear that further improvement in the sensitivity of screening mammography would be widely welcomed.

A study was undertaken by the authors to test the utility of a computer-aided detection (CAD) system developed to improve the sensitivity of screening mammography. ¹⁰ The system used in this study (Second Look, CADx Medical Systems, Quebec, Canada) is for investigational use only and is currently not for sale in the United States.

The radiologist uses the CAD system following initial image review. The films are loaded into the digitizer for computer processing. The system uses proprietary algorithms to detect potential areas of concern and generates a paper report of the screening examination, known as a Mammagraph, which highlights potentially suspicious findings (masses, architectural distortions, asymmetric densities, and microcalcifications). With the information provided by the CAD system, the radiologist decides whether the highlighted areas of concern warrant further investigation and retains the ability to make the final diagnosis. The CAD system represents a second-opinion reader that can assist the radiologist by confirming the detection of suspicious areas or identifying those that might otherwise have been missed.

This study was designed to assess the use of this CAD system in breast cancer detection. The system was used as a retrospective tool to analyze mammograms obtained from the Maroondah BreastScreen program. The sensitivity and specificity of breast cancer detection was determined based on the interpretation of films by two radiologists and by the CAD system.

Material and methods

Study design

This study was designed to evaluate the performance of this CAD system in a retrospective analysis of screening mammograms that led to the subsequent diagnosis of breast cancer, as well as prior screening mammograms of these screen-detected cancer cases. The mammograms analyzed were obtained from the Maroondah BreastScreen program, which provides screening and assessment services for women living in the outer eastern region of Melbourne, Victoria, Australia. Maroondah BreastScreen operates in accordance with the BreastScreen Australia Program and the associated National Accreditation Requirements. Maroondah BreastScreen employs a multidisciplinary team of health professionals, including radiologists, surgeons, pathologists, radiographers, and nurse/counselors, who are highly trained in the detection and assessment of the early signs of breast cancer.

Two separate substudies were undertaken (figure 1). In the first part, the CAD system was used to review 100 cases of biopsy-proven breast cancers collected from the BreastScreen program. The cases were chosen randomly from 100 pathology cases over the prior 2 years. Only cancers visible on mammography were selected for the study; cancers only seen on ultrasound or palpable clinically were excluded. The cancers included ductal carcinoma in situ (DCIS) and invasive carcinoma, both ductal and lobular. From the 100 biopsy-proven cancer cases, the sensitivity and specificity of the CAD system was evaluated based on the detection of masses and microcalcifications, and on the number of marks per image.

The second phase of this study was a review of 50 prior mammograms of biopsy-proven cancers that were reported as normal on the previous BreastScreen review prior to cancer diagnosis. Two experienced radiologists working as a team reviewed the 50 cases and identified 19 prior mammograms with retrospectively visible cancer. The 19 cases were then reviewed independently by the CAD system and by two other radiologists working separately. The sensitivity and specificity for breast cancer detection were calculated as per the first substudy.

Analysis of the mammograms by the CAD system

The screening films consisted of two types of projection images: cra-niocaudal (CC) and mediolateral oblique (MLO) views. The films were loaded into the CAD system and digitized for computer processing (figure 2). Using proprietary algorithms, the system detected potential areas of concern and generated a paper printout of the examination films on which the potentially suspicious findings were highlighted. The radiographic appearance of the cancers was either as clusters of microcalcifications or mass lesions. The term "mass" was used to refer to the usual classifications as per the BreastScreen program and included discrete mass, nonspecific density, architectural distortion, or stellate lesion.

Further analysis was performed to determine whether or not a lesion was detected by the CAD system (i.e., to evaluate the sensitivity of the system). When the CAD system detected a lesion, the case was classified as a true positive. A case with a lesion missed by the CAD system was termed a false negative. The number of marks by the CAD system and the number of films were evaluated. Any mark that was not a true-positive mark was a false-positive mark. The total number of false-positive marks and films were used to calculate the number of false positives per image; this allowed determination of the specificity of the system. False positives identified by a radiologist following the review of mammographic films usually result in further work-up (i.e., additional mammographic films are obtained, and ultrasound, or even biopsy, is performed). However, false positives marked by CAD do not necessarily result in further work-up. Rather, they prompt the radiologist to take a second look at the mammograms in the area corresponding to the false-positive marks. The radiologist can then decide whether or not the case warrants further work-up.

Results

Biopsy-proven cancer cases

Of 100 pathologically proven cancer cases examined by the CAD system, there were 73 masses and 31 microcalcifications detected, including 5 mixed mass/microcalcification lesions (figure 3). The pathology of the 100 cancers included 63 invasive ductal carcinomas, 8 invasive lobular carcinomas, 5 invasive mixed ductal and lobular carcinomas, 4 invasive NOS (not otherwise specified) carcinomas, and 20 DCIS (1 low-grade, 4 intermediate-grade, and 15 high-grade). In most cases, 4 films were used per case. There were 5 films used in 2 cases; 6 in 2 other cases, 7 films in 1 case, and 10 films in another case. The CAD system correctly marked the lesions in 88 of the 100 cancer cases, demonstrating a sensitivity of 88% (Table 1). A total of 631 false-positive marks were counted on 415 films, which corresponds to a rate of 1.5 false-positive marks per image by the system.

Screening mammograms previously considered normal

Two experienced readers working as a team assessed 50 previously interpreted screening mammograms. Of these, they detected 19 cases with retrospectively visible cancer. Those mammograms were subsequently evaluated separately by two other radiologists and by the CAD system. In most cases, there were 4 films per case. In one case, 6 films were used, and in another case, 5 films. The sensitivity of the readers and the CAD system was determined by dividing the number of true positives detected by the total number of cancers (i.e., 19).

For Reader I, there were 8 true positives, corresponding to a sensitivity of 42.1% (Table 2), and 9 false positives. Reader II had the same sensitivity as Reader I, detecting 8 true-positive lesions (sensitivity 42.1%), and 11 false positives. The CAD system demonstrated a higher sensitivity than the 2 readers and correctly identified 12 true positives out of the 19 cancers, a sensitivity of 63.2%. There were 328 false positives marked by the CAD system on 203 films, corresponding to a rate of 1.6 false-positive marks per image.

Further analysis indicated that Readers I and II together identified 10 lesions, 6 of which were the same. Reader I and the CAD system correctly highlighted 13 cases, 6 of which were the same. Combining the number of cancer cases marked correctly by Reader II and the CAD system revealed a total of 13 correct cases, 7 of which were the same. Altogether, the CAD system and the 2 readers correctly identified 14 cancer cases. There were 2 cases detected by the CAD system that were missed by both readers. Both cases were invasive ductal carcinoma. The CAD system, therefore, had a 10.5% higher sensitivity than did the radiologists.

Discussion

It is known that the sensitivity of mammography ranges from 70% to 90%. ^2,3 Thus, there is a 10% to 30% chance that breast cancer will be missed by mammography. Even though mammographic screening cannot detect all cancer cases, it can reduce mortality by approximately 30%. ^2,3 A recent study by Amos and colleagues ¹¹ suggests that of the 10% to 30% of cancers that are missed by mammography, up to two-thirds are potentially detectable. Thus, although mammography remains an effective tool to detect breast cancer and reduce mortality, further improvement in mammographic sensitivity (and eventually further reduction in breast cancer mortality) is clearly needed.

The search for such improvement has led to the development of CAD systems. ¹⁰ In the first part of this study, the CAD system under review showed a high sensitivity for the detection of masses and microcalcifications, suggesting that breast-cancer detection could be potentially improved using this system. The results also indicate that the sensitivity of the system compares favorably with the average radiologist sensitivity calculated at the Maroondah BreastScreen program.

For the second phase of the study, the CAD system detected 24% of cancer cases that were previously reported to be normal, while 2 radiologists detected 16% prospectively. Thus, when considering all 50 cases from that substudy, the CAD system provided an additional sensitivity of 8%. When the reading of the 19 retrospectively visible cancers is considered, a 10.5% increase in sensitivity is obtained with the CAD system. In addition, the CAD system detected 2 unique cases, indicating that it uses an alternate process that complements that of the radiologist for the detection of suspicious lesions.

Conclusion

This CAD system may therefore be viewed as a potential tool to help reduce the occurrence of missed cancers. CAD could also be helpful in screening mammography for earlier detection of breast cancer. This is particularly important, as breast cancers that are detected early are more amenable to treatment. Moreover, delays in breast cancer diagnosis are considered to be one of the leading causes of medical malpractice litigation in the United States. ¹² AR

Acknowledgment

The authors wish to thank Dr. Ilana Bush and Dr. Kerry Whyte for their assistance in mammographic interpretation and the staff of Maroondah Breast Screen for general assistance.