The authors report the results of their study of the use of a computer-aided detection (CAD) system in screening mammography. the system was used to analyze mammograms retrospectively; the sensitivity and specificity of breast cancer detection based on the interpretation of films by two radiologists and by teh CAD system were compared.
Dr. Taft is the Director of and Dr. Taylor is a Radiologist
at Maroondah BreastScreen, Victoria, Australia. Neither the
authors nor their colleagues have a direct financial or
corporate association with the equipment and vendor discussed
in this article.
Breast cancer is one of the most common types of cancer and is
the second leading cause of cancer death in women, exceeded only by
The American Cancer Society estimates that approximately 192,200
new breast cancer cases will be diagnosed and more than 40,000
deaths will occur in 2001.
Early and accurate detection is therefore important for better
prognosis and therapy of breast cancer.
Mammography is considered the gold standard for the early
detection of breast cancer and is effective in reducing
breast-cancer mortality by approximately 30%.
However, there appears to be a wide variability in the
interpretation of mammograms, mainly because of the complex
structure of the breast (particularly dense breast tissue), the
subtle nature of the early mammographic features of breast cancer,
and radiologists' varied experience, eye fatigue, or inattention.
Some studies indicate that up to 25% of malignancies are not
detected by mammography.
It has been shown that the accuracy of mammographic interpretation
can be increased when a second radiologist reads the mammogram;
such double reading improves breast cancer detection by up to 15%.
It is clear that further improvement in the sensitivity of
screening mammography would be widely welcomed.
A study was undertaken by the authors to test the utility of a
computer-aided detection (CAD) system developed to improve the
sensitivity of screening mammography.
The system used in this study (Second Look, CADx Medical Systems,
Quebec, Canada) is for investigational use only and is currently
not for sale in the United States.
The radiologist uses the CAD system following initial image
review. The films are loaded into the digitizer for computer
processing. The system uses proprietary algorithms to detect
potential areas of concern and generates a paper report of the
screening examination, known as a Mammagraph, which highlights
potentially suspicious findings (masses, architectural distortions,
asymmetric densities, and microcalcifications). With the
information provided by the CAD system, the radiologist decides
whether the highlighted areas of concern warrant further
investigation and retains the ability to make the final diagnosis.
The CAD system represents a second-opinion reader that can assist
the radiologist by confirming the detection of suspicious areas or
identifying those that might otherwise have been missed.
This study was designed to assess the use of this CAD system in
breast cancer detection. The system was used as a retrospective
tool to analyze mammograms obtained from the Maroondah BreastScreen
program. The sensitivity and specificity of breast cancer detection
was determined based on the interpretation of films by two
radiologists and by the CAD system.
Material and methods
This study was designed to evaluate the performance of this CAD
system in a retrospective analysis of screening mammograms that led
to the subsequent diagnosis of breast cancer, as well as prior
screening mammograms of these screen-detected cancer cases. The
mammograms analyzed were obtained from the Maroondah BreastScreen
program, which provides screening and assessment services for women
living in the outer eastern region of Melbourne, Victoria,
Australia. Maroondah BreastScreen operates in accordance with the
BreastScreen Australia Program and the associated National
Accreditation Requirements. Maroondah BreastScreen employs a
multidisciplinary team of health professionals, including
radiologists, surgeons, pathologists, radiographers, and
nurse/counselors, who are highly trained in the detection and
assessment of the early signs of breast cancer.
Two separate substudies were undertaken (figure 1). In the first
part, the CAD system was used to review 100 cases of biopsy-proven
breast cancers collected from the BreastScreen program. The cases
were chosen randomly from 100 pathology cases over the prior 2
years. Only cancers visible on mammography were selected for the
study; cancers only seen on ultrasound or palpable clinically were
excluded. The cancers included ductal carcinoma in situ (DCIS) and
invasive carcinoma, both ductal and lobular. From the 100
biopsy-proven cancer cases, the sensitivity and specificity of the
CAD system was evaluated based on the detection of masses and
microcalcifications, and on the number of marks per image.
The second phase of this study was a review of 50 prior
mammograms of biopsy-proven cancers that were reported as normal on
the previous BreastScreen review prior to cancer diagnosis. Two
experienced radiologists working as a team reviewed the 50 cases
and identified 19 prior mammograms with retrospectively visible
cancer. The 19 cases were then reviewed independently by the CAD
system and by two other radiologists working separately. The
sensitivity and specificity for breast cancer detection were
calculated as per the first substudy.
Analysis of the mammograms by the CAD system
The screening films consisted of two types of projection images:
cra-niocaudal (CC) and mediolateral oblique (MLO) views. The films
were loaded into the CAD system and digitized for computer
processing (figure 2). Using proprietary algorithms, the system
detected potential areas of concern and generated a paper printout
of the examination films on which the potentially suspicious
findings were highlighted. The radiographic appearance of the
cancers was either as clusters of microcalcifications or mass
lesions. The term "mass" was used to refer to the usual
classifications as per the BreastScreen program and included
discrete mass, nonspecific density, architectural distortion, or
Further analysis was performed to determine whether or not a
lesion was detected by the CAD system (i.e., to evaluate the
sensitivity of the system). When the CAD system detected a lesion,
the case was classified as a true positive. A case with a lesion
missed by the CAD system was termed a false negative. The number of
marks by the CAD system and the number of films were evaluated. Any
mark that was not a true-positive mark was a false-positive mark.
The total number of false-positive marks and films were used to
calculate the number of false positives per image; this allowed
determination of the specificity of the system. False positives
identified by a radiologist following the review of mammographic
films usually result in further work-up (i.e., additional
mammographic films are obtained, and ultrasound, or even biopsy, is
performed). However, false positives marked by CAD do not
necessarily result in further work-up. Rather, they prompt the
radiologist to take a second look at the mammograms in the area
corresponding to the false-positive marks. The radiologist can then
decide whether or not the case warrants further work-up.
Biopsy-proven cancer cases
Of 100 pathologically proven cancer cases examined by the CAD
system, there were 73 masses and 31 microcalcifications detected,
including 5 mixed mass/microcalcification lesions (figure 3). The
pathology of the 100 cancers included 63 invasive ductal
carcinomas, 8 invasive lobular carcinomas, 5 invasive mixed ductal
and lobular carcinomas, 4 invasive NOS (not otherwise specified)
carcinomas, and 20 DCIS (1 low-grade, 4 intermediate-grade, and 15
high-grade). In most cases, 4 films were used per case. There were
5 films used in 2 cases; 6 in 2 other cases, 7 films in 1 case, and
10 films in another case. The CAD system correctly marked the
lesions in 88 of the 100 cancer cases, demonstrating a sensitivity
of 88% (Table 1). A total of 631 false-positive marks were counted
on 415 films, which corresponds to a rate of 1.5 false-positive
marks per image by the system.
Screening mammograms previously considered normal
Two experienced readers working as a team assessed 50 previously
interpreted screening mammograms. Of these, they detected 19 cases
with retrospectively visible cancer. Those mammograms were
subsequently evaluated separately by two other radiologists and by
the CAD system. In most cases, there were 4 films per case. In one
case, 6 films were used, and in another case, 5 films. The
sensitivity of the readers and the CAD system was determined by
dividing the number of true positives detected by the total number
of cancers (i.e., 19).
For Reader I, there were 8 true positives, corresponding to a
sensitivity of 42.1% (Table 2), and 9 false positives. Reader II
had the same sensitivity as Reader I, detecting 8 true-positive
lesions (sensitivity 42.1%), and 11 false positives. The CAD system
demonstrated a higher sensitivity than the 2 readers and correctly
identified 12 true positives out of the 19 cancers, a sensitivity
of 63.2%. There were 328 false positives marked by the CAD system
on 203 films, corresponding to a rate of 1.6 false-positive marks
Further analysis indicated that Readers I and II together
identified 10 lesions, 6 of which were the same. Reader I and the
CAD system correctly highlighted 13 cases, 6 of which were the
same. Combining the number of cancer cases marked correctly by
Reader II and the CAD system revealed a total of 13 correct cases,
7 of which were the same. Altogether, the CAD system and the 2
readers correctly identified 14 cancer cases. There were 2 cases
detected by the CAD system that were missed by both readers. Both
cases were invasive ductal carcinoma. The CAD system, therefore,
had a 10.5% higher sensitivity than did the radiologists.
It is known that the sensitivity of mammography ranges from 70%
Thus, there is a 10% to 30% chance that breast cancer will be
missed by mammography. Even though mammographic screening cannot
detect all cancer cases, it can reduce mortality by approximately
A recent study by Amos and colleagues
suggests that of the 10% to 30% of cancers that are missed by
mammography, up to two-thirds are potentially detectable. Thus,
although mammography remains an effective tool to detect breast
cancer and reduce mortality, further improvement in mammographic
sensitivity (and eventually further reduction in breast cancer
mortality) is clearly needed.
The search for such improvement has led to the development of
In the first part of this study, the CAD system under review showed
a high sensitivity for the detection of masses and
microcalcifications, suggesting that breast-cancer detection could
be potentially improved using this system. The results also
indicate that the sensitivity of the system compares favorably with
the average radiologist sensitivity calculated at the Maroondah
For the second phase of the study, the CAD system detected 24%
of cancer cases that were previously reported to be normal, while 2
radiologists detected 16% prospectively. Thus, when considering all
50 cases from that substudy, the CAD system provided an additional
sensitivity of 8%. When the reading of the 19 retrospectively
visible cancers is considered, a 10.5% increase in sensitivity is
obtained with the CAD system. In addition, the CAD system detected
2 unique cases, indicating that it uses an alternate process that
complements that of the radiologist for the detection of suspicious
This CAD system may therefore be viewed as a potential tool to
help reduce the occurrence of missed cancers. CAD could also be
helpful in screening mammography for earlier detection of breast
cancer. This is particularly important, as breast cancers that are
detected early are more amenable to treatment. Moreover, delays in
breast cancer diagnosis are considered to be one of the leading
causes of medical malpractice litigation in the United States.
The authors wish to thank Dr. Ilana Bush and Dr. Kerry Whyte for
their assistance in mammographic interpretation and the staff of
Maroondah Breast Screen for general assistance.