A conversation with Timothy Freer, MD and Michael Ulissey, MD, the two doctors who conducted the first large prospective clinical trial of the use of computer-aided detection (CAD) in screening mammography.
A conversation with Timothy Freer, MD and Michael
Drs. Timothy Freer and Michael Ulissey conducted the first large
prospective clinical trial of the use of computer-aided detection
(CAD) in screening mammography at Women's Diagnostic & Breast
Health Center in Plano, TX. In their study of 12,860 women
undergoing screening mammography, they have reported a 19.5%
increase in cancers detected. Dr. Freer presented an abstract of
their results at the Radiological Society of North America meeting
in November 2000.
Their complete study results were published in the September 2001
While the study was designed primarily to measure the impact of
CAD on radiologists' detection rate for breast cancer, additional
data were collected to measure the collateral effects of using CAD,
such as its effect on recall and biopsy rates. In a conversation
, Drs. Freer and Ulissey discussed their results and their insights
into the use of CAD for screening mammography.
The promise of CAD
What role do you think CAD will play in radiology practices
Given our experience, we are no longer comfortable reading
mammograms without CAD. Let's face it, most radiologists reading
screening mammography are out there alone. We know that
double-reading mammograms increases cancer detection, but this
remains the exception because of the economic realities of medicine
in this country. It was certainly that way for us.
We now know that we can reduce mortality from breast cancer by
63% in women who are getting their annual mammogram.
Nonetheless, 21% of the time, we overlook the extremely subtle
signs of breast cancer and miss the opportunity for the earliest
possible detection. Despite advances in technology, the
false-negative rate of screening mammography is still far too high.
No matter how experienced and diligent the radiologist is, the
accuracy of mammography interpretation is limited by human
perception, and computer-aided detection has the potential to
eliminate the majority of these oversights.
Which factors led you to purchase a CAD system for your
The results of the premarket analysis of CAD were intriguing. They
documented that CAD had the potential to "mark" mammographic signs
of cancer initially missed by the radiologist, which should reduce
the false-negative rate of mammography. The study by Dr. Warren
Burhenne et al
reported that CAD marked the missed findings in 77% of
false-negative prior mammograms. It's hard to ignore data like
that. Most so-called "misses" are the result of a failure to
perceive subtle changes on the mammogram. If these are pointed out
to the radiologist, such a "miss" is less likely to occur, and CAD
clearly had the ability to mark 75% to 80% of these lesions.
What prompted you to conduct this study?
Frankly, we wanted to know if we'd made a good investment. A
"potential" is exactly that. We needed to know just how well CAD
actually performed in a clinical setting like ours, and what, if
any, were the downside effects of its use.
The previously published reports documented that CAD had the
potential to reduce the false-negative rate. But there was no data
demonstrating the effect of the use of CAD on radiologists'
sensitivity for detecting clinically occult breast cancers, on the
recall rate, or on the outcome of the patients who were recalled.
Each of these factors required prospective analysis to evaluate the
clinical effectiveness of CAD for screening mammography. Our study
was designed to assess exactly that.
Your article mentions that you waited 75 days after the system was
installed before beginning data collection for your study, in order
to become familiar with the system and develop the procedures for
using CAD. Can you comment on your experiences during this
We mainly noted that CAD marked a lot of things. As it turns out,
this is an important attribute of CAD. We learned not to obsess on
any one mark, but rather to re-inspect any marked area to ensure we
had not overlooked an important finding.
How CAD is used
How does CAD analysis work?
The CAD system consists of two units: the processing unit and the
display unit. The processing unit digitizes and analyzes the film
images, and the display unit displays digital images of the
mammogram on two monitors on the mammography autoviewer. The
digital images are electronically linked by means of a bar code to
the panels where the actual film images are mounted and are
displayed by pressing a button on the autoviewer control panel
(figure 1). Each digital image will display the marks, if any, that
indicate areas where the detection algorithm recognizes a pattern
that warrants closer evaluation by the radiologist. Two types of
marks are used: an asterisk, which indicates a pattern suggestive
of a mass or area of architectural distortion; and a solid
triangle, which indicates an area of clustered bright spots
suggestive of microcalcifications (figure 2).
During your study, how did you use CAD for mammography
How we used CAD in the mammography interpretation is particularly
important. Every mammogram must first be read in the usual manner.
Over our study year (February 2, 1998 to February 1, 1999), 12,860
women underwent routine screening mammography. When possible, each
mammogram was compared with the patient's oldest prior mammogram of
comparable quality or the most recent abnormal mammogram. In this
interpretation, the radiologist must make a primary decision:
either the study is "normal" or there are potential abnormalities
that require further investigation. Only after this initial
interpretation should CAD analysis be engaged, the mammogram
re-reviewed, and any change in the radiologists' decision made.
Our study protocol required that once we re-evaluated an area
marked by CAD, we could recall the patient or request prior
mammograms to review for a potential abnormality not initially
detected by the radiologist. However, the failure of the CAD system
to mark a potential abnormality originally detected by the
radiologist could not change the radiologist's decision to act on
This is a crucial point. A CAD analysis must never dissuade a
radiologist from recalling a patient for further evaluation even if
it has failed to "mark" an area that the radiologist believes is
potentially abnormal. It can only be allowed to initiate a recall
or request for prior films, by pointing out an area that the
radiologist initially overlooked and, on close inspection, clearly
merits further evaluation. The radiologist must be the final
arbiter on the merits of any finding, regardless of how it was
How specific were the results of the CAD interpretation of the
As Dr. Ulissey pointed out, the extremely low specificity of any
one "mark" was considered an important attribute of the detection
algorithm. Nearly 98% of the marks applied by CAD in this study
were ultimately dismissed as unimportant by the interpreting
Only 2.4% of the marks applied by CAD were recalled for
diagnostic evaluation, and most of those marks were on findings
already noted by the radiologist. The remaining marks were
dismissed quickly and easily. This may sound like a weakness of the
system, but it really affords us a two-fold advantage. First, the
value of any detection system lies primarily in its sensitivity. In
most systems, this requires a trade-off in specificity; the lower
the specificity, the more confidence we had in the likelihood that
most of the important lesions would be marked. Secondly, the low
marking specificity prevented us from obsessing on any CAD mark
that lacked visible mammographic findings, therefore not
influencing our judgment. It did what it was designed to do--that
is, to briefly focus our attention on an area of the image where a
subtle cancer might have been overlooked.
Sensitivity for malignancy
How did the results of the radiologists' interpretations compare
with the results of the CAD system?
The radiologist and the CAD system were statistically equal in
their ability to detect mammographic signs of malignancy. Of the 49
malignancies detected during the study period, the radiologists
detected 41 lesions and missed 8 lesions prompted by the CAD
system, increasing their sensitivity by 19.5%. The CAD system, in
turn, detected 40 lesions, and failed to mark 9 lesions detected by
the radiologist. More importantly, together, the radiologist and
CAD were 20% better than either was alone.
Seven of the 8 lesions initially missed by the radiologist
presented as clustered microcalcifications. All 9 malignancies that
CAD failed to mark presented as masses.
While we understand that radiologists will be interested in this
type of comparison data, we have to emphasize that current CAD
systems are not designed to act independently. The focus should be
placed on the performance of the radiologist when using CAD, not on
how either performed alone.
It seems clear to us that more refinement of the mass-detection
algorithms is needed, but we were concerned by the predominance of
microcalcifications missed by the radiologist. Many radiologists
may be overconfident in their ability to detect
microcalcifications. They can be very obscure and easily
This was certainly an eye-opener. In our study, 49% of the
actionable findings initially detected by CAD were clustered
microcalcifications. That may draw a number of critical comments.
To those who suggest that the early detection of such lesions may
have little impact on mortality reduction, we hasten to point out
that 3 of the 7 missed lesions presenting as microcalcifications
were moderate to high-grade ductal carcinoma in situ (DCIS) and 2
were invasive carcinoma. We all have increasing respect for the
lethality of higher-grade DCIS, particularly comedo types, and for
the fact that early invasive lesions may present as
microcalcifications without a visible mass.
Effect on recall rate
What effect did the CAD interpretation have on your recall
It is important to understand that CAD, as used in our practice,
can only increase the recall rate. This is because we did not allow
the lack of a CAD mark on a finding initially considered worrisome
to change our recommendation for further evaluation. CAD must never
be used to help determine the importance of a mammographic finding;
that job belongs to the radiologist. However, an increase in the
recall rate is not counter-productive if there is a commensurate
increase in cancers detected.
As Dr. Ulissey implied, if CAD had not led to an increased recall
rate, it would mean that the radiologist had been dissuaded from
recalling a patient because CAD failed to mark the radiologist's
finding. Given the proportional improvement in cancer detection (of
19.5%), we believe that the 20% increase in our recall rate (from
6.5% to 7.7%) was quite acceptable.
Effect on biopsies
With this increased recall rate, what change, if any, did you find
in the rate of biopsies being performed?
With CAD, the number of biopsies performed in the screening program
also increased by 20%. To measure the importance of this change, we
compared the positive predictive value (PPV) for biopsy for both
groups. The PPV for biopsies of lesions detected by the radiologist
and those detected only after review of the CAD prompts were both
38%, so no impact on overall PPV for biopsy was observed.
This might have been predicted, since the decision to biopsy was
based only on the merits of the lesion and not on the manner in
which it was detected. Frankly, we seldom knew by which means the
lesion was detected at the time of subsequent work-up. Therefore,
if the PPV were to have fallen, we would have to surmise that there
was something in the nature of the lesions prompted by CAD that
would have led to a disproportionate number of negative biopsies,
and we simply didn't observe this.
Effect on tumor stage
What did you discover about the lesions initially detected by
The most encouraging effects of CAD were a 19.5% increase in the
cancer detection rate, but we also measured an increase in the
proportion of malignancies detected at an early stage. All 8 of the
additional cancers detected by CAD were stage 0 or 1. It seems
intuitive to me the cancers we were missing were likely to be
early-stage lesions, and this notion is supported by our data.
With the current economic climate and the low reimbursement rate
for screening mammography, the financial pressures on mammography
centers continue to increase. How can radiologists justify the cost
of a CAD system?
Certainly, the cost of current CAD systems is substantial, which is
an important factor. Screening volume becomes the key element in
minimizing its cost-per-patient and the necessary reimbursement to
support its use. In our first 15 months of owning the CAD system,
my colleagues and I bore the entire expense of the system. After
completing our study, we recognized the benefit of CAD and felt
obligated to provide it to as many of our patients as possible, but
this required the purchase of a second system, which we simply
couldn't afford. We were left with little choice but to enlist the
support of our patients.
So, we offered the service of interpretation with CAD to our
non-Medicare patients for a small surcharge. Medicare patients were
given the service free-of-charge. The patients responded
enthusiastically, and 80% of the screening patients consistently
elected the service.
We were very gratified by the response of our patients.
Nonetheless, we all look forward to the day when we can provide
this service without asking for their support.
There has been a good deal of coverage in the industry on the need
for increased Medicare reimbursement for screening mammograms.
Where does reimbursement for CAD interpretation stand?
We have had some good news. Earlier this year, the Health Care
Finance Administration (HCFA) announced a Medicare reimbursement
for CAD of $15 and is scheduled to increase the rate to
approximately $19.50 on January 1, 2002. At that level, CAD should
be financially feasible to large- and moderate-volume centers.
Third-party payors are also beginning to take notice of the
technology, and we hope that Medicare will again prove a reliable
bellwether for private insurance companies. Further, the American
College of Radiology recently announced that the AMA has approved a
CPT I add-on code for CAD when used with screening mammography,
effective 2002, which should make it easier to obtain private
Current statistics confirm that breast cancer cases are the leading
cause of malpractice suits, especially claims of missed or late
has pointed out that many potential legal issues surrounding the
use of CAD have yet to be clearly addressed. The use of CAD in
screening mammography has not yet faced any legal challenges, but
clearly many mammographers must be concerned about their potential
medicolegal exposure. What can you suggest?
I must admit that it was one of my first concerns. What might the
legal consequences be of failing to act on a potential abnormality
marked by CAD? We hope that data such as ours will be useful in
defending the position that a potential "missed lesion" can be
judged only on its radiographic merits, and not on whether or not
it was marked by a computer. For myself, I think the small
malpractice risk that the use of CAD might pose is outweighed many
times by the advantage of improved cancer detection. In addition,
one can argue that by using a CAD system at your breast center, you
are doing your best to reduce your possible oversights.
Future of CAD
Since the FDA approved the first CAD system (R2 Technology, Inc.,
Los Altos, CA), more than 200 CAD systems have been installed in
the United States. What do you believe the future of CAD will
We think that the number of installed systems will increase
rapidly, particularly if adequate reimbursement can be obtained. We
really need to study several hundred thousand women to get the
statistical power necessary to validate or refute the findings of
our study. If future studies also confirm improved detection, CAD
will prove to be a true technological leap and, perhaps, the
greatest single advance in breast cancer detection in the last 20