Dr. Castellino
is the Chief Medical Officer of R2 Technology, Inc., Los Altos,
CA. He is also Professor of Radiology (emeritus), Stanford
Medical School, and Chairman (emeritus) of Radiology, Memorial
Sloan Kettering Cancer Center.
As radiologists, we have all experienced the following scenario.
On reading out a case, we observe an abnormality and, upon review
of the prior exam that had been read out as negative, we see that
the same--and rather obvious--finding was present. Wondering who
missed such an apparent abnormality, we look up the prior report
and--much to our dismay--find our name at the bottom of the
page!
No matter how diligent, well trained, and dedicated we are in
our review of radiographic studies, at times we will miss an
abnormality that, at any other time, we would most certainly have
reported. This "observational lapse" reflects the fact that we
simply did not perceive the finding at that read-out session. This
differs from having observed a finding that we dismiss as being
insignificant, which is an error in judgement (an interpretive
error).
It is also well known that errors in perception can be addressed
by "double-reading," e.g., having two observers review the same
studies independently and then compare the results of their
interpretations. Although both observers may detect the same number
of abnormalities (true positives), each will detect some that were
missed by the other, resulting in an increased number of
abnormalities detected (and thus a decrease in the false-negative
studies). In screening mammography, double-reading increases the
cancer detection rate by 5% to 15%.
1
Sophisticated computer algorithms have been developed to analyze
radiographic images to detect features associated with disease;
this technology is called
computer-aided detection
(CAD). The use of computer algorithms to analyze specific
radiographic features to develop a structured differential
diagnosis (e.g., malignant versus benign) is termed
computer-aided diagnosis
. Unfortunately, both technologies share the same acronym, CAD,
and, to some extent, are used interchangeably. In this article, the
technology under discussion is computer-aided detection.
CAD analysis
Image format
The CAD algorithms require that the image to be analyzed (figure
1) be in a digital data set format. Images captured on film (e.g.,
film-screen mammograms and chest radiographs [CXRs]) are first
digitized. Images that are acquired by digital techniques (e.g.,
computed tomography [CT], magnetic resonance imaging [MRI], digital
radiography [DR], and computed radiography [CR]) can be analyzed
directly by the CAD algorithms. In the former, the reproducibility
of the digitizer affects subsequent CAD performance; in the latter,
the technical acquisition of the data (e.g., low-dose versus
high-dose CT, CT slice thickness, pixel size on various full-field
digital mammography units) affects algorithm design and
performance.
Feature detection
The CAD algorithms must be written to detect the specific
radiographic features that the radiologist searches for (e.g.,
microcalcifications and masses or parenchymal distortions on
screening mammograms, lung nodules or masses on CXR and chest CT
examinations). There are various approaches to solving these
problems, but all require a large set of clinically proven cases
upon which the algorithms are trained. Since the algorithms are
trained on the test cases, their performance on the test cases
improves over time.
Therefore, in order to evaluate the performance of the
algorithms, a second, independent set of clinically proven cases
must be used. This second test set should be representative of the
mixture of cases encountered in clinical practice. If the cases
were selected for their degree of subtlety or obviousness, the
algorithm performance would incorrectly appear to be worse or
better, respectively, than it really is. Algorithm performance can
be assessed as follows:
1. The "true-positive" performance (sensitivity) of the
detection algorithms is determined by the percentage of known
lesions that are marked correctly.
2. The "false-positive" performance is determined by the number
of marks placed at locations other than the known lesions,
generally expressed as the "number of false marks per image or
case."
For additional discussion of this topic, see the comprehensive
review by Nishikawa in this supplement.
2
CAD display
The CAD marks should be made available to the radiologist in a
fashion that is intuitive to the read-out session and not
disruptive to the clinical workflow. When CAD is used with analog
images (i.e., X-ray film, such as a mammogram), the CAD output is
distinct and separate from the images being evaluated. A convenient
method to display the CAD marks is on small, low-resolution
cathode-ray tube (CRT) monitors or on a flat panel screen imbedded
in the multiviewer reading station, which the radiologist can
activate after initial review of the case. Alternately, the CAD
marks can be displayed on a paper printout and reviewed in
conjunction with the hard-copy images. When CAD is used with
digitally acquired images that will undergo soft copy review by the
radiologist, CAD marks can be displayed directly on the viewing
monitors.
Current CAD systems typically mark features by a symbol (e.g.,
asterisk) superimposed on the center of the detected feature, or by
placing a geometric shape (e.g, circle or square) around the area
of concern. In general, this works quite well since the feature
detected and marked by the CAD system is readily recognized by the
radiologist, who then must determine the significance of the
feature. At times, however, the CAD mark is in an area of the image
that appears unremarkable to the reviewing radiologist. This can
lead to unnecessary time spent trying to determine what the CAD
system is concerned about. An improved CAD display interface
provides the radiologist with an image of the specific feature that
was detected by the CAD algorithms (figure 2).
Current status of CAD in the USA
As of the time of this writing, the FDA has approved two CAD
devices for radiographic image analysis. Other devices are in
various stages of clinical trials and FDA review, and, presumably,
some will enter the marketplace shortly.
Mammography
To date, the FDA has approved only one CAD device for
mammography (ImageChecker, R2 Technology, Inc., Los Altos, CA). The
system was approved for screening studies in May 1998 and for
diagnostic mammography exams in May 2001. With the current
installed base of approximately 200 clinical units, at the current
utilization rate, approximately 1.9 million women will have their
screening mammograms interpreted with the aid of CAD each year.
This represents approximately 6% of the 33 million screening
mammograms performed last year.
The performance of the current FDA-approved (April 2000)
algorithms for breast cancer detection, with the false marker
rates, is shown in Table 1.
3,4
Based on this performance, the FDA approved the following claim:
"For every 100,000 breast cancers currently detected by screening
mammography, use of the ImageChecker could result in the early
detection of an additional 20,500 cancers." Freer and Ulissey
5
published the results of a prospective study in their community
breast center, concluding that the use of this CAD technology
increased their cancer detection rate by 19.5%.
Chest radiography
The second FDA approval, granted in July 2001, was for a CAD
device to detect solitary pulmonary nodules on film-based chest
radiographs (RapidScreen, Deus Technologies, Rockville, MD). The
sensitivity of the device, measured on 80 biopsy-proven lung cancer
cases, was shown to be 66%, with an accompanying 5.3 false marks
per image. The company showed that a physician using the CAD system
could increase his or her receiver operating characteristic ROC
performance on chest radiographs for lung cancers between 9.5 and
27.5 mm in size. The CAD algorithms were least effective for larger
lesions. These results were obtained using CXR from male heavy
smokers over 45 years of age.
6
CAD applications in radiology
The optimal CAD system should have the following performance
characteristics: high sensitivity in marking the abnormalities
being sought; low number of false-positive marks; "seamless"
integration into the clinical workflow (i.e., CAD input having
minimal impact on the technologist's workflow and CAD output being
easily accessible to the radiologist).
Although virtually all radiologic images could benefit from the
application of CAD, the potential benefits of CAD seem most
applicable to two categories of imaging: screening exams and
data-intense exams.
Screening exams
The interpretation of screening studies is particularly
challenging, since a large number of cases are viewed to detect a
small number of cancers. When the screening study is a conventional
two-dimensional radiographic image, such as a screening mammogram
or CXR, the cancer is often manifest by subtle alterations
superimposed upon the complex radiographic structure of the
surrounding and superimposed tissues. When the screening study is a
data-intense cross-sectional exam, such as CT virtual colonography
for colon cancer or low-dose chest CT for lung cancer, the multiple
image slices that need to be reviewed further compound this.
Data-intense exams
The amount of imaging data presented to the radiologist who is
interpreting scans generated on current CT and MRI protocols is
overwhelming. State-of-the-art CT and MRI scanners provide
exceptional anatomic display of normal and abnormal features
embedded in increasing numbers of scan slices with high spatial and
contrast resolution. The imaging data from these studies can lead
to increased observational oversights for even the experienced
observer. This is compounded when the task at hand is to identify
early, and therefore subtle, features of disease, in a setting in
which the incidence or prevalence of disease is low. Furthermore,
such features may be present on only one or several images of the
tens or even hundreds of scan slices in the data set.
Other applications
Frequently, and especially in oncologic imaging, measurements of
lesions are required for patient management. This is not only
time-consuming for the radiologist, but, as reported by Schwartz et
al,
7
the accuracy of measurements based on handheld calipers on film, or
placement of electronic calipers on a workstation, is inaccurate,
with errors up to 12%. It would be a great benefit if the CAD
system incorporated a set of computational tools to provide
accurate linear and volumetric measurements of a lesion and then
track lesion size over time to assess interval change (figure
3).
Conclusion
Computer-aided detection technology can be viewed as simply yet
another advance in the effort to present an image of greater
clarity to the radiologist to increase lesion conspicuity. Past
examples include advances that are taken for granted today, such as
refinements in film-screen combinations and X-ray generation
equipment in general radiography, and improved spatial and contrast
resolution in CT and MRI images. Or, consider the "low tech" but
frequently used hot-light and magnifying lens, which continue to
serve a similar purpose.
The potential applications for CAD technology in diagnostic
imaging are great. Although early attention has been directed to
those categories of imaging studies in which the likelihood for
observational lapses ("misses") are greatest, radiologists, and
their patients, could benefit from CAD in virtually all imaging
studies (figure 4). Further, as CAD algorithms improve,
radiologists may well be able to devote more of their efforts to
the interpretive aspects of image analysis as the burden for
detection is lessened, or shared, by the computer. *