Introduction
Pulmonary embolism (PE) remains a serious condition and is considered in some reports to be the third most common cause of death worldwide.1 Although many cases are mild or subclinical, mortality in more severe instances can reach as high as 35%.2 In the past 2 decades, CT pulmonary angiography (CTPA) has overtaken ventilation-perfusion scanning and invasive angiography as the favored imaging technique to establish the diagnosis of PE, accounting for approximately 90% of such examinations in the United States.3 CT scanners have become widely available in emergent and nonemergent settings. CTPA can be performed rapidly with minimal discomfort to the patient. CTPA has a reported pooled sensitivity and specificity of 0.94 and 0.98, respectively.4
The diagnosis of PE on CTPA relies primarily on the identification of a filling defect within the intravenous contrast column. Ancillary findings include the presence of a wedge-shaped peripheral density in the lung indicating pulmonary hemorrhage or infarction.5 Evidence of increased risk in patients with PE on CTPA includes findings of right ventricular (RV) dysfunction, such as RV enlargement relative to the left ventricle (LV), flattening of the interventricular septum, and reflux of contrast material into the inferior vena cava.6,7
Interpretation of CTPA for PE requires analysis of hundreds of thin-section CT images, as well as multiplanar reconstructions. The extent of PE varies widely, ranging from saddle emboli to isolated subsegmental emboli. A recently published classification system, PE-RADS, categorizes PE based on the centrality of embolus location and other findings on CTPA and MRI.5
Increasing imaging volumes amid a shortage of radiologists have created pressure to interpret studies more rapidly.8,9 Additional challenges include the small size and subtle appearance of some pulmonary emboli and variability in CT technique, including intravenous contrast administration and image acquisition parameters.5 These factors may increase the likelihood that pulmonary emboli will be overlooked by radiologists.
Artificial intelligence (AI) tools hold the prospect of assisting the radiologist at several levels. They may suggest the presence of PE that might otherwise remain undetected by the radiologist and assess PE severity. AI tools may also assist in prioritizing the reading order of CTPA studies such that those with findings suspicious for PE are evaluated immediately. More broadly, AI approaches may permit a reduction in reading time for CTPA studies that can lead to more efficiency and less radiologist burnout.
Here, we discuss the fundamentals of AI and its uses in the context of CTPA, including the basics of AI technology and research into its application of CTPA. We also describe its potential value in a clinical context (Table 1), based on our experience over a 4-year period.
Technical Aspects of AI on CTPA
AI tools have advanced rapidly across multiple areas of medical imaging. In PE detection, AI applications increasingly incorporate machine learning and deep learning methods trained on large CTPA datasets to improve diagnostic accuracy. Preprocessing may be performed to isolate the pulmonary vasculature. In one approach, Digital Imaging and Communications in Medicine data are used to generate a lung mask and a 3-dimensional (3D) bounding box encompassing the thorax. The resulting images are then resized and processed as a series of image slices.
A deep learning model based on a convolutional neural network (CNN), such as ResNet-50, is subsequently applied to extract imaging features associated with PE. The model undergoes training and validation using CTPA datasets to optimize its performance. Ultimately, the model generates a probability score indicating the likelihood of PE. Results may be displayed as probability values, heat maps, or notification icons triggered when a predefined threshold is exceeded.9-11
The stand-alone diagnostic accuracy of CNN-based AI algorithms for detecting PE on CTPA was evaluated in a systematic review and meta-analysis by Soffer et al.12 Seven studies comprising 36,847 CTPA examinations met the inclusion criteria. The pooled sensitivity was 88% (95% CI 80.3-92.7%), and the pooled specificity was 86% (95% CI 75.6-92.4%). The analysis included studies published through 2020. A more recent meta-analysis of 6 studies comprising 9101 CTPA examinations reported higher pooled sensitivity and specificity of 93% (95% CI 88-95%) and 98% (95% CI 93-100%), respectively.13
Applications of AI for CTPA and Other CT Studies
AI as a Second or Concurrent Reader for CTPA with the Radiologist
Based on current FDA requirements, AI algorithms cannot act as a primary interpreter of CTPA studies. Several studies have assessed the performance of radiologists for reading CTPAs vis-à-vis AI techniques, showing benefit for the latter over traditional radiologist reads (Figures 1, 2). One study of 1202 patients who had a PE prevalence on CTPA of 15.8% compared AI tools with emergency radiologists. The AI algorithm had superior sensitivity and negative predictive values (92.6% vs 90%; 98.6% vs 98.1%, respectively), whereas radiologists had better specificity and positive predictive value (99.1% vs 95.8% and 95% vs 80.4%, respectively). However, poor-to-average CTPA image quality decreased the advantage for radiologists and artifacts can be an issue (Figures 3, 4). Nearly 70% of radiologists agreed that AI increased their diagnostic comfort, suggesting the value of AI as a safety net for radiologists.14
A recent study that used real-time adjudication between radiologists and an AI algorithm in 32,501 CTPA studies demonstrated 97.8% concordance between the 2. In instances of discordance, adjudication favored the radiologist in 88.7% of cases.15 Another report of 3316 CTPA studies that involved either a radiologist or radiologist trainee showed significantly higher sensitivity and specificity for the AI tool (96.8% vs 91.6% and 99.9% vs 99.7%, respectively).16 The percentage of radiology trainees in this study was not stated.
A 3rd study used a sequential approach in which CTPA studies were interpreted by radiologists alone and later by radiologists with the assistance of an AI tool, which demonstrated a nonsignificant increase in sensitivity for PE detection (97.6% to 98.6%) and decrease in PE miss rate (12.3% to 6.1%). However, differences in composition between the initial and follow-up radiologist groups that performed the study may have impacted the results.17
The benefit of AI may be greatest among radiology trainees. In a study of 207 patients with suspected acute PE evaluated on CTPA, the sensitivity of PE diagnosis improved from 81.7% to 93.5% after using an AI algorithm. The concordance in interpretation between residents also improved significantly.18 A smaller study among junior trainees showed an increased sensitivity from 83% to 96% for diagnosis of PE on CTPA using AI.19
AI in the Detection of Incidental PE on CT
Incidental pulmonary embolism (IPE) is estimated to occur in 1.3-2.6% of patients undergoing contrast- enhanced chest CT, and AI may be of use in such instances (Figure 5).20,21 In a retrospective study involving 3003 conventional contrast-enhanced CT scans in which AI was utilized, AI showed significantly lower positive predictive value and specificity compared with radiologist readers, 86.8% vs 97.3%, and 99.8% vs 100.0%, respectively, and nonsignificant differences in sensitivity and negative predictive value, 82.5% vs 90.0%, and 99.8% vs 99.9%, respectively. AI detected 4 instances of PE that were overlooked by radiologists based on their reports. In a look-back study involving 1892 enhanced chest CT or chest/abdomen CT scans in 1069 patients with underlying malignancy, IPE was detected in 4% after application of AI, of which only 0.8% were initially reported.22
A study that assessed the effect of interval implementation of AI tools in patients with IPE in over 4000 contrast-enhanced body CT examinations, the vast majority of which were chest CTs, showed that the addition of AI to the workflow significantly increased sensitivity from 80.0% vs 96.2% without a change in specificity (99.9% vs 99.9%).23 Research has also demonstrated the value of AI techniques when applied to IPE on contrast-enhanced abdominal CT. In a study of 5515 contrast-enhanced abdominal CTs, AI tools identified 19 false-negative examinations (0.34%) overlooked on initial interpretation.21
IPE occurs in approximately 1.1% of patients undergoing cardiac CT angiography (CCTA).24 In one study of 1534 CCTA scans, 22 (1.4%) were found to have IPE after analysis with AI. 45% of these IPEs were overlooked on initial interpretation, all involving the segmental and subsegmental branches.25
It is important to note that even in areas where benefit for AI tools can be shown in the assessment of IPE, the downstream clinical value in terms of morbidity and mortality remains uncertain. For example, in the CCTA report described above, RV dysfunction portending an unfavorable prognosis was not found in any study in which IPE was missed. Graeve et al analyzed the impact of improved detection of IPE by AI techniques on 90-day mortality. In comparing cohorts before and after the implementation of AI, they found a decrease in missed IPE from 50% to 7.1%, without a significant change in 90-day mortality.26
Automation in the Assessment of PE Severity
Common CT-based approaches to assess the severity of PE are findings of RV dysfunction based on an increased RV/LV diameter ratio and thrombus burden based on vascular obstruction scores of which the Qanadli score is a widely used example.27 Djahnine used a database of 1386 CTPA examinations with image- and pixel-level annotations. The RV/LV diameter ratio was based on an AI 3D cardiac segmentation model and computer distance maps (distance to nearest boundary). The efficacy of the vascular obstruction score was obtained by plotting the Qanadli scores against the total volume of blood clots. Both indices showed good results on regression analysis, with R² = 0.723 (95% CI 0.668‐0.766) for RV/LV diameter and R² = 0.717 (95% CI 0.668‐0.760) for the Qanadli score, respectively.28
In a recent clinical study, 408 patients had positive CTPAs, 266 (65%) of whom demonstrated an increased RV/LV ratio (>1) determined based on an AI algorithm.29 In this enriched population, patients with an increased RV/LV ratio were significantly more likely to be admitted to the intensive care unit and had a higher rate of vascular intervention. However, the 30- and 90-day mortality was the same for patients with increased and normal RV/LV ratios.
Prioritization of CT Studies
With the increasing workload of radiologists, backlogs for certain types of studies are becoming more common. Included in this category are routine contrast studies that have the potential to demonstrate PE, particularly in oncology patients who are known to have an elevated risk of IPE. One pooled analysis identified IPE in 3.4% of patients.30
AI-triage techniques have the capability to analyze the enhanced CT scans in the queue for IPE and provide notification that allows the study to be prioritized for interpretation. One study that evaluated this potential application assessed 11,736 CT scans in 6447 oncology patients.31 The AI algorithm facilitated a reduction in detection and notification time to a median of 87 minutes from 7714 minutes without the use of AI tools in the routine setting. The IPE miss rate of radiologists in the study decreased from 44.8% to 2.6% with AI assistance.
A second prioritization study performed in a clinical setting without a significant backlog involved 2501 examinations of 2197 patients.20 Comparison of turnaround times before and after application of AI technology showed prioritization of 12.7% of studies in which AI detected PE. The mean report turnaround time decreased to 47.6 from 59.9 minutes, accounted for by a decrease in wait time rather than radiologist reading time. The effect was only noted for CT studies labeled as routine.
Reported Turnaround Time
One study of 11,252 CT studies evaluated the pre- and post-AI implementation turnaround times for CTPA studies.32 The PE-positive rate increased from 13.3% in the earlier cohort to 16.2% with the use of AI tools. During work hours, there was a significant decrease in turnaround time from 68.9 hours to 46.7 hours after AI implementation. However, the time-savings during off-hours were not significant (44.8 hours vs 42 hours).
Savage et al also evaluated report turnaround times before and after AI implementation in a setting without AI-based study prioritization.23 Overall report turnaround time decreased slightly, from 66 to 64.3 hours, although the difference was not statistically significant. However, the wait-time component decreased significantly, from 39.9 to 28.8 hours.
Chronic Pulmonary Embolism
The literature on the use of AI for detecting chronic PE on CTPA remains limited (Figure 6). Abdulaal et al conducted a systematic review that identified 5 studies meeting the inclusion criteria.33 These studies evaluated various manifestations of chronic PE, including vascular abnormalities such as webs, stenoses, and occlusions, as well as parenchymal findings such as mosaic attenuation and pulmonary infarction. One study reported a sensitivity of 62% and specificity of 63% for detecting chronic PE, although the number of CTPA examinations evaluated was not reported.34
Commercial AI Tools
Four commercially available AI tools for PE detection include Aidoc, Viz.ai, Avicenna.AI, and RapidAI. These platforms offer varying capabilities and limitations. Depending on the platform, reported benefits include rapid triage of suspected PE, streamlined communication and care coordination, detection of incidental PE, and automated calculation of the RV-to-LV diameter ratio.
Limitations and Challenges
Drawbacks of AI PE include false-positive results due to structures that mimic PE, motion artifact, and suboptimal contrast. False-negative results are often due to small emboli in distal vessels that are not identified by the algorithm. Chronic PE is also a cause of false-negative results because AI algorithms have not been optimized to detect the imaging manifestations such as webs and mural location.33 Automation bias, the tendency to overly rely on a machine-generated output, can be an issue in stressful, work-heavy environments. AI has been tested in research settings and some clinical settings, but its generalizability to the broader clinical world has not been fully tested.
Use of AI for Pulmonary Embolism on CT in a Clinical Setting
The University of Maryland Medical System consists of 13 hospitals, 9 urgent care centers, and over 150 other care locations across Maryland. Most of the hospitals implemented AI tools for cross-sectional imaging on a routine basis beginning in November 2022. The studies are read by fellowship-trained cardiothoracic radiologists and general radiologists. Prior to that time, researchers at the University of Maryland had published several studies on the use of computer-aided detection algorithms to aid in the diagnosis of PE on cross-sectional imaging.35-37 The following describes our institutional experience with AI PE.
The Aidoc AI tool is used at our institution to evaluate contrast-enhanced CT examinations of the chest and abdomen. The tool is integrated into the picture archiving and communication system and is accessible through a separate icon when a study is opened. A positive finding is indicated by a flashing, rounded orange icon that can be positioned within the workspace according to the interpreting radiologist’s preference.
The general impression among radiologists in our practice is that the AI tool for PE detection is highly effective, particularly compared with other AI tools currently used in the department. In our experience, the tool demonstrates high accuracy, even for subsegmental pulmonary emboli, with relatively few false-positive and false-negative results. Most observed errors involve subtle findings.
Several radiologists use the AI tool concurrently with CTPA interpretation, reviewing the AI results while interpreting the examination. When using this approach, it remains important to review the multiplanar reformatted images. Concurrent use has not been associated with an apparent increase in reporting time in our practice. Some radiologists believe it reduces interpretation time, although this has not been formally evaluated.
Abdominal radiologists in our practice have also noted the value of the AI tool for detecting IPE at the lung bases on contrast-enhanced abdominal CT examinations. We have not implemented AI-based prioritization of studies with suspected IPE, in part because our practice does not currently experience a substantial interpretation backlog. However, this capability may be useful during off-hours, when elective CT examinations are typically interpreted the following morning.
Trainees are permitted to use the AI tool but are instructed to formulate their interpretations based on the CT images before reviewing the AI results. Many have elected not to use the tool during their initial interpretation.
Future Directions
The recent release of the PE-RADS classification presents an opportunity to more closely integrate AI PE, including severity scoring and detection of right heart strain with this new system, facilitating an automatically generated PE-RADS score and providing improved standardization for reporting, quality improvement, and research.5 Current models have limited capability to detect chronic PE.33 Because 1-5% of patients with acute PE ultimately develop chronic PE, often with significant clinical consequences, automated detection of chronic PE would be advantageous.38 Another potential application to consider is the use of AI PE to determine the risk of recurrence. Finally, the use of multimodal AI modeling that allows integration of other data from the electronic health system, including clinical and other imaging data, may provide improved clinical risk stratification and workflow efficiency.
Conclusion
Substantial advances have been made in the application of AI to the detection and risk assessment of PE, as well as the prioritization of contrast-enhanced CT examinations. A growing body of research supports the utility of AI in several areas, including detection of acute and incidental PE, risk stratification, workflow optimization, and reduced interpretation turnaround times. Nevertheless, much of the evidence comes from single-center studies, relatively few studies are prospective, and randomized trials comparing outcomes with and without AI assistance present practical challenges. Consequently, although AI has demonstrated promise in improving PE detection and clinical workflows, whether these improvements translate into better patient outcomes remains uncertain. AI-assisted assessment of PE on contrast-enhanced CT is increasingly being integrated into clinical practice, although the extent of its future adoption remains to be determined.
| Application | Evidence Strength (1–4) | Clinical Utility |
|---|---|---|
| CTPA PE R/O | 3 | Detection/confirmation in a large volume of images |
| Incidental PE | 3 | Unexpected PE identification in lower IV contrast studies |
| Severity | 2 | Quantifies/standardizes RV/LV and clot burden scoring |
| Prioritization | 2 | Tags positive studies in a busy work environment — decrease turnaround |
| Report turnaround | 2 | More rapid delivery of results to the clinical team |
| Chronic PE | 1 | Earlier intervention to restore normal pulmonary artery pressures |
