Cognita Imaging Receives FDA Grant for AI Radiology Report Evaluation Research

Cognita Imaging, Inc. (Cognita), a subsidiary of Mosaic Clinical Technologies, Inc., has been awarded a $1.29 million research contract by the U.S. Food and Drug Administration (FDA) to investigate a novel approach for evaluating AI-generated radiology reports. The contract, effective June 22, 2026, is set to last for 18 months.

The initiative, entitled "Virtual Subject Matter Expert Agents for Radiology Report Evaluation," addresses a significant challenge in medical AI—accurately evaluating systems that produce complete radiology reports. Traditional reader studies are valuable but often cumbersome, making it difficult to analyze the extensive range of cases required to assess AI performance in routine clinical settings.

Under this contract, Cognita aims to create and validate a framework that leverages multiple large language models (LLMs) to assess both human-created and AI-generated radiology reports. The project will be spearheaded by Akshay Chaudhari, Ph.D., co-founder of Cognita and an associate professor at Stanford University, who will serve as the principal investigator. Louis Blankemeier, Ph.D., co-founder and CEO of Cognita, will take the role of co-investigator.

"Once an AI system starts writing the entire report, the evaluation problem changes," stated Chaudhari. He emphasized that while limited case studies can demonstrate a model's efficacy in specific contexts, they fail to capture the range of potential failures in actual practice. The project seeks to determine whether a panel of language models, with oversight from radiologists, can yield a more comprehensive and repeatable evaluation at scale.

Cognita refers to this approach as "LLMs-as-a-jury." Instead of having a single language model evaluate another's report, this method compares the assessments of several models. The intent is to enhance evaluation consistency, minimize reliance on individual model biases, and identify opportunities where automated reviews can supplement expert evaluations.

Upon developing and testing its framework, Cognita plans to apply it to about 1 million patient exams across a diverse U.S. cohort. The research will compare performance among various patient demographics, clinical environments, imaging technologies, and disease types, including rare conditions that may not be well-represented in smaller datasets. Additionally, the team will create smaller validation cohorts from this larger set to assess what critical information might be overlooked in limited studies.

Radiologists will evaluate clinically significant discrepancies to determine the source of issues—whether they originate from the AI-generated report, the language model evaluations, or the original radiologist’s interpretations. Findings from this review will inform enhancements to the framework and explore how automated processes and professional expertise can be integrated for both premarket validation and post-market surveillance.

Nina Kottler, M.D., chief medical AI officer at Mosaic Clinical Technologies, remarked, "Radiologists know the edge cases matter. A model can appear robust in a carefully curated study yet face challenges in different clinical settings or with atypical cases. This initiative allows us to scrutinize such variations at a scale that would be difficult through conventional reader studies, while ensuring radiologists remain integral where their expertise is essential."

This research builds on Cognita’s advancements in tailored language and vision-language models, including GREEN, a tool for analyzing reference and AI-generated radiology reports to highlight clinically significant differences. During the contract period, Cognita will provide the FDA with software code, insights for building LLM juries, comparative analyses of validation cohorts, radiologist-reviewed discrepancy studies, and comprehensive reports detailing the project's outcomes and limitations.