Can Uncertainty Scores Help Experts Trust AI Brain Tumor Analysis?
MRIs are among the most accurate imaging tests used to diagnose brain tumors, and MRI-based monitoring of tumors guides critical treatment decisions. The clinical standard for measuring tumors often relies on either subjective evaluation or simplified two-dimensional (2D) measurements.
However, for tumors with slow or irregular growth patterns, such as meningiomas, these 2D metrics can fail to capture the true tumor burden, with studies showing that 3D spatial analysis provides a more accurate picture of tumor progression. While deep neural networks can be trained to segment (i.e. delineate) tumors in 3D, there is a level of uncertainty that comes with the automated segmentation, limiting clinician trust and adoption of these diagnostic strategies. Due to tumor characteristics, locations, or imaging quality, that level of uncertainty is inherently higher in some cases than others, but current AI models treat each case the same, as if its outputs are definitive.
To measure levels of uncertainty when using AI to enhance MRI segmentation, UCSF researchers developed a deep learning framework that generated uncertainty estimates for meningioma segmentation on brain MRI. Their Evidential Deep Learning (EDL) AI model achieved a high level of accuracy and produced well-calibrated, credible measurements of tumor volume, supporting safer clinical AI deployment.
Their study appears July xx in npj Digital Medicine.
“The purpose of this study was to develop and validate a fully automated framework for tumor segmentation on real clinical brain MRIs, built on ensembles of EDL models, capable of producing clinically meaningful volumetric measurements and interpretable uncertainty maps,” said Andreas Rauschecker, MD, PhD, UCSF assistant professor of Radiology and Co-Chief of Intelligent Imaging Research. “Even with a good segmentation algorithm, there is always going to be some level of uncertainty about the volume of the tumor because AI can make mistakes in segmentation. Our objective was to capture a mathematical, quantifiable way of assessing the amount of uncertainty.”
High accuracy
The researchers focused their study on meningiomas, the most common primary brain tumor, accounting for over one-third of all intracranial tumors and nearly half of all primary brain tumors. While some meningiomas have very well-defined borders and model confidence would be expected to be high, others are adjacent to anatomical structures that obscure the tumor boundary, or demonstrate unusual shapes, leading to uncertainty in a model when measuring tumor volumes or when communicating tumor boundaries to clinicians.
Their deep learning framework was trained on 1,655 MRIs (788 patients) and included post-operative brain MRIs that added to uncertainty scores because treatment-related changes can exhibit similar patterns to tumor tissue. They also evaluated both homogeneous and heterogeneous AI ensembles on an independent test set of 68 MRIs (43 patients). The algorithm’s performance was assessed from the spatial agreement between uncertainty maps and neuroradiologist-identified ambiguous regions. Their model achieved high accuracy with uncertainty maps aligning with ambiguous regions and well-calibrated volume estimates. External validation in 353 patients confirmed generalizability.
“Our study produced calibrated uncertainty estimation that can be used for lesion segmentation beyond meningiomas,” said Rauschecker. “This capability has the potential to substantially increase trust in biomedical image segmentation, particularly in applications such as brain tumor volumetrics where decisions are sensitive to boundary ambiguities and subtle longitudinal changes in the context of heterogenous image quality.” It also advances the broader goal of developing transparent, safe, and trustworthy AI for medicine, paving the way for uncertainty-aware quantitative monitoring of tumor dynamics in routine clinical care.
Rauschecker notes that while their study was designed for deployment within a local workflow and trained exclusively on internal private data, the high segmentation performance on an independent external test set provides compelling evidence of cross-institutional generalization. Nevertheless, he adds that future studies should incorporate multi-center datasets and multi-rater annotations to fully validate model uncertainty against human inter-observer variability.
Additional UCSF Authors: Yassine Guennoun, Pierre Nedelec, Mark McArthur, Evan Bloch, Jinchi Wei, Leo Sugrue.
Additional Authors: Evan Calabrese
Funding: This work was supported by the Tianqiao and Chrissy Chen Institute Chen Scholars Program and the Foundation of the American Society of Neuroradiology.
Disclosures: The authors declare no competing interests. A patent application covering methods described in this manuscript has been filed by the University of California, San Francisco (UCSF). The named inventors are Yassine Guennoun, Pierre Nedelec, and Andreas M. Rauschecker. The application was filed with the U.S. Patent and Trademark Office on October 17, 2025 (Serial No. 63/901,286) and is currently pending. The patent application covers methods related to the uncertainty-aware brain tumor segmentation and volumetric estimation approaches described in this study.
About UCSF Health: UCSF Health is recognized worldwide for its innovative patient care, reflecting the latest medical knowledge, advanced technologies and pioneering research. It includes the flagship UCSF Medical Center, which is highly-ranked hospital, as well as UCSF Benioff Children’s Hospitals, with campuses in San Francisco and Oakland; two community hospitals, UCSF Health Stanyan and UCSF Health Hyde; Langley Porter Psychiatric Hospital; UCSF Benioff Children’s Physicians; and the UCSF Faculty Practice. These hospitals serve as the academic medical center of the University of California, San Francisco, which is world-renowned for its graduate-level health sciences education and biomedical research. UCSF Health has affiliations with hospitals and health organizations throughout the Bay Area. Visit www.ucsfhealth.org. Follow UCSF Health on Facebook or on Twitter.