An iterative artificial intelligence workflow could substantially reduce the time radiologists spend generating reference-standard medical image segmentations while maintaining high segmentation accuracy and yielding estimated cost savings, according to a study published in Radiology: Artificial Intelligence.

Researchers from the University of Cologne and collaborating institutions developed and evaluated an expert-guided annotation loop (XpertLoop), a human-in-the-loop framework that iteratively trains segmentation models using expert-corrected model predictions rather than repeated manual segmentation from scratch.

The retrospective study included 1,948 CT and MRI examinations from 1,520 patients across 10 datasets from University Hospital Cologne and public databases covering autosomal dominant polycystic kidney disease, prostate cancer, thyroid eye disease, uveal melanoma, and non-small cell lung cancer.

The researchers trained 57 nnU-Net deep learning segmentation models using two data selection strategies: random sampling and an active learning approach that combined foreground size with prediction uncertainty (SIZE-ENT).

In each iteration, radiologists corrected model-generated presegmentations, which were then incorporated into subsequent model training loops. Across all datasets, later-loop models showed improved agreement with the reference standard and reduced the need for manual corrections compared with early-loop models.

Final models achieved mean Dice scores of 0.93 for both kidney and liver segmentation, 0.90 for prostate segmentation, and 0.96 for eye bulb segmentation. Tumor segmentation was more challenging, with mean Dice scores of 0.62 on the internal lung tumor dataset and 0.61 on the external dataset.

The active learning strategy generally required fewer annotation cycles than random sampling for organ segmentation, indicating greater annotation efficiency. For example, optimal kidney segmentation performance was achieved after three annotation loops with SIZE-ENT compared with five loops using random sampling.

External validation demonstrated good generalizability for organ segmentation models, with mean Dice scores of at least 0.84 for most organs.

The researchers also compared their models with TotalSegmentator. XpertLoop models achieved higher Dice scores for kidney, liver, and prostate tasks while demonstrating similar performance for lung tumor segmentation.

The model was also evaluated for time and cost savings. Radiologists required an average of 5.6 minutes to manually segment kidneys from scratch compared with 0.56 minutes to correct model-generated kidney presegmentations, representing a maximum time savings of 90%. For lung tumors, annotation time decreased from 4.09 minutes to 2.12 minutes, corresponding to a 48% reduction in workload. Both reductions were statistically significant, according to the researchers.

Cost savings were analyzed for kidney and tumor examinations. Using national U.S. radiologist wage estimates, mean cost savings were $14.30 per kidney and $5.63 per lung tumor. While presegmentation was consistently cost saving for kidney segmentation, tumor segmentation yielded positive savings in approximately 78% of simulations because correcting inaccurate presegmentations occasionally required more time than manual annotation, the researchers noted.

The research team also evaluated the practical implementation of the XpertLoop models using Germany's RACOON national radiology research platform. The workflow integrated annotation software with AI training tools through a graphical interface without requiring programming expertise. In a feasibility study, a medical student without coding experience successfully used the workflow for adrenal gland segmentation. After initial model training, manual refinement time decreased from approximately 30 to 45 minutes per patient to 5 to 10 minutes for 86 of 98 patients.

The study had several limitations. Organ-level analyses may have been affected by within-patient correlation. The cost analysis did not include infrastructure, AI development, or operational expenses, limiting economic generalizability. Training multiple models substantially increased computational requirements, totaling approximately 1,375 GPU hours. Time savings were estimated using a single annotator for each task and only with optimized presegmentations. Finally, dedicated clinical deployment and lesion detection were outside the scope of the study.

"Future work will explore additional active learning strategies, including target-aware uncertainty sampling, and extend the framework to multiclass segmentation tasks," wrote first author Astha Jaiswal, of the Institute for Diagnostic and Interventional Radiology at the University of Cologne in Germany, and colleagues. "We also plan to evaluate XpertLoop in multicenter settings and integrate it with federated learning."

Conflicts of interest are reported in the study's supplemental ICMJE disclosure forms, which were not available for review.