The Influence of Large Language Model-Generated Artificial Intelligence Risk Estimates on Clinical Decision-Making in Breast Imaging: A Simulation-Based Study

Background/Objectives: Large language models (LLMs) are increasingly used as decision-support tools in radiology, yet their influence on clinical decision-making remains a significant concern. This study evaluates how artificial intelligence (AI)-generated malignancy risk estimates influence clinical decisions in breast imaging and assesses automation bias (AB) among radiology residents at different training levels.

Methods

In this simulation-based study, 20 residents reviewed radiology reports for 15 synthetic breast imaging cases generated by ChatGPT-4 Turbo, each presented in either a concordant or discordant format. Participants were stratified into two groups based on their completion of a breast imaging rotation and evaluated each case by assigning a BI-RADS score, recommending management, and rating their confidence in the AI risk estimate. Diagnostic accuracy and independent predictors of AB were analyzed using Generalized Estimating Equations (GEE) to account for reader-level clustering.