The AI Report Card: A Radiologist's Checklist for Safe, Fair, and Effective AI

Education Exhibit INEE-29, presented at the RSNA 2025 Annual Meeting (Radiological Society of North America), Chicago, 30 Nov - 4 Dec 2025. This exhibit provides radiologists with a practical framework to evaluate AI tools before clinical adoption. It simplifies key metrics (e.g., sensitivity, specificity, AUC-ROC), highlights the importance of diverse training data, and addresses issues like explainability, subgroup performance, and automation bias. By breaking down regulatory nuances, workflow fit, and long-term monitoring, it empowers clinicians to assess AI for safety, fairness, and real-world usability. Outline: Metrics That Matter; Generalizability; Black Box Breakdown; Bias Detection; Regulatory ≠ Ready; Workflow Fit; Model Drift; AI Evaluation Toolkit; QR Corner.