How Does ChatGPT Perform on the Urology Surgical Sciences Examination? Artificial Intelligence Performance Across the Australian Urology Training Pathway

Background/Objectives: Large language models (LLM) such as ChatGPT have demonstrated competitive performance on standardised medical examinations. Within the Australian urology training pathway, ChatGPT has been evaluated on the Generic Surgical Sciences Examination (GSSE), which gates entry to surgical training, and on the urology fellowship examination, the exit assessment. This study aimed to assess performance of ChatGPT on the intermediate urology-specific Surgical Sciences Examination (SSE).

Methods

A 120-question sample paper was constructed from a question bank compiled by previous urology SSE candidates and reviewed by a senior urology consultant and a urology trainee. Questions were administered to ChatGPT-5.5 across three content domains (anatomy, physiology, and pathology) and three question formats (Type A single-best-answer, Type B stem–reason, and Type X multiple true–false). Items spanned the major urological subspecialty domains. Overall accuracy was compared against the 62.8% pass mark from the July 2025 SSE sitting.