Canadian Urological Association Journal
Development of large language model-generated multiple-choice questions and flashcards to enhance the Canadian undergraduate urology curriculum
By Othmane Zekraoui, Tiange Li, David-Dan Nguyen and 12 others.
Abstract
Introduction
The Canadian Undergraduate Urology Curriculum (CanUUC) is a free, online curriculum that covers important topics in urology for medical students. While multiple-choice questions (MCQs) and flashcards are widely used for medical education, they can be resource-intensive and time-consuming to produce. Therefore, to strengthen the CanUUC with such tools, we developed a customized large language model (LLM)-enabled system to generate MCQs and flashcards from CanUUC material and evaluated the quality of its outputs.
Methods
We used OpenAI’s GPT-5 model and refined two system prompts to generate educational materials from two CanUUC topics (benign prostatic hyperplasia and urolithiasis). Six evaluators blinded to items’ origin were involved, and each item received two independent ratings for relevance, correctness, clarity, distractor quality, and suitability for education. Items with a mean score <3 on the suitability for education evaluation domain were excluded. Results were summarized descriptively.