BMC oral health
Diagnostic performance of multimodal large language models in detecting pulp stones on panoramic radiographs
By de Araujo BMM, Cavassin ME, Abuabara A and 7 others.
Abstract
Background
Pulp stones are calcified structures within the dental pulp that may increase the complexity of endodontic treatment and are commonly identified through radiographic examinations. Despite recent advances in artificial intelligence applied to dental imaging, the diagnostic capability of publicly available multimodal large language models (LLMs) for detecting pulp stones under zero-shot conditions remains unclear. Therefore, this study evaluated the diagnostic performance and response consistency of multimodal LLMs in detecting pulp stones on panoramic radiographs of posterior teeth.
Methods
This comparative and exploratory study evaluated the GPT-5 and Gemini 2.5 Flash multimodal LLMs using panoramic radiograph image crops containing posterior teeth. A total of 64 image crops, comprising 192 posterior teeth, were analyzed. The models were tested under standardized zero-shot conditions using independent sessions and a standardized English-language prompt. Each image was submitted six times to both models to assess response consistency. Diagnostic performance was evaluated using accuracy, sensitivity, specificity, positive predictive value, negative predictive value, and F1-score with 95% confidence intervals. Intra-model consistency was assessed using majority agreement, complete agreement across repetitions, and Fleiss' Kappa coefficient.