Reliability, Quality, and Readability of Freely Available Large Language Models for Clinical Decision Support in Perforated Peptic Ulcer: A Blinded, Multidimensional Study

Background/Objectives: Perforated peptic ulcer (PPU) is a common surgical emergency in which each hour before surgery increases mortality, and the decision is often made by a single surgeon on call. Large language models (LLMs) may offer rapid decision support, but their reliability in this setting is unknown. We compared five freely accessible LLMs in a blinded, multidimensional design.

Methods

Twenty-four structured questions derived from the WSES 2020 guideline, covering the preoperative, intraoperative, and postoperative phases, were posed to the free versions of ChatGPT (GPT-5.3 Instant), Claude (Sonnet 4.6), DeepSeek-V3, Gemini (2.0 Flash), and Grok 3 during April and May 2026, yielding 120 responses. Two general surgeons blinded to model identity scored scientific reliability with the modified DISCERN instrument (mDISCERN, 8–40) as the primary measure and overall quality with the Global Quality Score (GQS, 1–5) as a secondary measure. Because every question was answered by all five models, the Friedman test was primary.