Concordance Between Large Language Models and Multidisciplinary Tumor Board Recommendations in Treatment Allocation for Hepatocellular Carcinoma

Background/Objectives: Treatment allocation for hepatocellular carcinoma (HCC) is a highly complex task that requires multidisciplinary tumor board (MDT) input; however, its accessibility and reliability can vary across the healthcare system. Large language models (LLMs) have emerged as potential clinical decision-making tools to aid MDTs. Our aim was to compare different LLM-generated treatment recommendations for HCC cases with MDT decisions.

Methods

We retrospectively analyzed 100 HCC cases discussed during MDT meetings in a tertiary-care hospital in Cluj-Napoca, Romania. Identical prompts and structured clinical information were offered to four different LLMs (ChatGPT-5, a customized Tumor Board ChatGPT, Gemini 2.5 Flash, and Gemini 2.5 Pro), each of which was required to provide treatment recommendations. Concordance with the MDT decisions was assessed across the first, second, and third treatment options. Inter-model differences were evaluated using Cochran’s Q and Holm-adjusted exact McNemar tests. Chance-corrected agreement was assessed using Cohen’s kappa (κ), and generalized estimating equations evaluated associations between concordance and clinical complexity.