Comparative evaluation of ChatGPT and Gemini in answering patient-oriented questions about coronary artery bypass grafting: An expert-based cross-sectional study

Objectives: Large language models (LLMs) are increasingly used by patients to obtain information about high-risk procedures such as coronary artery bypass grafting (CABG).This study aimed to compare expert evaluations of responses generated by ChatGPT (GPT-5.5 Instant, OpenAI) and Gemini (Gemini 3.5 Flash, Google) to frequently asked patient-oriented questions about CABG. Patients and methods:In this expert-based cross-sectional study, responses generated by ChatGPT and Gemini to 29 frequently asked CABG-related questions developed by cardiovascular surgeons were evaluated.The questions were submitted in English to both models through free, publicly available web interfaces using a standardized prompt.Each question was entered in a separate new chat session, and only the first generated response was recorded.Optional web search and Deep Research features were not manually enabled.After model identities and model-specific elements were removed, the responses were presented separately in randomized order.Five cardiovascular surgeons independently rated the responses using a five-point Likert scale that jointly assessed clinical accuracy and usefulness.In the primary paired analysis, the sum of the five evaluators' ratings for each question was used, yielding 29 paired questionlevel observations.Results: Both models received high scores.The mean question-level total score was 23.31±1.54for ChatGPT and 23.00±1.31for Gemini.The mean paired difference was 0.31 points (bootstrap 95% confidence interval [CI], -0.07 to 0.69) and was not statistically significant (Wilcoxon signed-rank test, W=60.0; p=0.129; matched-pairs rank-biserial correlation=0.37).At the individual-rating level, 97.2% of ratings were 4 or 5 for each model.The average-measures absolute-agreement intraclass correlation coefficient was 0.50 (95% CI, 0.13-0.74)for ChatGPT and 0.15 (95% CI, -0.49-0.56)for Gemini.Conclusion: Under the reported evaluation conditions, responses generated by ChatGPT and Gemini received similarly high combined ratings for clinical accuracy and usefulness.These findings reflect the expert-rated clinical appropriateness of model responses generated at a specific time point; they do not establish guideline concordance, within-model reproducibility, patient comprehension, or clinical benefit.LLM-generated CABG information may have a supportive role in patient education; however, this possibility requires validation in patient-based studies.These tools should not replace individualized physician-led counseling.