Multidimensional evaluation of AI for integrated panvascular disease management: benchmarking against consensus guidelines on lipid, metabolic and antithrombotic therapy (2024-2025)

Abstract: To evaluate the performance of large language models (LLMs) in the comprehensive management (CM) of panvascular disease (PVD) across dimensions of language complexity, comprehensibility, actionability and information reliability.

Design

A cross-sectional benchmark evaluation study using a standardised clinical question bank derived from Chinese expert consensus statements on PVD management.

Setting

Four mainstream LLMs (DeepSeek-V.3.2, Doubao-1.8, Kimi-K2 and Qwen3-Max) were evaluated using 50 standardised clinical questions covering dyslipidaemia management, metabolic regulation, antithrombotic therapy and CM. The evaluation was conducted using the publicly available versions of the models in January 2026. Primary and secondary outcome measures: The primary outcomes were differences among LLMs in nine validated readability metrics, including the Automated Readability Index, Flesch Reading Ease Score, Gunning Fog Index, Flesch-Kincaid Grade Level, Coleman-Liau Index, SMOG Index, Linsear Write, Dale-Chall Readability Score and Spache Readability Formula. Secondary outcomes included understandability, actionability and overall scores assessed using the Patient Education Materials Assessment Tool for Printable Materials (PEMAT-P), information quality assessed using the DISCERN instrument, and correlations between readability and core evaluation metrics.