JAMIA open
Lightweight open-source reasoning large language model for evidence synthesis screening
Par Dávila-García DM, Gilbert A et Wilcox AB.
Contenu en anglais
Résumé
Objectives
To evaluate performance and efficiency of a lightweight open-source reasoning large language model (LLM) against an expert-derived reference standard for screening scientific publication databases.
Materials and methods
Two expert reviewers screened subsets of database records across 3 stages: (1) base eligibility (n = 200), (2) obesity focus (n = 200), and (3) women of reproductive age focus (n = 150). The LLM (gpt-oss-20b) screened all records (n = 20 118). Performance (F1-score and sensitivity [recall]) and efficiency (average screening rate) were assessed.