Academic Journal
Multicenter Evaluation of Large Language Models Versus Hepatologists for Prognostic Prediction in Drug-Induced Liver Injury.
| Τίτλος: | Multicenter Evaluation of Large Language Models Versus Hepatologists for Prognostic Prediction in Drug-Induced Liver Injury. |
|---|---|
| Συγγραφείς: | Fu H; Department of Infectious Diseases, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China., Du Y; Department of Infectious Diseases, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China., Zhao G; Department of Infectious Diseases, Ruijin North Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China., Xiao Y; Department of Infectious Diseases, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China., Song S; Department of Infectious Diseases, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China., Zang X; Department of Infectious Diseases, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China., Zhou T; Department of Infectious Diseases, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China., Zhuang Y; Department of Infectious Diseases, Wuxi Xinrui Hospital, Shanghai Jiao Tong University School of Medicine, Wuxi, China., Mo R; Department of Infectious Diseases, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China., Lai R; Department of Infectious Diseases, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China., Xie Q; Department of Infectious Diseases, Ruijin Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China. |
| Πηγή: | Liver international : official journal of the International Association for the Study of the Liver [Liver Int] 2026 Oct; Vol. 46 (10), pp. e70879. |
| Τύπος έκδοσης: | Journal Article; Multicenter Study; Comparative Study |
| Γλώσσα: | English |
| Στοιχεία περιοδικού: | Publisher: Wiley-Blackwell Country of Publication: United States NLM ID: 101160857 Publication Model: Print Cited Medium: Internet ISSN: 1478-3231 (Electronic) Linking ISSN: 14783223 NLM ISO Abbreviation: Liver Int Subsets: MEDLINE |
| Imprint Name(s): | Publication: Malden, MA : Wiley-Blackwell Original Publication: Oxford, UK : Blackwell Munksgaard, c2003- |
| Ιατρικοί όροι (MeSH): | Chemical and Drug Induced Liver Injury*/diagnosis , Chemical and Drug Induced Liver Injury*/mortality , Large Language Models* , Gastroenterology*, Humans ; Prognosis ; Male ; Female ; Middle Aged ; ROC Curve ; Clinical Decision-Making |
| Περίληψη: | Background & Aims: Drug-induced liver injury (DILI) would progress to chronicity or death. Large language models (LLMs) may enhance clinical decision-making, yet their utility relative to physicians in DILI remains unclear. Therefore, we evaluated their performance in predicting DILI outcomes. Methods: We enrolled 943 DILI patients from three centers as internal and external cohorts. Based on 12-month follow-up, outcomes were classified as recovery, 6/12-month chronicity, and death. LLMs (Gemini-2.5 Pro, GPT-5.1, DeepSeek-3.2), hepatologists (Junior, middle, senior), and models (Hy's Law, nHy's Law, MELD Score) estimated probabilities of outcomes. LLM-Rules (VOTE, OR, AND) were applied to enhance stability. Model performance was assessed. Results: For 6-month chronicity, the senior achieved highest AUROC (0.61) with an accuracy of 70%. Gemini-2.5 Pro and GPT-5.1 yielded AUROCs of 0.60 and 0.59, respectively, outperforming junior and middle hepatologists. Gemini-2.5 Pro demonstrated strongest agreement with senior (κ = 0.43). LLMs all exhibited lower accuracy and specificity than hepatologists. A similar result was observed in 12-month chronicity. For overall mortality, the senior achieved highest AUROC (0.87) with an accuracy of 83%. Gemini-2.5 Pro and GPT-5.1 achieved AUROCs of 0.86, outperforming junior hepatologist, Hy's Law, and nHy's Law. GPT-5.1 achieved strongest agreement with senior (κ = 0.25). LLM-Rules demonstrated stability for predicting outcomes across cohorts. OR and AND rules improved sensitivity and specificity, respectively. Conclusions: GPT-5.1 and Gemini-2.5 Pro showed AUROCs approaching senior hepatologists for DILI outcomes with limited accuracy and specificity. LLM-Rules demonstrated stable performance across cohorts with improved sensitivity or specificity, supporting the potential of multi-LLM approaches as clinician-supervised complementary tools. (© 2026 The Author(s). Liver International published by John Wiley & Sons Ltd.) |
| References: | Perspect Biol Med. 1968 Autumn;12(1):135-61. (PMID: 4387099) J Med Syst. 2025 May 15;49(1):64. (PMID: 40372503) JHEP Rep. 2025 Sep 03;7(12):101579. (PMID: 41321931) Clin Mol Hepatol. 2023 Jul;29(3):721-732. (PMID: 36946005) Hepatology. 2010 Dec;52(6):2065-76. (PMID: 20949552) J Hepatol. 2016 Sep;65(3):532-42. (PMID: 27184533) Gastroenterology. 2019 Nov;157(5):1245-1252.e3. (PMID: 31302142) Liver Int. 2019 Jan;39(1):31-41. (PMID: 30003672) Hepatology. 2000 Apr;31(4):864-71. (PMID: 10733541) Front Immunol. 2022 Apr 12;13:772368. (PMID: 35493499) Liver Int. 2025 Apr;45(4):e16112. (PMID: 39526465) Clin Gastroenterol Hepatol. 2015 Dec;13(13):2360-8. (PMID: 26122767) Nature. 2015 May 28;521(7553):452-9. (PMID: 26017444) J Med Internet Res. 2024 May 22;26:e53164. (PMID: 38776130) Nat Med. 2025 Jan;31(1):60-69. (PMID: 39779929) Liver Int. 2026 Oct;46(10):e70879. (PMID: 42757445) J Hepatol. 1990 Sep;11(2):272-6. (PMID: 2254635) Hepatology. 2017 Oct;66(4):1275-1285. (PMID: 28543844) Gastroenterology. 2015 Jun;148(7):1340-52.e7. (PMID: 25754159) Hepatol Commun. 2020 Feb 20;4(4):518-526. (PMID: 32258947) Liver Int. 2018 Jan;38(1):6-14. (PMID: 28771932) J Hepatol. 2023 Mar;78(3):663-671. (PMID: 36283499) Liver Int. 2025 Jun;45(6):e70115. (PMID: 40347005) J Hepatol. 2019 Jun;70(6):1222-1261. (PMID: 30926241) Gastroenterology. 2014 Jul;147(1):109-118.e5. (PMID: 24704526) Clin Pharmacol Ther. 2011 Jun;89(6):806-15. (PMID: 21544079) J Hepatol. 2022 Aug;77(2):293-295. (PMID: 35421424) Hepatol Int. 2023 Dec;17(6):1626-1636. (PMID: 37188998) Hepatol Res. 2020 Feb;50(2):224-232. (PMID: 31652370) Scand J Gastroenterol. 2015 Apr;50(4):439-46. (PMID: 25639449) Hepatol Int. 2023 Apr;17(2):488-498. (PMID: 36327052) JAMA Netw Open. 2023 Oct 2;6(10):e2336483. (PMID: 37782499) |
| Grant Information: | 82270618 National Natural Science Foundation of China; 2025ZD01906200 Prevention and Control of Emerging and Major Infectious Diseases-National Science and Technology Major Project |
| Contributed Indexing: | Keywords: chemical and drug induced liver injury; chronic disease; clinical outcome; large language models; liver failure; prognosis Local Abstract: [plain-language-summary] Drug‐induced liver injury (DILI) would progress to chronicity or death. We evaluate the utility of large language models (LLMs) in predicting DILI outcomes. We found GPT‐5.1 and Gemini‐2.5 Pro showed AUROCs approaching senior hepatologists for DILI outcomes with limited accuracy and specificity. LLM‐Rules demonstrated stable performance across cohorts with improved sensitivity or specificity, supporting the potential of multi‐LLM approaches as clinician‐supervised complementary tools. |
| Entry Date(s): | Date Created: 20260918 Date Completed: 20260918 Latest Revision: 20261005 |
| Update Code: | 20261005 |
| PubMed Central ID: | PMC13586842 |
| DOI: | 10.1111/liv.70879 |
| PMID: | 42757445 |
| Βάση Δεδομένων: | MEDLINE |
| ISSN: | 1478-3231 |
|---|---|
| DOI: | 10.1111/liv.70879 |