Academic Journal
Evaluation of the performance of large language models in the management of hip and knee osteoarthritis: analysis of EULAR 2023 recommendations.
| Τίτλος: | Evaluation of the performance of large language models in the management of hip and knee osteoarthritis: analysis of EULAR 2023 recommendations. |
|---|---|
| Συγγραφείς: | Mirza Y; Faculty of Health Science, Department of Physiotherapy and Rehabilitation, Necmettin Erbakan University, Konya, Turkey. Electronic address: yakkubak@gmail.com., Sari F; Bingol University, Faculty of Physical Therapy and Rehabilitation, Department of Physiotherapy and Rehabilitation, Bingöl, Turkey. Electronic address: fuldensari@hotmail.com., Arslan S; Faculty of Health Science, Department of Physiotherapy and Rehabilitation, Necmettin Erbakan University, Konya, Turkey. Electronic address: etsarslan@gmail.com. |
| Πηγή: | The Knee [Knee] 2026 Oct; Vol. 62, pp. 104524. Date of Electronic Publication: 2026 Jun 12. |
| Τύπος έκδοσης: | Journal Article |
| Γλώσσα: | English |
| Στοιχεία περιοδικού: | Publisher: Elsevier Country of Publication: Netherlands NLM ID: 9430798 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1873-5800 (Electronic) Linking ISSN: 09680160 NLM ISO Abbreviation: Knee Subsets: MEDLINE |
| Imprint Name(s): | Publication: [s.l.] : Elsevier Original Publication: Oxford, UK : Butterworth-Heinemann, c1994- |
| Ιατρικοί όροι (MeSH): | Osteoarthritis, Knee*/therapy , Osteoarthritis, Hip*/therapy , Large Language Models* , Practice Guidelines as Topic*, Humans ; Reproducibility of Results |
| Περίληψη: | Background: Clinical practice guidelines play a pivotal role in the management of complex and chronic conditions such as osteoarthritis (OA). However, the systematic evaluation of large language models (LLMs) against established guidelines-particularly in the context of knee and hip OA-remains limited. This study aimed to compare the responses generated by multiple LLMs to open-ended questions derived from the 2023 European League Against Rheumatism (EULAR) recommendations. Methods: This study transformed eight recommendations from the 2023 EULAR guideline on non-pharmacological management of knee and hip OA into clinically framed open-ended questions. Each question was posed to DeepSeek-R1, Gemini 2.5 Flash, and ChatGPT-5. The generated responses were evaluated in terms of informational quality, reliability, usability, readability, as well as semantic and lexical (surface-level) similarity to the reference recommendations. Results: DeepSeek-R1 achieved significantly higher scores in informational quality, reliability, usability, and readability compared with Gemini 2.5 Flash and ChatGPT-5 (p < 0.05). Lexical overlap with the reference recommendations was significantly lower for DeepSeek-R1 than for the other models (p = 0.001), indicating reduced verbatim similarity. No statistically significant differences were observed among the models with respect to semantic similarity (p = 0.409). Conclusion: All three models produced generally acceptable outputs; however, DeepSeek-R1 consistently outperformed Gemini 2.5 and ChatGPT-5. Although these models show promising informational quality, reliability, and usability, limited readability restricts their accessibility. LLMs are not yet appropriate for independent patient use and should currently function as supportive tools under the supervision of qualified healthcare professionals. (Copyright © 2026 Elsevier B.V. All rights reserved.) |
| Competing Interests: | Declaration of competing interest The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. |
| Contributed Indexing: | Keywords: Artificial intelligence; Guideline; Hip; Knee; Large language models; Osteoarthritis |
| Entry Date(s): | Date Created: 20260612 Date Completed: 20260909 Latest Revision: 20260909 |
| Update Code: | 20260910 |
| DOI: | 10.1016/j.knee.2026.104524 |
| PMID: | 42284853 |
| Βάση Δεδομένων: | MEDLINE |
καταχωρήστε σχόλιο πρώτοι!