Academic Journal
Προβολή σε EDS

Evaluation of the performance of large language models in the management of hip and knee osteoarthritis: analysis of EULAR 2023 recommendations.

Λεπτομέρειες βιβλιογραφικής εγγραφής
Τίτλος: Evaluation of the performance of large language models in the management of hip and knee osteoarthritis: analysis of EULAR 2023 recommendations.
Συγγραφείς: Mirza Y; Faculty of Health Science, Department of Physiotherapy and Rehabilitation, Necmettin Erbakan University, Konya, Turkey. Electronic address: yakkubak@gmail.com., Sari F; Bingol University, Faculty of Physical Therapy and Rehabilitation, Department of Physiotherapy and Rehabilitation, Bingöl, Turkey. Electronic address: fuldensari@hotmail.com., Arslan S; Faculty of Health Science, Department of Physiotherapy and Rehabilitation, Necmettin Erbakan University, Konya, Turkey. Electronic address: etsarslan@gmail.com.
Πηγή: The Knee [Knee] 2026 Oct; Vol. 62, pp. 104524. Date of Electronic Publication: 2026 Jun 12.
Τύπος έκδοσης: Journal Article
Γλώσσα: English
Στοιχεία περιοδικού: Publisher: Elsevier Country of Publication: Netherlands NLM ID: 9430798 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1873-5800 (Electronic) Linking ISSN: 09680160 NLM ISO Abbreviation: Knee Subsets: MEDLINE
Imprint Name(s): Publication: [s.l.] : Elsevier
Original Publication: Oxford, UK : Butterworth-Heinemann, c1994-
Ιατρικοί όροι (MeSH): Osteoarthritis, Knee*/therapy , Osteoarthritis, Hip*/therapy , Large Language Models* , Practice Guidelines as Topic*, Humans ; Reproducibility of Results
Περίληψη: Background: Clinical practice guidelines play a pivotal role in the management of complex and chronic conditions such as osteoarthritis (OA). However, the systematic evaluation of large language models (LLMs) against established guidelines-particularly in the context of knee and hip OA-remains limited. This study aimed to compare the responses generated by multiple LLMs to open-ended questions derived from the 2023 European League Against Rheumatism (EULAR) recommendations.
Methods: This study transformed eight recommendations from the 2023 EULAR guideline on non-pharmacological management of knee and hip OA into clinically framed open-ended questions. Each question was posed to DeepSeek-R1, Gemini 2.5 Flash, and ChatGPT-5. The generated responses were evaluated in terms of informational quality, reliability, usability, readability, as well as semantic and lexical (surface-level) similarity to the reference recommendations.
Results: DeepSeek-R1 achieved significantly higher scores in informational quality, reliability, usability, and readability compared with Gemini 2.5 Flash and ChatGPT-5 (p < 0.05). Lexical overlap with the reference recommendations was significantly lower for DeepSeek-R1 than for the other models (p = 0.001), indicating reduced verbatim similarity. No statistically significant differences were observed among the models with respect to semantic similarity (p = 0.409).
Conclusion: All three models produced generally acceptable outputs; however, DeepSeek-R1 consistently outperformed Gemini 2.5 and ChatGPT-5. Although these models show promising informational quality, reliability, and usability, limited readability restricts their accessibility. LLMs are not yet appropriate for independent patient use and should currently function as supportive tools under the supervision of qualified healthcare professionals.
(Copyright © 2026 Elsevier B.V. All rights reserved.)
Competing Interests: Declaration of competing interest The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Contributed Indexing: Keywords: Artificial intelligence; Guideline; Hip; Knee; Large language models; Osteoarthritis
Entry Date(s): Date Created: 20260612 Date Completed: 20260909 Latest Revision: 20260909
Update Code: 20260910
DOI: 10.1016/j.knee.2026.104524
PMID: 42284853
Βάση Δεδομένων: MEDLINE
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://www.doi.org/10.1016/j.knee.2026.104524?
    Name: ScienceDirect (all content) (s7799221)
    Category: fullText
    Text: View record from ScienceDirect
    MouseOverText: View record from ScienceDirect
  – Url: https://resolver.ebsco.com/c/fiv2js/result?sid=EBSCO:cmedm&genre=article&issn=18735800&ISBN=&volume=62&issue=&date=20261001&spage=104524&pages=104524&title=The Knee&atitle=Evaluation%20of%20the%20performance%20of%20large%20language%20models%20in%20the%20management%20of%20hip%20and%20knee%20osteoarthritis%3A%20analysis%20of%20EULAR%202023%20recommendations.&aulast=Mirza%20Y&id=DOI:10.1016/j.knee.2026.104524
    Name: Full Text Finder (for New FTF UI) (ns324271)
    Category: fullText
    Text: Full Text Finder
    MouseOverText: Full Text Finder
Header DbId: cmedm
DbLabel: MEDLINE
An: 42284853
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Evaluation of the performance of large language models in the management of hip and knee osteoarthritis: analysis of EULAR 2023 recommendations.
– Name: Author
  Label: Authors
  Group: Au
  Data: &lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Mirza+Y%22&quot;&gt;Mirza Y&lt;/searchLink&gt;; Faculty of Health Science, Department of Physiotherapy and Rehabilitation, Necmettin Erbakan University, Konya, Turkey. Electronic address: yakkubak@gmail.com.&lt;br /&gt;&lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Sari+F%22&quot;&gt;Sari F&lt;/searchLink&gt;; Bingol University, Faculty of Physical Therapy and Rehabilitation, Department of Physiotherapy and Rehabilitation, Bing&#246;l, Turkey. Electronic address: fuldensari@hotmail.com.&lt;br /&gt;&lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Arslan+S%22&quot;&gt;Arslan S&lt;/searchLink&gt;; Faculty of Health Science, Department of Physiotherapy and Rehabilitation, Necmettin Erbakan University, Konya, Turkey. Electronic address: etsarslan@gmail.com.
– Name: TitleSource
  Label: Source
  Group: Src
  Data: &lt;searchLink fieldCode=&quot;JN&quot; term=&quot;%229430798%22&quot;&gt;The Knee&lt;/searchLink&gt; [Knee] 2026 Oct; Vol. 62, pp. 104524. &lt;i&gt;Date of Electronic Publication: &lt;/i&gt;2026 Jun 12.
– Name: TypePub
  Label: Publication Type
  Group: TypPub
  Data: Journal Article
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: TitleSource
  Label: Journal Info
  Group: Src
  Data: &lt;i&gt;Publisher: &lt;/i&gt;&lt;searchLink fieldCode=&quot;PB&quot; term=&quot;%22Elsevier%22&quot;&gt;Elsevier &lt;/searchLink&gt;&lt;i&gt;Country of Publication: &lt;/i&gt;Netherlands &lt;i&gt;NLM ID: &lt;/i&gt;9430798 &lt;i&gt;Publication Model: &lt;/i&gt;Print-Electronic &lt;i&gt;Cited Medium: &lt;/i&gt;Internet &lt;i&gt;ISSN: &lt;/i&gt;1873-5800 (Electronic) &lt;i&gt;Linking ISSN: &lt;/i&gt;&lt;searchLink fieldCode=&quot;IS&quot; term=&quot;%2209680160%22&quot;&gt;09680160 &lt;/searchLink&gt;&lt;i&gt;NLM ISO Abbreviation: &lt;/i&gt;Knee &lt;i&gt;Subsets: &lt;/i&gt;MEDLINE
– Name: PublisherInfo
  Label: Imprint Name(s)
  Group: PubInfo
  Data: &lt;i&gt;Publication&lt;/i&gt;: [s.l.] : Elsevier&lt;br /&gt;&lt;i&gt;Original Publication&lt;/i&gt;: Oxford, UK : Butterworth-Heinemann, c1994-
– Name: SubjectMESH
  Label: MeSH Terms
  Group: Su
  Data: &lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Osteoarthritis%2C+Knee%22&quot;&gt;Osteoarthritis, Knee*&lt;/searchLink&gt;/&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Osteoarthritis%2C+Knee+therapy%22&quot;&gt;therapy&lt;/searchLink&gt; &lt;br /&gt;&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Osteoarthritis%2C+Hip%22&quot;&gt;Osteoarthritis, Hip*&lt;/searchLink&gt;/&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Osteoarthritis%2C+Hip+therapy%22&quot;&gt;therapy&lt;/searchLink&gt; &lt;br /&gt;&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Large+Language+Models%22&quot;&gt;Large Language Models*&lt;/searchLink&gt; &lt;br /&gt;&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Practice+Guidelines+as+Topic%22&quot;&gt;Practice Guidelines as Topic*&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Humans%22&quot;&gt;Humans&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Reproducibility+of+Results%22&quot;&gt;Reproducibility of Results&lt;/searchLink&gt;
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Background: Clinical practice guidelines play a pivotal role in the management of complex and chronic conditions such as osteoarthritis (OA). However, the systematic evaluation of large language models (LLMs) against established guidelines-particularly in the context of knee and hip OA-remains limited. This study aimed to compare the responses generated by multiple LLMs to open-ended questions derived from the 2023 European League Against Rheumatism (EULAR) recommendations.&lt;br /&gt;Methods: This study transformed eight recommendations from the 2023 EULAR guideline on non-pharmacological management of knee and hip OA into clinically framed open-ended questions. Each question was posed to DeepSeek-R1, Gemini 2.5 Flash, and ChatGPT-5. The generated responses were evaluated in terms of informational quality, reliability, usability, readability, as well as semantic and lexical (surface-level) similarity to the reference recommendations.&lt;br /&gt;Results: DeepSeek-R1 achieved significantly higher scores in informational quality, reliability, usability, and readability compared with Gemini 2.5 Flash and ChatGPT-5 (p&#160;&amp;lt;&#160;0.05). Lexical overlap with the reference recommendations was significantly lower for DeepSeek-R1 than for the other models (p&#160;=&#160;0.001), indicating reduced verbatim similarity. No statistically significant differences were observed among the models with respect to semantic similarity (p&#160;=&#160;0.409).&lt;br /&gt;Conclusion: All three models produced generally acceptable outputs; however, DeepSeek-R1 consistently outperformed Gemini 2.5 and ChatGPT-5. Although these models show promising informational quality, reliability, and usability, limited readability restricts their accessibility. LLMs are not yet appropriate for independent patient use and should currently function as supportive tools under the supervision of qualified healthcare professionals.&lt;br /&gt; (Copyright &#169; 2026 Elsevier B.V. All rights reserved.)
– Name: Abstract
  Label: Competing Interests
  Group: Ab
  Data: Declaration of competing interest The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
– Name: SubjectMinor
  Label: Contributed Indexing
  Group:
  Data: &lt;i&gt;Keywords: &lt;/i&gt;Artificial intelligence; Guideline; Hip; Knee; Large language models; Osteoarthritis
– Name: DateEntry
  Label: Entry Date(s)
  Group: Date
  Data: &lt;i&gt;Date Created: &lt;/i&gt;20260612 &lt;i&gt;Date Completed: &lt;/i&gt;20260909 &lt;i&gt;Latest Revision: &lt;/i&gt;20260909
– Name: DateUpdate
  Label: Update Code
  Group: Date
  Data: 20260910
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1016/j.knee.2026.104524
– Name: AN
  Label: PMID
  Group: ID
  Data: 42284853
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=cmedm&AN=42284853
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1016/j.knee.2026.104524
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        StartPage: 104524
    Subjects:
      – SubjectFull: Humans
        Type: general
      – SubjectFull: Reproducibility of Results
        Type: general
      – SubjectFull: Osteoarthritis, Knee therapy
        Type: general
      – SubjectFull: Osteoarthritis, Hip therapy
        Type: general
      – SubjectFull: Large Language Models
        Type: general
      – SubjectFull: Practice Guidelines as Topic
        Type: general
    Titles:
      – TitleFull: Evaluation of the performance of large language models in the management of hip and knee osteoarthritis: analysis of EULAR 2023 recommendations.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Mirza Y
      – PersonEntity:
          Name:
            NameFull: Sari F
      – PersonEntity:
          Name:
            NameFull: Arslan S
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 10
              Text: 2026 Oct
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-electronic
              Value: 1873-5800
          Numbering:
            – Type: volume
              Value: 62
          Titles:
            – TitleFull: The Knee
              Type: main
ResultId 1