Academic Journal
Προβολή σε EDS

Large language models as sources of patient information on robotic knee arthroplasty: a comparative evaluation.

Λεπτομέρειες βιβλιογραφικής εγγραφής
Τίτλος: Large language models as sources of patient information on robotic knee arthroplasty: a comparative evaluation.
Συγγραφείς: Ciftci MU; Sultan 2. Abdülhamid Han Training and Research Hospital, Department of Orthopedics and Traumatology, 34668 Istanbul, Turkey. Electronic address: mehmetutkuciftci.md@gmail.com., Koluman AC; Bakirkoy Dr. Sadi Konuk Training and Research Hospital, Department of Orthopedics and Traumatology, 34147 Istanbul, Turkey. Electronic address: md.alicankoluman@gmail.com., Aloğlu Çiftçi E; Istanbul Okan University, Faculty of Health Sciences, Department of Physiotherapy and Rehabilitation, Istanbul, Turkey; Istinye University, Institute of Graduate Education, Department of Physiotherapy and Rehabilitation, Istanbul, Turkey. Electronic address: ebru.aloglu@okan.edu.tr., Cakmur BB; Beylikduzu State Hospital, Department of Orthopedics and Traumatology, 34500 Istanbul, Turkey. Electronic address: basar_cakmur@hotmail.com., Ziroğlu N; Department of Orthopedic Prosthetics and Orthotics, Vocational School of Health Services, Acibadem Mehmet Ali Aydinlar University, 34638 Istanbul, Turkey; Department of Orthopaedics and Traumatology, Acibadem University Atakent Hospital, 34303 Istanbul, Turkey. Electronic address: nezih.ziroglu@yahoo.com.
Πηγή: The Knee [Knee] 2026 Oct; Vol. 62, pp. 104579. Date of Electronic Publication: 2026 Jul 30.
Τύπος έκδοσης: Journal Article; Comparative Study
Γλώσσα: English
Στοιχεία περιοδικού: Publisher: Elsevier Country of Publication: Netherlands NLM ID: 9430798 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1873-5800 (Electronic) Linking ISSN: 09680160 NLM ISO Abbreviation: Knee Subsets: MEDLINE
Imprint Name(s): Publication: [s.l.] : Elsevier
Original Publication: Oxford, UK : Butterworth-Heinemann, c1994-
Ιατρικοί όροι (MeSH): Arthroplasty, Replacement, Knee*/methods , Patient Education as Topic*/methods , Large Language Models* , Robotic Surgical Procedures*, Humans ; Reproducibility of Results ; Comprehension ; Generative Artificial Intelligence ; Artificial Intelligence
Περίληψη: Background: Robotic-assisted total knee arthroplasty (rTKA) is increasingly used because of its surgical precision. However, inconsistent outcomes and high costs often lead patients to seek additional information from artificial intelligence (AI) tools. Large language models (LLMs) such as ChatGPT-4o, Gemini-2.5-Flash, and DeepSeek-V3 are commonly used, but their reliability and readability in orthopaedics remain unclear.
Objectives: To compare the reliability, usefulness, quality, and readability of responses to common patient questions about rTKA generated by leading LLMs.
Methods: Three LLMs answered 20 frequently asked patient questions (n = 20) identified through Google Trends and expert validation. Three orthopaedic specialists (n = 3) evaluated reliability, usefulness, and overall quality using validated scales, while readability was assessed with standard indices.
Results: Inter-rater reliability was good to excellent (ICC = 0.728-0.879). Gemini-2.5-Flash achieved significantly higher reliability and usefulness scores than ChatGPT-4o and DeepSeek-V3 (all p < 0.05). ChatGPT-4o and DeepSeek-V3 produced more readable but less accurate content, revealing an inverse relationship between reliability and readability.
Conclusions: Gemini-2.5-Flash provided the most reliable responses, highlighting the need for supervised integration of LLMs in patient education.
(Copyright © 2026 The Author(s). Published by Elsevier B.V. All rights reserved.)
Competing Interests: Declaration of competing interest The authors declare no competing interests.
Contributed Indexing: Keywords: Artificial intelligence; Large language models; Patient education; Readability; Reliability; Robotic-assisted total knee arthroplasty
Entry Date(s): Date Created: 20260730 Date Completed: 20260909 Latest Revision: 20260909
Update Code: 20260910
DOI: 10.1016/j.knee.2026.104579
PMID: 42531925
Βάση Δεδομένων: MEDLINE
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://www.doi.org/10.1016/j.knee.2026.104579?
    Name: ScienceDirect (all content) (s7799221)
    Category: fullText
    Text: View record from ScienceDirect
    MouseOverText: View record from ScienceDirect
  – Url: https://resolver.ebsco.com/c/fiv2js/result?sid=EBSCO:cmedm&genre=article&issn=18735800&ISBN=&volume=62&issue=&date=20261001&spage=104579&pages=104579&title=The Knee&atitle=Large%20language%20models%20as%20sources%20of%20patient%20information%20on%20robotic%20knee%20arthroplasty%3A%20a%20comparative%20evaluation.&aulast=Ciftci%20MU&id=DOI:10.1016/j.knee.2026.104579
    Name: Full Text Finder (for New FTF UI) (ns324271)
    Category: fullText
    Text: Full Text Finder
    MouseOverText: Full Text Finder
Header DbId: cmedm
DbLabel: MEDLINE
An: 42531925
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Large language models as sources of patient information on robotic knee arthroplasty: a comparative evaluation.
– Name: Author
  Label: Authors
  Group: Au
  Data: &lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Ciftci+MU%22&quot;&gt;Ciftci MU&lt;/searchLink&gt;; Sultan 2. Abd&#252;lhamid Han Training and Research Hospital, Department of Orthopedics and Traumatology, 34668 Istanbul, Turkey. Electronic address: mehmetutkuciftci.md@gmail.com.&lt;br /&gt;&lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Koluman+AC%22&quot;&gt;Koluman AC&lt;/searchLink&gt;; Bakirkoy Dr. Sadi Konuk Training and Research Hospital, Department of Orthopedics and Traumatology, 34147 Istanbul, Turkey. Electronic address: md.alicankoluman@gmail.com.&lt;br /&gt;&lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Aloğlu+&#199;ift&#231;i+E%22&quot;&gt;Aloğlu &#199;ift&#231;i E&lt;/searchLink&gt;; Istanbul Okan University, Faculty of Health Sciences, Department of Physiotherapy and Rehabilitation, Istanbul, Turkey; Istinye University, Institute of Graduate Education, Department of Physiotherapy and Rehabilitation, Istanbul, Turkey. Electronic address: ebru.aloglu@okan.edu.tr.&lt;br /&gt;&lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Cakmur+BB%22&quot;&gt;Cakmur BB&lt;/searchLink&gt;; Beylikduzu State Hospital, Department of Orthopedics and Traumatology, 34500 Istanbul, Turkey. Electronic address: basar&amp;#95;cakmur@hotmail.com.&lt;br /&gt;&lt;searchLink fieldCode=&quot;AU&quot; term=&quot;%22Ziroğlu+N%22&quot;&gt;Ziroğlu N&lt;/searchLink&gt;; Department of Orthopedic Prosthetics and Orthotics, Vocational School of Health Services, Acibadem Mehmet Ali Aydinlar University, 34638 Istanbul, Turkey; Department of Orthopaedics and Traumatology, Acibadem University Atakent Hospital, 34303 Istanbul, Turkey. Electronic address: nezih.ziroglu@yahoo.com.
– Name: TitleSource
  Label: Source
  Group: Src
  Data: &lt;searchLink fieldCode=&quot;JN&quot; term=&quot;%229430798%22&quot;&gt;The Knee&lt;/searchLink&gt; [Knee] 2026 Oct; Vol. 62, pp. 104579. &lt;i&gt;Date of Electronic Publication: &lt;/i&gt;2026 Jul 30.
– Name: TypePub
  Label: Publication Type
  Group: TypPub
  Data: Journal Article; Comparative Study
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: TitleSource
  Label: Journal Info
  Group: Src
  Data: &lt;i&gt;Publisher: &lt;/i&gt;&lt;searchLink fieldCode=&quot;PB&quot; term=&quot;%22Elsevier%22&quot;&gt;Elsevier &lt;/searchLink&gt;&lt;i&gt;Country of Publication: &lt;/i&gt;Netherlands &lt;i&gt;NLM ID: &lt;/i&gt;9430798 &lt;i&gt;Publication Model: &lt;/i&gt;Print-Electronic &lt;i&gt;Cited Medium: &lt;/i&gt;Internet &lt;i&gt;ISSN: &lt;/i&gt;1873-5800 (Electronic) &lt;i&gt;Linking ISSN: &lt;/i&gt;&lt;searchLink fieldCode=&quot;IS&quot; term=&quot;%2209680160%22&quot;&gt;09680160 &lt;/searchLink&gt;&lt;i&gt;NLM ISO Abbreviation: &lt;/i&gt;Knee &lt;i&gt;Subsets: &lt;/i&gt;MEDLINE
– Name: PublisherInfo
  Label: Imprint Name(s)
  Group: PubInfo
  Data: &lt;i&gt;Publication&lt;/i&gt;: [s.l.] : Elsevier&lt;br /&gt;&lt;i&gt;Original Publication&lt;/i&gt;: Oxford, UK : Butterworth-Heinemann, c1994-
– Name: SubjectMESH
  Label: MeSH Terms
  Group: Su
  Data: &lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Arthroplasty%2C+Replacement%2C+Knee%22&quot;&gt;Arthroplasty, Replacement, Knee*&lt;/searchLink&gt;/&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Arthroplasty%2C+Replacement%2C+Knee+methods%22&quot;&gt;methods&lt;/searchLink&gt; &lt;br /&gt;&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Patient+Education+as+Topic%22&quot;&gt;Patient Education as Topic*&lt;/searchLink&gt;/&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Patient+Education+as+Topic+methods%22&quot;&gt;methods&lt;/searchLink&gt; &lt;br /&gt;&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Large+Language+Models%22&quot;&gt;Large Language Models*&lt;/searchLink&gt; &lt;br /&gt;&lt;searchLink fieldCode=&quot;MM&quot; term=&quot;%22Robotic+Surgical+Procedures%22&quot;&gt;Robotic Surgical Procedures*&lt;/searchLink&gt;&lt;br /&gt;&lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Humans%22&quot;&gt;Humans&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Reproducibility+of+Results%22&quot;&gt;Reproducibility of Results&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Comprehension%22&quot;&gt;Comprehension&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Generative+Artificial+Intelligence%22&quot;&gt;Generative Artificial Intelligence&lt;/searchLink&gt; ; &lt;searchLink fieldCode=&quot;MH&quot; term=&quot;%22Artificial+Intelligence%22&quot;&gt;Artificial Intelligence&lt;/searchLink&gt;
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Background: Robotic-assisted total knee arthroplasty (rTKA) is increasingly used because of its surgical precision. However, inconsistent outcomes and high costs often lead patients to seek additional information from artificial intelligence (AI) tools. Large language models (LLMs) such as ChatGPT-4o, Gemini-2.5-Flash, and DeepSeek-V3 are commonly used, but their reliability and readability in orthopaedics remain unclear.&lt;br /&gt;Objectives: To compare the reliability, usefulness, quality, and readability of responses to common patient questions about rTKA generated by leading LLMs.&lt;br /&gt;Methods: Three LLMs answered 20 frequently asked patient questions (n&#160;=&#160;20) identified through Google Trends and expert validation. Three orthopaedic specialists (n&#160;=&#160;3) evaluated reliability, usefulness, and overall quality using validated scales, while readability was assessed with standard indices.&lt;br /&gt;Results: Inter-rater reliability was good to excellent (ICC&#160;=&#160;0.728-0.879). Gemini-2.5-Flash achieved significantly higher reliability and usefulness scores than ChatGPT-4o and DeepSeek-V3 (all p&#160;&amp;lt;&#160;0.05). ChatGPT-4o and DeepSeek-V3 produced more readable but less accurate content, revealing an inverse relationship between reliability and readability.&lt;br /&gt;Conclusions: Gemini-2.5-Flash provided the most reliable responses, highlighting the need for supervised integration of LLMs in patient education.&lt;br /&gt; (Copyright &#169; 2026 The Author(s). Published by Elsevier B.V. All rights reserved.)
– Name: Abstract
  Label: Competing Interests
  Group: Ab
  Data: Declaration of competing interest The authors declare no competing interests.
– Name: SubjectMinor
  Label: Contributed Indexing
  Group:
  Data: &lt;i&gt;Keywords: &lt;/i&gt;Artificial intelligence; Large language models; Patient education; Readability; Reliability; Robotic-assisted total knee arthroplasty
– Name: DateEntry
  Label: Entry Date(s)
  Group: Date
  Data: &lt;i&gt;Date Created: &lt;/i&gt;20260730 &lt;i&gt;Date Completed: &lt;/i&gt;20260909 &lt;i&gt;Latest Revision: &lt;/i&gt;20260909
– Name: DateUpdate
  Label: Update Code
  Group: Date
  Data: 20260910
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1016/j.knee.2026.104579
– Name: AN
  Label: PMID
  Group: ID
  Data: 42531925
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=cmedm&AN=42531925
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1016/j.knee.2026.104579
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        StartPage: 104579
    Subjects:
      – SubjectFull: Humans
        Type: general
      – SubjectFull: Reproducibility of Results
        Type: general
      – SubjectFull: Comprehension
        Type: general
      – SubjectFull: Generative Artificial Intelligence
        Type: general
      – SubjectFull: Artificial Intelligence
        Type: general
      – SubjectFull: Arthroplasty, Replacement, Knee methods
        Type: general
      – SubjectFull: Patient Education as Topic methods
        Type: general
      – SubjectFull: Large Language Models
        Type: general
      – SubjectFull: Robotic Surgical Procedures
        Type: general
    Titles:
      – TitleFull: Large language models as sources of patient information on robotic knee arthroplasty: a comparative evaluation.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Ciftci MU
      – PersonEntity:
          Name:
            NameFull: Koluman AC
      – PersonEntity:
          Name:
            NameFull: Aloğlu Çiftçi E
      – PersonEntity:
          Name:
            NameFull: Cakmur BB
      – PersonEntity:
          Name:
            NameFull: Ziroğlu N
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 10
              Text: 2026 Oct
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-electronic
              Value: 1873-5800
          Numbering:
            – Type: volume
              Value: 62
          Titles:
            – TitleFull: The Knee
              Type: main
ResultId 1