Academic Journal
Προβολή σε EDS
Συνδεδεμένο Πλήρες Κείμενο

Assessing the performance of large language models when used to determine ASA status of cats and dogs and generate anaesthetic protocols.

Λεπτομέρειες βιβλιογραφικής εγγραφής
Τίτλος: Assessing the performance of large language models when used to determine ASA status of cats and dogs and generate anaesthetic protocols.
Συγγραφείς: Okur S; Department of Surgery, Faculty of Veterinary Medicine, Atatürk University, Erzurum, Turkey., Akçora Y; Department of Surgery, Faculty of Veterinary Medicine, Atatürk University, Erzurum, Turkey., Okur DT; Department of Obstetrics and Gynecology, Faculty of Veterinary Medicine, Atatürk University, Erzurum, Turkey., Şenocak MG; Department of Surgery, Faculty of Veterinary Medicine, Atatürk University, Erzurum, Turkey., Bedir AG; Department of Surgery, Faculty of Veterinary Medicine, Atatürk University, Erzurum, Turkey., Ersöz U; Department of Surgery, Faculty of Veterinary Medicine, Atatürk University, Erzurum, Turkey., Kartal T; Department of Surgery, Faculty of Veterinary Medicine, Atatürk University, Erzurum, Turkey., Kurt BK; Department of Surgery, Faculty of Veterinary Medicine, Aydın Adnan Menderes University, Aydın, Turkey.
Πηγή: The Veterinary record [Vet Rec] 2026 Oct 03; Vol. 199 (7), pp. e304-e311. Date of Electronic Publication: 2026 May 12.
Τύπος έκδοσης: Journal Article
Γλώσσα: English
Στοιχεία περιοδικού: Publisher: Wiley Country of Publication: England NLM ID: 0031164 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 2042-7670 (Electronic) Linking ISSN: 00424900 NLM ISO Abbreviation: Vet Rec Subsets: MEDLINE
Imprint Name(s): Publication: 2021- : [Oxford] : Wiley
Original Publication: -June 2009 : London : British Veterinary Medicine
Ιατρικοί όροι (MeSH): Anesthesia*/veterinary , Large Language Models*, Animals ; Dogs ; Cats ; Retrospective Studies ; Reproducibility of Results ; Clinical Protocols
Περίληψη: Background: Large language models (LLMs) are emerging as decision-support tools in human medicine; however, their evaluation in veterinary anaesthesiology remains limited.
Methods: We retrospectively analysed 225 anonymised feline and canine cases (American Society of Anaesthesiologists [ASA] classifications 1‒5) from Atatürk University Veterinary Hospital. ChatGPT-4o, ChatGPT-5 and Gemini 2.5 Pro independently assigned ASA classifications and generated anaesthetic protocols using standardised prompts. Protocol adequacy was evaluated for all cases, regardless of ASA classification agreement, by two experienced veterinary anaesthesiologists using a four-point scale. Statistical analyses included Friedman and Bonferroni-adjusted Wilcoxon tests, effect sizes and inter-panelist reliability (assessed by quadratic-weighted Cohen's kappa and intraclass correlation coefficient).
Results: ChatGPT-5 achieved the highest ASA classification accuracy (53.3%), followed by ChatGPT-4o (46.7%) and Gemini 2.5 Pro (30.7%). The performance was strongest for ASA 3‒5, whereas ASA 1 cases were frequently misclassified, mainly due to ASA overestimation. ChatGPT-5 generated the most clinically sufficient anaesthetic protocols, outperforming the other models.
Limitations: The retrospective, single-centre design and inclusion of only feline and canine cases may limit generalisability.
Conclusions: LLMs can generate clinically relevant ASA classifications and anaesthetic protocols in veterinary anaesthesiology, although performance varies across models. However, expert oversight remains essential.
(© 2026 The Author(s). Veterinary Record published by John Wiley & Sons Ltd on behalf of British Veterinary Association.)
References: Nat Rev Cancer. 2018 Aug;18(8):500-510. (PMID: 29777175)
J Clin Anesth. 2024 Sep;96:111475. (PMID: 38657530)
BMC Med Inform Decis Mak. 2020 Nov 30;20(1):310. (PMID: 33256715)
Vet Med Sci. 2025 May;11(3):e70315. (PMID: 40173266)
Vet Anaesth Analg. 2013 May;40(3):229-36. (PMID: 23402568)
Front Vet Sci. 2024 Jun 07;11:1395934. (PMID: 38911678)
Anesth Analg. 2024 Aug 1;139(2):349-356. (PMID: 38640076)
Domest Anim Endocrinol. 2020 Jul;72:106396. (PMID: 32006871)
Am J Emerg Med. 2025 Mar;89:27-29. (PMID: 39675178)
Anesthesiology. 2020 Feb;132(2):379-394. (PMID: 31939856)
Vet Med Sci. 2025 May;11(3):e70258. (PMID: 40145983)
Rev Cardiovasc Med. 2021 Dec 22;22(4):1095-1113. (PMID: 34957756)
BMC Med Inform Decis Mak. 2025 Mar 07;25(1):117. (PMID: 40055694)
J Am Vet Med Assoc. 2022 Mar 30;260(8):819-824. (PMID: 35358065)
J Equine Vet Sci. 2020 Jul;90:102973. (PMID: 32534764)
J Orthop Surg Res. 2024 Sep 18;19(1):574. (PMID: 39289734)
Histopathology. 2019 Feb;74(3):372-376. (PMID: 30270453)
JMIR Med Inform. 2024 May 10;12:e53787. (PMID: 38728687)
Biometrics. 1977 Mar;33(1):159-74. (PMID: 843571)
Nat Med. 2023 Aug;29(8):1930-1940. (PMID: 37460753)
Curr Anesthesiol Rep. 2023 Jun;13(2):31-40. (PMID: 38106626)
Contributed Indexing: Keywords: ASA classification; decision support; large language models; veterinary anaesthesiology
Entry Date(s): Date Created: 20260512 Date Completed: 20261002 Latest Revision: 20261004
Update Code: 20261004
PubMed Central ID: PMC13631851
DOI: 10.1002/vetr.70741
PMID: 42117364
Βάση Δεδομένων: MEDLINE
FullText Links:
  – Type: other
    Url: https://resolver.ebsco.com:443/public/rma-ftfapi/ejs/direct?AccessToken=4242A85F9F085CEC6979&Show=Object
Text:
  Availability: 0
Header DbId: cmedm
DbLabel: MEDLINE
An: 42117364
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Assessing the performance of large language models when used to determine ASA status of cats and dogs and generate anaesthetic protocols.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AU" term="%22Okur+S%22">Okur S</searchLink>; Department of Surgery, Faculty of Veterinary Medicine, Atatürk University, Erzurum, Turkey.<br /><searchLink fieldCode="AU" term="%22Akçora+Y%22">Akçora Y</searchLink>; Department of Surgery, Faculty of Veterinary Medicine, Atatürk University, Erzurum, Turkey.<br /><searchLink fieldCode="AU" term="%22Okur+DT%22">Okur DT</searchLink>; Department of Obstetrics and Gynecology, Faculty of Veterinary Medicine, Atatürk University, Erzurum, Turkey.<br /><searchLink fieldCode="AU" term="%22Şenocak+MG%22">Şenocak MG</searchLink>; Department of Surgery, Faculty of Veterinary Medicine, Atatürk University, Erzurum, Turkey.<br /><searchLink fieldCode="AU" term="%22Bedir+AG%22">Bedir AG</searchLink>; Department of Surgery, Faculty of Veterinary Medicine, Atatürk University, Erzurum, Turkey.<br /><searchLink fieldCode="AU" term="%22Ersöz+U%22">Ersöz U</searchLink>; Department of Surgery, Faculty of Veterinary Medicine, Atatürk University, Erzurum, Turkey.<br /><searchLink fieldCode="AU" term="%22Kartal+T%22">Kartal T</searchLink>; Department of Surgery, Faculty of Veterinary Medicine, Atatürk University, Erzurum, Turkey.<br /><searchLink fieldCode="AU" term="%22Kurt+BK%22">Kurt BK</searchLink>; Department of Surgery, Faculty of Veterinary Medicine, Aydın Adnan Menderes University, Aydın, Turkey.
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%220031164%22">The Veterinary record</searchLink> [Vet Rec] 2026 Oct 03; Vol. 199 (7), pp. e304-e311. <i>Date of Electronic Publication: </i>2026 May 12.
– Name: TypePub
  Label: Publication Type
  Group: TypPub
  Data: Journal Article
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: TitleSource
  Label: Journal Info
  Group: Src
  Data: <i>Publisher: </i><searchLink fieldCode="PB" term="%22Wiley%22">Wiley </searchLink><i>Country of Publication: </i>England <i>NLM ID: </i>0031164 <i>Publication Model: </i>Print-Electronic <i>Cited Medium: </i>Internet <i>ISSN: </i>2042-7670 (Electronic) <i>Linking ISSN: </i><searchLink fieldCode="IS" term="%2200424900%22">00424900 </searchLink><i>NLM ISO Abbreviation: </i>Vet Rec <i>Subsets: </i>MEDLINE
– Name: PublisherInfo
  Label: Imprint Name(s)
  Group: PubInfo
  Data: <i>Publication</i>: 2021- : [Oxford] : Wiley<br /><i>Original Publication</i>: -June 2009 : London : British Veterinary Medicine
– Name: SubjectMESH
  Label: MeSH Terms
  Group: Su
  Data: <searchLink fieldCode="MM" term="%22Anesthesia%22">Anesthesia*</searchLink>/<searchLink fieldCode="MM" term="%22Anesthesia+veterinary%22">veterinary</searchLink> <br /><searchLink fieldCode="MM" term="%22Large+Language+Models%22">Large Language Models*</searchLink><br /><searchLink fieldCode="MH" term="%22Animals%22">Animals</searchLink> ; <searchLink fieldCode="MH" term="%22Dogs%22">Dogs</searchLink> ; <searchLink fieldCode="MH" term="%22Cats%22">Cats</searchLink> ; <searchLink fieldCode="MH" term="%22Retrospective+Studies%22">Retrospective Studies</searchLink> ; <searchLink fieldCode="MH" term="%22Reproducibility+of+Results%22">Reproducibility of Results</searchLink> ; <searchLink fieldCode="MH" term="%22Clinical+Protocols%22">Clinical Protocols</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Background: Large language models (LLMs) are emerging as decision-support tools in human medicine; however, their evaluation in veterinary anaesthesiology remains limited.<br />Methods: We retrospectively analysed 225 anonymised feline and canine cases (American Society of Anaesthesiologists [ASA] classifications 1‒5) from Atatürk University Veterinary Hospital. ChatGPT-4o, ChatGPT-5 and Gemini 2.5 Pro independently assigned ASA classifications and generated anaesthetic protocols using standardised prompts. Protocol adequacy was evaluated for all cases, regardless of ASA classification agreement, by two experienced veterinary anaesthesiologists using a four-point scale. Statistical analyses included Friedman and Bonferroni-adjusted Wilcoxon tests, effect sizes and inter-panelist reliability (assessed by quadratic-weighted Cohen's kappa and intraclass correlation coefficient).<br />Results: ChatGPT-5 achieved the highest ASA classification accuracy (53.3%), followed by ChatGPT-4o (46.7%) and Gemini 2.5 Pro (30.7%). The performance was strongest for ASA 3‒5, whereas ASA 1 cases were frequently misclassified, mainly due to ASA overestimation. ChatGPT-5 generated the most clinically sufficient anaesthetic protocols, outperforming the other models.<br />Limitations: The retrospective, single-centre design and inclusion of only feline and canine cases may limit generalisability.<br />Conclusions: LLMs can generate clinically relevant ASA classifications and anaesthetic protocols in veterinary anaesthesiology, although performance varies across models. However, expert oversight remains essential.<br /> (© 2026 The Author(s). Veterinary Record published by John Wiley & Sons Ltd on behalf of British Veterinary Association.)
– Name: Ref
  Label: References
  Group: RefInfo
  Data: Nat Rev Cancer. 2018 Aug;18(8):500-510. (PMID: <searchLink fieldCode="PM" term="%2229777175%22">29777175)</searchLink><br />J Clin Anesth. 2024 Sep;96:111475. (PMID: <searchLink fieldCode="PM" term="%2238657530%22">38657530)</searchLink><br />BMC Med Inform Decis Mak. 2020 Nov 30;20(1):310. (PMID: <searchLink fieldCode="PM" term="%2233256715%22">33256715)</searchLink><br />Vet Med Sci. 2025 May;11(3):e70315. (PMID: <searchLink fieldCode="PM" term="%2240173266%22">40173266)</searchLink><br />Vet Anaesth Analg. 2013 May;40(3):229-36. (PMID: <searchLink fieldCode="PM" term="%2223402568%22">23402568)</searchLink><br />Front Vet Sci. 2024 Jun 07;11:1395934. (PMID: <searchLink fieldCode="PM" term="%2238911678%22">38911678)</searchLink><br />Anesth Analg. 2024 Aug 1;139(2):349-356. (PMID: <searchLink fieldCode="PM" term="%2238640076%22">38640076)</searchLink><br />Domest Anim Endocrinol. 2020 Jul;72:106396. (PMID: <searchLink fieldCode="PM" term="%2232006871%22">32006871)</searchLink><br />Am J Emerg Med. 2025 Mar;89:27-29. (PMID: <searchLink fieldCode="PM" term="%2239675178%22">39675178)</searchLink><br />Anesthesiology. 2020 Feb;132(2):379-394. (PMID: <searchLink fieldCode="PM" term="%2231939856%22">31939856)</searchLink><br />Vet Med Sci. 2025 May;11(3):e70258. (PMID: <searchLink fieldCode="PM" term="%2240145983%22">40145983)</searchLink><br />Rev Cardiovasc Med. 2021 Dec 22;22(4):1095-1113. (PMID: <searchLink fieldCode="PM" term="%2234957756%22">34957756)</searchLink><br />BMC Med Inform Decis Mak. 2025 Mar 07;25(1):117. (PMID: <searchLink fieldCode="PM" term="%2240055694%22">40055694)</searchLink><br />J Am Vet Med Assoc. 2022 Mar 30;260(8):819-824. (PMID: <searchLink fieldCode="PM" term="%2235358065%22">35358065)</searchLink><br />J Equine Vet Sci. 2020 Jul;90:102973. (PMID: <searchLink fieldCode="PM" term="%2232534764%22">32534764)</searchLink><br />J Orthop Surg Res. 2024 Sep 18;19(1):574. (PMID: <searchLink fieldCode="PM" term="%2239289734%22">39289734)</searchLink><br />Histopathology. 2019 Feb;74(3):372-376. (PMID: <searchLink fieldCode="PM" term="%2230270453%22">30270453)</searchLink><br />JMIR Med Inform. 2024 May 10;12:e53787. (PMID: <searchLink fieldCode="PM" term="%2238728687%22">38728687)</searchLink><br />Biometrics. 1977 Mar;33(1):159-74. (PMID: <searchLink fieldCode="PM" term="%22843571%22">843571)</searchLink><br />Nat Med. 2023 Aug;29(8):1930-1940. (PMID: <searchLink fieldCode="PM" term="%2237460753%22">37460753)</searchLink><br />Curr Anesthesiol Rep. 2023 Jun;13(2):31-40. (PMID: <searchLink fieldCode="PM" term="%2238106626%22">38106626)</searchLink>
– Name: SubjectMinor
  Label: Contributed Indexing
  Group:
  Data: <i>Keywords: </i>ASA classification; decision support; large language models; veterinary anaesthesiology
– Name: DateEntry
  Label: Entry Date(s)
  Group: Date
  Data: <i>Date Created: </i>20260512 <i>Date Completed: </i>20261002 <i>Latest Revision: </i>20261004
– Name: DateUpdate
  Label: Update Code
  Group: Date
  Data: 20261004
– Name: PubmedCentralID
  Label: PubMed Central ID
  Group: ID
  Data: PMC13631851
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1002/vetr.70741
– Name: AN
  Label: PMID
  Group: ID
  Data: 42117364
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=cmedm&AN=42117364
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1002/vetr.70741
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        StartPage: e304
    Subjects:
      – SubjectFull: Animals
        Type: general
      – SubjectFull: Dogs
        Type: general
      – SubjectFull: Cats
        Type: general
      – SubjectFull: Retrospective Studies
        Type: general
      – SubjectFull: Reproducibility of Results
        Type: general
      – SubjectFull: Clinical Protocols
        Type: general
      – SubjectFull: Anesthesia veterinary
        Type: general
      – SubjectFull: Large Language Models
        Type: general
    Titles:
      – TitleFull: Assessing the performance of large language models when used to determine ASA status of cats and dogs and generate anaesthetic protocols.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Okur S
      – PersonEntity:
          Name:
            NameFull: Akçora Y
      – PersonEntity:
          Name:
            NameFull: Okur DT
      – PersonEntity:
          Name:
            NameFull: Şenocak MG
      – PersonEntity:
          Name:
            NameFull: Bedir AG
      – PersonEntity:
          Name:
            NameFull: Ersöz U
      – PersonEntity:
          Name:
            NameFull: Kartal T
      – PersonEntity:
          Name:
            NameFull: Kurt BK
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 03
              M: 10
              Text: 2026 Oct 03
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-electronic
              Value: 2042-7670
          Numbering:
            – Type: volume
              Value: 199
            – Type: issue
              Value: 7
          Titles:
            – TitleFull: The Veterinary record
              Type: main
ResultId 1