Academic Journal
Performance Comparison of a Neuro-Symbolic Large Language Model System Versus Human Experts in Acute Cholecystitis Management.
| Τίτλος: | Performance Comparison of a Neuro-Symbolic Large Language Model System Versus Human Experts in Acute Cholecystitis Management. |
|---|---|
| Συγγραφείς: | Ekingen, Evren, Ucdal, Mete |
| Πηγή: | Journal of Clinical Medicine; Mar2026, Vol. 15 Issue 5, p1730, 31p |
| Θεματικοί όροι: | Cholecystitis, Clinical decision support systems, Language models, Diagnosis, Medical protocols, Artificial intelligence |
| Περίληψη: | Background/Objectives: Large language models (LLMs) have shown promising results in medical decision support; however, their effectiveness in managing acute cholecystitis and other gallbladder diseases remains insufficiently examined. This study evaluated the performance of a neuro-symbolic LLM system that integrates multiple AI agents with neural–symbolic reasoning for acute cholecystitis management and compared its diagnostic accuracy with that of human expert physicians across three clinical specialties. Methods: This multi-center cross-sectional study included 30 case-based questions covering acute cholecystitis and gallbladder diseases, stratified across eight predefined disease categories: acute calculous cholecystitis (n = 6), acute acalculous cholecystitis (n = 2), complicated cholecystitis including gangrenous, emphysematous, and perforated variants (n = 5), chronic cholecystitis and biliary colic (n = 4), gallbladder polyps and adenomyomatosis (n = 3), Mirizzi syndrome (n = 2), gallbladder carcinoma (n = 4), and post-cholecystectomy complications (n = 4). Questions were categorized into diagnosis (n = 10), treatment (n = 10), and complications/prognosis (n = 10). Gold standard answers were established through consensus by an expert panel consisting of two senior general surgery expert clinicians and one senior emergency medicine expert clinician, each with more than 20 years of clinical experience, utilizing the Tokyo Guidelines 2018 (TG18) as the reference standard for diagnostic criteria, severity grading, and management recommendations. The expert panel achieved unanimous consensus on all 30 gold standard answers. All responses were cross-referenced against the primary TG18 publications to ensure guideline-based rather than solely opinion-based reference standards. This consensus-based, guideline-anchored approach is consistent with established methodologies for gold standard establishment in AI diagnostic accuracy studies. Performance of a neuro-symbolic LLM system orchestrated via LangGraph v1.0 was compared against 10 general surgery specialists, 10 emergency medicine physicians, and 10 gastroenterology specialists from four tertiary centers in Turkey. The neuro-symbolic system incorporated the Tokyo Guidelines 2018 (TG18) as its symbolic knowledge base for diagnostic criteria, severity grading, and management algorithms. Results: The neuro-symbolic system attained the highest overall accuracy rate of 96.7% (29/30), markedly surpassing the performance of general surgery specialists (average 82.3% ± 6.8%), emergency medicine physicians (average 71.0% ± 8.2%), and gastroenterology specialists (average 78.7% ± 7.4%). Furthermore, the neuro-symbolic system exhibited superior performance across all clinical categories. Among human participants, general surgeons showed the highest accuracy in treatment decisions (88.0%), while gastroenterologists excelled in diagnostic questions (82.0%). Emergency medicine physicians showed comparable performance to other specialties in acute presentation scenarios. ROC analysis revealed excellent discrimination for the neuro-symbolic system (AUC = 0.983) compared to general surgery (AUC = 0.856), gastroenterology (AUC = 0.821), and emergency medicine (AUC = 0.764). Conclusions: The neuro-symbolic LLM system exhibited superior performance in standardized guideline-concordant case-based assessment of acute cholecystitis management compared to all human expert groups, reflecting its consistent application of encoded guideline criteria. These findings support its potential role as a clinical decision-support tool that augments, rather than replaces, physician expertise. The system's consistent application of standardized guidelines indicates its potential utility as a clinical decision support tool, particularly in settings where specialist expertise is limited. However, these results should be interpreted within the constraints of a structured case-based evaluation and do not imply global clinical superiority over human experts. [ABSTRACT FROM AUTHOR] |
| Copyright of Journal of Clinical Medicine is the property of MDPI and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.) | |
| Βάση Δεδομένων: | Biomedical Index |
| FullText | Text: Availability: 0 CustomLinks: – Url: https://resolver.ebsco.com/c/fiv2js/result?sid=EBSCO:edm&genre=article&issn=20770383&ISBN=&volume=15&issue=5&date=20260301&spage=1730&pages=1730-1760&title=Journal of Clinical Medicine&atitle=Performance%20Comparison%20of%20a%20Neuro-Symbolic%20Large%20Language%20Model%20System%20Versus%20Human%20Experts%20in%20Acute%20Cholecystitis%20Management.&aulast=Ekingen%2C%20Evren&id=DOI:10.3390/jcm15051730 Name: Full Text Finder (for New FTF UI) (ns324271) Category: fullText Text: Full Text Finder MouseOverText: Full Text Finder |
|---|---|
| Header | DbId: edm DbLabel: Biomedical Index An: 192623669 RelevancyScore: 1061 AccessLevel: 6 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 1060.7568359375 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Performance Comparison of a Neuro-Symbolic Large Language Model System Versus Human Experts in Acute Cholecystitis Management. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AR" term="%22Ekingen%2C+Evren%22">Ekingen, Evren</searchLink><br /><searchLink fieldCode="AR" term="%22Ucdal%2C+Mete%22">Ucdal, Mete</searchLink> – Name: TitleSource Label: Source Group: Src Data: Journal of Clinical Medicine; Mar2026, Vol. 15 Issue 5, p1730, 31p – Name: Subject Label: Subject Terms Group: Su Data: <searchLink fieldCode="DE" term="%22Cholecystitis%22">Cholecystitis</searchLink><br /><searchLink fieldCode="DE" term="%22Clinical+decision+support+systems%22">Clinical decision support systems</searchLink><br /><searchLink fieldCode="DE" term="%22Language+models%22">Language models</searchLink><br /><searchLink fieldCode="DE" term="%22Diagnosis%22">Diagnosis</searchLink><br /><searchLink fieldCode="DE" term="%22Medical+protocols%22">Medical protocols</searchLink><br /><searchLink fieldCode="DE" term="%22Artificial+intelligence%22">Artificial intelligence</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: Background/Objectives: Large language models (LLMs) have shown promising results in medical decision support; however, their effectiveness in managing acute cholecystitis and other gallbladder diseases remains insufficiently examined. This study evaluated the performance of a neuro-symbolic LLM system that integrates multiple AI agents with neural–symbolic reasoning for acute cholecystitis management and compared its diagnostic accuracy with that of human expert physicians across three clinical specialties. Methods: This multi-center cross-sectional study included 30 case-based questions covering acute cholecystitis and gallbladder diseases, stratified across eight predefined disease categories: acute calculous cholecystitis (n = 6), acute acalculous cholecystitis (n = 2), complicated cholecystitis including gangrenous, emphysematous, and perforated variants (n = 5), chronic cholecystitis and biliary colic (n = 4), gallbladder polyps and adenomyomatosis (n = 3), Mirizzi syndrome (n = 2), gallbladder carcinoma (n = 4), and post-cholecystectomy complications (n = 4). Questions were categorized into diagnosis (n = 10), treatment (n = 10), and complications/prognosis (n = 10). Gold standard answers were established through consensus by an expert panel consisting of two senior general surgery expert clinicians and one senior emergency medicine expert clinician, each with more than 20 years of clinical experience, utilizing the Tokyo Guidelines 2018 (TG18) as the reference standard for diagnostic criteria, severity grading, and management recommendations. The expert panel achieved unanimous consensus on all 30 gold standard answers. All responses were cross-referenced against the primary TG18 publications to ensure guideline-based rather than solely opinion-based reference standards. This consensus-based, guideline-anchored approach is consistent with established methodologies for gold standard establishment in AI diagnostic accuracy studies. Performance of a neuro-symbolic LLM system orchestrated via LangGraph v1.0 was compared against 10 general surgery specialists, 10 emergency medicine physicians, and 10 gastroenterology specialists from four tertiary centers in Turkey. The neuro-symbolic system incorporated the Tokyo Guidelines 2018 (TG18) as its symbolic knowledge base for diagnostic criteria, severity grading, and management algorithms. Results: The neuro-symbolic system attained the highest overall accuracy rate of 96.7% (29/30), markedly surpassing the performance of general surgery specialists (average 82.3% ± 6.8%), emergency medicine physicians (average 71.0% ± 8.2%), and gastroenterology specialists (average 78.7% ± 7.4%). Furthermore, the neuro-symbolic system exhibited superior performance across all clinical categories. Among human participants, general surgeons showed the highest accuracy in treatment decisions (88.0%), while gastroenterologists excelled in diagnostic questions (82.0%). Emergency medicine physicians showed comparable performance to other specialties in acute presentation scenarios. ROC analysis revealed excellent discrimination for the neuro-symbolic system (AUC = 0.983) compared to general surgery (AUC = 0.856), gastroenterology (AUC = 0.821), and emergency medicine (AUC = 0.764). Conclusions: The neuro-symbolic LLM system exhibited superior performance in standardized guideline-concordant case-based assessment of acute cholecystitis management compared to all human expert groups, reflecting its consistent application of encoded guideline criteria. These findings support its potential role as a clinical decision-support tool that augments, rather than replaces, physician expertise. The system's consistent application of standardized guidelines indicates its potential utility as a clinical decision support tool, particularly in settings where specialist expertise is limited. However, these results should be interpreted within the constraints of a structured case-based evaluation and do not imply global clinical superiority over human experts. [ABSTRACT FROM AUTHOR] – Name: Abstract Label: Group: Ab Data: <i>Copyright of Journal of Clinical Medicine is the property of MDPI and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.) |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edm&AN=192623669 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.3390/jcm15051730 Languages: – Code: eng Text: English PhysicalDescription: Pagination: PageCount: 31 StartPage: 1730 Subjects: – SubjectFull: Cholecystitis Type: general – SubjectFull: Clinical decision support systems Type: general – SubjectFull: Language models Type: general – SubjectFull: Diagnosis Type: general – SubjectFull: Medical protocols Type: general – SubjectFull: Artificial intelligence Type: general Titles: – TitleFull: Performance Comparison of a Neuro-Symbolic Large Language Model System Versus Human Experts in Acute Cholecystitis Management. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Ekingen, Evren – PersonEntity: Name: NameFull: Ucdal, Mete IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 03 Text: Mar2026 Type: published Y: 2026 Identifiers: – Type: issn-print Value: 20770383 Numbering: – Type: volume Value: 15 – Type: issue Value: 5 Titles: – TitleFull: Journal of Clinical Medicine Type: main |
| ResultId | 1 |