Academic Journal
Large language models' performance in answering common patient questions about colonoscopy: an expert-based evaluation.
| Τίτλος: | Large language models' performance in answering common patient questions about colonoscopy: an expert-based evaluation. |
|---|---|
| Συγγραφείς: | Güneş Aydemi̇r N; Department of Gastroenterology, Elazığ Fethi Sekin City Hospital, Elazığ, Türkiye., Yapar D; Institute of Health Science, Medical Informatics, Akdeniz University, Antalya, Türkiye.; Turkish Ministry of Health, Muratpasa District Health Directorate, Antalya, Türkiye., Demi̇r Avcı Y; Department of Public Health Nursing, Faculty of Nursing, Akdeniz University, Antalya, Türkiye. ydemir@akdeniz.edu.tr., Sonuvar ET; Institute for Medical Informatics, Statistics and Epidemiology, Department of Medical Data Science, Leipzig University, Leipzig, Germany., Aydemir AM; Department of Emergency medicine, Elazığ Fethi Sekin City Hospital, Elazığ, Türkiye. |
| Πηγή: | BMC gastroenterology [BMC Gastroenterol] 2026 Oct 01; Vol. 26 (1). Date of Electronic Publication: 2026 Oct 01. |
| Τύπος έκδοσης: | Journal Article; Observational Study; Comparative Study |
| Γλώσσα: | English |
| Στοιχεία περιοδικού: | Publisher: BioMed Central Country of Publication: England NLM ID: 100968547 Publication Model: Electronic Cited Medium: Internet ISSN: 1471-230X (Electronic) Linking ISSN: 1471230X NLM ISO Abbreviation: BMC Gastroenterol Subsets: MEDLINE |
| Imprint Name(s): | Original Publication: London : BioMed Central, [2001- |
| Ιατρικοί όροι (MeSH): | Large Language Models* , Colonoscopy*, Humans ; Adult ; Female ; Communication ; Turkey ; Middle Aged |
| Περίληψη: | Background: This study aimed to compare the performance of two large language models (LLMs) the ChatGPT-4 and the Google Bard in delivering medical information about colonoscopy. It also sought to evaluate the models' responses to common patient and caregiver questions in terms of accuracy, applicability, comprehensiveness, and communication quality, as assessed by experts using a rubric-based method. Methods: This comparative, expert-based observational study analyzed the responses of both models to 11 colonoscopy-related questions written in Turkish, simulating real patient inquiries. To ensure consistency and fairness, questions were asked in separate sessions using a standardized zero-shot approach without prior examples or context. The gastroenterology experts from various institutions in Türkiye were recruited using snowball sampling and evaluated the responses using a rubric-based scale across four domains. Data were collected online. Results: A total of 40 gastroenterologists participated (mean age 39.1 ± 6.4 years). For ChatGPT-4, more than 70% of responses were rated ≥ 4 across all domains, with median overall index scores ranging from 4.1 to 4.7. In contrast, Google Bard demonstrated lower performance, with median scores ranging from 3.6 to 4.0 and lower proportions of high ratings (≥ 4), particularly in comprehensiveness (55.7%) and applicability (58.4%). Comparative analysis showed that ChatGPT-4 significantly outperformed Google Bard across all domains: accuracy (p = 0.003, effect size = 0.47), applicability (p < 0.001, effect size = 0.64), comprehensiveness (p = 0.004, effect size = 0.46), and communication (p = 0.001, effect size = 0.52). No significant differences were observed across evaluator subgroups based on academic title, prior ChatGPT use, or age (all p-values > 0.05). Conclusions: ChatGPT-4 outperformed Google Bard across all domains, providing more accurate, comprehensive, and patient-centered responses. Despite these findings, human oversight remains essential to ensure safety and reliability in clinical communication. It should also be noted that different versions of LLMs may yield different results. (© 2026. The Author(s).) |
| Competing Interests: | Declarations. Ethics approval and consent to participate: In order to conduct the research, approval was obtained from the Akdeniz University Clinical Research Ethics Committee (Approval number: KAEK-682, Date:23.08.2023). And informed consent to participate was obtained from all of the participants in the study. Patient consent statement: Institutional permission was also received while informed and verbal consent was collected from the experts. Consent for publication: Not applicable. Competing interests: The authors declare no competing interests. |
| References: | Acta Orthop Traumatol Turc. 2025 Jul 18;59(4):222-229. (PMID: 40728044) Ther Adv Gastroenterol. 2025 Apr 1;18:17562848251328577. (PMID: 40171241) Aesthetic Plast Surg. 2026 Jun;50(12):4769-4780. (PMID: 41986550) Gastro Hep Adv. 2023 Oct 24;2(8):1138-1139. (PMID: 39131555) Nat Med. 2024 Sep;30(9):2613-2622. (PMID: 38965432) Commun Med (Lond). 2025 Jan 21;5(1):26. (PMID: 39838160) Diagn Interv Imaging. 2024 Jul-Aug;105(7-8):251-265. (PMID: 38679540) Bioengineering (Basel). 2025 Jun 10;12(6):. (PMID: 40564447) Gastro Hep Adv. 2023;2(8):1040-1043. (PMID: 37974564) J Med Internet Res. 2024 Dec 20;26:e66648. (PMID: 39705703) BMC Med Inform Decis Mak. 2024 Mar 12;24(1):72. (PMID: 38475802) J Med Internet Res. 2024 Jun 27;26:e54571. (PMID: 38935937) J Can Assoc Gastroenterol. 2022 Apr 28;5(4):161-168. (PMID: 35919762) J Hosp Med. 2011 Nov;6(9):519-25. (PMID: 22034196) Can Assoc Radiol J. 2024 May;75(2):344-350. (PMID: 37578849) NPJ Digit Med. 2024 Sep 28;7(1):258. (PMID: 39333376) NPJ Digit Med. 2025 Feb 5;8(1):85. (PMID: 39910315) Gastroenterology. 2023 Aug;165(2):509-511.e7. (PMID: 37150470) Int J Med Inform. 2025 Mar;195:105780. (PMID: 39753062) Jt Dis Relat Surg. 2024 Jan 1;35(1):169-176. (PMID: 38108178) JMIR Ment Health. 2025 May 15;12:e70014. (PMID: 40373033) Ther Adv Gastroenterol. 2024 Feb 22;17:17562848241227031. (PMID: 38390029) J Gastroenterol Hepatol. 2024 Dec;39(12):2487-2503. (PMID: 39165168) JMIR Diabetes. 2017 Jun 29;2(1):e13. (PMID: 30291084) J Clin Med. 2025 Jun 19;14(12):. (PMID: 40566123) Am J Audiol. 2026 Jun 2;35(2):668-679. (PMID: 41989339) |
| Contributed Indexing: | Keywords: ChatGPT; Colonoscopy; Expert evaluation; Gastroenterology; Gemini; Google Bard; Large language models; Rubric |
| Entry Date(s): | Date Created: 20261002 Date Completed: 20261002 Latest Revision: 20261008 |
| Update Code: | 20261008 |
| PubMed Central ID: | PMC13632202 |
| DOI: | 10.1186/s12876-026-05345-9 |
| PMID: | 42827234 |
| Βάση Δεδομένων: | MEDLINE |
καταχωρήστε σχόλιο πρώτοι!