Academic Journal
Can general purpose large language models assist pediatricians in predicting infants with serious bacterial infection?
| Τίτλος: | Can general purpose large language models assist pediatricians in predicting infants with serious bacterial infection? |
|---|---|
| Συγγραφείς: | Šimunović, I, Rezić, K, Franić, N, Boduljak, G, Batinić, M, Jukić, I, Jelovina, I, Biočić, J, Pogorelić, Z, Markić, J |
| Στοιχεία εκδότη: | BioMed Central |
| Έτος έκδοσης: | 2025 |
| Συλλογή: | Oxford University Research Archive (ORA) |
| Περιγραφή: | Background: Serious Bacterial Infection (SBI) in neonates and young infants often exhibit nonspecific symptoms and clinical signs in the early stages of illness, making early diagnosis challenging. Timely recognition and appropriate treatment are essential to prevent adverse outcomes. While several clinical algorithms are widely used for SBI risk stratification, these tools have limitations, particularly low positive predictive value. This study evaluates the diagnostic accuracy of general-purpose large language models (LLMs) in detecting SBI in neonates and infants under 90 days of age admitted to the emergency department. Our objective is to improve diagnostic precision, reduce unnecessary interventions, and enhance patient outcomes. LLM performance was compared against traditional machine learning models, state-of-the-art rule-based methods, and an ensemble of physicians to assess their potential as clinical decision-support tools in scenarios of diagnostic uncertainty. Results: On a dataset of 742 patients, LLMs demonstrated diagnostic accuracy comparable to traditional machine learning models and state-of-the-art rule-based methods. The optimized CatBoost (class-weighted) model achieved the best overall performance, with a PPV of 0.70, NPV of 0.90, sensitivity of 0.54, specificity of 0.95, F1-score of 0.60, and MCC of 0.54, outperforming the baseline CatBoost model and achieving results on par with large language models (LLMs) and physicians. When optimally prompted, LLMs performed on par with ensembles of experienced clinicians. Additionally, LLMs exhibited effective medical reasoning and provided credible diagnostic predictions, particularly valuable in cases of clinician uncertainty. The models achieved balanced performance across multiple evaluation metrics, including PPV, NPV, sensitivity, specificity, F1-score, and Matthew’s correlation coefficient (MCC). ChatGPT-4o achieved a sensitivity of 0.65 and specificity of 0.83, with an MCC of 0.41. Claude Sonnet 3.5 reached a sensitivity of 0.60 and ... |
| Τύπος εγγράφου: | article in journal/newspaper |
| Γλώσσα: | English |
| DOI: | 10.1186/s12911-025-03258-3 |
| Διαθεσιμότητα: | https://doi.org/10.1186/s12911-025-03258-3 https://ora.ox.ac.uk/objects/uuid:fc425c13-fb5e-4008-ab69-157c70ff56cb |
| Rights: | info:eu-repo/semantics/openAccess ; CC Attribution-NonCommercial-NoDerivatives (CC BY-NC-ND) |
| Αριθμός Καταχώρησης: | edsbas.A35FE8F1 |
| Βάση Δεδομένων: | BASE |
καταχωρήστε σχόλιο πρώτοι!