Academic Journal
Extraction of Pain Severity and Functional Interference From Clinical Narratives Using Domain-Informed Large Language Models: Protocol for a Development and Validation Study.
| Τίτλος: | Extraction of Pain Severity and Functional Interference From Clinical Narratives Using Domain-Informed Large Language Models: Protocol for a Development and Validation Study. |
|---|---|
| Συγγραφείς: | Zhang X; Department of Biomedical Informatics and Medical Education, University of Washington, Seattle, WA, United States., Wilson CR; Seattle-Denver Center of Innovation for Veteran-Centered and Value-Driven Care, VA Puget Sound Health Care System, 1660 S. Columbian Way, Seattle, WA, 98108, United States, 1 2062774175., Eyre H; Seattle-Denver Center of Innovation for Veteran-Centered and Value-Driven Care, VA Puget Sound Health Care System, 1660 S. Columbian Way, Seattle, WA, 98108, United States, 1 2062774175., Reed Ii DE; Seattle-Denver Center of Innovation for Veteran-Centered and Value-Driven Care, VA Puget Sound Health Care System, 1660 S. Columbian Way, Seattle, WA, 98108, United States, 1 2062774175.; Department of Psychiatry and Behavioral Sciences, University of Washington, Seattle, WA, United States., Hee Wai TY; Seattle-Denver Center of Innovation for Veteran-Centered and Value-Driven Care, VA Puget Sound Health Care System, 1660 S. Columbian Way, Seattle, WA, 98108, United States, 1 2062774175., Kloehn A; Seattle-Denver Center of Innovation for Veteran-Centered and Value-Driven Care, VA Puget Sound Health Care System, 1660 S. Columbian Way, Seattle, WA, 98108, United States, 1 2062774175., Rosser EW; Seattle-Denver Center of Innovation for Veteran-Centered and Value-Driven Care, VA Puget Sound Health Care System, 1660 S. Columbian Way, Seattle, WA, 98108, United States, 1 2062774175., Luo G; Department of Biomedical Informatics and Medical Education, University of Washington, Seattle, WA, United States., Zeliadt SB; Seattle-Denver Center of Innovation for Veteran-Centered and Value-Driven Care, VA Puget Sound Health Care System, 1660 S. Columbian Way, Seattle, WA, 98108, United States, 1 2062774175.; Department of Health Systems and Population Health, School of Public Health, University of Washington, Seattle, WA, United States. |
| Πηγή: | JMIR research protocols [JMIR Res Protoc] 2026 Oct 07; Vol. 15, pp. e96346. Date of Electronic Publication: 2026 Oct 07. |
| Τύπος έκδοσης: | Journal Article; Validation Study |
| Γλώσσα: | English |
| Στοιχεία περιοδικού: | Publisher: JMIR Publications Country of Publication: Canada NLM ID: 101599504 Publication Model: Electronic Cited Medium: Internet ISSN: 1929-0748 (Electronic) Linking ISSN: 19290748 NLM ISO Abbreviation: JMIR Res Protoc Subsets: MEDLINE |
| Imprint Name(s): | Original Publication: Toronto : JMIR Publications |
| Ιατρικοί όροι (MeSH): | Pain Measurement*/methods , Chronic Pain*/diagnosis , Natural Language Processing*, Veterans/statistics & numerical data ; United States Department of Veterans Affairs/organization & administration ; Humans ; Large Language Models ; Retrospective Studies ; Patient Reported Outcome Measures ; Electronic Health Records ; United States ; Narration |
| Περίληψη: | Background: Chronic pain is a leading cause of disability and requires multidimensional assessment of pain intensity and functioning, yet electronic health records rarely capture these measures systematically. By contrast, surveys collecting patient-reported outcomes can assess pain over multiple dimensions but remain resource-intensive and difficult to scale for continuous population-level monitoring. Objective: The objective of this study is to develop and validate a domain-informed natural language processing framework to derive pain severity and functional interference outcomes from unstructured clinical narratives. We aim to demonstrate that natural language processing-derived outcomes can serve as a reliable, scalable surrogate for resource-intensive patient-reported surveys. Methods: This study uses a retrospective cohort of 3725 Veterans with chronic musculoskeletal pain initiating complementary and integrative health therapies across 18 Veterans Health Administration Whole Health Flagship sites (2021-2023). The dataset encompasses longitudinal patient-reported outcome surveys serving as the benchmark, linked with unstructured clinical narratives from the Veterans Health Administration electronic health record. Guided by established psychometric instruments and subject matter expert (SME) input, we developed a seed lexicon and annotation guidelines to identify language distinguishing 3 pain domains: pain severity, interference with enjoyment of life, and interference with general activities. Preliminary large language model (LLM) prompting was used to identify 600 candidate encounters (200 per domain) from 6747 notes across 260 patients for SME annotation, forming a ground truth validation sample. Two candidate LLMs will be evaluated on this sample; the best-performing LLM will generate a large library of span-level annotations to train a scalable, lightweight language model. The study uses a 3-stage validation process: (1) documentation completeness of pain interference in clinical narratives against SME-annotated references; (2) inference accuracy of the LLM-as-annotator and the fine-tuned lightweight model against SME annotations across note-level classification and span-level localization; and (3) concordance between the lightweight model's output and patient-reported pain, enjoyment, and general activity scores across a range of temporal windows. Results: As of July 2026, the cohort of 3725 Veterans has been identified and linked to clinical notes. The seed lexicon and annotation guidelines have been developed. Applying a developmental LLM to screen 6747 text notes in 6642 unique encounters over a 7-month period for 260 patients, at least 1 of the 3 pain domains was identified in 75% of notes and 99% of patients. SME validation at the encounter level is in progress. Final results from the subsequent knowledge distillation and validation stages are expected in the first half of 2027. Conclusions: This protocol outlines a framework for identifying severe pain intensity and interference from clinical narratives, addressing a critical gap in health care system surveillance. To our knowledge, this is the first study to validate clinical text-based pain outcome extraction against patient-reported outcomes in a nationwide longitudinal cohort. If successful, this approach will enable health care systems to continuously monitor reports of pain-related functional interference and support more holistic, patient-centered pain management at scale. (© Xiaoyi Zhang, Christopher R Wilson, Hannah Eyre, David E Reed II, Travis Y Hee Wai, Alexander Kloehn,Ethan W Rosser, Gang Luo, Steven B Zeliadt. Originally published in JMIR Research Protocols (https://www.researchprotocols.org).) |
| References: | medRxiv. 2025 May 06;:. (PMID: 40313300) J Am Med Inform Assoc. 2023 Feb 16;30(3):570-587. (PMID: 36458955) J Pain. 2024 May;25(5):104420. (PMID: 37952861) BMJ. 2024 Apr 16;385:e078378. (PMID: 38626948) J Gen Intern Med. 2007 Oct;22(10):1453-8. (PMID: 17668269) J Med Internet Res. 2021 Nov 3;23(11):e26777. (PMID: 34730546) Med Biol Eng Comput. 2018 Jul;56(7):1285-1292. (PMID: 29280092) J Pain. 2006 Feb;7(2):129-33. (PMID: 16459278) J Gen Intern Med. 2018 May;33(Suppl 1):31-37. (PMID: 29633139) Med Care. 2025 Sep 1;63(9):679-685. (PMID: 40730349) Nature. 2023 Aug;620(7972):172-180. (PMID: 37438534) Pain Manag Nurs. 2018 Apr;19(2):125-129. (PMID: 29249620) J Gen Intern Med. 2009 Jun;24(6):733-8. (PMID: 19418100) J Pain. 2022 Mar;23(3):370-378. (PMID: 34508905) Pain Med. 2021 Oct 8;22(10):2235-2241. (PMID: 33749760) BMJ Open. 2024 Apr 19;14(4):e079923. (PMID: 38642997) J Gen Intern Med. 2006 Jun;21(6):607-12. (PMID: 16808744) Pain Physician. 2022 Mar;25(2):E245-E254. (PMID: 35322976) Pain. 2001 Apr;91(3):317-322. (PMID: 11275389) Glob Adv Integr Med Health. 2025 Jul 8;14:27536130251358757. (PMID: 40657238) NCHS Data Brief. 2024 Oct;(518):. (PMID: 39751180) Pain. 2005 Jan;113(1-2):9-19. (PMID: 15621359) Pain. 2010 Jul;150(1):173-182. (PMID: 20554116) Clin Trials. 2020 Aug;17(4):351-359. (PMID: 32522024) AMIA Annu Symp Proc. 2022 Feb 21;2021:438-447. (PMID: 35308962) Lancet Digit Health. 2023 Oct 26;:. (PMID: 39492289) Expert Rev Pharmacoecon Outcomes Res. 2024 Apr;24(4):467-475. (PMID: 38383308) Pain. 2022 Jun 1;163(6):e715-e724. (PMID: 34724683) Front Psychol. 2016 Sep 30;7:1466. (PMID: 27746750) Pain. 1985 Dec;23(4):345-356. (PMID: 4088697) Pain Med. 2020 Dec 12;21(Suppl 2):S100-S109. (PMID: 33313736) Physiotherapy. 1980 Aug;66(8):271-3. (PMID: 6450426) Br J Anaesth. 2019 Aug;123(2):e273-e283. (PMID: 31079836) Med Care. 2026 May 01;64(5):283-292. (PMID: 41771006) Glob Adv Integr Med Health. 2024 Apr 6;13:27536130241241259. (PMID: 38585239) JMIR Med Inform. 2020 Nov 16;8(11):e18659. (PMID: 33108311) Nat Med. 2025 Mar;31(3):943-950. (PMID: 39779926) J Med Internet Res. 2025 Mar 31;27:e65903. (PMID: 40163858) Pain. 1995 May;61(2):277-284. (PMID: 7659438) |
| Contributed Indexing: | Keywords: Veterans; chronic pain; electronic health records; knowledge distillation; large language models; natural language processing; pain assessment; patient-reported outcomes |
| Entry Date(s): | Date Created: 20261007 Date Completed: 20261007 Latest Revision: 20261009 |
| Update Code: | 20261009 |
| PubMed Central ID: | PMC13645315 |
| DOI: | 10.2196/96346 |
| PMID: | 42842900 |
| Βάση Δεδομένων: | MEDLINE |
| ISSN: | 1929-0748 |
|---|---|
| DOI: | 10.2196/96346 |