Academic Journal
Sentence-Level Provenance for AI Medical Record Summarization in a Click-to-Inspect Interface: Formative Usability Evaluation.
| Τίτλος: | Sentence-Level Provenance for AI Medical Record Summarization in a Click-to-Inspect Interface: Formative Usability Evaluation. |
|---|---|
| Συγγραφείς: | Parambath A; Stanford Medicine, 900 Welch Road, Suite 350, Palo Alto, CA, 94304, United States, 1 2672979144., Pulpo G; Abstractive Health, New York, NY, United States., Hartman V; Abstractive Health, New York, NY, United States. |
| Πηγή: | JMIR human factors [JMIR Hum Factors] 2026 Sep 08; Vol. 13, pp. e95644. Date of Electronic Publication: 2026 Sep 08. |
| Τύπος έκδοσης: | Journal Article |
| Γλώσσα: | English |
| Στοιχεία περιοδικού: | Publisher: JMIR Publications Inc Country of Publication: Canada NLM ID: 101666561 Publication Model: Electronic Cited Medium: Internet ISSN: 2292-9495 (Electronic) Linking ISSN: 22929495 NLM ISO Abbreviation: JMIR Hum Factors Subsets: MEDLINE |
| Imprint Name(s): | Original Publication: Toronto : JMIR Publications Inc, [2014]- |
| Ιατρικοί όροι (MeSH): | Artificial Intelligence* , Electronic Health Records* , User-Computer Interface*, Humans |
| Περίληψη: | Background: Large language models can generate fluent summaries of longitudinal medical records, but in high-stakes clinical settings, verification burden remains a barrier to trust. Existing provenance mechanisms such as document-level citations and section references often require manual search within long, fragmented notes, limiting their usefulness during time-constrained workflows for clinicians. Objective: This study aimed to design and evaluate a sentence-level provenance interface ("click to inspect") that enables rapid verification of AI-generated longitudinal medical record summaries at the level of individual statements. Methods: Between November 2023 and January 2024, we conducted a formative usability study using remotely moderated usability sessions via Zoom to evaluate a web-based sentence-level provenance interface for AI-generated longitudinal medical record summaries. A convenience sample of clinicians was recruited through email outreach to academic and professional networks across the United States. Formative usability testing was conducted with 46 clinician interactions using synthetic longitudinal patient charts. Participants included medical students, residents, and attending physicians across multiple specialties, including internal medicine, dermatology, radiology, plastic surgery, anesthesiology, interventional radiology, obstetrics and gynecology, and family medicine. Usability was assessed using the System Usability Scale and net promoter score, alongside qualitative feedback. Results: Clinicians reported high usability (mean System Usability Scale score 86.25, SD 7.77; 95% CI 83.96-88.54 from 46 participants) and a positive overall experience (net promoter score of 35; 22/46, 47.8% promoters; 18/46, 39.1% passives; and 6/46, 13% detractors). Participants described rapid access to supporting evidence as critical for trust calibration during first-pass chart review. Qualitative feedback identified friction in traditional citation-based interfaces and supported sentence-level inspectability as a low-friction verification mechanism. Conclusions: Sentence-level provenance transforms AI-generated summaries from static narratives into interactive verification tools. An approach that enables rapid, selective inspection of individual claims during longitudinal chart review may reduce verification burden and support calibrated reliance in high-risk clinical contexts. (© Andrew Parambath, Giordana Pulpo, Vince Hartman. Originally published in JMIR Human Factors (https://humanfactors.jmir.org).) |
| References: | J Am Med Inform Assoc. 2023 Nov 17;30(12):1995-2003. (PMID: 37639624) Ann Fam Med. 2017 Sep;15(5):419-426. (PMID: 28893811) JAMA. 2024 Feb 27;331(8):637-638. (PMID: 38285439) Ann Intern Med. 2018 Jul 3;169(1):50-51. (PMID: 29801050) Ann Intern Med. 2020 Feb 4;172(3):169-174. (PMID: 31931523) NPJ Digit Med. 2025 May 13;8(1):274. (PMID: 40360677) J Am Med Inform Assoc. 2015 Mar;22(2):263-74. (PMID: 25352564) JAMIA Open. 2025 Sep 02;8(5):ooaf096. (PMID: 40904519) PLOS Digit Health. 2025 Jun 17;4(6):e0000899. (PMID: 40526634) J Biomed Inform. 2011 Aug;44(4):688-99. (PMID: 21440086) JMIR AI. 2024 Oct 30;3:e53207. (PMID: 39476365) J Am Med Inform Assoc. 2015 Sep;22(5):938-47. (PMID: 25882031) Nat Med. 2024 Apr;30(4):1134-1142. (PMID: 38413730) |
| Contributed Indexing: | Keywords: AI; artificial intelligence; clinical decision support systems; electronic health records; health IT; human-centered design; medical informatics; natural language processing; trust calibration; usability testing |
| Entry Date(s): | Date Created: 20260908 Date Completed: 20260908 Latest Revision: 20260914 |
| Update Code: | 20260914 |
| PubMed Central ID: | PMC13552831 |
| DOI: | 10.2196/95644 |
| PMID: | 42709992 |
| Βάση Δεδομένων: | MEDLINE |
| ISSN: | 2292-9495 |
|---|---|
| DOI: | 10.2196/95644 |