Academic Journal
Προβολή σε EDS

Ensuring Reliability of Curated Electronic Health Record–Derived Data: The Validation of Accuracy for Large Language Model–/Machine Learning–Extracted Information and Data (VALID) Framework.

Λεπτομέρειες βιβλιογραφικής εγγραφής
Τίτλος: Ensuring Reliability of Curated Electronic Health Record–Derived Data: The Validation of Accuracy for Large Language Model–/Machine Learning–Extracted Information and Data (VALID) Framework.
Συγγραφείς: Estevez, Melissa, Singh, Nisha, Dyson, Lauren, Adamson, Blythe, Krismer, Konstantin, Magee, Kelly, Yuan, Qianyu, Hildner, Megan W., Fidyk, Erin, Williams, Tori, Mbah, Olive, Khan, Farhad, Seidl-Rathkopf, Kathi, Cohen, Aaron B.
Πηγή: JCO Clinical Cancer Informatics; 3/16/2026, Vol. 10, p1-11, 11p
Θεματικοί όροι: Language models, Data quality, Quality assurance, Oncology, Machine learning, Statistical bias, Electronic health records, Medical records
Περίληψη: Large language models (LLMs) are increasingly used to extract clinical data from electronic health records, offering significant improvements in scalability and efficiency for real-world data (RWD) curation in oncology. However, the adoption of LLMs introduces new challenges in ensuring the reliability, accuracy, and fairness of extracted data, which are essential for research, regulatory, and clinical applications. Existing quality assurance frameworks for RWD and artificial intelligence (AI) do not fully address the unique error modes and complexities associated with LLM-extracted data. In this paper, we propose a comprehensive framework for evaluating the quality of clinical data extracted by LLMs. The framework integrates variable-level performance benchmarking against expert human abstraction, verification checks for internal consistency and plausibility, and replication analyses comparing LLM-extracted data to human-abstracted data sets or external standards. This multidimensional approach enables the identification of variables most in need of improvement, systematic detection of latent errors, and confirmation of data set fitness-for-purpose in real-world research. Additionally, the framework supports bias assessment by stratifying across demographic subgroups. By providing a rigorous and transparent method for assessing LLM-extracted RWD, this framework advances industry standards and supports the trustworthy use of AI-powered evidence generation in oncology research and practice. [ABSTRACT FROM AUTHOR]
Copyright of JCO Clinical Cancer Informatics is the property of American Society of Clinical Oncology and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)
Βάση Δεδομένων: Complementary Index
FullText Text:
  Availability: 0
Header DbId: edb
DbLabel: Complementary Index
An: 192315043
RelevancyScore: 1041
AccessLevel: 6
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 1041.072265625
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: Ensuring Reliability of Curated Electronic Health Record–Derived Data: The Validation of Accuracy for Large Language Model–/Machine Learning–Extracted Information and Data (VALID) Framework.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Estevez%2C+Melissa%22">Estevez, Melissa</searchLink><br /><searchLink fieldCode="AR" term="%22Singh%2C+Nisha%22">Singh, Nisha</searchLink><br /><searchLink fieldCode="AR" term="%22Dyson%2C+Lauren%22">Dyson, Lauren</searchLink><br /><searchLink fieldCode="AR" term="%22Adamson%2C+Blythe%22">Adamson, Blythe</searchLink><br /><searchLink fieldCode="AR" term="%22Krismer%2C+Konstantin%22">Krismer, Konstantin</searchLink><br /><searchLink fieldCode="AR" term="%22Magee%2C+Kelly%22">Magee, Kelly</searchLink><br /><searchLink fieldCode="AR" term="%22Yuan%2C+Qianyu%22">Yuan, Qianyu</searchLink><br /><searchLink fieldCode="AR" term="%22Hildner%2C+Megan+W%2E%22">Hildner, Megan W.</searchLink><br /><searchLink fieldCode="AR" term="%22Fidyk%2C+Erin%22">Fidyk, Erin</searchLink><br /><searchLink fieldCode="AR" term="%22Williams%2C+Tori%22">Williams, Tori</searchLink><br /><searchLink fieldCode="AR" term="%22Mbah%2C+Olive%22">Mbah, Olive</searchLink><br /><searchLink fieldCode="AR" term="%22Khan%2C+Farhad%22">Khan, Farhad</searchLink><br /><searchLink fieldCode="AR" term="%22Seidl-Rathkopf%2C+Kathi%22">Seidl-Rathkopf, Kathi</searchLink><br /><searchLink fieldCode="AR" term="%22Cohen%2C+Aaron+B%2E%22">Cohen, Aaron B.</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: JCO Clinical Cancer Informatics; 3/16/2026, Vol. 10, p1-11, 11p
– Name: Subject
  Label: Subject Terms
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Language+models%22">Language models</searchLink><br /><searchLink fieldCode="DE" term="%22Data+quality%22">Data quality</searchLink><br /><searchLink fieldCode="DE" term="%22Quality+assurance%22">Quality assurance</searchLink><br /><searchLink fieldCode="DE" term="%22Oncology%22">Oncology</searchLink><br /><searchLink fieldCode="DE" term="%22Machine+learning%22">Machine learning</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+bias%22">Statistical bias</searchLink><br /><searchLink fieldCode="DE" term="%22Electronic+health+records%22">Electronic health records</searchLink><br /><searchLink fieldCode="DE" term="%22Medical+records%22">Medical records</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Large language models (LLMs) are increasingly used to extract clinical data from electronic health records, offering significant improvements in scalability and efficiency for real-world data (RWD) curation in oncology. However, the adoption of LLMs introduces new challenges in ensuring the reliability, accuracy, and fairness of extracted data, which are essential for research, regulatory, and clinical applications. Existing quality assurance frameworks for RWD and artificial intelligence (AI) do not fully address the unique error modes and complexities associated with LLM-extracted data. In this paper, we propose a comprehensive framework for evaluating the quality of clinical data extracted by LLMs. The framework integrates variable-level performance benchmarking against expert human abstraction, verification checks for internal consistency and plausibility, and replication analyses comparing LLM-extracted data to human-abstracted data sets or external standards. This multidimensional approach enables the identification of variables most in need of improvement, systematic detection of latent errors, and confirmation of data set fitness-for-purpose in real-world research. Additionally, the framework supports bias assessment by stratifying across demographic subgroups. By providing a rigorous and transparent method for assessing LLM-extracted RWD, this framework advances industry standards and supports the trustworthy use of AI-powered evidence generation in oncology research and practice. [ABSTRACT FROM AUTHOR]
– Name: Abstract
  Label:
  Group: Ab
  Data: <i>Copyright of JCO Clinical Cancer Informatics is the property of American Society of Clinical Oncology and its content may not be copied or emailed to multiple sites without the copyright holder's express written permission. Additionally, content may not be used with any artificial intelligence tools or machine learning technologies. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract.</i> (Copyright applies to all Abstracts.)
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=edb&AN=192315043
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1200/CCI-25-00215
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 11
        StartPage: 1
    Subjects:
      – SubjectFull: Language models
        Type: general
      – SubjectFull: Data quality
        Type: general
      – SubjectFull: Quality assurance
        Type: general
      – SubjectFull: Oncology
        Type: general
      – SubjectFull: Machine learning
        Type: general
      – SubjectFull: Statistical bias
        Type: general
      – SubjectFull: Electronic health records
        Type: general
      – SubjectFull: Medical records
        Type: general
    Titles:
      – TitleFull: Ensuring Reliability of Curated Electronic Health Record–Derived Data: The Validation of Accuracy for Large Language Model–/Machine Learning–Extracted Information and Data (VALID) Framework.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Estevez, Melissa
      – PersonEntity:
          Name:
            NameFull: Singh, Nisha
      – PersonEntity:
          Name:
            NameFull: Dyson, Lauren
      – PersonEntity:
          Name:
            NameFull: Adamson, Blythe
      – PersonEntity:
          Name:
            NameFull: Krismer, Konstantin
      – PersonEntity:
          Name:
            NameFull: Magee, Kelly
      – PersonEntity:
          Name:
            NameFull: Yuan, Qianyu
      – PersonEntity:
          Name:
            NameFull: Hildner, Megan W.
      – PersonEntity:
          Name:
            NameFull: Fidyk, Erin
      – PersonEntity:
          Name:
            NameFull: Williams, Tori
      – PersonEntity:
          Name:
            NameFull: Mbah, Olive
      – PersonEntity:
          Name:
            NameFull: Khan, Farhad
      – PersonEntity:
          Name:
            NameFull: Seidl-Rathkopf, Kathi
      – PersonEntity:
          Name:
            NameFull: Cohen, Aaron B.
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 16
              M: 03
              Text: 3/16/2026
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-print
              Value: 24734276
          Numbering:
            – Type: volume
              Value: 10
          Titles:
            – TitleFull: JCO Clinical Cancer Informatics
              Type: main
ResultId 1