Academic Journal
Προβολή σε EDS

A jackknife approach to estimate the prediction uncertainty from binary classifiers under right-censoring.

Λεπτομέρειες βιβλιογραφικής εγγραφής
Τίτλος: A jackknife approach to estimate the prediction uncertainty from binary classifiers under right-censoring.
Συγγραφείς: Jahn-Eimermacher A; Department of Mathematics and Natural Sciences, University of Applied Sciences, Darmstadt, Germany.; European University of Technology, European Union, Darmstadt, Germany., Klein L; Department of Mathematics and Natural Sciences, University of Applied Sciences, Darmstadt, Germany.; European University of Technology, European Union, Darmstadt, Germany.; Faculty of Medicine, Martin-Luther-University Halle-Wittenberg, Halle (Saale), German., Grieser G; European University of Technology, European Union, Darmstadt, Germany.; Department of Computer Science, University of Applied Sciences, Darmstadt, Germany.
Πηγή: Statistical methods in medical research [Stat Methods Med Res] 2026 Jan; Vol. 35 (1), pp. 79-100. Date of Electronic Publication: 2025 Nov 14.
Τύπος έκδοσης: Journal Article
Γλώσσα: English
Στοιχεία περιοδικού: Publisher: SAGE Publications Country of Publication: England NLM ID: 9212457 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 1477-0334 (Electronic) Linking ISSN: 09622802 NLM ISO Abbreviation: Stat Methods Med Res Subsets: MEDLINE
Imprint Name(s): Publication: London : SAGE Publications
Original Publication: Sevenoaks, Kent, UK : Edward Arnold, c1992-
Ιατρικοί όροι (MeSH): Machine Learning* , Models, Statistical* , Uncertainty*, Kidney Transplantation/mortality ; Humans ; Computer Simulation ; Likelihood Functions ; Probability
Περίληψη: Clinical prediction models are developed to estimate a patient's risk for a specific outcome, and machine learning is frequently employed to improve prediction accuracy. When the outcome is some event that happens over time, binary classifiers can predict the risk at specific time points if right-censoring is addressed by inverse-probability-of-censoring-weighting . Assessing prediction uncertainty is crucial for interpreting individual risks, but there is limited knowledge on how to consider inverse-probability-of-censoring-weighting when estimating this uncertainty. We propose an adjustment of the infinitesimal jackknife estimator for the standard error of predictions that incorporates inverse-probability-of-censoring-weighting. By using a nonparametric approach, it is broadly applicable, especially to machine learning classifiers. For a simple tractable example, we show that the proposed adjustment reveals unbiased standard error estimates. For other situations, we evaluate performance through simulation studies under both parametric models with inverse-probability-of-censoring-weighting-customized log-likelihood and machine learning with inverse-probability-of-censoring-weighting-customized loss function. We illustrate the methods by predicting post-transplant survival probabilities, using national kidney transplant registry data. Our findings show that the proposed estimator is useful for quantifying prediction uncertainty of inverse-probability-of-censoring-weighting classifiers. Applications to simulated and real data show that prediction uncertainty increases when employing binary classifiers on dichotomized data compared to predictions from survival models.
Competing Interests: Declaration of conflicting interestThe authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Data availability statementAn implementation of the applied analysis methods is available as an R package on GitHub: https://github.com/IDEN-Project-UAS-Darmstadt/IPCWJK. The data analysis presented in this article uses retrospective and anonymized data of the German Transplant Registry. These data have been supplied by the Transplant Registry Agency represented by the Gesundheitsforen Leipzig GmbH. Data can be requested for research purposes here https://transplantations-register.de/en/research.
References: BMC Med Inform Decis Mak. 2021 Feb 6;21(1):43. (PMID: 33549087)
J Mach Learn Res. 2014 Jan;15(1):1625-1651. (PMID: 25580094)
Kidney Int. 2023 May;103(5):936-948. (PMID: 36572246)
Clin Transplant. 2021 Aug;35(8):e14388. (PMID: 34155697)
Lifetime Data Anal. 2023 Apr;29(2):441-482. (PMID: 35799026)
Stat Med. 2016 May 30;35(12):2016-30. (PMID: 26712471)
Transplantation. 2009 Jul 27;88(2):231-6. (PMID: 19623019)
Lancet Digit Health. 2021 May;3(5):e295-e305. (PMID: 33858815)
Stat Methods Med Res. 2019 Dec;28(12):3755-3768. (PMID: 30514179)
Sci Rep. 2022 Nov 14;12(1):19525. (PMID: 36376402)
BMC Med Res Methodol. 2018 Feb 26;18(1):24. (PMID: 29482517)
NPJ Digit Med. 2022 Jul 11;5(1):89. (PMID: 35817953)
Stat Methods Med Res. 2017 Aug;26(4):1654-1670. (PMID: 25934643)
Epidemiology. 2010 Jan;21(1):13-5. (PMID: 20010207)
Stat Med. 2024 Mar 30;43(7):1384-1396. (PMID: 38297411)
Ann Intern Med. 2015 Jan 6;162(1):W1-73. (PMID: 25560730)
Stat Med. 2023 Dec 30;42(30):5657-5675. (PMID: 37947168)
Transplant Proc. 2020 Jan - Feb;52(1):246-258. (PMID: 31926745)
Kidney Int. 2023 Nov;104(5):1035-1036. (PMID: 37863623)
J Biomed Inform. 2016 Jun;61:119-31. (PMID: 26992568)
Stat Med. 2019 May 20;38(11):2074-2102. (PMID: 30652356)
Contributed Indexing: Keywords: Prediction; inverse-probability-of-censoring; machine learning; survival analysis; uncertainty
Entry Date(s): Date Created: 20251114 Date Completed: 20260109 Latest Revision: 20260112
Update Code: 20260130
PubMed Central ID: PMC12783380
DOI: 10.1177/09622802251393626
PMID: 41236860
Βάση Δεδομένων: MEDLINE
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://resolver.ebsco.com/c/fiv2js/result?sid=EBSCO:cmedm&genre=article&issn=14770334&ISBN=&volume=35&issue=1&date=20260101&spage=79&pages=79-100&title=Statistical methods in medical research&atitle=A%20jackknife%20approach%20to%20estimate%20the%20prediction%20uncertainty%20from%20binary%20classifiers%20under%20right-censoring.&aulast=Jahn-Eimermacher%20A&id=DOI:10.1177/09622802251393626
    Name: Full Text Finder (for New FTF UI) (ns324271)
    Category: fullText
    Text: Full Text Finder
    MouseOverText: Full Text Finder
Header DbId: cmedm
DbLabel: MEDLINE
An: 41236860
AccessLevel: 3
PubType: Academic Journal
PubTypeId: academicJournal
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: A jackknife approach to estimate the prediction uncertainty from binary classifiers under right-censoring.
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AU" term="%22Jahn-Eimermacher+A%22">Jahn-Eimermacher A</searchLink>; Department of Mathematics and Natural Sciences, University of Applied Sciences, Darmstadt, Germany.; European University of Technology, European Union, Darmstadt, Germany.<br /><searchLink fieldCode="AU" term="%22Klein+L%22">Klein L</searchLink>; Department of Mathematics and Natural Sciences, University of Applied Sciences, Darmstadt, Germany.; European University of Technology, European Union, Darmstadt, Germany.; Faculty of Medicine, Martin-Luther-University Halle-Wittenberg, Halle (Saale), German.<br /><searchLink fieldCode="AU" term="%22Grieser+G%22">Grieser G</searchLink>; European University of Technology, European Union, Darmstadt, Germany.; Department of Computer Science, University of Applied Sciences, Darmstadt, Germany.
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="JN" term="%229212457%22">Statistical methods in medical research</searchLink> [Stat Methods Med Res] 2026 Jan; Vol. 35 (1), pp. 79-100. <i>Date of Electronic Publication: </i>2025 Nov 14.
– Name: TypePub
  Label: Publication Type
  Group: TypPub
  Data: Journal Article
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: TitleSource
  Label: Journal Info
  Group: Src
  Data: <i>Publisher: </i><searchLink fieldCode="PB" term="%22SAGE+Publications%22">SAGE Publications </searchLink><i>Country of Publication: </i>England <i>NLM ID: </i>9212457 <i>Publication Model: </i>Print-Electronic <i>Cited Medium: </i>Internet <i>ISSN: </i>1477-0334 (Electronic) <i>Linking ISSN: </i><searchLink fieldCode="IS" term="%2209622802%22">09622802 </searchLink><i>NLM ISO Abbreviation: </i>Stat Methods Med Res <i>Subsets: </i>MEDLINE
– Name: PublisherInfo
  Label: Imprint Name(s)
  Group: PubInfo
  Data: <i>Publication</i>: London : SAGE Publications<br /><i>Original Publication</i>: Sevenoaks, Kent, UK : Edward Arnold, c1992-
– Name: SubjectMESH
  Label: MeSH Terms
  Group: Su
  Data: <searchLink fieldCode="MM" term="%22Machine+Learning%22">Machine Learning*</searchLink> <br /><searchLink fieldCode="MM" term="%22Models%2C+Statistical%22">Models, Statistical*</searchLink> <br /><searchLink fieldCode="MM" term="%22Uncertainty%22">Uncertainty*</searchLink><br /><searchLink fieldCode="MH" term="%22Kidney+Transplantation%22">Kidney Transplantation</searchLink>/<searchLink fieldCode="MH" term="%22Kidney+Transplantation+mortality%22">mortality</searchLink> ; <searchLink fieldCode="MH" term="%22Humans%22">Humans</searchLink> ; <searchLink fieldCode="MH" term="%22Computer+Simulation%22">Computer Simulation</searchLink> ; <searchLink fieldCode="MH" term="%22Likelihood+Functions%22">Likelihood Functions</searchLink> ; <searchLink fieldCode="MH" term="%22Probability%22">Probability</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Clinical prediction models are developed to estimate a patient's risk for a specific outcome, and machine learning is frequently employed to improve prediction accuracy. When the outcome is some event that happens over time, binary classifiers can predict the risk at specific time points if right-censoring is addressed by inverse-probability-of-censoring-weighting . Assessing prediction uncertainty is crucial for interpreting individual risks, but there is limited knowledge on how to consider inverse-probability-of-censoring-weighting when estimating this uncertainty. We propose an adjustment of the infinitesimal jackknife estimator for the standard error of predictions that incorporates inverse-probability-of-censoring-weighting. By using a nonparametric approach, it is broadly applicable, especially to machine learning classifiers. For a simple tractable example, we show that the proposed adjustment reveals unbiased standard error estimates. For other situations, we evaluate performance through simulation studies under both parametric models with inverse-probability-of-censoring-weighting-customized log-likelihood and machine learning with inverse-probability-of-censoring-weighting-customized loss function. We illustrate the methods by predicting post-transplant survival probabilities, using national kidney transplant registry data. Our findings show that the proposed estimator is useful for quantifying prediction uncertainty of inverse-probability-of-censoring-weighting classifiers. Applications to simulated and real data show that prediction uncertainty increases when employing binary classifiers on dichotomized data compared to predictions from survival models.
– Name: Abstract
  Label: Competing Interests
  Group: Ab
  Data: Declaration of conflicting interestThe authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article. Data availability statementAn implementation of the applied analysis methods is available as an R package on GitHub: https://github.com/IDEN-Project-UAS-Darmstadt/IPCWJK. The data analysis presented in this article uses retrospective and anonymized data of the German Transplant Registry. These data have been supplied by the Transplant Registry Agency represented by the Gesundheitsforen Leipzig GmbH. Data can be requested for research purposes here https://transplantations-register.de/en/research.
– Name: Ref
  Label: References
  Group: RefInfo
  Data: BMC Med Inform Decis Mak. 2021 Feb 6;21(1):43. (PMID: <searchLink fieldCode="PM" term="%2233549087%22">33549087)</searchLink><br />J Mach Learn Res. 2014 Jan;15(1):1625-1651. (PMID: <searchLink fieldCode="PM" term="%2225580094%22">25580094)</searchLink><br />Kidney Int. 2023 May;103(5):936-948. (PMID: <searchLink fieldCode="PM" term="%2236572246%22">36572246)</searchLink><br />Clin Transplant. 2021 Aug;35(8):e14388. (PMID: <searchLink fieldCode="PM" term="%2234155697%22">34155697)</searchLink><br />Lifetime Data Anal. 2023 Apr;29(2):441-482. (PMID: <searchLink fieldCode="PM" term="%2235799026%22">35799026)</searchLink><br />Stat Med. 2016 May 30;35(12):2016-30. (PMID: <searchLink fieldCode="PM" term="%2226712471%22">26712471)</searchLink><br />Transplantation. 2009 Jul 27;88(2):231-6. (PMID: <searchLink fieldCode="PM" term="%2219623019%22">19623019)</searchLink><br />Lancet Digit Health. 2021 May;3(5):e295-e305. (PMID: <searchLink fieldCode="PM" term="%2233858815%22">33858815)</searchLink><br />Stat Methods Med Res. 2019 Dec;28(12):3755-3768. (PMID: <searchLink fieldCode="PM" term="%2230514179%22">30514179)</searchLink><br />Sci Rep. 2022 Nov 14;12(1):19525. (PMID: <searchLink fieldCode="PM" term="%2236376402%22">36376402)</searchLink><br />BMC Med Res Methodol. 2018 Feb 26;18(1):24. (PMID: <searchLink fieldCode="PM" term="%2229482517%22">29482517)</searchLink><br />NPJ Digit Med. 2022 Jul 11;5(1):89. (PMID: <searchLink fieldCode="PM" term="%2235817953%22">35817953)</searchLink><br />Stat Methods Med Res. 2017 Aug;26(4):1654-1670. (PMID: <searchLink fieldCode="PM" term="%2225934643%22">25934643)</searchLink><br />Epidemiology. 2010 Jan;21(1):13-5. (PMID: <searchLink fieldCode="PM" term="%2220010207%22">20010207)</searchLink><br />Stat Med. 2024 Mar 30;43(7):1384-1396. (PMID: <searchLink fieldCode="PM" term="%2238297411%22">38297411)</searchLink><br />Ann Intern Med. 2015 Jan 6;162(1):W1-73. (PMID: <searchLink fieldCode="PM" term="%2225560730%22">25560730)</searchLink><br />Stat Med. 2023 Dec 30;42(30):5657-5675. (PMID: <searchLink fieldCode="PM" term="%2237947168%22">37947168)</searchLink><br />Transplant Proc. 2020 Jan - Feb;52(1):246-258. (PMID: <searchLink fieldCode="PM" term="%2231926745%22">31926745)</searchLink><br />Kidney Int. 2023 Nov;104(5):1035-1036. (PMID: <searchLink fieldCode="PM" term="%2237863623%22">37863623)</searchLink><br />J Biomed Inform. 2016 Jun;61:119-31. (PMID: <searchLink fieldCode="PM" term="%2226992568%22">26992568)</searchLink><br />Stat Med. 2019 May 20;38(11):2074-2102. (PMID: <searchLink fieldCode="PM" term="%2230652356%22">30652356)</searchLink>
– Name: SubjectMinor
  Label: Contributed Indexing
  Group:
  Data: <i>Keywords: </i>Prediction; inverse-probability-of-censoring; machine learning; survival analysis; uncertainty
– Name: DateEntry
  Label: Entry Date(s)
  Group: Date
  Data: <i>Date Created: </i>20251114 <i>Date Completed: </i>20260109 <i>Latest Revision: </i>20260112
– Name: DateUpdate
  Label: Update Code
  Group: Date
  Data: 20260130
– Name: PubmedCentralID
  Label: PubMed Central ID
  Group: ID
  Data: PMC12783380
– Name: DOI
  Label: DOI
  Group: ID
  Data: 10.1177/09622802251393626
– Name: AN
  Label: PMID
  Group: ID
  Data: 41236860
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=cmedm&AN=41236860
RecordInfo BibRecord:
  BibEntity:
    Identifiers:
      – Type: doi
        Value: 10.1177/09622802251393626
    Languages:
      – Code: eng
        Text: English
    PhysicalDescription:
      Pagination:
        StartPage: 79
    Subjects:
      – SubjectFull: Kidney Transplantation mortality
        Type: general
      – SubjectFull: Humans
        Type: general
      – SubjectFull: Computer Simulation
        Type: general
      – SubjectFull: Likelihood Functions
        Type: general
      – SubjectFull: Probability
        Type: general
      – SubjectFull: Machine Learning
        Type: general
      – SubjectFull: Models, Statistical
        Type: general
      – SubjectFull: Uncertainty
        Type: general
    Titles:
      – TitleFull: A jackknife approach to estimate the prediction uncertainty from binary classifiers under right-censoring.
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Jahn-Eimermacher A
      – PersonEntity:
          Name:
            NameFull: Klein L
      – PersonEntity:
          Name:
            NameFull: Grieser G
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 01
              Text: 2026 Jan
              Type: published
              Y: 2026
          Identifiers:
            – Type: issn-electronic
              Value: 1477-0334
          Numbering:
            – Type: volume
              Value: 35
            – Type: issue
              Value: 1
          Titles:
            – TitleFull: Statistical methods in medical research
              Type: main
ResultId 1