Report
Προβολή σε EDS

An Applied Researchers' Guide to Estimating Effects from Blocked Cluster Randomized Trials: Estimands, Estimators, and Estimates. EdWorkingPaper No. 26-1537

Λεπτομέρειες βιβλιογραφικής εγγραφής
Τίτλος: An Applied Researchers' Guide to Estimating Effects from Blocked Cluster Randomized Trials: Estimands, Estimators, and Estimates. EdWorkingPaper No. 26-1537
Γλώσσα: English
Συγγραφείς: Luke Miratrix, Michael J. Weiss, Sophie Litschwartz, Colin Hill, Kayla Warner, Annenberg Institute for School Reform at Brown University
Πηγή: Annenberg Institute for School Reform at Brown University. 2026.
Διαθεσιμότητα: Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/
Peer Reviewed: N
Page Count: 128
Ημερομηνία έκδοσης: 2026
Sponsoring Agency: Institute of Education Sciences (ED)
Contract Number: R305D220046
Intended Audience: Researchers
Τύπος εγγράφου: Reports - Research
Descriptors: Computation, Randomized Controlled Trials, Error of Measurement, Effect Size, Regression (Statistics), Hierarchical Linear Modeling, Statistical Analysis, Selection, Statistical Bias
Περίληψη: Blocked cluster randomized trials (CRTs) are widely used to evaluate educational interventions. In such trials, researchers face critical choices about how to define and estimate the average treatment effect (ATE). These choices--both the estimand (e.g., person- vs. cluster-weighted) and the estimator (e.g., regression models, multilevel models)--can influence study conclusions. This paper provides an applied guide to estimands and estimators for blocked CRTs and empirically examines their performance using 26 large-scale blocked CRTs. We estimate ATEs and standard errors for 50 outcomes using 19 estimators and compare results across and within estimands. Findings show that estimator choice can substantially affect point estimates and precision: ranges in ATE estimates exceed 0.10 SD in 18% of cases, and standard error estimates differ by more than 50% in 16% of cases. Some estimators fail or produce unstable results in common real-world designs. These findings underscore the importance of pre-specifying estimation strategies in a public pre-analysis plan. We provide practical guidance for researchers planning and analyzing blocked CRTs.
Abstractor: As Provided
IES Funded: Yes
Entry Date: 20260101
Αριθμός Καταχώρησης: ED682079
Βάση Δεδομένων: ERIC
FullText Text:
  Availability: 0
CustomLinks:
  – Url: https://eric.ed.gov/contentdelivery/servlet/ERICServlet?accno=ED682079
    Name: ERIC Full Text
    Category: fullText
    Text: Full Text from ERIC
Header DbId: eric
DbLabel: ERIC
An: ED682079
AccessLevel: 3
PubType: Report
PubTypeId: report
PreciseRelevancyScore: 0
IllustrationInfo
Items – Name: Title
  Label: Title
  Group: Ti
  Data: An Applied Researchers' Guide to Estimating Effects from Blocked Cluster Randomized Trials: Estimands, Estimators, and Estimates. EdWorkingPaper No. 26-1537
– Name: Language
  Label: Language
  Group: Lang
  Data: English
– Name: Author
  Label: Authors
  Group: Au
  Data: <searchLink fieldCode="AR" term="%22Luke+Miratrix%22">Luke Miratrix</searchLink><br /><searchLink fieldCode="AR" term="%22Michael+J%2E+Weiss%22">Michael J. Weiss</searchLink><br /><searchLink fieldCode="AR" term="%22Sophie+Litschwartz%22">Sophie Litschwartz</searchLink><br /><searchLink fieldCode="AR" term="%22Colin+Hill%22">Colin Hill</searchLink><br /><searchLink fieldCode="AR" term="%22Kayla+Warner%22">Kayla Warner</searchLink><br /><searchLink fieldCode="AR" term="%22Annenberg+Institute+for+School+Reform+at+Brown+University%22">Annenberg Institute for School Reform at Brown University</searchLink>
– Name: TitleSource
  Label: Source
  Group: Src
  Data: <searchLink fieldCode="SO" term="%22Annenberg+Institute+for+School+Reform+at+Brown+University%22"><i>Annenberg Institute for School Reform at Brown University</i></searchLink>. 2026.
– Name: Avail
  Label: Availability
  Group: Avail
  Data: Annenberg Institute for School Reform at Brown University. Brown University Box 1985, Providence, RI 02912. Tel: 401-863-7990; Fax: 401-863-1290; e-mail: annenberg@brown.edu; Web site: https://annenberg.brown.edu/
– Name: PeerReviewed
  Label: Peer Reviewed
  Group: SrcInfo
  Data: N
– Name: Pages
  Label: Page Count
  Group: Src
  Data: 128
– Name: DatePubCY
  Label: Publication Date
  Group: Date
  Data: 2026
– Name: SourceSuprt
  Label: Sponsoring Agency
  Group: SrcSuprt
  Data: Institute of Education Sciences (ED)
– Name: NumberContract
  Label: Contract Number
  Group: NumCntrct
  Data: R305D220046
– Name: Audience
  Label: Intended Audience
  Group: Audnce
  Data: Researchers
– Name: TypeDocument
  Label: Document Type
  Group: TypDoc
  Data: Reports - Research
– Name: Subject
  Label: Descriptors
  Group: Su
  Data: <searchLink fieldCode="DE" term="%22Computation%22">Computation</searchLink><br /><searchLink fieldCode="DE" term="%22Randomized+Controlled+Trials%22">Randomized Controlled Trials</searchLink><br /><searchLink fieldCode="DE" term="%22Error+of+Measurement%22">Error of Measurement</searchLink><br /><searchLink fieldCode="DE" term="%22Effect+Size%22">Effect Size</searchLink><br /><searchLink fieldCode="DE" term="%22Regression+%28Statistics%29%22">Regression (Statistics)</searchLink><br /><searchLink fieldCode="DE" term="%22Hierarchical+Linear+Modeling%22">Hierarchical Linear Modeling</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+Analysis%22">Statistical Analysis</searchLink><br /><searchLink fieldCode="DE" term="%22Selection%22">Selection</searchLink><br /><searchLink fieldCode="DE" term="%22Statistical+Bias%22">Statistical Bias</searchLink>
– Name: Abstract
  Label: Abstract
  Group: Ab
  Data: Blocked cluster randomized trials (CRTs) are widely used to evaluate educational interventions. In such trials, researchers face critical choices about how to define and estimate the average treatment effect (ATE). These choices--both the estimand (e.g., person- vs. cluster-weighted) and the estimator (e.g., regression models, multilevel models)--can influence study conclusions. This paper provides an applied guide to estimands and estimators for blocked CRTs and empirically examines their performance using 26 large-scale blocked CRTs. We estimate ATEs and standard errors for 50 outcomes using 19 estimators and compare results across and within estimands. Findings show that estimator choice can substantially affect point estimates and precision: ranges in ATE estimates exceed 0.10 SD in 18% of cases, and standard error estimates differ by more than 50% in 16% of cases. Some estimators fail or produce unstable results in common real-world designs. These findings underscore the importance of pre-specifying estimation strategies in a public pre-analysis plan. We provide practical guidance for researchers planning and analyzing blocked CRTs.
– Name: AbstractInfo
  Label: Abstractor
  Group: Ab
  Data: As Provided
– Name: CodeSource
  Label: IES Funded
  Group: SrcInfo
  Data: Yes
– Name: DateEntry
  Label: Entry Date
  Group: Date
  Data: 20260101
– Name: AN
  Label: Accession Number
  Group: ID
  Data: ED682079
PLink https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=eric&AN=ED682079
RecordInfo BibRecord:
  BibEntity:
    Languages:
      – Text: English
    PhysicalDescription:
      Pagination:
        PageCount: 128
    Subjects:
      – SubjectFull: Computation
        Type: general
      – SubjectFull: Randomized Controlled Trials
        Type: general
      – SubjectFull: Error of Measurement
        Type: general
      – SubjectFull: Effect Size
        Type: general
      – SubjectFull: Regression (Statistics)
        Type: general
      – SubjectFull: Hierarchical Linear Modeling
        Type: general
      – SubjectFull: Statistical Analysis
        Type: general
      – SubjectFull: Selection
        Type: general
      – SubjectFull: Statistical Bias
        Type: general
    Titles:
      – TitleFull: An Applied Researchers' Guide to Estimating Effects from Blocked Cluster Randomized Trials: Estimands, Estimators, and Estimates. EdWorkingPaper No. 26-1537
        Type: main
  BibRelationships:
    HasContributorRelationships:
      – PersonEntity:
          Name:
            NameFull: Annenberg Institute for School Reform at Brown University
      – PersonEntity:
          Name:
            NameFull: Luke Miratrix
      – PersonEntity:
          Name:
            NameFull: Michael J. Weiss
      – PersonEntity:
          Name:
            NameFull: Sophie Litschwartz
      – PersonEntity:
          Name:
            NameFull: Colin Hill
      – PersonEntity:
          Name:
            NameFull: Kayla Warner
    IsPartOfRelationships:
      – BibEntity:
          Dates:
            – D: 01
              M: 07
              Type: published
              Y: 2026
          Titles:
            – TitleFull: Annenberg Institute for School Reform at Brown University
              Type: main
ResultId 1