Academic Journal
Comparative Effectiveness Research Using Randomized Trials and Observational Studies: Validity and Feasibility Considerations.
| Title: | Comparative Effectiveness Research Using Randomized Trials and Observational Studies: Validity and Feasibility Considerations. |
|---|---|
| Authors: | Bikdeli B; Division of Cardiovascular Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States.; Thrombosis Research Group, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States.; YNHH/Yale Center for Outcomes Research and Evaluation (CORE), New Haven, Connecticut, United States., Ross JS; YNHH/Yale Center for Outcomes Research and Evaluation (CORE), New Haven, Connecticut, United States.; Section of General Internal Medicine, Department of Internal Medicine, Yale School of Medicine, New Haven, Connecticut, United States., Bukhari S; Department of Cardiology, Johns Hopkins University, Baltimore, Maryland, United States., Jeffery MM; Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, Minnesota, United States.; Division of Health Care Delivery Research, Mayo Clinic, Rochester, Minnesota, United States.; Department of Emergency Medicine, Mayo Clinic, Rochester, Minnesota, United States., Lip GYH; Liverpool Centre for Cardiovascular Science at University of Liverpool, Liverpool John Moores University and Liverpool Heart and Chest Hospital, Liverpool, United Kingdom.; Department of Clinical Medicine, Aalborg University, Aalborg, Denmark., You SC; Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, Republic of Korea., Cohen DJ; Cardiovascular Research Foundation, New York, New York, United States.; St Francis Hospital and Heart Center, Roslyn, New York, United States., Januzzi JL Jr; Division of Cardiology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, United States.; Baim Institute for Clinical Research, Boston, Massachusetts, United States., Wallach JD; Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, Georgia, United States. |
| Source: | Thrombosis and haemostasis [Thromb Haemost] 2026 Apr; Vol. 126 (4), pp. 335-345. Date of Electronic Publication: 2025 Jul 24. |
| Publication Type: | Journal Article; Review |
| Language: | English |
| Journal Info: | Publisher: Thieme Country of Publication: Germany NLM ID: 7608063 Publication Model: Print-Electronic Cited Medium: Internet ISSN: 2567-689X (Electronic) Linking ISSN: 03406245 NLM ISO Abbreviation: Thromb Haemost Subsets: MEDLINE |
| Imprint Name(s): | Publication: 2018- : Stuttgart : Thieme Original Publication: Stuttgart, Schattauer. |
| MeSH Terms: | Observational Studies as Topic*/methods , Observational Studies as Topic*/standards , Randomized Controlled Trials as Topic*/methods , Randomized Controlled Trials as Topic*/standards , Comparative Effectiveness Research*, Humans ; Reproducibility of Results ; Feasibility Studies ; Research Design ; Treatment Outcome |
| Abstract: | In comparative effectiveness research (CER), ensuring internal, construct, and external validity is crucial. Internal validity determines whether observed outcomes are causally linked to an intervention; construct validity assesses whether a study measures what it intends to; and external validity relates to generalizability in routine practice. Double-blind randomized trials optimize internal validity by minimizing bias and confounding, while construct validity is strengthened through pre-specified protocols and standardized data collection. However, controlled conditions limit external validity. Pragmatic RCTs improve generalizability but may compromise internal validity due to open-label designs. Observational CER studies-including observational studies following the target trial emulation framework-offer broader external validity and feasibility in less time and at lower cost. However, due to lack of random assignment, these studies are susceptible to measured and unmeasured confounding. Several techniques help mitigate these concerns, including a detailed pre-specified protocol, tools such as propensity score matching to balance measured confounders, falsification endpoint testing for assessing the presence of unmeasured confounders, and quasi-experimental designs (including instrumental variable analysis), which may be able to address both. Pre-specified sensitivity analyses and triangulation with complementary data sources further enhance robustness. Construct validity in observational CER depends on accurate patient profiling and validated computational phenotypes for identifying patients, exposures, and outcomes. Thoughtful study design and analytic rigor are essential for balancing these validity considerations. This brief review highlights these issues with examples from thrombosis research. (Thieme. All rights reserved.) |
| Competing Interests: | Outside the submitted work, B.B. was supported by a Career Development Award from the American Heart Association and VIVA Physicians (#938814). B.B. was supported by the Scott Schoen and Nancy Adams IGNITE Award and is supported by the Mary Ann Tynan Research Scientist award from the Mary Horrigan Connors Center for Women's Health and Gender Biology at Brigham and Women's Hospital, and was supported by the Heart and Vascular Center Junior Faculty Award from Brigham and Women's Hospital. B.B. reports that he was a consulting expert, on behalf of the plaintiff, for litigation related to two specific brand models of IVC filters. B.B. has not been involved in the litigation from 2022 to 2025 nor has he received any compensation from 2022 to 2025. B.B. reports that he is a member of the Medical Advisory Board for the VascuLearn Network, and serves in the Data Safety and Monitory Board of the NAIL-IT trial funded by the National Heart, Lung, and Blood Institute, and Translational Sciences. B.B. is a collaborating consultant with the International Consulting Associates and the US Food and Drug Administration in a study to generate knowledge about utilization, predictors, retrieval, and safety of IVC filters. B.B. receives compensation as an Associated Editor for the New England Journal of Medicine Journal Watch Cardiology, as an Associate Editor for Thrombosis Research, and as an Executive Associate Editor for JACC, and is a Section Editor for Thrombosis and Haemostasis (no compensation). S.C.Y. reports grants from Daiichi Sankyo. He is a coinventor of granted Korea Patent DP-2023–1223 and DP-2023–0920, and pending Patent Applications DP-2024–0909, DP-2024–0908, DP-2022–1658, DP-2022–1478, DP-2022–1365, PATENT-2025–0039190, PATENT-2025–0039191, PATENT-2025–0039192, PATENT-2025–0039193, and PATENT-2025–0039194 unrelated to current work. S.C.Y. is a chief executive officer of PHI Digital Healthcare. D.J.C. reports institutional research grant support from Edwards Lifesciences, Boston Scientific, Abbott, Medtronic, Corvia, Cathworks, Philips, Zoll Medical, I-Rhythm, JenaValve, and ANCORA as well as consulting income from Medtronic, Edwards Lifesciences, Boston Scientific, Abbott, Zoll Medical, and Elixir Medical. J.L.J. is supported in part by the Adolph Hutter Professorship at Harvard Medical School. J.L.L. reports a board position with Imbria Pharma and equity in Jana Care. He is a Deputy Editor at JACC and receives current/recent grant support from Abbott, AstraZeneca, BMS, HeartFlow, and Novartis Pharmaceuticals, consulting income from Abbott Diagnostics, AstraZeneca, Beckman-Coulter, Boehringer Ingelheim, Eli Lilly, Janssen, Novartis, Prevencio, Quidel, and Roche Diagnostics, and serves on clinical endpoint committees/data safety monitoring boards for Abbott, AbbVie, Amgen, CVRx, Medtronic, Pfizer, and Roche Diagnostics. J.S.R. currently receives research support through Yale University from Johnson and Johnson to develop methods of clinical trial data sharing, from the Food and Drug Administration for the Yale-Mayo Clinic Center for Excellence in Regulatory Science and Innovation (CERSI) program (U01FD005938), from the Agency for Healthcare Research and Quality (R01HS022882), and from Arnold Ventures; formerly received research support from the Medical Device Innovation Consortium as part of the National Evaluation System for Health Technology (NEST) and from the National Heart, Lung and Blood Institute of the National Institutes of Health (NIH) (R01HS025164, R01HL144644); and in addition, J.S.R. was an expert witness at the request of Relator's attorneys, the Greene Law Firm, in a qui tam suit alleging violations of the False Claims Act and Anti-Kickback Statute against Biogen Inc. that was settled September 2022. J.D.W. reported receiving grants from the National Institute on Alcohol Abuse and Alcoholism of the National Institutes of Health (under award 1K01AA028258), Johnson & Johnson (through the Yale Open Data Access Project), Arnold Ventures, and the FDA, as well as former consulting fees from Hagen Berman Sobol Shapiro LLP and Dugan Law Firm APLC outside the submitted work. M.M.J. has received unrelated funding from National Institute on Drug Abuse (NIDA), the United States Food and Drug Administration (FDA), and the Agency for Healthcare Research and Quality (AHRQ). |
| Entry Date(s): | Date Created: 20250728 Date Completed: 20260323 Latest Revision: 20260323 |
| Update Code: | 20260324 |
| DOI: | 10.1055/a-2664-7887 |
| PMID: | 40719151 |
| Database: | MEDLINE |
| FullText | Text: Availability: 0 |
|---|---|
| Header | DbId: cmedm DbLabel: MEDLINE An: 40719151 AccessLevel: 3 PubType: Academic Journal PubTypeId: academicJournal PreciseRelevancyScore: 0 |
| IllustrationInfo | |
| Items | – Name: Title Label: Title Group: Ti Data: Comparative Effectiveness Research Using Randomized Trials and Observational Studies: Validity and Feasibility Considerations. – Name: Author Label: Authors Group: Au Data: <searchLink fieldCode="AU" term="%22Bikdeli+B%22">Bikdeli B</searchLink>; Division of Cardiovascular Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States.; Thrombosis Research Group, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts, United States.; YNHH/Yale Center for Outcomes Research and Evaluation (CORE), New Haven, Connecticut, United States.<br /><searchLink fieldCode="AU" term="%22Ross+JS%22">Ross JS</searchLink>; YNHH/Yale Center for Outcomes Research and Evaluation (CORE), New Haven, Connecticut, United States.; Section of General Internal Medicine, Department of Internal Medicine, Yale School of Medicine, New Haven, Connecticut, United States.<br /><searchLink fieldCode="AU" term="%22Bukhari+S%22">Bukhari S</searchLink>; Department of Cardiology, Johns Hopkins University, Baltimore, Maryland, United States.<br /><searchLink fieldCode="AU" term="%22Jeffery+MM%22">Jeffery MM</searchLink>; Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, Minnesota, United States.; Division of Health Care Delivery Research, Mayo Clinic, Rochester, Minnesota, United States.; Department of Emergency Medicine, Mayo Clinic, Rochester, Minnesota, United States.<br /><searchLink fieldCode="AU" term="%22Lip+GYH%22">Lip GYH</searchLink>; Liverpool Centre for Cardiovascular Science at University of Liverpool, Liverpool John Moores University and Liverpool Heart and Chest Hospital, Liverpool, United Kingdom.; Department of Clinical Medicine, Aalborg University, Aalborg, Denmark.<br /><searchLink fieldCode="AU" term="%22You+SC%22">You SC</searchLink>; Department of Biomedical Systems Informatics, Yonsei University College of Medicine, Seoul, Republic of Korea.<br /><searchLink fieldCode="AU" term="%22Cohen+DJ%22">Cohen DJ</searchLink>; Cardiovascular Research Foundation, New York, New York, United States.; St Francis Hospital and Heart Center, Roslyn, New York, United States.<br /><searchLink fieldCode="AU" term="%22Januzzi+JL+Jr%22">Januzzi JL Jr</searchLink>; Division of Cardiology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, United States.; Baim Institute for Clinical Research, Boston, Massachusetts, United States.<br /><searchLink fieldCode="AU" term="%22Wallach+JD%22">Wallach JD</searchLink>; Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta, Georgia, United States. – Name: TitleSource Label: Source Group: Src Data: <searchLink fieldCode="JN" term="%227608063%22">Thrombosis and haemostasis</searchLink> [Thromb Haemost] 2026 Apr; Vol. 126 (4), pp. 335-345. <i>Date of Electronic Publication: </i>2025 Jul 24. – Name: TypePub Label: Publication Type Group: TypPub Data: Journal Article; Review – Name: Language Label: Language Group: Lang Data: English – Name: TitleSource Label: Journal Info Group: Src Data: <i>Publisher: </i><searchLink fieldCode="PB" term="%22Thieme%22">Thieme </searchLink><i>Country of Publication: </i>Germany <i>NLM ID: </i>7608063 <i>Publication Model: </i>Print-Electronic <i>Cited Medium: </i>Internet <i>ISSN: </i>2567-689X (Electronic) <i>Linking ISSN: </i><searchLink fieldCode="IS" term="%2203406245%22">03406245 </searchLink><i>NLM ISO Abbreviation: </i>Thromb Haemost <i>Subsets: </i>MEDLINE – Name: PublisherInfo Label: Imprint Name(s) Group: PubInfo Data: <i>Publication</i>: 2018- : Stuttgart : Thieme<br /><i>Original Publication</i>: Stuttgart, Schattauer. – Name: SubjectMESH Label: MeSH Terms Group: Su Data: <searchLink fieldCode="MM" term="%22Observational+Studies+as+Topic%22">Observational Studies as Topic*</searchLink>/<searchLink fieldCode="MM" term="%22Observational+Studies+as+Topic+methods%22">methods</searchLink> <br /><searchLink fieldCode="MM" term="%22Observational+Studies+as+Topic%22">Observational Studies as Topic*</searchLink>/<searchLink fieldCode="MM" term="%22Observational+Studies+as+Topic+standards%22">standards</searchLink> <br /><searchLink fieldCode="MM" term="%22Randomized+Controlled+Trials+as+Topic%22">Randomized Controlled Trials as Topic*</searchLink>/<searchLink fieldCode="MM" term="%22Randomized+Controlled+Trials+as+Topic+methods%22">methods</searchLink> <br /><searchLink fieldCode="MM" term="%22Randomized+Controlled+Trials+as+Topic%22">Randomized Controlled Trials as Topic*</searchLink>/<searchLink fieldCode="MM" term="%22Randomized+Controlled+Trials+as+Topic+standards%22">standards</searchLink> <br /><searchLink fieldCode="MM" term="%22Comparative+Effectiveness+Research%22">Comparative Effectiveness Research*</searchLink><br /><searchLink fieldCode="MH" term="%22Humans%22">Humans</searchLink> ; <searchLink fieldCode="MH" term="%22Reproducibility+of+Results%22">Reproducibility of Results</searchLink> ; <searchLink fieldCode="MH" term="%22Feasibility+Studies%22">Feasibility Studies</searchLink> ; <searchLink fieldCode="MH" term="%22Research+Design%22">Research Design</searchLink> ; <searchLink fieldCode="MH" term="%22Treatment+Outcome%22">Treatment Outcome</searchLink> – Name: Abstract Label: Abstract Group: Ab Data: In comparative effectiveness research (CER), ensuring internal, construct, and external validity is crucial. Internal validity determines whether observed outcomes are causally linked to an intervention; construct validity assesses whether a study measures what it intends to; and external validity relates to generalizability in routine practice. Double-blind randomized trials optimize internal validity by minimizing bias and confounding, while construct validity is strengthened through pre-specified protocols and standardized data collection. However, controlled conditions limit external validity. Pragmatic RCTs improve generalizability but may compromise internal validity due to open-label designs. Observational CER studies-including observational studies following the target trial emulation framework-offer broader external validity and feasibility in less time and at lower cost. However, due to lack of random assignment, these studies are susceptible to measured and unmeasured confounding. Several techniques help mitigate these concerns, including a detailed pre-specified protocol, tools such as propensity score matching to balance measured confounders, falsification endpoint testing for assessing the presence of unmeasured confounders, and quasi-experimental designs (including instrumental variable analysis), which may be able to address both. Pre-specified sensitivity analyses and triangulation with complementary data sources further enhance robustness. Construct validity in observational CER depends on accurate patient profiling and validated computational phenotypes for identifying patients, exposures, and outcomes. Thoughtful study design and analytic rigor are essential for balancing these validity considerations. This brief review highlights these issues with examples from thrombosis research.<br /> (Thieme. All rights reserved.) – Name: Abstract Label: Competing Interests Group: Ab Data: Outside the submitted work, B.B. was supported by a Career Development Award from the American Heart Association and VIVA Physicians (#938814). B.B. was supported by the Scott Schoen and Nancy Adams IGNITE Award and is supported by the Mary Ann Tynan Research Scientist award from the Mary Horrigan Connors Center for Women's Health and Gender Biology at Brigham and Women's Hospital, and was supported by the Heart and Vascular Center Junior Faculty Award from Brigham and Women's Hospital. B.B. reports that he was a consulting expert, on behalf of the plaintiff, for litigation related to two specific brand models of IVC filters. B.B. has not been involved in the litigation from 2022 to 2025 nor has he received any compensation from 2022 to 2025. B.B. reports that he is a member of the Medical Advisory Board for the VascuLearn Network, and serves in the Data Safety and Monitory Board of the NAIL-IT trial funded by the National Heart, Lung, and Blood Institute, and Translational Sciences. B.B. is a collaborating consultant with the International Consulting Associates and the US Food and Drug Administration in a study to generate knowledge about utilization, predictors, retrieval, and safety of IVC filters. B.B. receives compensation as an Associated Editor for the New England Journal of Medicine Journal Watch Cardiology, as an Associate Editor for Thrombosis Research, and as an Executive Associate Editor for JACC, and is a Section Editor for Thrombosis and Haemostasis (no compensation). S.C.Y. reports grants from Daiichi Sankyo. He is a coinventor of granted Korea Patent DP-2023–1223 and DP-2023–0920, and pending Patent Applications DP-2024–0909, DP-2024–0908, DP-2022–1658, DP-2022–1478, DP-2022–1365, PATENT-2025–0039190, PATENT-2025–0039191, PATENT-2025–0039192, PATENT-2025–0039193, and PATENT-2025–0039194 unrelated to current work. S.C.Y. is a chief executive officer of PHI Digital Healthcare. D.J.C. reports institutional research grant support from Edwards Lifesciences, Boston Scientific, Abbott, Medtronic, Corvia, Cathworks, Philips, Zoll Medical, I-Rhythm, JenaValve, and ANCORA as well as consulting income from Medtronic, Edwards Lifesciences, Boston Scientific, Abbott, Zoll Medical, and Elixir Medical. J.L.J. is supported in part by the Adolph Hutter Professorship at Harvard Medical School. J.L.L. reports a board position with Imbria Pharma and equity in Jana Care. He is a Deputy Editor at JACC and receives current/recent grant support from Abbott, AstraZeneca, BMS, HeartFlow, and Novartis Pharmaceuticals, consulting income from Abbott Diagnostics, AstraZeneca, Beckman-Coulter, Boehringer Ingelheim, Eli Lilly, Janssen, Novartis, Prevencio, Quidel, and Roche Diagnostics, and serves on clinical endpoint committees/data safety monitoring boards for Abbott, AbbVie, Amgen, CVRx, Medtronic, Pfizer, and Roche Diagnostics. J.S.R. currently receives research support through Yale University from Johnson and Johnson to develop methods of clinical trial data sharing, from the Food and Drug Administration for the Yale-Mayo Clinic Center for Excellence in Regulatory Science and Innovation (CERSI) program (U01FD005938), from the Agency for Healthcare Research and Quality (R01HS022882), and from Arnold Ventures; formerly received research support from the Medical Device Innovation Consortium as part of the National Evaluation System for Health Technology (NEST) and from the National Heart, Lung and Blood Institute of the National Institutes of Health (NIH) (R01HS025164, R01HL144644); and in addition, J.S.R. was an expert witness at the request of Relator's attorneys, the Greene Law Firm, in a qui tam suit alleging violations of the False Claims Act and Anti-Kickback Statute against Biogen Inc. that was settled September 2022. J.D.W. reported receiving grants from the National Institute on Alcohol Abuse and Alcoholism of the National Institutes of Health (under award 1K01AA028258), Johnson & Johnson (through the Yale Open Data Access Project), Arnold Ventures, and the FDA, as well as former consulting fees from Hagen Berman Sobol Shapiro LLP and Dugan Law Firm APLC outside the submitted work. M.M.J. has received unrelated funding from National Institute on Drug Abuse (NIDA), the United States Food and Drug Administration (FDA), and the Agency for Healthcare Research and Quality (AHRQ). – Name: DateEntry Label: Entry Date(s) Group: Date Data: <i>Date Created: </i>20250728 <i>Date Completed: </i>20260323 <i>Latest Revision: </i>20260323 – Name: DateUpdate Label: Update Code Group: Date Data: 20260324 – Name: DOI Label: DOI Group: ID Data: 10.1055/a-2664-7887 – Name: AN Label: PMID Group: ID Data: 40719151 |
| PLink | https://search.ebscohost.com/login.aspx?direct=true&site=eds-live&db=cmedm&AN=40719151 |
| RecordInfo | BibRecord: BibEntity: Identifiers: – Type: doi Value: 10.1055/a-2664-7887 Languages: – Code: eng Text: English PhysicalDescription: Pagination: StartPage: 335 Subjects: – SubjectFull: Humans Type: general – SubjectFull: Reproducibility of Results Type: general – SubjectFull: Feasibility Studies Type: general – SubjectFull: Research Design Type: general – SubjectFull: Treatment Outcome Type: general – SubjectFull: Observational Studies as Topic methods Type: general – SubjectFull: Observational Studies as Topic standards Type: general – SubjectFull: Randomized Controlled Trials as Topic methods Type: general – SubjectFull: Randomized Controlled Trials as Topic standards Type: general – SubjectFull: Comparative Effectiveness Research Type: general Titles: – TitleFull: Comparative Effectiveness Research Using Randomized Trials and Observational Studies: Validity and Feasibility Considerations. Type: main BibRelationships: HasContributorRelationships: – PersonEntity: Name: NameFull: Bikdeli B – PersonEntity: Name: NameFull: Ross JS – PersonEntity: Name: NameFull: Bukhari S – PersonEntity: Name: NameFull: Jeffery MM – PersonEntity: Name: NameFull: Lip GYH – PersonEntity: Name: NameFull: You SC – PersonEntity: Name: NameFull: Cohen DJ – PersonEntity: Name: NameFull: Januzzi JL Jr – PersonEntity: Name: NameFull: Wallach JD IsPartOfRelationships: – BibEntity: Dates: – D: 01 M: 04 Text: 2026 Apr Type: published Y: 2026 Identifiers: – Type: issn-electronic Value: 2567-689X Numbering: – Type: volume Value: 126 – Type: issue Value: 4 Titles: – TitleFull: Thrombosis and haemostasis Type: main |
| ResultId | 1 |