Pre-analysis plans: an early stocktaking

Ofosu, George K. ORCID: 0000-0003-4112-0749 and Posner, Daniel N. (2023) Pre-analysis plans: an early stocktaking. Perspectives on Politics, 21 (1). 174 - 190. ISSN 1537-5927

Text (pre-analysis-plans-an-early-stocktaking) - Published Version Available under License Creative Commons Attribution. Download (546kB)

• Author
• Scopus publication

Abstract

Pre-analysis plans (PAPs) have been championed as a solution to the problem of research credibility, but without any evidence that PAPs actually bolster the credibility of research. We analyze a representative sample of 195 PAPs registered on the Evidence in Governance and Politics (EGAP) and American Economic Association (AEA) registration platforms to assess whether PAPs registered in the early days of pre-registration (2011–2016) were sufficiently clear, precise, and comprehensive to achieve their objective of preventing “fishing” and reducing the scope for post-hoc adjustment of research hypotheses. We also analyze a subset of ninety-three PAPs from projects that resulted in publicly available papers to ascertain how faithfully they adhere to their preregistered specifications and hypotheses. We find significant variation in the extent to which PAPs registered during this period accomplished the goals they were designed to achieve. We discuss these findings in light of both the costs and benefits of preregistration, showing how our results speak to the various arguments that have been made in support of and against PAPs. We also highlight the norms and institutions that will need to be strengthened to augment the power of PAPs to improve research credibility and to create incentives for researchers to invest in both producing and policing them.

Item Type:	Article
Official URL:	https://www.cambridge.org/core/journals/perspectiv...
Additional Information:	© 2021 The Authors
Divisions:	Government
Subjects:	H Social Sciences > H Social Sciences (General) J Political Science
Date Deposited:	26 Nov 2021 15:39
Last Modified:	30 Sep 2024 04:21
URI:	http://eprints.lse.ac.uk/id/eprint/112747

Actions (login required)

View Item