Channeling Fisher: randomization tests and the statistical insignificance of seemingly significant experimental results

Young, Alwyn (2019) Channeling Fisher: randomization tests and the statistical insignificance of seemingly significant experimental results. Quarterly Journal of Economics, 134 (2). 557 - 598. ISSN 0033-5533

Text (ChannellingFisher) - Accepted Version Download (1MB)

• Author

Abstract

I follow R. A. Fisher's The Design of Experiments (1935), using randomization statistical inference to test the null hypothesis of no treatment effects in a comprehensive sample of 53 experimental papers drawn from the journals of the American Economic Association. In the average paper, randomization tests of the significance of individual treatment effects find 13% to 22% fewer significant results than are found using authors’ methods. In joint tests of multiple treatment effects appearing together in tables, randomization tests yield 33% to 49% fewer statistically significant results than conventional tests. Bootstrap and jackknife methods support and confirm the randomization results.

Item Type:	Article
Official URL:	https://academic.oup.com/qje
Additional Information:	© 2018 The Author
Divisions:	Economics
Subjects:	H Social Sciences > HB Economic Theory Q Science > QA Mathematics
JEL classification:	C - Mathematical and Quantitative Methods > C1 - Econometric and Statistical Methods: General > C12 - Hypothesis Testing C - Mathematical and Quantitative Methods > C9 - Design of Experiments > C90 - General
Date Deposited:	19 Aug 2019 15:36
Last Modified:	03 Nov 2025 22:36
URI:	http://eprints.lse.ac.uk/id/eprint/101401

Actions (login required)

View Item