Generalizability of heterogeneous treatment effect estimates across samples

Coppock, Alexander, Leeper, Thomas J. and Mullinix, Kevin J. (2018) Generalizability of heterogeneous treatment effect estimates across samples. Proceedings of the National Academy of Sciences of the United States of America, 115 (49). 12441 - 12446. ISSN 0027-8424

Text (Generalizability of heterogeneous treatment effect estimates across samples) - Published Version Download (1MB)

• Scopus publication
• Author

Abstract

The extent to which survey experiments conducted with nonrepresentative convenience samples are generalizable to target populations depends critically on the degree of treatment effect heterogeneity. Recent inquiries have found a strong correspondence between sample average treatment effects estimated in nationally representative experiments and in replication studies conducted with convenience samples. We consider here two possible explanations: low levels of effect heterogeneity or high levels of effect heterogeneity that are unrelated to selection into the convenience sample. We analyze subgroup conditional average treatment effects using 27 original–replication study pairs (encompassing 101,745 individual survey responses) to assess the extent to which subgroup effect estimates generalize. While there are exceptions, the overwhelming pattern that emerges is one of treatment effect homogeneity, providing a partial explanation for strong correspondence across both unconditional and conditional average treatment effect estimates.

Item Type:	Article
Additional Information:	© 2018 National Academy of Sciences
Divisions:	Government
Subjects:	H Social Sciences > H Social Sciences (General)
Date Deposited:	26 Mar 2019 11:27
Last Modified:	15 Nov 2024 06:18
URI:	http://eprints.lse.ac.uk/id/eprint/100355

Actions (login required)

View Item