Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics

Bartoš, František, Maier, Maximilian, Wagenmakers, Eric Jan, Nippold, Franziska, Doucouliagos, Hristos, Ioannidis, John P.A., Otte, Willem M., Sladekova, Martina, Deresssa, Teshome K., Bruns, Stephan B., Fanelli, Daniele ORCID: 0000-0003-1780-1958 and Stanley, T. D. (2024) Footprint of publication selection bias on meta-analyses in medicine, environmental sciences, psychology, and economics. Research Synthesis Methods, 15 (3). 500 - 511. ISSN 1759-2887

Text (Fanelli_footprint-of-publication--published) - Published Version Available under License Creative Commons Attribution. Download (1MB)

• Scopus publication
• Author

Abstract

Publication selection bias undermines the systematic accumulation of evidence. To assess the extent of this problem, we survey over 68,000 meta-analyses containing over 700,000 effect size estimates from medicine (67,386/597,699), environmental sciences (199/12,707), psychology (605/23,563), and economics (327/91,421). Our results indicate that meta-analyses in economics are the most severely contaminated by publication selection bias, closely followed by meta-analyses in environmental sciences and psychology, whereas meta-analyses in medicine are contaminated the least. After adjusting for publication selection bias, the median probability of the presence of an effect decreased from 99.9% to 29.7% in economics, from 98.9% to 55.7% in psychology, from 99.8% to 70.7% in environmental sciences, and from 38.0% to 29.7% in medicine. The median absolute effect sizes (in terms of standardized mean differences) decreased from d = 0.20 to d = 0.07 in economics, from d = 0.37 to d = 0.26 in psychology, from d = 0.62 to d = 0.43 in environmental sciences, and from d = 0.24 to d = 0.13 in medicine.

Item Type:	Article
Official URL:	https://onlinelibrary.wiley.com/journal/17592887
Additional Information:	© 2024 The Authors
Divisions:	Methodology
Subjects:	Z Bibliography. Library Science. Information Resources > Z665 Library Science. Information Science
Date Deposited:	26 Feb 2024 10:45
Last Modified:	15 Nov 2024 03:51
URI:	http://eprints.lse.ac.uk/id/eprint/122107

Actions (login required)

View Item