Methodological concerns underlying a lack of evidence for cultural heterogeneity in the replication of psychological effects

Schimmelpfennig, Robin, Spicer, Rachel ORCID: 0000-0002-2807-8796, White, Cindel J. M., Gervais, Will, Norenzayan, Ara, Heine, Steven J., Henrich, Joseph and Muthukrishna, Michael ORCID: 0000-0002-7079-5166 (2024) Methodological concerns underlying a lack of evidence for cultural heterogeneity in the replication of psychological effects. Communications Psychology. ISSN 2731-9121 (In Press)

Text (commsPsych_ML2_preprint) - Accepted Version Pending embargo until 1 January 2100. Download (824kB) | Request a copy

Abstract

The multi-site replication study, Many Labs 2, concluded that sample location and setting did not substantially affect the replicability of findings. Here, we examine theoretical and methodological considerations for a subset of the analyses, namely exploratory tests of heterogeneity in the replicability of studies between “WEIRD and less-WEIRD cultures”. We conducted a review of literature citing the study, a re-examination of the existing cultural variability, a power stimulation for detecting cultural heterogeneity, and re-analyses of the original exploratory tests. Findings indicate little cultural variability and low power to detect cultural heterogeneity effects in the Many Labs 2 data, yet the literature review indicates the study is cited regarding the moderating role of culture. Our reanalysis of the data found that using different operationalizations of culture slightly increased effect sizes but did not substantially alter the conclusions of Many Labs 2. Future studies of cultural heterogeneity can be improved with theoretical consideration of which effects and which cultures are likely to show variation as well as a priori methodological planning for appropriate operationalizations of culture and sufficient power to detect effects.

Item Type:	Article
Additional Information:	© 2024 Springer Nature Limited
Divisions:	Psychological and Behavioural Science
Subjects:	H Social Sciences B Philosophy. Psychology. Religion > BF Psychology
Date Deposited:	17 Sep 2024 08:36
Last Modified:	17 Sep 2024 08:42
URI:	http://eprints.lse.ac.uk/id/eprint/125418

Actions (login required)

View Item