Methodological concerns underlying a lack of evidence for cultural heterogeneity in the replication of psychological effects

Schimmelpfennig, Robin, Spicer, Rachel ORCID: 0000-0002-2807-8796, White, Cindel J. M., Gervais, Will, Norenzayan, Ara, Heine, Steven J., Henrich, Joseph and Muthukrishna, Michael ORCID: 0000-0002-7079-5166 (2024) Methodological concerns underlying a lack of evidence for cultural heterogeneity in the replication of psychological effects. Communications Psychology, 2. ISSN 2731-9121

Text (s44271-024-00135-z) - Published Version Available under License Creative Commons Attribution. Download (1MB)

Abstract

The multi-site replication study, Many Labs 2, concluded that sample location and setting did not substantially affect the replicability of findings. Here, we examine theoretical and methodological considerations for a subset of the analyses, namely exploratory tests of heterogeneity in the replicability of studies between “WEIRD and less-WEIRD cultures”. We conducted a review of literature citing the study, a re-examination of the existing cultural variability, a power stimulation for detecting cultural heterogeneity, and re-analyses of the original exploratory tests. Findings indicate little cultural variability and low power to detect cultural heterogeneity effects in the Many Labs 2 data, yet the literature review indicates the study is cited regarding the moderating role of culture. Our reanalysis of the data found that using different operationalizations of culture slightly increased effect sizes but did not substantially alter the conclusions of Many Labs 2. Future studies of cultural heterogeneity can be improved with theoretical consideration of which effects and which cultures are likely to show variation as well as a priori methodological planning for appropriate operationalizations of culture and sufficient power to detect effects.

Item Type:	Article
Additional Information:	© 2024 The Author(s)
Divisions:	Psychological and Behavioural Science
Subjects:	H Social Sciences B Philosophy. Psychology. Religion > BF Psychology
Date Deposited:	17 Sep 2024 08:36
Last Modified:	20 Dec 2024 00:57
URI:	http://eprints.lse.ac.uk/id/eprint/125418

Actions (login required)

View Item