A metric of knowledge as information compression reflects reproducibility predictions for biomedical experiments

Fanelli, Daniele ORCID: 0000-0003-1780-1958, Tan, Pedro Batista, Amaral, Olavo B. and Neves, Kleber (2025) A metric of knowledge as information compression reflects reproducibility predictions for biomedical experiments. Royal Society Open Science, 12 (7). ISSN 2054-5703

Text - Published Version Available under License Creative Commons Attribution. Download (1MB)

Abstract

Forecasting the reproducibility of research findings is one of the key challenges of metascience. Above-chance predictions have mainly been achieved by pooling the subjective ratings of experts, and how these predictions are formed remains to be understood. Here, we show that reproducibility forecasts made for the Brazilian Reproducibility Initiative (BRI), a large-scale replication of experiments in the life sciences, are significantly correlated with K, a principled metric of knowledge as information compression. For each study in the BRI sample, we calculated K by dividing the effect size, measured in bits of Shannon entropy, by the descriptive length (a proxy of the complexity) of the study’s methodology, calculated as the optimal Shannon encoding of a conceptual graph representing the replication protocol. We found that experts’ predictions about reproducibility were statistically associated with K values and with the complexity of protocols. This relation was robust to controlling for study methodology and other possible confounding factors. These results suggest that expert raters partially judge the reproducibility of findings by assessing the ratio between the information yielded and the information required by a study, and they support the hypothesis that scientific knowledge may be understood and studied through the lenses of information compression.

Item Type:	Article
Additional Information:	© 2025 The Authors
Divisions:	Methodology
Subjects:	H Social Sciences > H Social Sciences (General) Q Science > Q Science (General)
Date Deposited:	10 Jul 2025 11:48
Last Modified:	10 Jul 2025 11:48
URI:	http://eprints.lse.ac.uk/id/eprint/128767

Actions (login required)

View Item