Assessing the significance of model selection in ecology

Wheatcroft, Edward ORCID: 0000-0002-7301-0889 (2020) Assessing the significance of model selection in ecology. European Journal of Ecology, 6 (2). pp. 87-103. ISSN 1339-8474

Text (document) - Published Version Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (823kB)

• Author

Abstract

Model Selection is a key part of many ecological studies, with Akaike's Information Criterion (AIC) being by far the most commonly used technique for this purpose. Typically, a number of candidate models are defined a priori and ranked according to their expected out-of-sample performance. Model selection, however, only assesses the relative performance of the models and, as pointed out in a recent paper, a large proportion of ecology papers that use model selection do not assess the absolute fit of the 'best' model. In this paper, it is argued that assessing the absolute fit of the 'best' model alone does not go far enough. This is because a model that appears to perform well under model selection is also likely to appear to perform well under measures of absolute fit, even when there is no predictive value. This paper proposes a model selection permutation test that assesses the probability that the model selection statistic of the 'best' model could have occurred by chance alone, whilst taking account of dependencies between the models. It is argued that this test should always be performed as a part of formal model selection. The test is demonstrated on two real population modelling examples of ibex in northern Italy and wild reindeer in Norway. In both cases, the model selection permutation test gives a highly significant result, indicating that the performance of the 'best' model is unlikely to be through chance alone. R code is provided with which to perform the tests.

Item Type:	Article
Additional Information:	© 2020 The Author.
Divisions:	Statistics
Subjects:	H Social Sciences > HA Statistics
Date Deposited:	24 Jun 2022 13:33
Last Modified:	12 Dec 2024 03:05
URI:	http://eprints.lse.ac.uk/id/eprint/115434

Actions (login required)

View Item