How Do You Select The Best Statistical Model?

But sometimes there are problems. In a paper published this week in The British Journal for the Philosophy of Science, Emeritus Curator Scott Lidgard and Beckett Sterner of Arizona State University show how different scientists’ prior beliefs can fail to produce the same result, even for the same data and candidate models. The article, “Objectivity and Underdetermination in Statistical Model Selection,” focuses on a particular kind of underdetermination, the idea that available evidence is insufficient to identify which judgement one should make about that evidence. Various model selection methods weight goodness-of-fit or similar criteria differently when considering the relationships of models to data. Researchers may have assumptions about a given model selection method, or convictions about whether the set of candidate models contain the true distribution. When the available data leave uncertainty about the relation of the candidate models to the true distribution, in many cases scientists attempt to settle the issue by invoking prior beliefs about nature, including general expectations about how their systems of study behave. Scott and Beckett contend that the problem is unlikely to go away soon, because no single model selection method serves equally well for different scientists’ views on the adequacy of the candidate model set.
September 6. 2024