A Strong Case for Rethinking Causal Inference

In this invited commentary, John Deke discusses two methodological studies examining the mistakes that arise from the misuse of statistical significance in the field of education. He argues that Simpson (2022) and Sims et al (2022) ably contribute to our growing understanding of inferential errors that arise when filtering research findings using statistical significance. Drawing on the 'Type M' (magnitude) and 'Type S' (sign) errors described by Gelman and Carlin (2014), both articles demonstrate that large Type M errors almost certainly exist when applying the statistical significance filter in the field of education. Deke discusses the pros and cons of four of the more actionable suggestions offered by the authors of both articles in their discussion sections. He also provides his own recommendations for drawing inferences about causal relationships that will empower researchers and decision makers to more productively use evidence while avoiding the types of inferential mistakes examined by Sims et al (2022) and Simpson (2022). Specifically, he recommends researchers use the BASIE framework to interpret impact estimates from evaluations.