Enhancing Education Impact Studies with the '4R Lenses' Approach

Mathematica is partnering with the Society for Research on Educational Effectiveness (SREE) and the MDRC to share an approach for prioritizing and selecting context features when conducting high-quality impact studies of education interventions to provide the strongest evidence about whether interventions improve academic outcomes. Information about the contexts in which interventions are implemented may provide further insight into impact findings, such as why impacts occurred in some sites but not others. Practitioners also want to know the contexts in which an intervention was studied and assess whether it is a good fit for their situation. Yet the many possible context features to collect, analyze, and report in a study are numerous and are likely to differ from intervention to intervention and from study to study. This leaves researchers with the daunting task of deciding which context features to prioritize.

In this SREE webinar, Carolyn Hill from MDRC and Lauren Scher from Mathematica will describe a general ‘4R lenses’ approach that researchers can draw on to help them make informed decisions about the context features to include in their studies. The approach, which begins during study design, involves identifying and assessing (1) Requirements of interest holders, (2) prior Research, (3) Relevance to the current study settings, and (4) whether the features are Realistic to collect for the current study. The webinar builds on the 2023 Institute of Education Sciences’ guide Conducting Implementation Research in Impact Studies of Education Interventions.