Tips for Planning an Impact Evaluation

The Regional Partnership Grant (RPG) program supports partnerships between providers of child welfare services, substance abuse disorder (SUD) treatment, and other social services that enhance the safety and well-being of children who are in, or at risk of, out-of-home placement because of a parent’s or caretaker’s SUD. The Children’s Bureau in the Administration for Children and Families directs the RPG program, which is a key federal program that the United States Congress authorizes to solely and specifically address the issues that arise at the intersection of child welfare and substance use.

The Children’s Bureau funded this brief for groups that receive RPG grants or other grants and want to conduct an evaluation that can identify the causal impacts of a program designed for families involved with—or at risk of involvement with—the child welfare system. It draws on the experiences of previous cohorts of RPG grantees to give tips about how to plan an impact evaluation. The tips focus on impact evaluation designs in which groups were formed either through a randomized controlled trial (RCT) or a quasi-experimental design (QED). The discussion of these two strategies highlights pitfalls and proposed solutions to them.