As states step forward to test solutions to urgent social challenges, Colorado leads the way in identifying and improving innovations in its human services programs. At a recent conference in the state, Mathematica showcased our collaborations with policy and program staff to provide insights into developing and testing changes in service delivery and making them stick.
As part of the Rocky Mountain Workforce Development SkillUP! Conference, Mathematica experts highlighted the importance of reimagining the way we gather and use data. They showed how doing so can help policymakers and program administrators see clearly, act quickly, and respond effectively so that science can inform practice, and stakeholders can build on lessons learned to test and make small changes to program operations along the way. Our experts also reviewed practical ways to make research more accessible.
Starting at the beginning
Many state and local agencies want to use evidence to improve their services but don’t know where to start. Using an assessment tool grounded in a review of the literature for improving outcomes for low-income youth, Senior Researcher Annalisa Mastri showed how programs can be more research-based by developing indicators tied to specific outcomes to work toward. The assessment can also serve to rate an ongoing program and flag areas for improvement. Mastri discussed a study in Denver of career and educational support for out-of-school youth and how the assessment can help to rate the use of evidence-informed interventions. The program provides young people and their families with wraparound services to help them achieve their employment, education, and personal aspirations. The services are offered through a partnership between the Office of Economic Development and Denver Public Schools. Read her presentation, “Using Evidence-Informed Practices to Advance Youth Self-Sufficiency and Well-Being.”
Conceptualizing, developing, and road testing
Researchers and policymakers alike struggle to connect evidence to practice and improve programs. Too often, policy initiatives set ambitious goals that might be hard to achieve. Senior Researcher Michelle Derr and Data Analytics Lead Jonathan McCay partnered with a group of Colorado counties to assess the counties’ programs, develop targeted strategies for change, and test and refine new ideas before scaling them up. The process, called Learn, Innovate, Improve, or LI2, can be used with any program to bring a systematic and evidence-informed method to uniting research with practice and achieving lasting program changes. The Learn stage examines the problem and assesses the program’s readiness for change. The Innovate stage develops promising interventions, drawing on the science, existing evidence, and wisdom of practitioners. The Improve stage uses small-scale analytic pilots and techniques such as rapid-cycle evaluation to road test evidence-based ideas. The research team has used LI2 to help more than 15 urban and rural counties increase engagement and improve other outcomes for Colorado Works families. Read their presentation, “Learn, Innovate, Improve (LI2): A New Approach to Using Evidence for Continuous Program Improvement.”
Putting the pieces together
Building on the LI2 process, Derr and McCay worked with practitioners and social scientists to develop a practice model called Goal4It!™ to engage and motivate customers in employment programs and spur their success. Goal4 It!™ is rooted in a three-step process for setting and achieving goals linked to a broader set of strategies for improving child and family outcomes: (1) strengthening core skills, (2) reducing sources of stress, and (3) supporting responsive relationships. Goal4 It!™ centers on teaching people this specific approach to pursuing goals and intentionally building the self-regulation skills necessary to achieve those goals. The process, grounded in behavioral and social science, places customers’ aspirations at the center of goal setting, goal planning, putting plans into action, and reviewing and learning from progress. On the organizational side, Goal4It!™ relies on committed program leaders, a culture of trust and respect, and a flexible service delivery process.
Read more about Mathematica’s work to build the bridge between research and practice and translate scientific evidence into practical tools for improving programs.
