Youth At-Risk of Homelessness: Foundational Analytic Plan for the Summative Evaluation of Pathways to Success

Purpose

This analytic plan serves as the foundation for a summative evaluation of the Colorado Pathways to Success comprehensive service model that Mathematica and ACF are conducting as part of YARH-3 in partnership with the Colorado Department of Human Services and the Center for Policy Research. This plan describes the implementation study analytic approach, the impact study analytic approach, and the research questions guiding the studies and the data sources that will inform analyses. This is one of a series of public documents describing and documenting the summative evaluation.

The YARH-3 summative evaluation includes an implementation study and an impact study. The implementation study will use multiple methods to collect, analyze, and report on comprehensive data to address the study’s two broad objectives, supporting interpretation of Pathway’s impacts on outcomes and assessing Pathways implementation. Mathematica will collect data from a range of stakeholders using interviews and administrative data. Mathematica will take a structured approach, guided by the conceptual framework, to analyze the data across Pathways sites.

The impact study will be the first rigorous impact evaluation of Colorado’s Pathways comprehensive service model. The impact study will conduct traditional inferential tests. It will examine evidence of program effectiveness on a large number of policy-relevant outcomes, including stable housing, education, employment, permanent connections to caring adults, and social-emotional well-being. It will explore the effectiveness of Pathways in short- and long-term follow-up periods and estimate the extent to which the program is more or less effective for key subgroups. Finally, the study originally proposed to explore linking features of program implementation (for example, dosage, quality, or adherence of the program delivery) to youth and young adult outcomes. However, this last analysis might no longer be feasible, given the limited variability in youth and young adult Pathways experiences and projections for the number of youth and young adults expected to enroll in Pathways by the end of the study.