Advancing Educational Equity: Tools for Evidence Building

The right evidence at the right time can help organizations design and rapidly improve solutions to pressing social challenges—whether these solutions are new or evolving programs, products, or practices. Mathematica’s measurement and evaluation (M&E) approach and associated toolkit can fuel evidence-based, equitable innovation and maximize learning across multiple solutions. The M&E approach starts by identifying a need, problem, or challenge in partnership with a community in focus—the community whose goals should drive design—and then engages this community as collaborators throughout all phases of research and design.

Keep reading to learn more about these tools and the M&E process or click on an icon below to jump to a step in the process.

 

What is the Measurement and Evaluation process?

In partnership with the Bill & Melinda Gates Foundation, Mathematica developed a set of M&E tools to implement this M&E approach and help education grantees generate evidence about their solutions to education’s most persistent problems. To expand the reach of this work and promote the adoption of evidence-based solutions more broadly, we adapted the tools for use in many fields and by a diverse set of users—organizations, funders, research partners, and other interested parties.

Some tools were created for a specific phase of solution development, but many of the tools apply across phases—from designing a solution all the way to validating its effectiveness. For definitions of terms used in the tools, please refer to this glossary. You can also download a list of the tools or the complete toolkit.

A Road Map for Building Evidence that Drives Learning and Improvement

Mathematica’s measurement and evaluation (M&E) approach tailors learning in four phases of a solution’s development: designing the solution, refining the solution, assessing for early evidence of success, and validating effectiveness. Explore the primary objective of each phase, alongside example study designs and research questions, and a case study of this approach in action.

Learn More
 

Step 1: Assess Measurement and Evaluation Needs and Interests

These tools introduce the iterative approach to evidence building and help organizations and their community partners articulate their M&E goals, interests, and needs.

Expand to See Tools for Step 1
result(s) for are listed below
Select View
Summary

In this video, Mathematica experts introduce an iterative approach to evidence building that they created to help organizations design, refine, and test solutions that seek to address educational and other social challenges.

View Video
Summary

The Measurement and Evaluation Needs Inventory is a tool designed to assist an organization in conducting a self-assessment of its existing measurement and evaluation capacity, with the goal of identifying priority areas for support from an external research partner.

Download
Summary

The Prior Research Table is a tool designed to help organizations document previous research on solutions to inform future evaluations.

Download

Step 2: Plan Measurement and Evaluation

These tools help organizations and their community partners develop their own M&E or design research plan. The checklists provide guidance on working through the evidence-building phases.

Expand to See Tools for Step 2
Filter
Hide
result(s) for are listed below
Select View
Summary

This resource provides educators, school district leaders, researchers, and funders a set of measures and approaches to assess the extent that math instruction and interventions change what students know and how they feel about math, as well as how they use math outside of the classroom.

Download
Summary

This menu describes a set of teacher- and student-level measures and approaches to understand the extent that writing instruction and interventions change how teachers think about and teach students about writing, and what students know and how they feel about writing.

Download
Summary

The Measurement and Evaluation Template is a road map organizations and funders can use to develop a detailed measurement and evaluation plan with support from a research partner.

Download
Summary

This template is for organizations that want to incorporate the needs and preferences of the communities they intend to reach into their program, practice, or product’s design. It includes space for organizations to document a description of their program, practice, or product; their theory of change; their learning objectives and approach; and a timeline to carry out the work.

Download
Summary

Phase 1 of the Measurement and Evaluation Checklist guides users through the process of co-designing a solution with a partner and developing a well-defined theory of change for how that solution is expected to improve outcomes for students in a priority community.

Download
Summary

Phase 2 of the Measurement and Evaluation Checklist guides users through the steps to refine a solution based on lessons learned during implementation. By following these steps, users will develop descriptive evidence that the refined solution was successfully implemented in a priority community.

Download
Summary

Phase 3 of the Measurement and Evaluation Checklist guides users through the steps to conduct a small pilot study and demonstrate that a solution is associated with improved outcomes for students in a priority community.

Download
Summary

Phase 4 of the Measurement and Evaluation Checklist guides users through the steps to conduct a large pilot study and demonstrate with a high degree of confidence that a solution leads to improved outcomes for students in a priority community.

Download
Summary

This guide is designed for organizations implementing solutions, with support from a research partner. It provides guidance on when and how to measure usability, usefulness, and utilization—key indicators of success during early implementation.

Download
Summary

This guide is designed to support organizations at different stages of solution development and implementation define a set of targets for each research question they will answer. Targets enable organizations or their funders to define success.

Download
Summary

This guide is designed for funders and organizations implementing solutions with support from a research partner. It provides an overview of the sample size considerations for each of the evidence-building phases.

Download
Summary

This tool offers strategies for engaging community members as part of a research team and including their experiences, perspectives, and expertise throughout the research process.

Download

Step 3: Conduct Measurement and Evaluation

These tools assist organizations and their community partners execute their M&E plan, including collecting the data needed to address each research question.

Expand to See Tools for Step 3
Filter
Hide
result(s) for are listed below
Select View
Summary

In this video, Mathematica experts introduce an iterative approach to evidence building that they created to help organizations design, refine, and test solutions that seek to address educational and other social challenges.

View Video
Summary

In this video, Mathematica experts provide a deep dive into each of the five measurement and evaluation criteria that are part of an evidence-building pipeline they developed to support organizations, research partners, and funders develop, refine, and test solutions.

View Video
 

Step 4: Analyze and Report Results

These tools guide organizations and their community partners through the final step of the M&E process, which includes analyzing collected data, determining next steps, and reporting the findings.

Expand to See Tools for Step 4
Filter
Hide
result(s) for are listed below
Select View
Summary

This resource is intended to guide an organization’s approach to data interpretation, with the goal of interpreting study data collaboratively with members of the community or communities where research is taking place.

Download
Summary

Organizations, research partners, and funders can use the Measurement and Evaluation Reporting Template to summarize and share findings from a study.

Download
Summary

This tool offers strategies for integrating community perspectives when disseminating research. It outlines key steps for co-interpretation—or collaborating with community members to interpret data—along with strategies for sharing findings that are accessible to and reach communities.

Download

Related Staff

To connect with one of our experts on this project or to get more information about these tools, please send an email to info@mathematica-mpr.com.

Kate Place

Kate Place

Senior Researcher

View Bio Page
Virginia Knechtel

Virginia Knechtel

Director

View Bio Page
Kara Conroy

Kara Conroy

Senior Researcher

View Bio Page
Megan Shoji

Megan Shoji

Principal Researcher

View Bio Page
Ryan Ruggiero

Ryan Ruggiero

Researcher

View Bio Page
Gregory Chojnacki

Gregory Chojnacki

Senior Researcher

View Bio Page
Lindsay Fox

Lindsay Fox

Senior Researcher

View Bio Page
Kaleen Healey

Kaleen Healey

Senior Researcher

View Bio Page
George Smith

George Smith

Senior Researcher

View Bio Page
Marykate  Zukiewicz

Marykate Zukiewicz

Senior Researcher

View Bio Page
Francesca Venezia

Francesca Venezia

Researcher

View Bio Page