Although performance measurement and program evaluation are both ostensibly about assessing the effectiveness of government, they have historically meant different things in terms of what gets assessed and who does the assessing. Performance measurement is more commonly associated with ongoing monitoring and reporting of program accomplishments and is typically conducted by program or agency staff. Program evaluation, on the other hand, is more commonly associated with periodic or ad hoc studies conducted by experts outside of an agency or program.
But are those distinctions still relevant today? That’s one of the questions journalists Katherine Barrett and Richard Greene discuss in their new book, The Promises and Pitfalls of Performance-Informed Management.
In this episode of On the Evidence, I talked with Barrett and Greene about how state and local governments use performance measurement and program evaluation to inform management decisions, providing contemporary case studies along with historical context about how the field has evolved over their three decades covering the topic.
We discussed the following topics:
- The integration of different but related disciplines of performance auditing, performance measurement, and program evaluation (19:42–22:23)
- The increasing availability of data and its effect on performance-informed management (22:26–26:40)
- Changes over time in how states value, understand, and use data in decision making (28:40–30:15)
- What the book might have covered about the two major stories of 2020—the COVID-19 pandemic and concern over persistent racism in the United States—if it had been published a few months later (30:14–37:05)
In the book’s final chapter, dedicated to public policy research and evaluation, Mathematica’s Chief Executive Officer Paul Decker tells Barrett and Greene that in the past, “we tended to have this attitude that on the one hand there was performance measurement and on the other, there was evaluation … I think more and more these two concepts have become intertwined. We’re blending with each other and the idea of frenemies becomes passé.”
Listen to the full episode below.
A version of the conversation with closed captioning is also available on Mathematica’s YouTube channel here.
Want to hear more episodes of On the Evidence? Visit our podcast landing page or subscribe for future episodes on Apple Podcasts or SoundCloud.
Show notes
Read Barrett and Greene’s Q&A with Mathematica’s Paul Decker, based on an interview they conducted for their book, here.
In their book, Barrett and Greene quote from or cite past episodes of On the Evidence about the following topics:
- Cities’ use of evidence-based policymaking, featuring Katherine Klosek of the Center for Government Excellence at Johns Hopkins University
- Challenges and opportunities in states’ use of administrative data, featuring Elizabeth Weigensberg of Mathematica
- Lessons from two Colorado counties’ use of rapid-cycle evaluations to make quick, evidence-based improvements in their employment assistance programs, featuring Jonathan McCay of Mathematica
