Every day, city leaders face pressing challenges that require a rapid response. In an ideal world, they would spring into action with proven programs and tactics that are guaranteed to work. But sometimes leaders don’t know how to find those proven programs and tactics, and they need help finding useful information in a timely fashion. In her digital book, A Roadmap for Policy Change, Katherine Klosek provides a step-by-step guide for time-strapped government officials who still want the best available information about potential solutions to challenges in their communities.
Klosek is the director of applied research at the Center for Government Excellence (GovEx) at Johns Hopkins University, which helps cities across the country manage and use data to support decision making. Before she was advising government staff about how to use data and evidence, she was in their shoes. Under Martin O’Malley, the former governor of Maryland, she was a senior analyst for a stat program, helping executives manage their agencies more effectively by gathering performance data on a regular basis, discussing the results with staff, and then using the data to allocate resources as wisely as possible.
On this episode of On the Evidence, I spoke with Klosek about the roadmap and the state of evidence-based policymaking in cities. Click here to listen to the full interview. You can also read an edited excerpt of the interview in the following transcript.
What’s the shorthand description for GovEx?
Our mission is to help governments use data and evidence to make better decisions to improve the lives of residents.
How does GovEx go about trying to do that?
For our first three years, we were working with city governments, with populations between a hundred thousand and a million, to help assess their current practices and their capacity to improve their practices around data management and data use. That was the first step. Then, with some of those governments, we moved forward with technical assistance projects lasting about six months or so, in which our team would dive deep into our bag of tricks and provide resources and guidance that was free of charge. Now, we’ve added this applied research role, so we’re trying to connect some of the research that comes out of Johns Hopkins University and other places to that technical assistance work with cities.
What is the connection between stat programs and the work that you do today?
We’ve seen the stat performance management programs really evolve from [how it was early on:] [a] pragmatic city or state leader using it to improve efficiency and effectiveness of service delivery on a departmental basis. That’s how it started out, and it’s really evolved to being more outcome focused. City leaders want to look at things like trash pickup or responsiveness to 311 calls about rats—which is a very real issue in Baltimore—but now a city leader is [also] looking at more of an outcome goal, like community health, where those two indicators might be measured toward an overall goal of the health of a neighborhood or community. Leaders are recognizing that performance management frameworks and programs are a great way to monitor progress toward whatever outcome they’re trying to achieve and to see whether their policies and practices are getting the desired outcome.
How do you think evidence-based policy has evolved at the local level over the last 10 years?
City leaders want to look at what other cities are doing, what other governments are doing, and figure out how that might apply in their city. There’s this natural tension that’s arising where decision makers need timely data points, they need performance measures to indicate what to do almost on a daily basis, [but] rigorous policy analysis takes time. So there’s this tension between wanting a really robust body of evidence and needing performance metrics to indicate what to do and where [to] invest right now. That’s one thing that we tried to address in the roadmap, almost reassuring city leaders that you don't need randomized controlled trials for everything that you’re going to try in your city. Sometimes you can generate your own evidence.
Why write this roadmap and why write it right now?
When I started this position, I wanted to dig into the effective and ineffective policies that cities were trying. Then I decided that it might make more sense to take a step back and figure out where there are barriers to using evidence and data in decision making, and why some policies spread and are used and why others are not. We started out with a series of interviews with cities in our network and asked them some open-ended questions about their policymaking and decision process. We dug into that process, and each step ended up being a chapter in the roadmap.
I think the time is right, not just because [GovEx is] taking a deeper dive into policy, but there’s a lot of discussion about science and evidence in government and decision making. There have been several [recent] marches for science. When we’re talking about science, we’re talking about evidence and vice versa. So, it makes sense from the national perspective as well.
What’s your advice for local government officials who want to create a demand for evidence?
Find a partner either inside or outside of city government. It’s probably going to be outside of city government, like a university partner or local nonprofit [that] can be a partner in evaluation. Developing long-term partnerships between city government and universities is going to be key. There’s a lot in the policy change [and] political science literature about how partnerships on one-time evaluations aren’t the most sustainable. [If you] create a peer network or participate in a peer network that brings practitioners and researchers together, [that] can hopefully help to shift the conversation so that the evidence coming out of universities is more helpful and meaningful to governments.
For private research organizations that want to help build a demand for evidence, is there anything that they can do?
Be deferential to residents [and let] governments know that the needs of residents and outcomes of residents are why you’d want to partner with them. [Make] it clear that there’s no other agenda. I think that’s part of the distrust or concern: that an evaluation organization might be coming in to test the population.
What's next for the roadmap?
What we’re doing now is designing what we’re calling communities of practice. We are trying to seed some mini networks and bring together governments [and] nonprofits, school districts, and other partners that are focused on a certain issue and help them use some of the tools and guidance in the roadmap. [We’ll] figure out what’s helpful [and] what’s not and then iterate on the roadmap itself, developing new tools. Most importantly, we want to capture the results of those communities and any stories that come out of that.
Want to hear more episodes of On the Evidence? Visit our podcast landing page or subscribe for future episodes on Apple Podcasts or SoundCloud.
Show Notes
Klosek, Katherine. A Roadmap for Policy Change: A New Approach to Evidence-Based Policymaking. Baltimore, MD: Center for Government Excellence at Johns Hopkins University, May 2018. Available at https://govex.jhu.edu/wiki/a-roadmap-for-policy-change/
