Elizabeth Linos on Using Evidence to Bolster the Public Sector Workforce and Improve Government Services

[Elizabeth Linos]

There’s a lot of research in public management that suggests that if the people who are delivering those services look like the communities they’re called to serve, that outcomes are better. We’re still trying to figure out exactly why that is, whose behavior is changing for those outcomes to be better.

But you can imagine a world where if we brought in more of the people into the government bureaucracy that have the same lived experience as the people that they’re called to serve, that might improve those mini interactions that happen.

[J.B. Wogan]

I’m J.B. Wogan from Mathematica. Welcome back to On the Evidence.

My guest for this episode is Elizabeth Linos, a public management scholar and behavioral scientist who studies, designs, and tests innovations in how government operates. Elizabeth is currently an Associate Professor of Public Policy and Management at the Harvard Kennedy School; and she’s the Faculty Director of the People Lab at Harvard, which seeks to empower the public sector by producing cutting-edge research on the people of government and the communities they are called on to serve. Elizabeth is this year’s winner of the David N. Kershaw Award and Prize, which is awarded to scholars under 40 whose contributions to research-based knowledge have advanced the design, implementation, and evaluation of public policies.

Long-time listeners may remember our past interviews with the last three winners of the Kershaw Award: Abigail Aiken, Sanya Carly, and Kirabo Jackson. Elizabeth is the 23’rd recipient of the prize.

Elizabeth, welcome to the show.

[Elizabeth Linos]

Thank you for having me. It’s great to be here.

[J.B. Wogan]

I wanted to start by asking about your time as a public servant. We’ve had guests on the show before who conduct research for government agencies, and we’ve had guests who work in government and guide the kind of research that Mathematica does; but you’ve dabbled in both worlds. Early in your career, you served as a policy advisor to the Prime Minister of Greece during the Greek financial crisis. I’d love to hear more about your experience in government and any influence it has on the work that you do now.

[Elizabeth Linos]

Absolutely, I think it’s been a privilege to be able to see public policy from both sides – so both from the side of government and now in my role working with government agencies. I was fortunate enough to have the opportunity to work with Prime Minister of Greece George Papandreou; and I came into government with a cohort of young, data-driven, evidence-based people who really were excited to bring those skills into government and into government innovation.

Now, I won’t go through the full story of how we went from wanting to make Greece data-driven into finding out that the previous government had faked government statistics to a global financial crisis. Suffice to say my job requirements changed, and my job role changed quite rapidly as the situation changed in Greece What that meant was that we were tasked with leading a huge public sector reform very quickly without a lot of resources and support internally but with a lot of bright minds from across the world thinking about policy design and thinking about broad-reform structures.

It occurred to me as we were going through this process that even though we had the smartest people in the world thinking about policy design and what public policies should be, and I knew from my previous career at J-PAL that there are bright minds across the world thinking about evaluation of those policies. I felt like we didn’t have enough evidence about what works to support the people who actually were called to make those changes – so the career civil servants who are tasked with changing the way they work, who are being called lazy or corrupt in the media, and who are really trying to do the best that they can to move Greece forward.

So that really inspired me when I went back into academia to focus my research on the people of government and to really think about how we can support the people who choose to work for government and how we can bring in more talent into the government workforce. That’s of course expanded to other areas of research now, but it has been central to how we think about the work of the people and it has been central to how I think about [evidence].

[J.B. Wogan]

You were at J-PAL before your time with the Greek Government, and then you went back into academia. Did you know that you would always want to go back into policy research? Did you know it was kind of a short-term deal with working in government?

[Elizabeth Linos]

Oh, no, I don’t have long-term plans in my life (laughing). Like many people who work in public policy, I’ve always been unsure about whether or not the right way to make a difference in the world is from the research side, thinking side, or from within government. So I think there’s many people – more people than we imagine, actually, even in academia – who will be going in and out of research and in and out of government in the next years to try to make a difference in policy work.

[J.B. Wogan]

You referenced your interest in the public sector workforce, and how do you recruit routine and talented, smart people to work in government. I was wondering; do you think that the public understands why it is so important to recruit routine and talented people in the public sector? Like if we were just to poll people randomly on the street, what do you think the average citizen might not appreciate about this issue that you invest so much time in understanding and trying to improve?

[Elizabeth Linos]

Well, I think that the narrative about who public sector workers are and what they do really changes over time. So I think there’s been periods in U.S. history where it was certainly the case that people thought the best and the brightest were working in government. Then there’s periods in U.S. history – more recently, I would say – where even our politicians who depend on that career civil service spread a narrative of disdain for the people who work in government. I think that does seep into the general kind of popular belief that either we don’t need innovators and smart thinkers in government or that the people who currently work in government don’t have those skills – both of which I think are false.

I really think that there’s a fundamental misunderstanding, if you just poll people on the street, of how important it is to support the talent that already works in government and to attract and retain talent in government if we want public services to deliver the way we expect them to.

[J.B. Wogan]

You mentioned that you were part of this iteration of people who are data-oriented and interested in evidence who were coming into the Greek government at the time that you did. Do you know, was there a specific reason why people like you were entering government at that time?

[Elizabeth Linos]

Yeah, I think there was a couple of things happening in the world at the same time. On the one hand, there was a really remarkable leader in George Papandreou that many of us saw and thought would be kind of the future of innovation and success in Greece. He brought with him a message about climate change and being progressive and being more innovative in government that really resonated, I think, with a lot of people from my background. But this wasn’t happening in a vacuum. At the same time, you would see a report in the news of Obama’s speech writer being 27 or something. So I think there was this broad sense that kids were entering government in some sense or like young people could come and really make a difference. So for me, Papandreou was like Obama was for many Americans at that time.

There was also an ongoing movement in the innovations space because this was more than 10 years ago, where the narrative was if you just bring in innovators or people with these different data skills we’re going to transform government. Now, I would argue that that was an incorrect narrative about how to do innovation in government. We really thought that the right way to effect change was to bypass the bureaucracy and work around government to make change because we didn’t have time to waste with the bureaucracy. But inherent in that narrative is a continued disdain for the people who work in government and the people who have 30 years of experience trying the same things that were trying as well.

So I don't think the narrative worked, but the premise that you can bring in people with all sorts of skills and all sorts of education who want to make a difference to support the vision of a political leader to change government, I think, holds and will hold again in the future in a bunch of different countries if we create the systems for young people to be able to enter government, which oftentimes is one of the main barriers.

[J.B. Wogan]

I’ve noticed that in some of your research you go beyond just asking the question of how we can either – various aspects of improving the public workforce, whether it’s reducing burnout or trying to diversify part of the workforce like a police workforce. But you also sometimes are asking questions about what the relationship is between improving the condition of the workforce and then the services it provides to the public. What do you – could you say a little bit more about that connection that you make and why you think it’s important to investigate not only sort of what is the state of the workforce but then what sort of a better-maintained workforce might mean for the public and maybe for taxpayers?

[Elizabeth Linos]

Yeah, I think it’s a very important question that we need to do a lot more research on to better understand kind of the causal link between what it means to invest in the workforce and how that translates into better service delivery. But at least in some of my research, we do see that connection popping up. So during COVID, we did quite a few projects related to burnout for frontline workers which, as you can imagine, was a significant problem and continues to be a significant challenge for a lot of government agencies.

What we find in at least some of our work is that if you invest in the mental health of the workforce, that might change how they then view their role vis-à-vis the people that they’re serving. Just to give you an example, we’ve done some work with Jesse Harvey in the Bangor Sheriff’s Department that focuses on the burnout challenges and mental health challenges of correctional officers. What we find is that if we invest in improving that through peer support programs that doesn't only have benefits for the employees themselves but that also changes how they view the incarcerated population. So I’m excited to do more research in this space because I think that could translate into better resident/state interactions more broadly if we do more of this work.

[J.B. Wogan]

Are there other projects you have underway or that you will have underway soon that might explore this in other contexts?

[Elizabeth Linos]

Well, in the service delivery states, we’ve been doing a lot of work with my team on how do you think about reducing the administrative burden that people face when they have to interact with government to receive benefits – so think about the broader social safety net, the child tax credit, the EITC, dental assistance. A lot of our early work in that space focused on improving outcomes by providing information in different ways for reducing some of the compliance hurdles that people face. The next stage of research that we’re thinking about now is kind of linking those two worlds – so thinking about who can provide higher-touch support for low-income households – who the people are, how we support them, what kind of hotline options we have, what is the background of the people who are making those calls and providing that assistance and how that might change the willingness or ability for low-income households to access benefits.

[J.B. Wogan]

That’s a really interesting topic. I’m looking forward to seeing what evidence you bring to bear. I mean, I think the intuition makes sense to me. Like I’m thinking about just in my day-to-day experiences going to a coffee shop or a grocery store. There are certain businesses where it seems like they treat their employees better, and they seem to be better run; and it spills over in terms of my experience as a customer, where I feel like I’m being treated better because they’re happier. So I’ll be curious to see if there’s any kind of translation in a public sector context.

[Elizabeth Linos]

Yeah, absolutely, and there’s an added component to that when we think about the government bureaucracy. There’s a lot of research in public management that suggests that if the people who are delivering those services look like the communities they’re called to serve, whether that’s demographically or in terms of experience, that outcomes are better. We’re still trying to figure out exactly why that is, whose behavior is changing for those outcomes to be better.

But you can imagine a world where if we brought in more of the people into the government bureaucracy that have the same lived experience as the people that they’re called to serve, that might improve those kind of mini interactions that happen when a low-income household is showing up for an interview to get access to benefits or when you have to call a household to say, okay, you missed your appointment. You can imagine those micro interactions looking very different if both sides acknowledge that they come from similar backgrounds.

[J.B. Wogan]

Yeah, I can imagine a stronger sense of trust.

[Elizabeth Linos]

Mm-hm.

[J.B. Wogan]

All right, so having worked on the research side and on the government side, what do you think each camp could stand to learn from one another; and do you think there are any, I guess in addition to anything that could benefit from learning about one another, are there any key misconceptions you think that government has about researchers and that researchers have about government?

[Elizabeth Linos]

Yeah, it’s a great question. It’s funny because one of the misconceptions that I also have had myself in different roles but I certainly see coming out on both sides is that both sides seem to think that the other side is too slow – which is kind of a funny idea because both sides can’t be too slow. When I didn’t work in academia, I thought, look, we have serious problems today. We need to deliver services today. You can’t imagine how many challenges a given public sector worker has to face and what level of resource they have to deliver those services. So don’t come to me, academic, and tell me you’re going to spend six months doing the lit review. That’s completely useless.

On the other side as an academic, I often hear from fellow academics, “Yeah, I really want to be helpful and I want to provide services for free and I want to provide my insights and be an extra set of brains, but they’re taking too long to get back to me,” or “It takes too long to set up an MOU or a data-sharing agreement.” So what I’m seeing is basically a community of people who feel the urgency of making a difference today. Both sides feel that urgency, but they haven't really been able to figure out appropriate ways to work together.

Now, I should say there’s a lot of exceptions to that rule. I’m so lucky that the people we get to work with are the exceptions. We are very fortunate to have partners who work in government who have managed to bridge those gaps and who really take on the risk of working with academic partners. So if you ask me kind of whose behavior needs to change, I would argue it’s on the side of academia to become more useful and to answer question that are in fact useful for government rather than the other way around. I think it’s going to take a lot of work.

[J.B. Wogan]

Actually, you touched on something. I accidentally skipped a question I was going to ask you earlier, so thank you for the reminder. But I wanted to ask about how you decide on the questions that you pursue because you do seem to select such timely and policy-relevant topics for research. The recruitment and retention of police officers and diverse police officers comes to mind. So at least in your case, how do you decide what to research next?

[Elizabeth Linos]

Yeah, it’s a good question; and it’s not by accident. So the way I’ve set up my team and the way I think about my own usefulness in the world is to really try to answer questions that a government can’t ask. So the first step is to say what are the kinds of questions that are holding people up in delivering the services that they want to deliver, what are the barriers that they’re facing, but also what keeps them up at night. I do that by having kind of made a large investment in listening to government workers. I spend a lot of my time on phone calls with department heads or with people in the mayor’s office or people who work in state and federal government and ask them kind of what are they worried about or what’s keeping them up at night.

The patterns become very clear. So all the questions that I answer because I focus on public management are horizontal questions. But when the fourth or fifth or sixth person from a different department tells you, “The thing that keeps me up at night is my vacancy challenge because we’re having a demographic shift in our workforce,” then to me that’s a signal that we’re going to need more research in that area. The fourth or fifth time in different programs that someone says, “We’re trying to get people to access this program, but they’re not showing up,” or “We’re not able to get everyone who’s eligible for this program to take it up,” that is a signal for me and my team that we’re going to need to do a lot more research on how to do that better.

So it really is starting with a government challenge as the starting point for our research and then pulling out what is an interesting research question and what can kind of contribute to generalizable knowledge in academia as well.

[J.B. Wogan]

I noticed that this week as we were preparing for today’s interview the local public radio station, WAMU, had a story about how the police department here in D.C., the Metropolitan Police Department, has had like a 50-year low for police officers in the district; and they were trying to explore what it is, what are the factors that are contributing to problems of retention and recruitment. I was thinking your research is especially relevant in a place like the District of Columbia.

Let’s see, I’d like to wrap up by talking about a column that you wrote for the B&G Report earlier this year in which you describe an initiative involving more than 70 cities to use evidence in addressing local challenges, like diversifying the police department and improving code enforcement. The column raises some important questions about how to scale up and sustain pilot programs even when those programs are successful. So would you mind sharing some of the key takeaways from that initiative and the questions it has raised for you in your research partnerships with government agencies going forward?

[Elizabeth Linos]

Yeah, absolutely, so the initiative that you’re referring to is the What Works Cities initiative, which was a large (inaudible) philanthropy initiative to support cities across the U.S. to essentially do evidence-based or data-driven government in a bunch of different ways. The part that I was involved with through the Behavioral Insights Team was related to evaluations. So we were working with mid-sized cities who were interested in doing rigorous randomized controlled trials to test some sort of change in a program or a service in their city. So in some ways, this is kind of the dream scenario for an academic policy/practitioner partnership where you work together to codesign a potential intervention and you test if it works in that context.

So in that department, were you able to change code enforcement? Were you able to get more people to access the program? Were you able to get more police officers or different police officers to apply? The kind of status quo in these types of partnerships is that we work together really intensively; and then we get our results and we put together our policy brief or some sort of presentation to the leadership of thee department and we say, “Look, it worked; great job,” or “Look, we got some significant findings or statistically significant findings.” Then we pat ourselves on the back, and we move on to the next question.

What we did in this kind of research portfolio is we just went back five years later and said, “Did you actually use the treatments that we tested in those interventions?” So if you go back and you know that you have some point between 2015 and 2019 where a rigorous evaluation was done, it was in the right context, there was a treatment group and a control group -- so we have all the pieces there -- do we know if it worked or not? Did you actually adopt the treatment?

It turns out, much to my heartbreak, that it is not the case that just because we found statistically significant evidence that that magically translates into adoption afterwards. In fact, we find that about 30% of those treatments were adopted in the following years and that the strength of the evidence is not a predictor of whether or not something gets adopted.

[J.B. Wogan]

And those were—

[Elizabeth Linos]

I was—

[J.B. Wogan]

And those were--

[Elizabeth Linos]

Oh, go ahead.

[J.B. Wogan]

Those were also inexpensive, right? So most of these things were things like changing the language on a postcard or a text message, not the – expense probably wasn’t a factor in the decision not to move forward.

[Elizabeth Linos]

Yes, exactly, so one potential thing you could have imagined is the (inaudible) cost. But even the things that have zero cost, like changing an e-mail, we still see the same effect. So it’s not cost. It’s not the strength of the evidence. It’s not even staff retention, which I felt was going to be the main thing. So if the same person who worked in government at the time is still there, maybe the evidence is adopted, although we see some directional evidence.

The main predictor in whether or not something gets adopted is whether or not the treatment or the intervention was a tweak to a preexisting communication or infrastructure. So what that means is if something is kind of a small tweak to an existing process that’s already going out, then the evidence is used. Then the better version of the letter gets implemented. If it’s kind of a new innovation that there isn’t any preexisting infrastructure, the likelihood that that’s going to get adopted five years later is very, very low.

Now to me, that opens up a world of really important research questions that are behavioral in nature. So rather than saying what kind of projects or what kinds of treatments improve the outcome, now I’m asking questions around what kind of projects improve adoption over a time. So the outcome is no longer just about did you improve applications or take up, but what are the kinds of interventions that are going to be adopted over time. My sense is that over the next few years, we’re going to see a lot more RCTs – at least from my team – on how to support practitioners in adopting evidence. Because we’re behavioral scientists, we know that just giving people information is not enough to change their behavior.

So we’re going to have to do a whole bunch of work around what kind of support, what kinds of technical assistance, what kind of training, what kind of reminders do we give to people at the end of an RCT or when evidence does exist to support adoption at scale.

[J.B. Wogan]

Does that – my initial read when I read that column was, oh, so we should be focusing more on incremental improvements and smaller tweaks rather than instituting an entirely new program or replacing a program that doesn't work with one we think would work. But maybe that’s too far a leap. Do you think there are ways that we might be able to increase adoption and sustainability of big new changes? Do we have to settle for an incrementalist approach?

[Elizabeth Linos]

That’s a very good question. I’ve gone through all those feelings myself. When we first saw the results, that was my first reaction too – which was like, oh god, are we saying that incrementalism is the only thing that sticks?

My current read is essentially if you’re not going to invest in adoption as a thing, then the only things that will organically get adopted are the incremental changes. But I don't think that means we should settle for only making incremental changes. Rather, I think we should invest heavily in how to support government in adopting evidence once it exists.

We’ve done this before. So 10 years ago or 15 years ago, there was a huge investment both from the philanthropic sector and government itself, and potentially academia, to invest in running these RCTs in government. So the legacy of J-PAL or the Evidence Act or all these kind of important initiatives was to say we’re going to invest in running rigorous evaluations in government – not just taking things from other contexts. So we’ve made that investment. It’s time for a second investment now that we have 10 years of evidence in adoption as a thing that we invest in as heavily and with the same thoughtfulness and purpose as we did to produce the evidence in question.

[J.B. Wogan]

I think those findings are just so relevant to organizations that you’ve worked with in the past, like J-PAL, but also organizations like Mathematica and our clients where we’re often involved in these big demonstration projects. Yeah, it’s great to know that even if the demonstration proves successful that that’s not the end of the story. There’s still work to be done to make sure that the things that worked, that the evidence shows worked, making sure that that continues and it can be scaled up.

I was even thinking in terms of the American Rescue Plan, which is enabling a bunch of localities right now to pilot various things like the guaranteed basic income and to evaluate them. I’m wondering in the contest of your research how cities could actually continue anything that they find is effective with the ARPA dollars.

[Elizabeth Linos]

Yeah, it’s a great example; and it’s also an exciting time to be working in the space because of ARPA funds. We are making huge investments in trying new things and trying bold new approaches to delivering services. So I think the main message from this new line of research around evidence adoption is really to say let’s just not stop at producing the evidence. Let’s also think about adoption and the bottlenecks or the barriers to adoption as things that are worthy of research and attention.

[J.B. Wogan]

All right, I think that’s a great place to end today’s conversation. Thank you so much for your time, Elizabeth.

For anyone interested in learning more about Elizabeth’s work, we list links for additional reading and listening in our show notes. Elizabeth is a frequent podcast guest, and I would highly recommend you listen to some of her other interviews that go deeper into the details of specific papers.

If you liked this episode, please consider subscribing. We’re on YouTube, Apple Podcasts, Spotify, as well as other podcasting platforms. To learn more about the show, visit us at Mathematica.org/OnTheEvidence.