Jennifer Pahlka on Government in the Digital Age

[Jennifer Pahlka]:

There's a couple of laws in the book. One of them is Wong's Law, where he says, you know, wherever you are in government, there's always a new system. It's always going to fix all of our problems, and it'll be here in 18 to 24 months. But the joke of that is that it's always 18 to 24 months away no matter when you ask. There's a different approach that says, let's have a team that can improve this system every day, every week, every month in small ways.That kind of thinking really, you know, can get us a lot further than the, you know, holdout for the thing that's going to meet all the new requirements.

[J.B. Wogan]:

Welcome back to On the Evidence. I’m J.B. Wogan from Mathematica and I’m glad to be back from paternity leave.

This episode is another in an occasional series on the show that we call Evidence in Government, where we talk about what’s happening in the halls of government and the role evidence can or should play in decisions that could improve people's lives.

My colleague Mike Burns leads these episodes. Mike is the Senior Director of Communications and Public Affairs here at Mathematica. Our guest for this episode is Jennifer Pahlka, who served as the U.S. deputy chief technology officer under President Barack Obama and founded Code for America, a nonprofit that works to improve government digital services. In her new book, Recoding America: Why Government is Failing in the Digital Age and How We Can Do Better, Pahlka takes us beyond the basics to explore why policy implementation goes wrong and how we can improve delivery of government services to achieve better outcomes for the American public.

On this episode of On the Evidence, Mike and Jennifer discuss Recoding America, the nexus of evidence-based policy and delivery-driven policy, and how we can close the gap between policy intentions and real-world outcomes.

I hope you enjoy the episode.

[Mike Burns]:

Well, Jennifer, thank you so much for joining On The Evidence. Really looking forward to our conversation.

[Jennifer Pahlka]:

Thank you.

[Mike Burns]:

Let's start with your new book, Recoding America. So this is a book about tech, but more than that, it's really about implementation of government policies and programs. And tech, digital, that's the vehicle. And you have a quote that really stands out, that's really stood out to me, as I'm sure it did to a lot of people who read the book. And I would say it's really a main theme of the book. And that is, when systems or organizations don't work the way you think they should, it's generally not because the people in them are stupid or evil. It's because they're operating according to structures and incentives that aren't obvious from the outside. What exactly are those structures and incentives?

[Jennifer Pahlka]:

Well, I don't think there's one set of them that's easy to describe. But I do think this is what people who are trying to get policy to be more based on evidence run up against, when the evidence is clear, but the policy doesn't change, or it's not adopted. And one of those things that I talk about is this accountability trap, where the incentives of public servants are very contested. So they're certainly supposed to get outcomes, but they have all these processes in place. And their careers depend on following the process, even if it conflicts with getting the outcome.

But I think there's just something beneath that. People complain about government being risk averse. And certainly it is. It would seem, of course, that following the evidence would be less risky than doing the status quo. But the reality is that we have people who are attached in many ways to their current processes, not just because they're not going to get in trouble for doing what has come before, but something we talk about later in the book, which is that that has become very much who they are in some ways. And I don't mean this to say it in a demeaning manner. I have enormous respect for public servants.

But I think any of us working to change policy or change how government operates have to fundamentally respect that public servants have come to their views honestly, authentically. And understanding what's driving them is really critical to making those changes. We've all tried to sort of bowl over resistance. And has it worked? Not very well for me. I don't know about your other listeners. But those are just two, I think, of the dynamics that are at play when we go for change and we don't see it.

[Mike Burns]:

And there was one example, I think, that really exemplified this. And that's the story you gave about the Supplemental Nutrition Assistance Program in California, where the application form was over 212 questions. And interestingly enough, you actually cite a Mathematica report that looked at California and other efforts around access to web-based benefits. And this story stood out to me. Well, that was one of the reasons. But also because SNAP is a very effective federal program. It's popular in California. And this form was not intentionally burdensome. So you have this evidence that the lengthy form isn't working for many folks who are trying to access it on mobile or at a public library. They're not getting on. How is that evidence then leveraged to say, look, we need to do this differently?

[Jennifer Pahlka]:

Yeah, I think it's a great example of your earlier question, too. What were the incentives of the representatives of the counties in the consortium that was responsible for this particular form? So it's a little bit less than half of California counties were in this particular consortium. And in that case, the incentives were for each county to represent their needs in deciding on what features were added to this app or how the application would work. And I think that's sort of how they got to this 212-question form. It wasn't just that it didn't work on mobile. It's that it was very, very, very long. And they each had their own unique legacy business processes. Because before the internet, they didn't have to come together and pool resources to make a form that they could put online.

So what happens in that is that everybody is doing their job to the best of their ability, often with enormous integrity and diligence. But the system has been set up to represent what I would call government needs. And the evidence that the team at Code for America, including Jake Solomon, brought to the table was, right, those all need to be accounted for. But how are we understanding the needs of the users of this system? It's as if you have a legislature voting, but none of them understands their constituents. Right? They're just trying to vote for their own interests. I'm not trying to draw a parallel here.

And so I find a lot of the times in my work that the evidence isn't actually a set of numbers. It's a story, or a person, or actually experiencing something that the numbers actually can't get across. You see this elsewhere in -- well, I should finish that story. So on SNAP, what happened was Jake Solomon was able to present to this consortium and show them what it's like to actually use the system if you're a low-income person, if you don't have a mobile phone, the number of places that there was drop off, much more to the story than that. And they decided to make it mobile friendly. And California's come a really, really long way since that story, which was over 10 years ago.

But another example of this would be earlier in the book, we talk about a team at the VA bringing screen capture and audio of a veteran who has been trying to get on veterans benefits for years. And there was no evidence prior to that that could convince the VA to fix this broken form. They would look at the paperwork and say, here are the requirements. The requirements have been fulfilled. There is nothing we can do. The evidence that we have inside the building is nothing is wrong. And unfortunately, what was wrong was that that form was set up so that you had to have the exact combination of Internet Explorer and Adobe Reader, or it wouldn't load. That was the exact combination that all the VA computers were set up with, but hardly any computers outside. And so they had to bring different evidence, which was not about requirements or testing or anything. It was simply this one guy named Dominic talking about how he'd been trying, and actually using it, and narrating not just how frustrating it was not to get through, but how it made him feel. And I think I would love for the evidence-based community to make a lot of room for qualitative research, which is a nice name for actual stories of people that can capture the hearts and really change the conversation that needs to be changed in order for the rest of that evidence to be acted upon.

[Mike Burns]:

So Ezra Klein, and this is high praise, said that you've written one of the best policy books he's ever read. And I agree, but it's funny.

[Jennifer Pahlka]:

Thank you.

[Mike Burns]:

It's funny, though, because one of the points you make is that what's valued in government is policy and arguably politics, but not delivery. You say that systems that are used to deliver government services are viewed as merely a detail of implementation, not as a matter of policy. But what makes government effective is the delivery. So you can't really neatly separate implementation and policy. But where does this view of separation come from, and how do we change that?

[Jennifer Pahlka]:

I think that separation is really deeply rooted in our culture, and even the cultures from which some of our government is derived, right? When I was writing the book, I shared the chapter about my own experience realizing this huge resistance to dealing with delivery or implementation from the most powerful parts of government. I happen to be working in one of them at the White House. I was in the Office of Science and Technology Policy, but since my boss was off saving healthcare.gov, or I should say working on healthcare.gov, which was saved by some of his team and many, many dedicated career civil servants, I was mostly working with the Office of Management and Budget while my boss was away. And that is very much the center of power. And their resistance to standing up what became the United States Digital Service really resonated with an experience that when I read about previous attempts to get digital and implementation into the White House, it really resonated with my own experience.

So back in 1995, or I guess it was '96, during the hearings to debate what became the Clinger-Cohen Act, which had much stronger provisions in it originally for the White House to take on digital strategy, the deputy director of OMB at the time said, we don't want this. It's operational in nature, and therefore inconsistent with the policy role of this institution, which is totally understandable. It's just that the world has been changing such that the delivery really does matter more. And '96 would have been a good time for the White House to take that on and bring on core implementation capacity, which would, of course, include but not be limited to digital. And we didn't. We got to it a little later, but we're getting there.

But when I say that it goes back really deep, I was saying that I showed this chapter to my friend who ran the government digital service in the UK, which I'm sure many of your listeners are familiar with. And Mike Bracken, he looked at me and said, yeah, what you're talking about there is what we call in the British service the difference between the intellectuals and the mechanicals. It's really in there. It's called out in a way that we would never call it out in the US. But it is lurking there, this sense that people doing strategy and policy have the important jobs and there's not much need for them to listen to those doing implementation. And yet, that attitude has clearly been enormously destructive to our ability to keep the trust and faith of the public in the end.

[Mike Burns]:

And that raises an interesting question. We're seeing, because it's very much evidence-based, you're pointing out this concrete evidence that the current structures and culture isn't working. Meanwhile, we're seeing a big push in government around evidence-based policy making and decision making, whether it's federal evaluation, homing in on policies and programs that are delivering the best results. There's a real culture shift around it. And it's certainly very important. But why aren't we seeing a similar culture shift around digital delivery and implementation? It seems to me that delivery-driven policy is really a natural fit with evidence-based policy for all the reasons that you've said.

[Jennifer Pahlka]:

I think delivery-driven policy is just another form of evidence-based policy making. And it happens to come from the bottom up instead of the top down. And it comes in real time. And I do actually think we are seeing a greater adoption of that and more openness to it. When I went back to DC after -- the book hadn't come out yet, but a number of people had read it in its next to final form. And I was afraid no one would talk to me, because I'd told stories that were not all positive.

And so often in government, we just want to talk about the positive. And instead, people said to me, no, this is great. And guess what? I see this changing already, before the book had come out. Because of the work of people like Yadira Sanchez, who's going to be a heroine in the book, and Natalie Cates, and Marina Neto, all of these people who've been doing this, I think really are changing it. And I think when it doesn't change as fast as we want, I think it's often because of language. We're using language that people don't understand. There's even a rift, I think, between the evidence-based policy community and the digital delivery community that doesn't really need to exist. We really ought to be able to talk about these things in very plain language and/or bring them to life. That's why I mentioned this story about Dominic. It's like we use words that sound academic. And because of that, we make assumptions about what the other person means.

I think that there's a great opportunity to integrate traditional evidence-based policy with the kinds of real-time insights and ongoing insights that digital delivery people are just swimming in. Right? I heard once about sort of a conflict between a team doing A-B testing on an app. I think it was like a student loan form or something. And in the digital delivery world, you want to, if you get A-B test results, what? You can get them an hour later. You can get them two days later. Your instinct immediately is then to shift the rest of the messaging to the winning test. And there was conflict between those who wanted to do a much more thorough evaluation of this and even, in some cases, get academic peer review on it before it was changed. And those are some kinds of the culture clash that we need to bring together. There's a good middle ground there where the practices of each of these communities, A, start sounding more like each other, and B, is more intelligible and friendly to the traditional policymaking community that's handling all the politics side of it.

[Mike Burns]:

And do you think part of that is just people wanting to achieve the perfect when really we should be trying to achieve the good, what's good for the most number of people? You share some stories about that in your book. I believe, was at the healthcare.gov forum?

[Jennifer Pahlka]:

About how it could have served 85% of the people?

[Mike Burns]:

Yeah.

[Jennifer Pahlka]:

Yeah, yeah. I think that's true of a lot of them, but yes, that's the one I mentioned in the book. Well, yeah, I think that's exactly right. We go for perfect instead of better. And a little bit better every day is better than waiting for the perfect thing. And that's true across so many different domains. There's a saying that I mentioned, I think only in the footnotes. There's a couple of laws in the book. One of them is Wong's Law, where he says, you know, wherever you are in government, there's always a new system. It's always going to fix all of our problems, and it'll be here in 18 to 24 months. But the joke of that is that it's always 18 to 24 months away no matter when you ask. That time never shortens. And in the meantime, we accept that the system we're using isn't going to be any better, right? It's just, we're always going to wait for the improvements, and then we'll wait and we'll wait. And there's a different approach that says, let's have a team that can improve this system every day, every week, every month in small ways instead of waiting for some giant new, you know, fancy thing to be delivered. And I think that kind of thinking really, you know, can get us a lot further than the, you know, holdout for the thing that's going to meet all the new requirements.

[Mike Burns]:

So back on the topic of implementation, you say that while government needs its own basic digital competency, contractors are a valuable piece of the implementation puzzle. How can contractors best support effective implementation and help keep the focus on and achieve positive outcomes?

[Jennifer Pahlka]:

That's a great question. And I thank you for asking it because so many people have read the book and sort of seen it as anti-vendor, which is just not authentic to who I am or what I believe. And I do know so many vendors that do a great job partnering with their clients. Again, you know, last couple of chapters, I feature a woman named Yadira Sanchez at Centers for Medicare and Medicaid Services. And she has a fantastic relationship with vendors. And I think that what she would say is they provide product management and partner on product management, which means that I think a traditional vendor approach is maybe that vendor or usually a different vendor has done an enormous amount of requirements development.

And the vendor then is going to just fulfill all of those requirements. And one of the things that some large vendors like about that is those projects can get very big. But when you have a partnership with a vendor, somebody like Yadira or a product manager on her team and a great vendor, what they're saying is we've got to prioritize these. We've got to understand what is most important, what really meets user needs. We're deciding what to do in the first place instead of just trying to do all of it. When vendors come to the table with that kind of approach and can partner, they're going to get -- unfortunately maybe for them, I think actually fortunately, do smaller projects or at least projects that start smaller, deliver value earlier and then change, get incrementally better over time. But they have to accept that they're building a long-term relationship that's gonna get evaluated after the initial launch, which is maybe just a minimum viable product instead of, I've gotten a hold of this big contract that I get to continue to manage for the next 12 years because we're locked into 12 years. So I think it's a different approach. And I will say that the vendors who take that approach, I think have happier and more dedicated employees because they know the value they're providing to the public.

[Mike Burns]:

That makes a ton of sense. You've gotten a lot of well-deserved attention for your book and this idea that we need better implementation of digital services to ensure the success of policies and programs. I think that really resonates with a lot of people. How do you hope your book moves the conversation in government? How do you keep up this momentum? And do you think five, 10 years from now, government is going to look different?

[Jennifer Pahlka]:

If it does, I think it will be because of the efforts of the community that are proving that these approaches are better. But I did hope, and I think I'm seeing some evidence that we can start to include higher-level policymakers and even electeds in this conversation. In the very beginning of the book, I talk about my time with the Employment Development Department in California. It was the first summer of the pandemic and there was obviously an enormous backlog of claims as there were in all the other states, unemployment insurance. And it was actually pretty eye-opening for me because it was the first time where my job was to sort of be a liaison between a digital team that I had hired to sort of figure out the problem or they had hired at my recommendation, the head of EDD. But then I also was the liaison with a lot of legislative offices. So I got to see how they were thinking about it. And it's understandable that they wanted to layer on more mandates and quote unquote, hold the department accountable.

But I could just turn around in my chair and be on the other side of that divide and see how profoundly unuseful it can be to throw more mandates on a department that A, doesn't have the capacity to fulfill what it's being asked of it today. And B, is just being crushed under the weight of, in that case, almost 90 years of policy cruft that's accumulated and accreted. It's not like there's one mastermind across those 90 years that says, here's how we will make sure that these departments in the States have the right policy that makes sense for today's world that they can execute on. It's just like every additional person over time across two different levels of government, state and federal, and three branches, executive, legislative and judicial, all of this is just opportunities for people to sort of pile stuff on that makes it so hard for the agencies to actually operate. And none of those folks take responsibility for pulling back and saying, how do we simplify the policy and process and regulations that govern this?

And until they see their role in holding back digital delivery, we're only gonna get so far. I do think we can go a long way with the bottom-up push and with sort of administrative agencies advocating for what they need. But the legislative branch in particular has a lot of learning to do. I'm really proud to say that there's one, so far I can say exactly one person from the legislative branch has come to me and said, I see everything differently now. I get what you're saying, I'm gonna try to act differently. And I hope he's not the last. I have some more meetings set up, but it's a real battle I think, given all else that goes on in lawmaking today and all of the divisiveness and all of the fear to make room for this. But the reality is if we don't make room for this and we don't get better delivery and we don't start to regain the trust and faith of the people, there's gonna be a lot worse problems in legislatures than even, believe it or not, we have today. So it's hard, but we've got to do it and we've got to engage beyond the community that's been talking about this so far.

[Mike Burns]:

So there's reasons to be optimistic?

[Jennifer Pahlka]:

Oh, absolutely. But I think I personally need a lot of education in how to talk to these folks.

[Mike Burns]:

Well, I'm sure you're doing great. I think your book was very compelling. I would not worry about that. But you've written other things besides this book. Back in June, you had a Washington Post column, an op-ed titled Better Government Tech Starts With People, New Jersey Shows How. And as someone originally from New Jersey and who works for an organization that's based in New Jersey, this really caught my attention. What exactly is New Jersey showing us, and how can other states argue what you say is an effective example?

[Jennifer Pahlka]:

So I had the opportunity to visit the New Jersey Department of Labor several months ago now. And interestingly, of course, had the ability to contrast it with what I saw in California in its parallel agency there. And I was super impressed with what they're doing. And it gets back to what I was saying before about Wong's Law. They, in part because they have the Office of Innovation and in part because I think they have fantastic leadership in that agency, had the resources to really understand their own system when COVID hit and it caused the same backlogs everywhere. But New Jersey recovered better. And the recovery is not just -- it's not just that they got a bunch of claims actually paid, but that they're making the experience better for people. And those two things are deeply connected, right?

Like you're gonna have more of a backlog if your users don't understand the letters that have been sent out, you know, or if they don't get them. If they can reply knowing what's asked of them, you reduce the churn on both the administrators and the users. So because they had sufficient internal competence, both the attention of the leadership, you know, all the way down in the organization to the, you know, the most junior staff and these folks from the Office of Innovation who'd already started partnering with them before the pandemic hit, so they had trust between these two organizations. They were able to troubleshoot their way through the pandemic crisis. And then coming out of that say, let's just keep going. We know what we're doing now. We can make these forms easier. We can, you know, adjust all these problems where we see people not being able to file. They had that situational awareness, which I think gets back to, again, the earlier question about evidence. They weren't waiting for a study. They weren't waiting for statistics. They were enmeshed in it and then could see on a day-to-day basis where are people getting stuck? You know, what is confusing about this? And we're simply able to make the system a little bit better every week and every month.

And they keep doing that. And that outfit came out at a time when the feds were clawing back money that had been promised to states for UI modernization through CARES -- no, ARPA -- I can't remember. Something that got clawed back because of the debt ceiling deal. And it really is a tragedy for a lot of states not to have that. But a state like New Jersey is going to be far less hurt by it because they're not counting on some big procurement. And I think all states that got that hit from the clawback ought to be turning and looking at how you develop that competency just in your staff.

Now, obviously, some states with that hit then don't have the money for their own internal staff, which is really, really tragic, which is why we need to educate our electeds not to just give money for contracts. I talk in the end of the book about this idea of op-ex first and then cap-ex. You can get good money out of your capital expense, your contracting, if you have the right internal team before you go do that contract. And they know what they're getting, they're product managing it. That's what New Jersey has done. And other states are going to struggle to do it now with the clawbacks. They all ought to work on their op-ex first and then their cap-ex.

[Mike Burns]:

So let's end where we started on Recoding America. You present a lot of evidence about why things go wrong in government and how we can try to make them right, both through firsthand experience and in-depth research. First, where can interested people find Recoding America? And for folks interested in further reading or listening, what do you recommend?

[Jennifer Pahlka]:

Well, there's a website in addition to the books. You can get the book anywhere books are sold. The audio book has gotten good praise too, though I hesitate to say that because I am the one who read it, so I'm tooting my own horn a bit. Recodingamerica.us is where I put a lot of the stuff that I wasn't allowed to put in the book because it's meant to be general non-fiction and not a manual for practitioners. But there's stuff for people in digital service delivery, for policymakers, for the general public, for people in oversight roles. There's a whole lot of additional resources there on the website. And there will also soon be, if you go on the concepts page, it's called Core Concepts, I'm gonna be giving out little stickers of each of those core concepts. You can see some of them up there now.

So I guess schwag that always helps with spreading the word on concepts that are important to you and might help you in your job. There's so much else to read, and my list of additional reading recommendations is also up on the website. Might start with something like Power to the Public by Hannah Schunk and Tara McGinnis, Hack Your Bureaucracy by Marina Nitze and Nick Sinai. But the list goes on from there, and there's just tons of reading and resources for anyone interested.

[Mike Burns]:

So I'm gonna backtrack with one more question. Your book came out, I think it was about three months ago now.

[Jennifer Pahlka]:

Yeah.

[Mike Burns]:

You don't strike me as someone who is not going to stay busy in this space. You mentioned going to the Hill, talking to legislators. What else is next?

[Jennifer Pahlka]:

I'm gonna try to pull the thread on some of the things I suggest at the end of the book. And when I say pull the thread, I mean learn a lot in addition to -- I don't wanna just preach at people. When I talk about things like oversight, creating a more risk-averse culture, which I don't think anybody intends. I think it's an important thing to understand that I actually don't know how to fix it. So I wanna go talk to people in oversight roles and sort of get conversations together so that they can see it from the point of view of those who they're overseeing who may have that additionally risk-averse response afterwards. I talk about funding. I don't know how to fix funding, but I think it's a critical piece of this whole puzzle.

Part of it, of course, is what I said, like why do we keep giving money for contracts without actually staffing these agencies appropriately? And it means number of people, but also the kinds of people. And it also means being able to hire the right people. When I visit agencies right now and I ask what their problems are, they all say hiring. The time to hire is going up and they can't use the assessments they wanna use and they absolutely should be able to. We've just gotta like dig in and pull apart what's holding them back. And I wanna be an advocate for that kind of change. And I wanna come to it with humility and curiosity.

[Mike Burns]:

Well, Jennifer, thank you. Again, fantastic book, definitely a great read. And I wish you all the best in these efforts. They're definitely appreciated by a lot of people.

[Jennifer Pahlka]:

Well, thank you. It was great to be here and I hope your listeners reach out to me and tell me their stories of change.

[Mike Burns]:

Thank you, Jennifer.

[Jennifer Pahlka]:

Take care.

[J.B. Wogan]:

Thanks to our guest, Jennifer Pahlka. Any resources that we discussed on the episode can be found in the episode’s show notes. As always, thank you for listening to On the Evidence, the Mathematica podcast. This episode was hosted by Mike Burns and produced by Rick Stoddard. Subscribe for future episodes on YouTube, Apple Podcasts, Spotify, or wherever you listen to podcasts. You can also learn more about the show by visiting us at mathematica.org/ontheevidence.