The Role of Evidence in How Local Governments Spend Federal Pandemic Relief Funds

[CANDACE MILLER]

This is transformational money, $350 billion to local state governments. This is such a potential transformative windfall if it can be used well.

[J.B. WOGAN]

I'm J.B. Wogan from Mathematica, and welcome back to "On the Evidence," a show that examines what we know about today's most urgent challenges and how we can make progress in addressing them. On this episode, we're going to talk about the role that evidence is playing in ensuring that federal coronavirus relief funds are going to the most effective and innovative uses.

As background, the American Rescue Plan Act, which President Biden signed in May of 2021, included $350 billion for state, local, and tribal governments to support their response to and recovery from the COVID-19 pandemic. State, local, and tribal governments have a lot of latitude in terms of how they spend that money. They can replace lost public sector revenue caused by the pandemic. They can support public health programs and economic assistance programs. They can offer higher pay for essential workers who bear the highest health risks during the pandemic. They can also use the money to support water, sewer, and broadband infrastructure projects.

One of the eligible uses of the corona virus state and local fiscal relief funds from the American Rescue Plan Act is to improve the efficacy of public health and economic programs through tools like program evaluation and improved data management capacity. The idea there is that we maximize the impact of these funds if they're going towards either evidence-based uses or helping us to evaluate whether a promising program actually worked as intended.

My guests for this episode are Christy McFarland, Zach Markovits, and Candace Miller. Christie is the Research Director at the National League of Cities, which has developed several free online tools that track how the state and local fiscal relief funds from the American Rescue Plan Act are being spent.

Zach is the Vice President and local practice lead at Results for America. Candace is a Principle Researcher at Mathematica. Zach and Candace have collaborated on the American Rescue Plan Data and Evidence Dashboard, which is another free online tool that uses publicly-available plans from local governments to analyze trends and assess the extent to which localities are being guided by data and evidence as they spend their corona virus fiscal relief funds. In the show notes, I will include links to resources from all three organizations.

We recorded this conversation in late January, about eight months after the U.S. Department of Treasury made available the first half or tranche of funds to state, local, and tribal governments. The second half will be made available in May of this year. In our interview, Christy, Zach, and Candace take stock of how the money has been spent, or is being earmarked to be spent, so far and what may change in the future, especially with the release of the second tranche of funds this spring.

We also discuss the value of digital resources, like the American Rescue Plan Data and Evidence Dashboard, for informing local government decisions around the use of these pandemic relief funds that can make a material difference in people's lives. I hope you find the conversation useful.

I wanted to start by asking about after the American Rescue Plan Act was signed into law, all of our organizations – Results for America, Mathematica, National League of Cities – created digital resources about the use, or planned use, of federal funds for from ARPA. Tell me a little bit about the motivations behind these resources. What did you think local government leaders needed to know, and how did you want to help?

Christy, could we maybe perhaps start with you?

[CHRISTY MCFARLAND]

Sure, thanks, J.B.

You know, one thing we realized relatively quickly at NLC was that although cities were very excited for these funds and the flexibility that it allowed, the internal capacity to manage the funds, understanding eligible uses, and the ability really to think beyond a deficit mindset – which many states have been in for quite some time, very significantly across the country. So at NLC, we created resources and data tools to address the different types of needs and aspirations that we recognized in cities across the country.

So first, we transitioned our local COVID policy tracker to begin tracking ARPA implementation plans. In this way, it really helped other cities understand how their counterparts are thinking about the funds. The data tool served as a hub and resources for programs and policies across the country.

Again, the spectrum of cities related to ARPA implementation itself runs from simply needing to better understand the law and eligible uses through highest and best use of the funds and innovations and the like. NLC represents and serves cities of all sizes from New York to Thurmond, West Virginia; and we recognize that although many cities have a solid handle on the basics of the ARPA law and how it works, this isn't the case for all cities.

So to allow NLC to advance our support for cities in terms of best practices for innovative solutions, we also needed to make sure that we weren't leaving anyone behind. So for this, we created a Treasury Guidance Playbook, which was just recently updated with the recently-published Final Rule from Treasury. It's designed to educate local leaders on the value of the historic funding program.

In addition to that and most recently, we're working on a local government ARPA Investment Tracker with Brookings Metro and National Association of Counties that provides a detailed picture of ARPA investments in larger cities and counties. It's extremely interactive, easy to use for cities interested in understanding strategies and approaches to the particular uses of ARPA.

The bottom line is that what we've recognized at NLC is that for ARPA to be successful, we need to make sure that the least-resourced communities have the support that they need. So NLC has been there to fill the gaps and help communities think creatively and smartly about these funds.

[J.B. WOGAN]

Candace and Zach, you also created an online resource, the ARP Data and Evidence Dashboard. And, like National League of Cities, you all are engaging with local decision-makers and are able to get a sense of what they need to know. So what were your intentions/motivations, and what were you responding to in creating this dashboard?

[ZACHARY MARKOVITS]

Well, thanks first, J.B., and Christy and Candace. To have this conversation today is exciting.

Results for America, just as a little background, works at all levels of government – federal, state, and local – to help imbue data and evidence into decision-making with the goal of improving the lives of residents and belief in the power of government to help deliver real change. Over the last seven years, we've worked on our local side with over 250 local governments in a variety of different capacities, from sort of measuring how well they are using data and managing their cities through the What Works city certification through actual sort of direct technical assistance and support alongside them.

In those relationships, as for that year throughout the pandemic and as the American Rescue Plan was passed, we had heard most of our sort of local governments kind of sync three big messages. This dovetails really well with, I think, what Christy has just said.

One, the first thing was, "I'm burned out. I am exhausted from the year-and-a-half since the pandemic started and the rapid response we have to do, and now we're faced with incredible opportunity and yet challenged at the same time with just an unprecedented amount of dollars flowing from the federal government into our local coffers, which are sorely needed, and yet we want to make sure we're doing it strategically." This leads to sort of the next two things, which was, two, what are other people planning to spend their money on? What trends are you seeing? Who stands out, even if it's not a trend as like a best actor in this space?

Then, third is what actually works? What is sort of strong or leading evidence in programs that we can copy, borrow, and steal? There is no stealing in local government-support work, so how can we take what's best out there and try and implement it in a way that is going to be respectful to our own community?

So we reached out to Candace and the team at Mathematica to partner with us on what was initially a small project to inform our own internal priorities. We work with government and have worked with many governments, and we wanted to step up in this kind of call of help but did want to do so strategically. How can we make sure that we are not pushing an agenda but really working hand in glove with what is needed at the local level?

Very, very quickly and supported by the Bill and Melinda Gates Foundation, we realized that this data needed to get into the hands of practitioners who were needing to make fast decisions and couldn't wait on us or really anybody to try and internalize this data and figure out a program that could be set out. So we quickly turned what was a kind of internal-facing dataset into something that could be really, we hoped, useful for practitioners with kind of three big things:

One, what is the planned spending that exists out there? We looked at 150 of the Treasury performance reports that were sent at the end of August. So what was that planning/plan spending out there?

Two, how are cities and counties and tribal governments aligning those with the best practices around data and evidence? There was some guidance, which I can talk about in a second, from Treasury, around certain key provisions around data and evidence and equity capacity.

Then, how do we shorten the distance between somebody in, let's say, a county CEO's office or in a mayor's office and having an idea that we want to spend on workforce or we want to spend on housing because it's a critical priority for our community and the ability to connect with somebody in another jurisdiction that they may not even know to have an understanding of like what are they doing so they can see how they can kind of pivot and borrow and steal. So I'm excited about doing something in workforce. I don't know any of the new traits, but I've heard really cool things. I see they're highlighted on this dashboard. How do I connect with them in a real way and bring them together?

[J.B. WOGAN]

Candace, I'm curious both from your standpoint as somebody who's been working on contact tracing and public health during the pandemic, but also as somebody who has experience as a local government elected official -- you're on a school board -- when you were thinking about how this money might be spent and where the gaps were, where the needs were, what came to mind for you? Where did you think local governments could use more information?

[CANDACE MILLER]

Great question – so let me say that I've been working in public health within the pandemic now since March of 2020 and really understanding the limitations of our current public health infrastructure. I serve on a local school committee, so I have been very much involved in sort of the politics and the questions around...what do we do? What do we do around testing? What do we do around contact tracing? What do we do around keeping students in school? What do we do about the mental health issues that are coming up not only in students but in parents and staff and sort of the whole community?

I also have the viewpoint of being a foster parent and so have been interacting with the Massachusetts Department of Children and Families throughout this. What I see is such exhaustion and burnout and such a deep need for investment, but investment that is not too hard to get. I think Christy, in fact, made some really important points about how policymakers are exhausted. This is transformational money, $350 billion dollars to local state governments. This is such a potential transformative windfall if it can be used well. I think that's where we all sort of come in and say how do we help these locations? How do we help these cities/counties/states make the best use of money?

There is some time; but of course, time is short and the challenges are massive. So I'm thinking about the context all the time, right? We've got this incredible political divide, and some are less interested in focusing on the power of the money and more just sort of critical. So I think it puts an additional burden to say how do we make sure that we use this money in a way that truly yields the outcomes that we all want to see.

I think that by focusing on evidence and focusing on data and focusing on what works, it gives policymakers information to make the best decision; and it gives the accountability necessary for the country to feel good about this type of investment.

[J.B. WOGAN]

Okay, that's perfect.

Zach, I know you've put out a blog post. There was an op ed in The Hill talking about, well, localities have a fair amount of flexibility in how they use this money; but Results for America has some views on how the money should be spent. Could you talk a little bit about what is your position? Ideally, what would you like to see from localities and states and tribal governments as they're starting to put this money to use?

[ZACHARY MARKOVITS]

Yeah, sure, so as I had mentioned previously, Results for America believes in the power of government to deliver dramatic improvement in the lives of all residents; and that, A, is not the critical tool in being efficient and effective at reaching that goal is the use of data in decision-making and the alignment or the shifting of dollars to program evidence – that evidence shows what really works away from what's been less effective towards what we would say is more effective.

We were really heartened to see that U.S. Treasury in their guidance documents to jurisdictions on how to spend the American Rescue Plan state and local funds included these like five key provisions within its guidance that incentivized state and local and tribal governments to invest these dollars in ways that helped to advance economic recovery, economic ability, and racial equity but using an evidence-based approach.

So we, as you'll see evidenced in the tool that we've put out with Mathematica, highlight these five provisions, which include kind of building data and evidence capacity; the use of data and evidence; investing in evaluation and evaluation of programs; engaging communities in this process; and ensuring equitable outcomes. So obviously, the primary goal of these funds is to ensure that jurisdictions are able to maintain their workforce and actively fight this pandemic. But we believe that these dollars also provide a way to do this and at the same time building that long-term capacity and resilience to use data to align resources and evaluate programs, proving what works, and sort of use that to fight this pandemic and at the same time have that long-term opportunity for your local government.

So for RFA's perspective, there are two common challenges that exist across all local governments in some ways; but in lots of ways, every challenge is local and there are specific priorities that mayors and county executives and tribal leaders are going to be facing. So we're not sitting in the way of that or pushing any one priority to any local government. There are too many, and many organizations out there saying exactly what you should or should not be spending money on right now.

However, what we do believe is that however you're going to be allocating these dollars programmatically, it should be done in a way that's helping on this long-term capacity that allows you to kind of build out the ability of your government to do more and helps as you look at this long-term opportunity to spend dollars effectively.

One last thing, earlier this year I think many experts assumed that cities and counties would be using these recovery funds to avoid service cuts and layoffs of local government employees. While that's true and they are filling budget holes, what our data has shown – and what we've been excited to see – is that many cities and counties are now also investing in innovative and data-driven solutions that could have lasting impacts for residents.

For example, we saw that one-third of jurisdictions that we reviewed are piloting new programs with these recovery funds showing that local government is continuing to be these new laboratories of democracy to sort of shift that Brandeis sort of term there. What we want to be able to see is that they're able to then evaluate these programs so that not only do we have a better sense of whether they are working – and piloting is an extremely exciting opportunity these funds can help to deliberate – is not only to evaluate them so that after this money is spent, the larger community of cities and counties can learn from any individual projects that are happening across the country and more people can get more efficient with these dollars even when they go away.

[J.B. WOGAN]

I did notice last week that the city of Minneapolis announced how it was going to spend some of its American Rescue Plan money for a guaranteed basic income pilot. In the dashboard that Mathematica and Results for America created, there were 19 jurisdictions across the country just in the 150 plans that we reviewed that had said they either were spending or planned to spend towards a guaranteed basic income pilot. So that is at least one interesting example. I think it's going to get a lot of coverage and a lot of attention – is what we learn from those pilots.

Christy and Candace, what were you looking for in terms of how this money would be spent?

[CHRISTY MCFARLAND]

Yeah, I think a really interesting point about what Zach was saying in terms of the sort of guidance principles behind ARPA, particularly around racial equity in particular -- what we're finding so far is that many cities are using a large portion of their funds on "revenue replacement," which really is sort of stabilizing operations. As time goes on, we are seeing sort of more innovative and sort of larger-scale projects; and we anticipate to see that as local governments stabilize their operations, get things sort of straight in the house first, right?

One thing that we've noticed that sort of – it's not necessarily a broad-based trend necessarily, but we're seeing places that are dedicating a larger share of their ARPA dollars to revenue replacement are typically those that have been hardest hit both economically and fiscally as a result of the pandemic. We know, for example, Philadelphia is one of those communities. The impact from the pandemic led to severe cuts in services and the need to then apply ARPA dollars to restore those services.

But again getting back to those core principles although service levels need to be restored within the city, there's still an opportunity for the community to think about those basic principles within ARPA, which is prior to the pandemic which communities already had depressed service levels? Can we think about restoring service levels in a way that's more equitable? So even within approaches in terms of application of ARPA that may seem less innovative, per se, there are still opportunities to increase equity within the cities based on how those new strategies are being applied.

[J.B. WOGAN]

Candace, anything you would like to add?

[CANDACE MILLER]

I think that we looked at the 150 municipal and county plans, and then we looked at 50 states. We really just wanted to see, looking across the different categories, how were governments filling these. I think it's important to note the incredible variation across the reports. Some are very extensive, and some are a little bit thin. I think digging into the details about how the location sees its priorities and sees its challenges and then the plans they have, I think it's really endearing to see the sort of hopes and dreams that these locations have. The ones that really have dug in, you really want them to have the supports necessary in order to make them happen.

[ZACHARY MARKOVITS]

If I could just build on that a second, I think that between what Candace and Christy just mentioned there are, as Christy mentioned, there are a bunch of communities that have been hard hit that are spending a lot of their money on making sure that they can restore services, making sure they maintain staffing levels in really important ways. And everyone through these plans in some ways are conveying their hopes and dreams, and trying to be ambitious even if they have to be devoting most of their state and local fiscal relief funds to maintaining service levels and getting help to people that need it.

But the ability to look at the second tranche of funding and where it should go next, and by looking at this larger community of cities and counties and tribal nations that are trying different things and sharing these practices, it will hopefully have set up well for this next tranche of spending and this funding to be innovative in broad and new ways.

I mentioned we were excited to see the piloting. As a part of this -- Candance mentioned this -- some of these plans are really ambitious; some of them are not, which we can talk about sort of what that looks like. A lot of that are products that have just sort of been deadlined by when they had to submit this report. But because of that, we think that the data that's here is really a baseline. There's a lot more innovation, a lot more exciting programs, that are happening across the country that are just not captured by the formal report that they had to submit to Treasury. So we'll see more over the next year of what innovation is happening out there.

So that one-third of jurisdictions doing that -- I think it's a much, much larger number. And yet even despite what Christy was saying about cities like Philadelphia really having to spend a lot of their money on restoring services, the focus she mentioned on equity is so important. About 77% of jurisdictions that we reviewed are in some way targeting their funds to help ensure equitable outcomes for residents – including places like King County, Washington, that are using an equity impact tool, strategic plan, and dashboard to ensure these dollars are leading to equitable outcomes for these residents.

These are really important not necessarily innovations, but important framing and sort of tools to put around the use of these funds; and it's very exciting to see. I think that that opportunity – the ambition that's here – is pretty broad; and I expect to see more over the next year as additional funds, additional (inaudible).

[J.B. WOGAN]

Okay, great, so I want to dig a little bit more into the online resources that both the National League of Cities and our organizations at Mathematica and Results for America have created around ARPA. I want to start with the ARPA Data and Evidence Dashboard. So we've put this out, and then you've had some workshops and different stakeholder engagement events since then. I'm curious, after we put out this tool what kind of response did you get? What kind of feedback have you gotten? To what extent has it influenced the ways that localities are actually spending the money or thinking about spending the money?

[ZACHARY MARKOVITS]

So, one, we know that it's being – the surveys are showing that it's being really well-read. What's exciting to see is people are actually spending a fair bit of time on the site – five, six, seven minutes – before clicking off to go often to another city or county report. So one of that key goals is sort of like spending time to see what's there and then going to the place, to the jurisdiction, that has something innovative or new. That's an exciting sort of just like data trend we're seeing.

We also know it's begun a conversation among those who are planning for tranche two, the second big tranche of money -- and for those who are not aware, the state and local and fiscal relief funds are coming in two tranches to local governments. One came last May; one should be coming this spring, likely around May. So although it's anecdotal from those that have reached out to us to get some deeper knowledge, we do know they're using it for those planning purposes. We at Results for America, or Results for America, are hoping to launch a set of supports over the next few months that will work alongside jurisdictions who are looking to kind of spend their money in strategic ways that align deeper with the state and local data and evidence use.

One interesting thing on this quickly is one analysis we did on the data was seeing how previous building of capacity actually led to stronger plans. So one thing that we thought was quite interesting was in fact local governments that had previously invested in building data and evidence-based capacity for decision-making seemed to have created stronger, higher-potential American Rescue Plans spending plans. We used as a proxy for that data and evidence decision-making the What Works city certification, which is a measurement of how well city governments are managing using data and evidence.

So that correlation we're seeing is very exciting because as cities are looking to invest more now, it's a way to say if you invest more now, it will actually help you set up for the future and not just in meeting the moment as we are.

[J.B. WOGAN]

Candace, I saw you nodding a bunch. Is there anything that you wanted to add to what Zach said or does that sort of capture the gist of it?

[CANDACE MILLER]

I think I would emphasize that as we reviewed all of these plans, it was very, very clear that those locations that have invested in data evidence capacity – their plans are much stronger. They are using evidence-based programs, so they know that the program that they plan to fund has an evidence base that they know it's going to work.

They may need to tweak it, but they know it's got a strong likelihood of being successful. They have performance management systems. They have metrics that are defined. They have a timeline. They know the outcomes that they want to look at. They know that it's a process of looking at the data and making mid-course corrections and so forth. They have some rigorous evaluation plans. So all of these different processes really come from governments investing in evidence and data capacity.

So this is not only – as was said, not only this great opportunity to do some incredible programming, but it's a great opportunity to build data and evidence systems that will be long-standing for decades to come. Having that capacity is such an asset, and it will help decision-makers make the best decision across a range of different types of programs and resources and outcomes for many years to come.

[J.B. WOGAN]

One thing that I know Results for America does really well is this kind of recognition, lifting up of north stars, both through the What Works certification process but also the state and federal standard of excellence reports that come out every year where you're actually calling attention to the stellar examples in federal and state and local government who are kind of leaders in the use of data and evidence.

Zach, it may be too early to know in this case; but I know that this dashboard does have a section where there's like a composite score where it is essentially casting a spotlight on some of the best in show from the plan. Do you have any sense of whether localities have appreciated that positive attention, whether localities who didn't are eager to get a higher score in the dashboard – anything along those lines?

[ZACHARY MARKOVITS]

Well, we have heard from a number of communities that are higher up in that sort of scoring that they really appreciated and actually spent a lot of time focusing on that as a part of their key plans. Especially this is true, I think, among the communities that are often not necessarily always lifted up to the top of the heap on this. There are lots of really forward thinking communities in Washington D.C. and New York that you sort of hear about a lot when it comes to this work.

Washington County, Oregon, as an example of a place that has really been aggressive in the way in which they're trying to approach this work. It's exciting to see them also be recognized in the way in which we did that, but also have other people look to them as possible exemplars of ways that they can sort of borrow and steal.

The last piece I'll say is I tend to look at this, and hope other people do, as this is not a place to cast doubt or castigate those who are lower on that score because of the limitations in the plans that we're actually looking at. It's more about those that are higher up on that who expressed it in their plan, in a way can serve as an exemplar, can serve as an opportunity to look towards what's happening right there, and can share amongst each other as we're trying to share amongst counties and cities what works best out there.

So I look less at those that are on the lower end because there's lots of reasons why they could be there that have nothing to do with their actual plans for their success there. The ones that are higher have a really good explanation of what they're trying to do, how it aligns with data and evidence.

[CHRISTY MCFARLAND]

Zach, I think that's a really good point, particularly about exemplars and being able to use the data tools that are best for your organization, our organization, and the ways that cities are actually using these tools, one, obviously to identify these exemplars. But one thing we've noticed in particular is that over the past six months, many cities have been less than 100% clear about how they can use the funds. So there's a lot of flexibility in the funds. There is also only an interim rule that they were working from, from Treasury for the longest time.

So the Final Rule about very specific but about how funds can be used just came out in recent weeks. So because of that, although cities wanted to commit funds and go forward with projects, they were still a little bit hesitant because they wanted to make sure that they were doing the right thing and using the federal funds in a way that was responsible.

Being able to see exemplars, being able to identify cities within those tools, gives them a sense of, yes, we want to do something similar; and other places are doing it. So this gives us a little bit more assurance that this is something that we are allowed to do, that we're permitted to do, that will be seen sort of as a good use of these funds. So just being able to provide a certain level of assurance and certainty that their ideas and their programs and policies that they've been envisioning can move forward, I think having examples from other places across the country has really helped move that along.

[J.B. WOGAN]

Christy, you've started it on what I was going to ask about next; so thank you. I wanted to ask if you could kind of give us and the listeners a status update of how the funds – where the funds are today. I know only the first tranche of two has been made available; but remind us how much money has been released, if we know how much has actually been spent by localities, and any notable trends. I think you actually referenced a couple of trends in an earlier answer – but any notable trends in terms of who's been able to spend that money and how they are spending that money.

[CHRISTY MCFARLAND]

So the Treasury Department has released the initial tranche of funding, which for cities in particular is half the amount of funding. Then in May of 2022, they'll release the second tranche of funding. Most of what has been available have been city plans, so understanding how cities want to spend the funds. There are some cities that have also provided plans about how they've actually spent the funds, so it just depends on the different reports that are acquired through Treasury; and that's the gold mine and sort of the data standard of how we're getting our information.

There are some fine lines around sort of how cities are planning to spend the funds and how much has actually been spent. We continue to attract spending plans and funds spent, and we will be releasing that in our new data tool. We're looking primarily at the largest metro cities, as well as sort of over 50 of the smaller municipalities. What we've noted is that nearly 70% of the funds that have been allocated have been sort of either planned or expended as of now, which is pretty good.

So within that, when we look at the largest municipalities that have eligible funds, what we're seeing in terms of trends, again, is that most of the top three spending categories, at least to date, government operations and investments. Again, that gets back to revenue replacement, service levels, as well as filling budget cuts. So we know that, again, this is sort of what we were expecting initially out of the gate – that cities would be stabilizing their operations.

We're also seeing a very large portion of funds going towards community aid, including youth and family support has the most allocated funds in the spending category. So many cities are funding youth and family support programs. Housing as sort of the third for largest cities; homelessness is sort of when you think about housing as a big policy area, sort of what are some of the sub policy areas that we're seeing rise up, and homelessness is sort of key to that.

When we look at smaller communities, the trends are very different. In smaller communities, most of funding – the vast majority of funding -- is going towards different types of infrastructure. As you could imagine, smaller communities have greater needs as well when it comes to a lot of these infrastructure challenges and just not having past access to some of the funds that they do have now. So those are some of the trends that we're seeing, particularly for the smaller communities, looking at things like water and sewer projects; broadband, obviously, with the larger cities focused more on community aid support, homelessness, and government operations.

[J.B. WOGAN]

All right. Candace, based on the plans that Mathematica and Results for America reviewed, what role is evidence playing in helping cities spend wisely now and in terms of those long-term transformational uses as well? If you don't mind, give a few examples of cities that are implementing evidence-based programs or gathering new evidence on promising pilot programs. I know we've already talked about guaranteed basic income, so I don't know if you want to continue on that specific example or if you have others in mind.

[CANDACE MILLER]

Sure, there are some great examples – there really are a host of interesting examples of evidence and data evaluation capacity. I think they fit into a number of buckets. So there are some locations that are doing learning agendas. So they're looking across either their ARPA funds or programming areas or just looking across government services and saying what are our learning gaps? What information do we know, or do we need to know, in order to make the decisions?

So Boulder County, Pima County – they're good examples of places that have a learning agenda, and they're sort of building their evidence needs based on that. There are some locations that are requiring logic models. For example, if a county or a state or a city is asking community organizations to apply for money, they are requiring that the community organization has a logic model and shows what are the inputs, what are the outputs, what are the activities, what are the expected results, so that they know that the organization applying for money has a real idea about what they want to do.

Baltimore is a city that is working closely with community-based organizations to really train them so that they can fill out these applications for the money. So that's a great example of using this money in a thoughtful way.

There are places that have performance management frameworks and performance management systems. Seattle is a city that has a performance management framework, and that framework really guides the thinking about all other program implementation and the data that's collected – again, the indicators and the metrics and the timeline and the outputs and how they're going to use that information to make changes along the way.

There are some locations that have standardized data systems so that they're looking to collect data really in a standardized way so that it can be more useful to more people and transparent. Washington County in Oregon is an example of a location that's doing that.

There are a number of data dashboards that different localities are using. Maricopa County is an example of a location that is putting all of their information about program outcomes online, navigable, so that the community and policymakers can look and see what the results are.

New York City has a plan to create shared data systems, so the data that comes in from these different programs can be used among partners and shared in a way that fosters different improvements.

One of my favorite examples of an innovative program is in San Diego, where they have a system of community health workers that have been used to promote health outcomes – particularly have been used during COVID. But San Diego is doing additional evaluation, and they are trying to be innovative in thinking about how can they come up with a sustainable community health worker program that extends far beyond COVID and is something that really is integrated into the county and promotes a range of health outcomes across the life course.

So there's many more examples, but I think those are the buckets that we're seeing; and it's great to see. I hope that other cities/counties/states that are receiving this next tranche of money are thinking along these lines. Starting with that learning agenda to really identify what you want to know and having a strong performance management system are really key to success.

[J.B. WOGAN]

Okay and, Candace, if I could just stick with you for one more. The other question that I wanted to ask was as cities receive the second tranche of funding, the corona virus fiscal relief funding for state and localities, and they start to invest more of the federal relief dollars, where might there be room for improvement when it comes to ensuring that the money is actually supporting the most impactful uses?

Zach and Christy, I would welcome your input on this as well.

[CANDACE MILLER]

When we look at some of these great examples, what we're seeing is that probably about one-third of local/county/state governments are really pushing the envelope and doing best practices around data and evidence. There is a tremendous opportunity for these other locations to also be digging in and diving in and thinking about how can they build their data and evidence capacity.

So there are some wonderful examples, but there are more opportunities I think for growth. There are some localities that are doing maybe one or two of these things; but they don't have a full, robust program that is – you know, and all the capacity that they need. So we know, for example, that technology across public health/human services has been under invested in; and it's just not fit for purpose. So creating really fit-for-purpose data systems that are user friendly for the people who have to manage them, I think that's an additional opportunity.

There really are an endless amount of opportunities. It doesn't have to be perfect. Perfect is the enemy of the good, right? But there's so much opportunity to push the envelope and build on a better practice around data and evidence.

[ZACHARY MARKOVITS]

Really quickly, just one other thing I want to add because I thought Candace's answer was great, is that one thing we saw is that less than half of jurisdictions have shown clear and promising investments in evaluating the effectiveness of their investments. Evaluation is really important, especially for places that are piloting new things. I mentioned this before. But cities and counties and tribal governments that invest in evaluation will just help to ensure that their programs and policies are working and helping residents to recover better from this pandemic and will feed into the larger knowledge base that we have across cities and counties about what works in this space.

So again, to the point that I made previously, it's an amazing opportunity to give back to the larger group -- to have a better sense of after this money goes and it's no longer there, how do we take our limited budget dollars and put it towards what works. We have an opportunity now to do an incredible multilayer research project, in some ways, across the country to try and see what is actually going to be working here so that later on when these dollars are not there, we can shift those dollars to the most effective use out there.

[CHRISTY MCFARLAND]

Yeah, Zach, and I think in terms of that too, I think one of the key components to ensuring success and that I think cities probably could use a lot of help with in terms of being, again, more impactful going forward is stepping back and looking at the big picture a little bit. I know they have a lot of pressure on their time, but understanding that the local fiscal relief and the state fiscal relief is just one stream of funding that's coming into their communities. There's a whole host of funding that's coming both from the American Rescue Plan, other federal sources, as well as massive new investments now coming from the infrastructure law.

So with that, it's really incumbent on local governments to understand what is their role and how can they be most impactful with their dollars in terms of providing a value add, leveraging private funding, as well as other sources of federal funding that may be going to, for example, a small business. Is it most impactful for local governments to set up a grant program for small businesses who may also be getting funds from other sources, or do they have choices to make about how they create longer-term fixes around maybe vastly updating their permitting process through technological solutions?

I think there are a lot of ways that cities can be mindful of additional funding coming into their communities, both to them directly or to small businesses and residents from the federal government and private sector; thinking about how can they leverage and add value to these additional funds; and then also using their funds in a way that will provide long-term value. Obviously, there are going to be a number of short-term fixes that are required; but also, what is the longer-term value add that the public sector can add?

[J.B. WOGAN]

Christy, thank you, that is such a really good point. Places like NLC and DACO have like such good resources on the various maps of funds that exist on the federal level, and it's important. I would just add to her excellent points there, which is there is also an opportunity to enter jurisdictional elements here -- the city, county, school district, private sector, nonprofit sector – that are all in some ways benefitting from this but need to be working not in cross purposes but in collaboration. It gives us an opportunity to make these dollars go further.

It also makes sure from a collective impact perspective that we're not stepping on toes and are working in a large way towards the collective well-being here. This is an amazing opportunity. We saw over the pandemic kind of an unprecedented collaboration between levels of government, city and county especially, when it comes to kind of tackling these health department staff by the data, staff in a mayor's office that are coming over to provide support. Some of those (inaudible), a lot of those kind of muscles if they're just exercised again can be bulked up again; and I think this is a great time to do it.

So as we look at where is there an opportunity to do more, although it is hard in this exhausting time having that second-level conversation, making sure that we're aligning, is going to be so, so helpful and critical if we're going to take advantage of these resources.

[CANDACE MILLER]

Yeah, Zach, and I think the data tools that our organizations have developed can help advance that in some way. As we begin to sort of grow the number of places that we're able to track, I think having sort of a composite picture of what it looks like in a region in terms of the different plans that have been provided – cities, counties, other entities within that region can look in fuller aggregate to say, okay, these are the other things that are happening. Obviously, they should be starting from the ground up and really understanding that from the local and regional perspective; but I think having some of those other plans collected in a way where they can see all of the activities that are planned for, that region is going to be really impactful.

[J.B. WOGAN]

Christy, earlier you had mentioned the Final Rule for Treasury that just came out recently, within the last month; and it provides some additional guidance. I think it seems pertinent/relevant to our conversation about the role that evidence might play in the use of ARPA dollars. My understanding is that it does put a thumb on the scale of making sure that evidence and data are not an afterthought in the use of these dollars.

This is a question to the group, but what are the implications of this Final Rule; and how might it change the planned use of those corona virus fiscal relief funds?

[CHRISTY MCFARLAND]

One of the biggest changes and implications of the final rule is on the sort of tests that cities need to utilize in order to determine how they spend their funds. So this is a huge opportunity for smaller jurisdictions, particularly those with limited staff resources. The interim final rule, local governments had to perform sort of a four-part test to determine their lost revenue or revenue replacement in particular. Now the Final Rule has a standard allowance of $10 million that can be used toward government services, which includes any service traditionally provided by the local government.

So according to our data, 26,000 local governments have an award through the American Rescue Plan of less than $10 million. So it's an illustration of how many governments will be impacted by this change. So that really is just an additional flexibility for small places, which I think then also goes to the need for additional data-driven approaches and ensuring that the flexibility is matched with an evidence-based approach.

Other changes to the Final Rule include additional flexibilities and great flexibilities for local governments in allocating funds to specific water and broadband infrastructure projects, as well as housing and housing assistance programs. I think having this additional clarity as well as additional flexibilities is really important, again now that we have additional influx of funding related specifically to infrastructure through the Infrastructure Bill. It allows cities to really understand the landscape in a more holistic way.

[J.B. WOGAN]

Zach, is there anything you would add to what Christy has just said about the ways that the Final Rule changes the picture of how evidence might inform or influence use of ARPA funds?

[ZACHARY MARKOVITS]

The Final Rule also clarifies that ARPA dollars can be used for kind of all aspects of building and using data, outcomes-based budgeting, outcomes-based procurement, performance management, and to make these much more impactful in local jurisdictions. It specifically, actually, called out the ability for state and local governments to hire staff and build a capacity for their teams to do this work, which was hinted at in the early guidance; but the Final Rule really cemented this.

They specifically said in order to push this, and I'm quoting here, "Governments with a high capacity to use data and evidence to (inaudible) programs are more likely to be responsive towards the needs of their community or transparent about their community impact and more resilient to crises, such as the pandemic, and its economic impacts."

So it's clear Treasury is trying to work alongside state and local governments in a way that allows them to do this and to build this effectively. It helps to buttress these five key data and evidence and outcome provisions in the ARP program to respond more effectively to crises and to improve the impact of using these (inaudible) investments.

In general, we just have heard that they made clear what's allowable and what is not is so important to jurisdictions. So it makes it clear that attacking this pandemic by investing in one's capacity is paramount. For those who have previously been on the fence -- especially in the smaller jurisdictions as Christy was highlighting before, which are many – about how to spend and what to spend on, this should really be a breath of fresh air as they enter into the heat of the planning process and are trying to figure out exactly what they should be doing not only with the first tranche but this upcoming second tranche.

[J.B. WOGAN]

The last question I wanted to ask – when the American Rescue Plan Act became law last year, the vaccine rollout was picking up speed; and some of us – I'll just say I was really hoping that by early 2022 we would be moving out of the public health emergency and focusing more fully on the economic recovery.

I was wondering, does the rise of first the Delta variant and now the Omicron variant, change the picture in terms of how localities need to spend this money; and do you think we might see a shift toward pandemic response and a shift away from other imagined uses – long-term economic/recovery type uses?

[CHRISTY MCFARLAND]

That's a tricky question. I think Omicron has fully exhausted all of us, and Delta exhausted us. Omicron further exhausted us. But I think more than changing, I think it really clarifies and underscores what governments I think maybe knew they had to do anyway. So we know, again, public health infrastructure is really in need of modernization; and Delta and Omicron have just further confirmed that. We know that our essential workers need more support; they need policies; they need economic relief.

The neighborhoods/communities need to have investments to mitigate the impacts of Omicron. We have these ongoing health disparities – health care disparities, economic disparities, homelessness, mental health issues. All of those things existed before, and they have been further exacerbated; and I think that the lens on what needs to happen to reduce those issues, reduce those problems, build the public health systems/human services systems – it's sharper in focus. We really understand. We're sort of doomed to repeat mistakes until we solve some of these problems.

So I don't know that things are going to change, but I do think there's a greater sense of urgency to meet these challenges with effective solutions.

[J.B. WOGAN]

Zach, do you think that the picture has changed at all?

[ZACHARY MARKOVITS]

I don't know if it's changed at all. I do think that the pandemic, as many people have said before, has just laid bare many of the challenges that our public systems and our society has to face – both the sort of challenges to think about long-term capacity in our local governments and within our public institutions, the inequities and inequalities that exist within our communities and that are exacerbated by an epidemic like this.

So the ability to invest in the delivery of services, whether it's a specific pandemic response or long-term service orientation for our community, in some ways it's six in one/half a dozen in the other. In some ways it's an investment in the ability to get people the help or the services they need in order to improve their lives and the ability to direct resources in a way that is efficient, effective, and treats residents both as the recipients of services and as taxpayers who are part of a larger social contract that they believe in and want to have delivered to them the best services possible.

So we may see some shifts on the margins. We may see some shifts here or there. Ideally, if by investing back in the institution itself in order to build a more (inaudible) equal, we should see better government, better services, and we hope improvement in the lives of people and hopefully a shortening of this pandemic in a way that allows us to (inaudible).

[J.B. WOGAN]

Thanks to Christy McFarland, Zach Markovits, and Candace Miller. In the show notes, I'll include links to the online tools and dashboards we discussed in the episode.

As always, thank you for listening to another episode of "On the Evidence," the Mathematica podcast. There are a few ways that you can keep up-to-date on future episodes. You can subscribe on Apple podcasts, Spotify, Stitcher, YouTube, or anywhere else you find podcasts. You can also follow us on Twitter.

I'm @JBWogan. Mathematica is @Mathematicanow.