Advancing Equity and Addressing the Climate Crisis in 2022, the Year of Evidence for Action

[CHRIS TRENHOLM]

Fundamentally then was that Mathematica was working at a distance from underserved populations who brought expertise about the programs and the evidence that we were creating that we were not recognizing.

Below that is a root cause that is rooted in systemic racism and that that cause is something that an organization like Mathematica, and many others like us, need to recognize and work in not merely an acknowledgement of but in an effort to further deconstruct and impact.

There are parts of me at 56 years old that wonder why at 26 years old I wasn't able to see it in the way that I do now, and it requires a lot of hard work and humility on our part.

[J.B. WOGAN]

I'm J.B. Wogan from Mathematica, and welcome back to On the Evidence, a show that examines what we know about today's most urgent challenges and how we can make progress in addressing them.

For the final episode of 2022, we’re continuing our annual tradition of inviting three guests from Mathematica to help us make sense of the past year and the role that evidence played in it. This time, our guests are Nancy Murray, Jill Constantine, and Chris Trenholm. Together, they oversee Mathematica’s International, Human Services, and Health business units, respectively.

As some listeners may already know, 2022 was an important year in the movement to embed evidence in public policy and decision making. The White House declared 2022 the “Year of Evidence for Action,” which involved a series of events, culminating in a White House summit co-hosted by the Office of Science and Technology Policy and the Office of Management and Budget.

When the White House announced the initiative, it explained the motivation for promoting evidence-based policies and programs this way: “Making policies based on the best-available research and data, with scientific integrity at the heart of an evidence-based approach, is critical to keeping the American public safe, healthy, informed, and economically prosperous.” The announcement went on to say that “people benefit when evidence informs U.S. government decisions.”

That’s music to our podcast-listening ears.

The White House’s activities around its Year of Evidence for Action built on several other important milestones in recent years, including the passage of the bipartisan Foundations for Evidence-Based Policy Act of 2018, and the Presidential Memorandum on Restoring Trust in Government Through Scientific Integrity and Evidence-Based Policymaking.

Those milestones are important backdrops for the conversation I had with Nancy, Jill, and Chris.

Through the International Unit that Nancy oversees, Mathematica works with partners in more than 50 countries across Africa, Asia, and Latin America to address health and social challenges related to child literacy, clean energy, agricultural development, maternal and sexual reproductive health, water and sanitation services, and more.

Through the Human Services Unit that Jill oversees, Mathematica works on a wide range of domestic social policy issues, including nutrition, employment, criminal justice, child care, child welfare, child support, K-12 education, higher education, and more.

Through the Health Unit that Chris oversees, Mathematica works with public and private sector partners on improving programs, such as Medicare and Medicaid, and employing health information education technology to address the quality, efficiency, delivery, affordability, equity, and financing of health care.

On this episode, Nancy, Jill, Chris, and I discuss where we see encouraging signs of leadership and other forms of progress in the use of evidence. We talk about the increasingly important role of evidence in measuring and addressing health and social inequities. And we give reasons for being optimistic about the use of evidence in improving people’s lives in 2023. Spoiler alert: the increased attention being paid to developing solutions to climate change and its knock-on effects is one of them.

Like last year, this episode is being released the same week as our annual year-in-review web page, which features a curated list of videos, podcasts, blogs, interactive data visualizations, and other research products that capture some of our most impactful work of 2022. I’ll include a link to the page in the show notes and I highly recommend you check it out.

A full transcript of our conversation is also available on the Mathematica blog.

I hope you enjoy the episode.

[J.B. WOGAN]

All right, great, well the first thing I wanted to talk about as we're reflecting on the year that was is any signs of progress that you all have seen in the use of evidence. I think there are a lot of different ways we can take this.

In terms of just putting this question into context, I've been thinking about all the federal activity that we've seen. I know we don't just work in the U.S. We don't just work with federal agencies, but certainly there was the 2018 Evidence Act. There was the White House Memorandum on Restoring Trust in Government Through Scientific Integrity and Evidence-Based Policymaking. There's been some other more incremental steps related to those two actions. So, yeah, are you seeing any bright spots in terms of how people or communities are showing leadership in their use of evidence?

Why don't we start – Jill, if I could pick on you first, perhaps you could kick us off for this question.

[JILL CONSTANTINE]

Yeah, and I'll start with the Federal Memorandum in January 2021 because it was so transformative. It was built off the evidence, like you said; but it was a very clear directive to department and agency heads, right? And basically what it said or what it does, it's such a sea change in how the government looks at its programs, assesses its programs. It moved just from what we think of as performance reporting...oh, I served this many this and that or, oh, I had this many of my clients show up once or twice a month...into true possibility for what we think of as learning and then improving and strengthening programs.

So it's a very clear directive. It's a lot of "you shalls," right? So the agencies had to consider, "How am I going to gather data or make sure there's any information on my programs?" Again, in Human Service space, that was so important because human services are out there delivered through many thousands of state and local agencies, community programs, grantees. So the idea that a Federal Government was on the hook for some sort of process and creation, way of bringing together data – and literally, it's putting in chief data officers or data information officers. So that then let them take on the next must-do steps, which are basically to create learning agendas.

Okay, you have these programs out there. How are you going to know they're doing what you intended? Not only serving who you intended – that kind of looks like the old performance reporting – but that they're actually effective and they're having that impact that the agency intends. So learning agendas then started...okay, what's the model of your program, how would it effect change? Okay, that's how; how do you know you're measuring those things? Okay, you're putting that in place. So then finally, how would you know that that program is leading to those changes that were desired?

So really a sea change all the way from the, "We need a data infrastructure," all the way to all sorts of learning, evaluation, and improvement. And it could be little learnings, right? Let's experiment with different ways to deliver that service so we can get it to more clients – what we call these smaller, kind of rapid-cycle learnings – to the bigger learnings. Was that program effective? What did we learn from this kind of evaluation?

So we see this in so many of our agencies. What's been so interesting is to watch it roll out. There were some agencies we worked with in Human Services that were pretty well-grounded in this already but to see it roll out to so many other agencies. So, yes, we have a long time working with various agencies of the Department of Education and Health and Human Services. But now we work with the Department of Homeland Security and the Department of Office and Community Services and HHS and the Department of Justice to watch all of the agencies really go on that path of transforming themselves beyond just, "We're going to monitor and check our programs' performance," to really learning, strengthening, and improving.

So I'm so optimistic. That's really just year two of the directive, and we see it being embraced and change everywhere. So I'll start with the federal because I think that's been pretty transformative.

[J.B. WOGAN]

Okay, and that's interesting that while we may have seen some pockets of leadership in past years – maybe even before the directive -- that you're seeing an expansion in terms of the number of agencies and how they're doing it. Also, I think I heard you say it used to be performance reports; and now the data is being used for learning and improving. So there's a little bit of a change in terms of how that information is being used.

Chris, I want to turn to you. From your vantage point in the Health Unit and with the various actors that Unit partners with on health, where are you seeing signs of progress and bright spots in leadership in the use of evidence?

[CHRIS TRENHOLM]

I'd pick up right where Jill went and maybe just even carry it a step further. I think what Jill described seems incremental. It is incremental on the surface. It's a step, and it's a step from data and information moving from – you know, in a sense I'd call it the "elites," the decision makers at the top of an organizational chart at HHS, Health and Human Services, or the federal agencies, or people who are experts in understanding how to think about evidence, and then an organization like us that has long contracted with them to provide that evidence and thinking. And that engagement will occur right there and will help inform and guide the policy decisions that are happening.

I think what Jill is describing is a long-standing siloing that's existing not just in federal but I think in many decision environments where that's that organization; and then right next door there's a technical assistance or a community-based organization or there's agencies that are focused on taking that information at some level and putting it into practice and then allowing those communities to make informed decisions and be guided. And I think what we're seeing happening is those silos in a sense are beginning to break down.

The term "democratizing data" has been around for a while, and I think maybe that's a reach in what we're describing here because it's still residing in the hands of decision makers. But the decision makers now that are able to be part of that data evidence ecosystem are among the community. They're in communities and organizational structures that are very close and very passionate and very much reflect the communities that they're working in.

I think one example I can think of, of that is we've been working for five or ten years building the first generation of data that is accessible and can inform states about what's going on across the United States with the Medicaid program. So we're very close to the Federal Medicaid Program. They've historically been often a regulatory agency and a financial instrument to support the states in running Medicaid. They've had little sight lines into how Medicaid is actually operating in the states, and they've had almost no capacity to help the states help themselves; and the communities and the individuals that Medicaid serves recognize where the best practices are, where the progress is, what gaps and needs are in their communities, in their states across the United States.

I think the data and the work that we've done alongside many others to begin to institute that first generation of ongoing, available information that can help people understand how the program is operating, where they money's being spent, who's being served, how effective that is, that's now leading to a recognition and a dialog and an embracing of data and evidence close to the community, close to low-income families, close to populations who are disabled, close to the frail elderly that I don't think has ever existed before.

So it's the lead edge of a transformative change. So it looks like a small step; but it's really the point of that executive order, that executive statement. I think we're right at the cusp of that turn. It's very exciting because an organization that has long been in a sense apart from a lot of those communities, how we've seen ourselves – I think we're much closer and much more able to deliver on a mission that we believe in because the data and our participation in those data are now much closer to those communities than they've ever been.

[J.B. WOGAN]

Well, something that I've noticed with the Health Unit is some of the partners – I mean, you're talking about sort of a more traditional -- CMS being a sort of a traditional kind of federal agency kind of partner. But I notice that we're also working with foundations; we're working with some private sector companies. And I'm curious about both -- Chris, what you have to say and, Nancy, what you have to say from the international perspective, about the growth or expansion in the use of data, the increased appetite for data, among these sort of nongovernmental partners that are not necessarily siloed or not working with government but they are themselves. Is that something else where you're seeing some signs of improvement or anything exciting in that regard?

[CHRIS TRENHOLM]

Absolutely, I'll be quick because I think Nancy and I have a lot of close partnership in public health data. Whether it's domestic or international, the expertise we have and the passion we have for it is so connected.

But one example of that, J.B., would be work that we have for the Robert Wood Johnson Foundation that is really exciting, a project called "Transforming Public Health Data Systems." It really is a close collaboration with the Foundation but also working in support of and on behalf of communities of color and organizations that are working in the communities to recognize important questions like:

How are data collected locally?

What kinds of data are collected locally by the public health system?

Who collects those data?

Why are they collecting those data; what are those data used for?

Then when you begin those questions about current state, you begin to ask next questions about, "Well, what should this look like; and why and how does it need to be different?"

One really important example of that in that project is a recognition that when you start to understand and monitor community health -- health of low-income populations, sort of the underserved populations – and you look across the well-being of that community over time, the experience of individuals, you begin to recognize some really blatant gaps in the system of data collection. One really obvious one that is just there is incarcerated adults. There is really very little in the public health care system locally or at the state level that connects the data of the health care system that resides in the incarceration system of the United States and what the experience of that is in relationship to that community.

There are people who are moving in and out of those institutions and organizations; and essentially, they become gaps in the story and understanding of the well-being of someone over the course of their lifetime. That's blatant and present every day in the United States, and that is a really good example where if you can – that's just shining a light by communities who recognize this issue and are struggling because of it; and they're showing, frankly, Mathematica something that we haven't recognized or seen very well ourselves, let alone the policymakers and decision makers.

So that's a really good example of working with communities and organizations that are – we're seeing ourselves differently. We’re not the experts in that room. And as a result of drawing on their expertise from lived experience, we're seeing a contribution we can make in building those health systems better and differently than they've been built in the past.

[J.B. WOGAN]

All right.

So, Nancy, I want to put this question to you.

Then, Jill, I asked that question to Chris and Nancy; but I would also be interested to hear from your perspective about Human Services...like what is the role of these nongovernmental actors, and what progress are we seeing, signs of progress, there?

But first, let me turn to Nancy. What are you seeing on the international front here?

[NANCY MURRAY]

So maybe I'll go back to the federal space for just a moment, but I do want to say something about our Foundation partners as well.

I guess for us, maybe it's not so much the impact of the Evidence Act and its consequences as it was the creation of MCC almost 15 years ago with a very strong commitment to generating evidence to inform programs, understanding the impact that programs were having, and following well after the programs ended to assess long-term impacts.

MCC had an outside influence on the development community in terms of its thinking and its commitment to the generation of evidence, which actually was followed just a few years later by a really important change in USAID's evaluation policy, which very much borrowed from MCC's in terms of elevating the importance of being able to say something about programmatic effects – comparing them to a counterfactual, for example – which was not at all state-of-the-art thinking about the kind of evidence that we would like to generate to assess our international development programs.

But maybe – I might just fast forward and talk about a bright spot this year that I think may have been indirectly influenced by the Evidence Act, but probably more directly by the change in leadership at USAID, which is the recent announcement of the appointment of Dean Karlan to the Chief Evaluation Officer position, the Chief Economic Officer position at USAID, and what that signifies for the Agency and what it signifies about how serious the Agency is about embedding evaluations in a lot of spaces where it hasn't lived...so behavioral nudges, programmatic A/B testing to improve programs. There's a commitment here at the highest level of the Agency to start thinking about evaluation much more expansively than it has before.

And as to Jill's point, what can we do to improve programs and take them to scale, being the most robust programs they can be, by doing those sorts of things along the way as we build the program and make it accessible to more folks once we find out that it's achieving its objectives?

But to go back to our Foundation partners in the international space, I think also we're seeing this corollary on the Foundation side where, as Chris mentioned -- and Jill, I think -- our Foundation partners are extremely interested in demonstrating what they're achieving, not only to inform strategy refreshes and investments but also to inform the field, right? So they're very interested in contributing to the field of knowledge in the ecosystems in which they work, which also influence government and other multilateral donors.

So they're seeing that they need to also think about the transparency of data and making data available to sort of legitimize the findings or document the findings in a way that is as transparent as it is with MCC or even now with USAID with their publicly-available datasets that back up all these evaluations that they're funding. Our Foundation partners are doing that as well – for different reasons, but it's an interesting trend.

So I guess I would say our bright spots are the elevation in both spaces of the use of evidence and the transparency of the use of evidence. I think Chris was trying to get at that a little bit with his examples in the Public Health space, but we're seeing it more broadly.

[J.B. WOGAN]

Is that true, Jill, in Human Services as well? Are you seeing Human Services organizations increasingly see the value of evidence beyond just the federal agencies we talked about earlier?

[JILL CONSTANTINE]

Yeah, that's right. Remember, in the Human Services space, the Federal Government is not a big player, right? There are not many human services that are delivered at the federal level...that are the equivalent of Medicare or anything like that. It is state and local level.

I want to really pick up on Chris' point around the use of democratization of data even at the federal level. What we see in our space is those federal agencies are now and a very wonderful area of growth and way we're partnering with both state, local agencies, grantees, is to build the data systems for them. So they may have hundreds or thousands of state and local agency grantees that they're working with, and they appreciate now, "We need a data stream both for hose state and local agencies," or grantees or whatever they are -- they can be a community organization – "to be able to enter their data and have a data stream that they can work from and learn from."

But also up at the federal level that the federal agencies can look and learn, and then they can also learn across state and local agencies and entities. So in the last three years, to Nancy and Chris's point and to your point, this did predate the formal memorandum from the White House. But we've really seen an expansion in the acknowledgement and the embracing. We can't learn – we, at all levels, right down to that community level who's actually delivering services -- unless we have a data infrastructure.

And, yes, you can go all the way down to learning and evaluation; but I would say data or evidence, right? They show you what's going on right out there in front of you...who are you serving, how are you serving, who's getting access, who isn't getting access? If somebody isn't getting access, why? So this work we're now doing with multiple agencies within Health and Human Services -- we do this work for the Office of Population Assistance; for SAMHSA, the Substance Abuse and Mental Health Services Administration. That's thousands of grantees.

So to me, that's the great place of both optimism and power, to Chris's point – that you're creating infrastructure so that program operators at all levels can look and see what's going on within their own programs and across other programs. That's what's really going to leverage the learning and the improvement.

[J.B. WOGAN]

Okay.

I want to talk about the importance of equity in the work that we do. This year, Mathematica helped launch an Evidence and Equity Collaborative, along with eight research organizations; and our company's 2035 vision statement, which came out a couple years ago, also calls for Mathematica to be "shaping an equitable and just world." So I'm wondering as we center equity in the way that we improve public well-being, how is that changing the work that we do?

Chris, perhaps you could start us off on this one?

[CHRIS TRENHOLM]

Sure and, J.B., I think one of the first opportunities I had to talk with you in this context of looking back at the year was during the pandemic. There was a change underway, I think, for us within Mathematica; and it was already underway but like so much with the pandemic, it accelerated the pace of change and the recognition of the need for change. I think the change that was underway was Mathematica has long seen itself alongside organizations like ours that bring evidence to bear as an arbiter of evidence and, as a result, saw ourselves requiring a level of distance between the creation of that evidence and the responsibility of providing insights from that evidence in the communities or organizations or individuals who that evidence was ultimately intended to serve.

What that meant fundamentally then was that Mathematica was working a distance from underserved populations who brought expertise about the programs and the evidence that we were creating that we were not recognizing. I think we recognized that. I don't know that we were as intentional about what the implications of that were as we became -- during the pandemic. So there was an acceleration of a recognition of our responsibility in seeing ourselves working for underserved communities and experts in the field with the lived experience and expertise that that represented.

So it wasn't that, "Oh, you're now going to be a part of our team; and we'll collect data from you." It was, "No, the work that we do is a contribution to the expertise that you bring to those communities." So it's an inversion of the model in a sense; and it's something that, depending on the context, has permutations of many kinds. But I think that transformation was important for us; and I think inside the pandemic, which we talked about, J.B. – the murder of George Floyd was a further accelerator of that. We brought new eyes to ourselves and to our responsibility in doing work in the way that I'm describing.

Also, we recognized the scale and enormity of the issue. I think at the time we spoke, I quoted from what I still think remains one of the best things that I read, which was Alan Weil, the editor of Health Affairs, had written about there's a lot of continued recognition of the critical importance of social determinants of health being fundamental to effecting the health and well-being. It's not just about health care; it's about all of those factors that influence health and well-being. It's homelessness, transportation, education, income. But what Alan was recognizing in that article was that below that is a root cause that is rooted in systemic racism and that that cause is something that an organization like Mathematica, and many others like us, need to recognize and work in not merely an acknowledgement of but in an effort to further deconstruct and impact.

I think that is a remarkable recognition for us, and there are parts of me at 56 years old that wonder why at 26 years old I wasn't able to see it in the way that I do now. But the work that we've done as a result of that, I think, has changed, both in the nature of it, the kinds of work we're doing. I think my practice is much more invested in public health and work in communities than it's ever been. We've had opportunities that we've never had, but we've really been intentional about that.

The example I gave of the transforming public health systems data is a good example of many project engagements we have where I think we're seeing ourself working on behalf of organizations and experts in those communities. And there's lots of layers to this. Jill and I, the Human Services and Health Unit, have partnered extensively, for example, with the Assistant Secretary of Planning and Evaluations Office. That is the arm of the Secretary. It's very influential.

And we have partnered with – Jill and I have supported the expertise of individuals in the two units to work closely with ASPE to help federal agencies for the first time really do the hard work of understanding when we put federal policies or practices into place, what are the implications of that for equity and outcomes? It's a different level of engagement than what I'm describing at the community level, but it's a lens and a recognition that I don't think the Federal Government has ever had in the same – in kind of the intentional way that I think that project or that engagement represents.

I think it represents a parallel journey that the public sector, the government, has been on, alongside Mathematica, about seeing our work with different eyes and not seeing ourselves with some sort of global expertise that can solve a public – understand how to implement or design a policy without that lens. So I see it at many levels. It has a lot of contexts, but it's been transformative.

I have to say though, we're kind of babies at it, J.B. I'm looking forward – it may not be me, but in 2035, I can't wait to talk about this stuff or listen to this podcast because the progress I hope that we're able to make in this space in particular – it can be enormous for a mission-driven organization. I think in some ways we're just getting started, and it requires a lot of hard work and humility on our part. So it's been a fascinating couple years.

[J.B. WOGAN]

Okay, so yeah, we're making progress. We're still at the beginning of the continuum, but we are seeing some changes in terms of the clients we work with or what kind of work our clients or partners want to do.

I'm curious...is the story the same in the international context, Nancy? Like I'm very aware, say, like the White House establishing a – was it like a Climate and Health Equity Office? Equity is certainly a priority in the domestic sphere here in the United States. Both with the federal agencies you all work with and foundations, what are we seeing in terms of equity in those areas?

[NANCY MURRAY]

Yeah, so I like the way Chris put it. We're on a journey, and there are so many layers to the journey that we're on.

But I guess if you think back on how you described how we used to work, Chris, right – at a distance, sort of impartially, not diving deep into listening to local voices ourselves or empowering the community to participate in even designing a study that was going to impact them. I think we're seeing all the same trends in the international space. So they can be exemplified in a couple of different ways.

I think both in the foundation space as well as with our sort of federal partners, MCC and USAID, there's a huge emphasis on how US-based organizations need to think about their partnerships in the countries and communities in which we work. Because if we were at a distance doing our work in the United States, imagine what a distance we're at doing our work in the countries around the globe, right? And we had partners, but those partners back in the day when we started this work were actually relegated to data collection tasks. They weren't truly partners in the full sense of the work, right? They didn't help us do the evaluation design. They didn't help us except in a very superficial way -- because we didn't ask them to -- interpret the data, right?

Now all of our partners, and we ourselves, are looking for ways to fully empower our in-country partners in research and in the interpretation of research and the findings but also the beneficiaries in the same way that we're trying to do in the United States. And, I mean, what we're seeing is that our – especially at USAID, they're actually setting goals for themselves in terms of the amount of resources that those partners should have, how much money should stay in country in terms of the agencies' investments in development and not sort of flow back to U.S.-based organizations in quite the same way as it has in the past many decades, right?

So the current administrator of USAID is saying, "We're going to change the fact that 6% of development resources are going to in-country partners and organizations, and we're going to set a high bar goal of getting that up to 25% and 50% in the next four to five years. That's a very ambitious target, but it tells you that the agency is really serious about redirecting resources and ensuring that the partners in country that the agency exists to serve are going to be getting the bulk of the resources that the agency is distributing.

We'll see how quickly they can effect this change; but it has lots of implications for how we do our work, which are the right ones, right? If we're in development to do ourselves out of a job, this is the direction that things need to go and quickly.

So I guess similarly on the Foundation side, since you asked me, we're seeing the same trends. And our Foundation partners might not all have exactly the same sort of percentages of resources that they'd like to see flowing to the groups they're working with in country, but the sentiments are exactly the same. And I think there what we see is many of our Foundation partners are articulating this commitment to equity around who leads, right? And they're saying, "We want to see prime organizations that come from the countries and regions where we're working."

So we're starting to see – and I'm sure we'll continue to see – more arrangements where Mathematica plays an important technical capacity-building sort of quality assurance/communications role across partners maybe spread across a region. But local partners are going to be in the driver's seat, and they are increasingly in the driver's seat. And we are increasingly told, like, "No, we really want to have this initiative led by a coalition or a consortium or one strong local organization; and we want you guys to be part of that mix, but you're not going to be the lead because it's not appropriate." For the voices to be elevated, the voices need to be leading.

So I think we're seeing exactly the same kinds of things as we're seeing in the states, and it's the right thing to have happen.

[J.B. WOGAN]

Gotcha, okay, so one of the things I'm hearing you say, Nancy, is you're talking about the how of research or the who and the how of how research is conducted, like thinking more about conducting research in an equitable manner. But there's also a flip side where I think – or not a flip, but another part of this, which is having evidence inform policies and programs that drive equity in terms of outcomes and how people are affected. Those are both important pieces of the conversation.

I think of this with our own mission, with Mathematica's mission, which is to improve public well-being. But it seems like over the time that I've been here, almost five years, that equity and advancing equity has become a much more explicit component in the definition of how we're defining improving public well-being. If equity – if it was resulting in sort of everyone being better but some groups being worse off, that wouldn't be what we were going for.

Jill, I don't think we've gotten to you yet. So tell me a little bit about in terms of Human Services, where are we seeing equity manifest in our work?

[JILL CONSTANTINE]

I'll try. It's so hard because, as I said, all of the communities we serve are communities/families/children/students that have suffered structural racism barriers. So I'll try not to go on and on so we have the time for a few other questions.

But I'm going to pick up on what Chris said; for sure in this country, the murder of George Floyd was an inflection point, a racial reckoning for the country in terms of Mathematica's work. And at the same time, we create our 2020-2035 vision to go with our mission, which is Mathematica creates an equitable and just world where evidence drives a decision for global impact, right? So the words "equity" and "justice" appear in our vision statement for the first time.

But to Nancy and others' points, at that point in time – mid-2020 – really it was only our Foundation partners stepping up, leading the way in this space, demanding two things; and I'll get to both of them in a minute because it really all transformed within a year of that time, not just the what you're studying. To your point, who was impacted? Was it everybody, or were some groups disproportionately affected? But it's the how we're studying it to get to Nancy's point.

So my all-time favorite Executive Order – yes, I line up executive orders and memorandums and things like that in terms of my favorites -- 2021 was big because I had two favorites there, the Memorandum on Trust in the Government and Data and Evidence, but also the Executive Order promoting racial equity. So that is transformative because it does two things. And I'm going to talk about those two things taking – I swear I didn't ask them to run this today – a New York Times headline that says, "Is New York's Child and Welfare System Racist? Some of its own workers say, 'Yes.'"

So what we're looking at here – and you can put any state or any locality in there – what we're looking at here is not just the, "Well, what are we doing here?" We're trying to protect children. We're trying to help people released from prison re-enter their population. We're trying to get children and families fed. We're trying to promote learning, particularly making up for a lot of interrupted learning in schools due to COVID." So it's not enough just to say, "Did we try something, and did it work? Did it help everybody?

It's that recognition that some families/students/children/communities are disproportionately or don't have the same access to programs as everybody.

So we are on the hook – to your point, J.B. – how did it help this population? Or let's look at our Child Welfare system, very well-documented. Let's look at the data. Children from black families are much more likely to be taken away from their families and put into foster placement, right? In communities, black men are much more likely to be arrested and charged for the same crime than white men. So that's your first thing. You're going to look at your data and evidence to say, "What's happening here? Is this disproportional?" And we're going to ask that question to document the disproportionality.

But then we're going to ask ourselves, "Why – why is that happening? Why are black children more disproportionately removed? Why are black men more disproportionately stopped and charged with a crime?" So when you go into the why, you're trying to understand then what's happening with the program and the community.

What's been so transformative about the Executive Order? Well, the great thing – the thing we've embraced at Mathematica – is it so aligns with that vision, that 2035 vision. That's right where we want to be. But at the same time, our federal agencies/local programs are kind of in all different places in terms of how they're going to figure out the why. You see that commitment to figuring out the why; but we see everything from some agencies going, "I don't know, how do we figure out the why? Why is this happening," to some being very directive, "You will engage the voice of the community. You will make sure you understand how community stakeholders are experiencing that program. You will engage in all sorts of actions where you go out in the community – and Chris referenced it very much in his public health example – to understand what's going on, which means bringing in the voice of that community and the stakeholders.

That's so important because you're very unlikely to get to the why unless you do that, right? So we have so many examples of projects we're starting this year that bring that all together. I mentioned the Department of Justice one. That's to work with their community relations, eventually their community policing organizations, to understand the programs they are implementing and to make sure we're learning from those.

Chris mentioned one of our lovely – Chris mentioned ASPE, joint across Health and Human Services. We have a project to take the Child Welfare data – it goes back to also data, bringing it together, which sits all over the place in states -- the state Medicaid data, which sits somewhere else -- bring it together to actually do those predictive analytics around which children are around risk of being placed in foster care; and is that right, and is that equitable?

So again, to me it's just transformative in what we can do in terms of meeting that Mathematica 2035 vision – what is actually happening, who is being impacted, are they being impacted disproportionately, and why is that? So that, to me, is really what the centering of equity is bringing and bringing that vision to all of our clients and programs that we're working with.

[J.B. WOGAN]

Okay, perfect.

All right, so last question – I want to look ahead here. We've been talking about 2022; but this is December of 2022, so we're also looking ahead to the year that will be not just the year that was. So considering the potential for evidence to help address pressing challenges, where do you find reason for optimism in the year ahead?

Nancy, maybe you can kick us off on this one.

[NANCY MURRAY]

Yeah, I'm happy to.

So I am feeling so much optimism about the year ahead for a number of reasons. But maybe I'll take a step back and say much like we centered equity in our 2035 vision, we also said climate change is one of the most urgent issues of our time; and we're going to address it.

So our deep commitment to climate and to finding answers on how to mitigate the worst effects of climate change – increased resilience, global warming, all through identification of solutions that rely on evidence – is remarkable. I feel like we made that commitment before we really knew how the clients in our ecosystems were going to address climate change. So I'm proud and optimistic that we sort of stepped into that space because we felt strongly as an organization that it was the right thing to do. But our clients, at least in the international development space, are stepping right in there too.

So I see so many opportunities for us to be part of the solution or at least using evidence to identify parts of different ways we can solve climate-related problems. We see across our Foundation partners, as well as USAID and MCC, they're all diverting or finding additional funds to invest in climate. And USAID and MCC have put out policies specifying how they're integrating climate into their organizations at every level. We've seen lots of opportunities in the climate-adjacent spaces in which we work – so agriculture, health, energy. These are all spaces that allow us to participate in identifying different ways of doing business that can bring about the kind of positive change in the climate space that we're all hoping for.

So maybe I'll stop there and let Chris and Jill chime in.

[J.B. WOGAN]

Okay, that's great.

Yeah so, Jill, why don't you pick up where Nancy left off? One of the reasons I like to ask this question is we work on so many sort of big, tricky problems that can be kind of depressing to think about. So it's nice to think about like where are we going to find – where is the reason for optimism in the year ahead?

[JILL CONSTANTINE]

Yeah, so I actually am going to go back to one of your earlier questions. I have such a good – it's just my experience last week generated so much optimism around this particular subject. So the Evidence and Equity collaborative, the firms that are part of that, hosted for the Equity and Inclusion Fellows APPAM, our Association for Public Policy and Management. organization and conference hosted a breakfast that was really around career paths and thinking about that and thinking where do I want to be.

So I met – actually through a fun run in the morning and through that lunch – three young grad students – some in master's, some focused more on policy administration, some focused more on research, some Ph.D. students – all focused on climate but as it applies to inequitable impacts of climate events in localities and how that affects lots of things, Human Service agencies, health agencies. One was focused on – was studying actually coastal flooding and risks there and how you measure that and how agencies better prepare their communities. How are we going to help you when you get displaced by a flood and all your services – your food, your supports?

Another very focused on air quality and what that meant for children – disproportionately black children in her city with asthma. So such a focus on what in the Health and Human Services agencies in state and local communities environmental justice, right? Here again is another phenomenon that's occurring that is and will certainly in the future disproportionately impact communities of color or communities experiencing poverty, communities who have not had access to services.

And I'm just like, okay, this is -- half of the folks I'm talking to here are very focused on this topic. So that gives me lots of optimism. The young early-career students and early-career people interested in policy know this is the issue coming forward in our time, and we see it also in our work. We've been working with the Association of Public Human Service Agencies. We've kind of worked with them at some of their convenings and things, where they're very much focused on this issue. Yes, resilience after climate-induced disasters but also this notion of environmental justice. How do we prepare or protect our communities in a way that these impacts aren't so disproportional?

So we've had lots of good work with them to think through how they would organize themselves and their services. Everybody is struggling under resources for this work, right? It's so unfortunate, and it requires such transformative/foundational thinking and resourcing of how we would tackle these issues. So that I actually find daunting as we see the struggle at all levels to prepare for and fund this issue appropriately. But, okay, we're doing it around data and evidence; we're doing it around equity. Hopefully, we'll get there on climate too is how I think about it. But I very much appreciated and felt the energy and the spirit of the new researchers and new policy analysts who want to focus on this.

[J.B. WOGAN]

That's great. So one reason for optimism is that the best and the brightest – we're putting the right people on the right problems. I don't know what a frivolous problem would be, but we're at least having – we're seeing the next generation focusing on exactly the big, daunting challenges ahead.

Chris, I'm going to give you the final word here. Where do you find a reason for optimism in the year ahead?

[CHRIS TRENHOLM]

I love what Nancy and Jill just shared. And I think the point around a generation that brings a passion and skill set and capability that the world has never seen is worth a tremendous amount of optimism. So I would completely reiterate that. We see that within Mathematica. I mean, the talent and the passion and the leadership that will emerge that replaces people like Jill and Nancy and Chris is going to be remarkable. So I totally echo that point.

I think that also some of the discussion earlier around the equity journey of the country and Mathematica – yeah, this is not a one-year effort. This is about getting to that vision for 2035. And we've all been through a collective trauma of the pandemic. There's just no other way to describe it. The mental health data that are coming out now and the realities of families are very, very real and very, very painful.

Any experience of trauma is complex because when trauma emerges, there's opportunity to change. I marry the comments made around the generation that's coming forward and their talent and vision and ability to see with new eyes, the recognition of Mathematica seeing itself with new eyes, and then the recognition that we need to change and the opportunity is there for us. I think the coalescence of those ideas I think are powerful and will lead to change. I think words like "transformation" get used a lot. It rarely happens. But I have an optimism that I haven't had before that we could see transformative change.

[J.B. WOGAN]

Nancy, Jill, Chris, thank you so much for taking the time to talk with me today and continuing this tradition of checking in at the end of the year and talking about, reflecting on, the year that was and the year ahead.

[CHRIS TRENHOLM]

J.B., thanks.

[JILL CONSTANTINE]

Thank you, J.B.

[NANCY MURRAY]

Thank you.

[J.B. WOGAN]

I want to thank all three of my guests for coming on the show, Nancy Murray, Jill Constantine, and Chris Trenholm. We'll post this podcast on the Mathematica blog, along with lots of links to the research and tools that Nancy, Jill, and Chris referenced in the interview. A full transcript of our conversation is also available on the blog. As I said before, we just published our annual year-in-review web feature, which we call Inquiry to Insight. I’ll include a link to the page in the show notes and I’d greatly appreciate you’re giving it a read.

If you're a fan of this podcast, please share it widely. We want to make sure we're reaching everyone who cares as much about objective, data-driven policy as we do. You can also help us by giving us a rating and review. We're on Apple podcasts, Spotify, Stitcher, Google podcasts, and pretty much everywhere else you find podcasts.

We'll keep covering interesting work from Mathematica and our partners on the show, but another way you can stay up to date on everything Mathematica is doing is by following us on Twitter. I'm at J.B. Wogan. Mathematica is at MathematicaNow.