Twenty Years of Insights from the Millennium Challenge Corporation on Reducing Poverty Through Evidence-Based Solutions

[ALICIA PHILLIPS MANDAVILLE]

I think our one challenge is going to be, because we are so evidence centered. There's a desire to say, “well show me where this has worked somewhere else.” And that's an appropriate and good reflex. But some of the things we're trying to address just haven't happened somewhere else, or they haven't been done somewhere else. And, you know, I think we're trying to be like proactive about making sure we stay evidence-focused and not evidence-addicted because there are places where it's just never been done. And so you can bring to bear the relevant evidence of other places and learn from them, but still be willing to take that risk and address the problem that just needs solving.

[J.B. WOGAN]

I’m J.B. Wogan from Mathematica and welcome back to On the Evidence.

Twenty years ago, the U.S. Congress created a new federal agency that represented a bold experiment in international aid. The Millennium Challenge Corporation strives to reduce global poverty through economic growth strategies. Grants from the agency flow to low and middle-income countries that are committed to good governance, economic freedom, and investing in their citizens.

Part of what makes the Millennium Challenge Corporation unusual in the international development space is its evidence-based approach. From the outset, the agency focused on evaluating its impacts and fostering a culture of learning and accountability to improve aid effectiveness.

On this episode of On the Evidence, my Mathematica colleague Jeff Bernson interviews Alicia Phillips Mandaville of the Millennium Challenge Corporation, or MCC for short. Jeff leads our international business unit known as Mathematica Global. Alicia is the vice president of the Department of Policy and Evaluation at MCC.

Among other topics, Jeff and Alicia discuss how MCC approaches locally led development, how MCC’s evidence-based approach to learning has evolved over time, and what MCC has learned about how to reduce poverty in developing countries.

I hope you enjoy the episode. If you’re new to the show, please considering subscribing. More information on how to subscribe on your podcasting app of choice is available at mathematica.org/ontheevidence.

[JEFF BERNSON]

So, greetings, Alicia. Welcome to On the Evidence. We're thrilled, of course, to have you. I'm particularly excited because, you've had such a great career at Millennium Challenge Corporation. And you've seen so much transpire in the agency. So, but I am wondering if maybe you could kind of take us back a little bit and, because some of our listeners obviously can span the, not just the international policy areas, but also domestic. And, I was hoping you could orient them to a little bit of the description around MCC’s creation, its background, its mission. And, of course, take us back 20 years. Given your experience and, help us get to get oriented.

[ALICIA PHILLIPS MANDAVILLE]

Excellent. Thank you. It's it's a pleasure to be here. I always, I enjoy talking about MCC, and it's good work and even more with folks who want to talk about evidence. So this feels like the perfect place to do that. No, and it's actually also really enjoyable to talk with a group of people who are not all already deeply immersed in kind of the international economic development space.

So maybe that's the place to start with, what is Millennium Challenge Corporation? Because we might be the world's worst named, like most effective, worst named economic development agency on the planet. It's a United States Foreign Assistance Agency. So our job as part of the United States government, it’s the smaller one, the one you haven't heard of necessarily.

Most people have heard of USAID. It's the largest of the United States foreign aid agencies. We are one specifically designed to focus on making grant investments in poverty reduction through economic growth. And we were set up 20 years ago, and I'll come back to this a little bit, a bit to test out some learning and some lessons from 50 years of international economic assistance, whether that was the World Bank or USAID at the time or other efforts around the world.

And so, our, in a beautiful and frustrating to some people way, we're super focused exclusively on how do we make large-scale, apparent investments in economic infrastructure, typically, in ways that promote economic growth and poverty reduction in countries. And we work only in countries at the low- and low- and middle- income level. So countries that, where additional money going into the economy and into public infrastructure or public systems like health or education, but also roads and energy and transit networks, can have a really meaningful effect on people.

So that's, that's our job. We work in about 30 countries right now. We invest at kind of a funny scale, only 2 or 3 countries a year, but somewhere between $300 million and a half $1 billion a piece. So it's a it's a great place, and, I think really committed to how we actually make sure that we're using that money for the best possible outcomes in the interest of poverty reduction.

[JEFF BERNSON]

That's great. And also insightful, of course, you mentioned very humbly that MCC is, you know, the, the organization maybe, or the agency that, folks haven't heard of. At Mathematica, we've also we've heard a lot about MCC. We follow MCC pretty carefully, but I am wondering if you could kind of, double-click a little bit into what makes MCC an agency so distinctive, maybe from other agencies. It could be U.S. or even other international development agencies, because you guys do operate. You were you were founded, on on different principles and operate and have operated very differently.

[ALICIA PHILLIPS MANDAVILLE]

No, absolutely. And I think it actually really informs, our kind of origin story for as much as it was 20 years ago. And that feels dated to people know, we the world had learned a tremendous amount about what does it mean to use money in pursuit of economic outcomes all over the world, and in particular in countries with smaller economies or lower income economies, what was called the developing world for a long time, the global South, however, we're describing it now, we use the technical term because we're technical people, low and low and middle income countries.

But the World Bank had been working for a long time. IMF had said a whole Bretton Woods arrangements had been focused on this, the creation of USAID, you know, the Western Europe as it, came to the end of the millennium, was also each of them running their own bilateral aid agencies. And there was a lot of learning, in particular in the kind of final years of the Washington consensus being. So the World Bank and the IMF have this clear definition of how the economy worked.

And through the 90s, there was a lot of learning around when you press countries on certain types of policies, what actually happens? It was putting into practice a lot of what economic theory had agreed worked for developed economies. What happens in lesser developed economies. And so having done that through USAID during the Cold War and having done that through the World Bank and the IMF during that same window of time, but as a multilateral body, MCC was set up to test out, what if you did the opposite of things that didn't work? And that's really the best way to describe it. And there's really three key pieces that that inform that. And the first is being selective. So USAID, the multilateral banks, they work in every country that has a need of some sort. And they have a huge mandate. Everything from humanitarian work to, you know, financial, financial services for, for people.

What if you're selective both about where you work and you use clear public criteria to decide which countries it's going to be. So it's not politically driven. It's not fad driven, it's not mission. It's just mission driven. And it's clear and transparent. And then what if you're selective about what you do and you set up all of these rules internally so that you only make investments in things that you think will have meaningful economic impact.

So again, not politically driven, but economically driven. So selective was the first. Country ownership was the second. And we can talk more about that because I think that's a word that's less clear to folks who haven't worked in international development recently. But basically, in the same way that when you think about health or education or other kind of municipal or community services, you worry about how the population that it's serving is invested in and served by the work and how are they leading on that work.

And so the same thing can apply at an international level. How do you think about the way the country partner with whom you're working leads and owns the work that you're doing together? So how do you center country ownership? That was the second lesson that we were asked to look at. And then third was be accountable for results.

So it's not just spending the money, it's not just making a friend. It's not just even getting the outputs, which I know this crowd will understand. It's what's the actual outcome and what's the actual impact and can you structure an aid agency, a foreign assistance agency, to only spend money selectively to make sure the countries are leading on key decisions and implementation, and then to set yourself up to be publicly accountable for the results you actually achieve.

And the evidence to date is that that's worked. We've got 20 years of now evaluations and we think at this point, we expect 400 million people to have benefited from our work in 47 countries. It's a remarkable positive outcome to an experiment. And so, every day, we kind of think about, okay, what's the next step in the experiment? Because we found some things that didn't work, too, we've like to stop doing those things. What what are we going to learn next?

[JEFF BERNSON]

Yeah. No, the results have been impressive. You know, as you mentioned, this is the 20th anniversary of MCC. Yeah. This is, a $17 billion investment, right? That has been made. I'm wondering if you could give a few examples of investments that maybe our listeners haven't heard about or would be interesting to them about how people are benefiting from MCC’s investments.

[ALICIA PHILLIPS MANDAVILLE]

Yeah. No, I'd be delighted to, one to start that is easily to resonate, a couple that resonate easily whether you're working domestically or internationally. So health and education are the sectors we make a lot of investments in all over, you know, Africa, Latin America and Asia. But I know a lot of folks may be listening work on that here in the U.S. as well.

So let me start with a couple of those. And then I'll throw in a port and some power sector work, if that sounds good. And so maybe, on the health side, so we, had our board had identified Lesotho as a country, MCC should make an investment with, the way our process works. We asked the country what they're interested in focusing on.

And so Lesotho came to us, because at that point in time, the United States government as a whole and a lot of other donors, were really worried about the AIDS and HIV epidemics in Southern Africa and had invested a huge amount of money in addressing that. And so U.S. PEPFAR work, a bunch of things from, USAID as well, other donors, were really focused on that side of the health sector because it really was an epidemic, and they were addressing it. And then in good news, they addressed it really well. It's been an incredibly successful effort. But it meant that in Lesotho their health sector was it was almost two layers. There was the one that was designed to address the epidemic and it functioned extremely well, was really effective.

And then there was everything else that people needed, right. Like, you know, giving birth and just getting sick and dealing with bacterial things and surgery and basic, you know, child health. Right. And so Lesotho came to us and said we want to rebuild, we want to build that system. And there's some infrastructure pieces of that. And there's some systems pieces of that and there's some training pieces of that. But we want the basic health care pieces to get closer to where our epidemic addressing is going. And so we worked with them both on building some health clinics, putting in place some of the systems so those clinics could communicate with each other ,so that what, what was known in one place could be known elsewhere, and that the materials and, items that are needed, medical supplies that are needed could get where they need to go.

There were some training pieces. There was community engagement pieces. And you see, really we're actually making a second investment in Lesotho now. And we see some of the results of that. And people able to access now not just good HIV and AIDS, you know, treatment and prevention, but also a lot of other, you know, core health services that really matter to people.

Some similar kind of examples around, in, say, the Republic of Georgia in the Caucasus, where we work in education with them, they wanted to make a transition to building a workforce that was ready for science and mathematics and heavy engineering work, because they didn't feel like they had that in place. And so we worked with them on curriculum design for those types of, you know, not just high school and vocational, but how do you think about it at the university level?

We made some investments in the university in Batumi, worked with some U.S. universities, actually with them alongside that teacher training for the high school level. So kids were ready to go into those kinds of field when they got to college. And you see similarly that change in the employment system, or the opportunities for employment and the qualifications that that Georgian graduates have now. So those are the ones that I think are familiar to people, and it's the kind of thing we care a lot about at home, and we, you know, people everywhere else care about it there, too. And so we work with them to support it.

And we'll just tag, evaluate all of this work and then publish those evaluations. So if you want to find them, you can go to our evidence platform and find them at evidence dot mcc dot gov. But if you give me if you'll indulge me one more minute, I'll give you an example that's not something we spend a lot of time talking about at home, which is something like, the power sector. Right? So, for the most part, the United States, everyone is connected to electricity in some way, shape, or form.

There are people who don't have it, but it's really rare in the U.S. So we don't spend a lot of time thinking about its effect on families, on individuals, on economies, on communities, on schools. And so MCC has worked in kind of almost every country in West Africa, like most, a lot of the countries along the coast and a lot in the interior as well in the Sahel.

And in many of them, we've made investments in the power sector, whether that's generation like a generation facility, a water-based generation facility that we worked on in, Liberia or in Malawi or transmission, which is how do you get the power from where you build it to where it's needed. Right? Or distribution? How do you get it from this, where you just got it to actually into people's homes?

And when we do that, we see it isn't just okay. Yes. Obviously consumption of electricity goes up. You see a lot more economic activity. But now, like kids who need to go to school can go with, like having they have the opportunity to do homework at night if they want. Parents and families who are providing for folks can rely on being able to charge their phones so that they actually can do the things that they need to do.

Refrigeration. Right? Like you can keep your food safe and clean and cold for a block of time. Really core things that I think we all really take for granted that we see millions of people now able to do, in part because of these base infrastructure investments, but they really make a huge difference in people's lives.

[JEFF BERNSON]

Well, it's excellent points. The knock-on effects of power, right? I mean, well-documented evidence that supports the health and the wealth, and the well-being of populations and families.

[ALICIA PHILLIPS MANDAVILLE]

And if I can just add, like, I think this is one of the things that MCC finds complicated, which is, we do build these massive things, like physically, if you've ever stood in a port like a, you know, an oceanfront port and it's like you can feel the economy around you. The cranes are enormous, the ships are enormous, even the containers that come off them, when you see them from far away, they look like Legos. They're enormous. They're just enormous. And when you stand there, you can feel a whole economy. And we've invested in rehabilitating ports and dredging them and all these things, in Benin, for example. And it feels like a non-personal investment. Right? Like the economy will benefit, but what about the people? Except that in Benin where we invested the number of trucks that can go through it now daily doubled. And so it means you've got twice as many people employed in the trucking industry, got twice as many goods getting to people. You've got all the people whose employment is based on those goods either going in or coming out, going out or coming in.

And so the like income, income generation for truckers, income generation for [inaudible] income generation for producers, all of those things have the knock on effects for their family. And we sometimes struggle to put our kind of big, concrete, literally concrete investments into human terms. But the evidence that that happens is actually there. And it's, I think, one of the coolest things about making infrastructure investments, in particular in Africa.

[JEFF BERNSON]

And the fact that you guys spend so much energy in trying to evaluate these huge, complex projects. Oftentimes, it does take complex evaluation methods and approaches to actually do that. That kind of helps me turn, a little bit to some of the learning. I mean, anniversaries are great events, in general, but also they are natural opportunities to reflect on lessons learned. And I am curious, you know, given your two stints and the fact that you saw MCC take off at such an early stage and now you're back, obviously, and, helping it, shepherd through its next iteration and evolution. I'm wondering, are there things that you point to in terms of what you've learned about what it takes to reduce poverty?

[ALICIA PHILLIPS MANDAVILLE]

I will say, you're right. Anniversaries are great. They make you more reflective. And so people ask us these kinds of questions. So we actually have spent a lot of time thinking, okay, well, so, we spend a lot of time learning about specifics, like when we look across the board, what can we summarize?

And in some ways, there's a couple of really satisfying answers. And then there's a really unsatisfying answer. So I'll give you the satisfying ones first and then the other. And I think on the satisfying side, what does it take to reduce poverty? Like could there be a bigger question? Right. On the satisfying side, I do think we've learned in early days, MCC made investments that cut across lots of different sectors.

So like take our first Morocco compact, it I think had like seven different kinds of projects, everything from fisheries and livelihoods to, you know, almond farm, like almond orchards, I think you're supposed to call it like almond farming, essentially, for lack of. So you can reveal my lack of agricultural technical skills. But like, that's such a range, right? And some when you spread yourself really thin like that, it is hard to be as holistic as you need to, to actually work with a country to address all of the moving parts of the need in that place. And so one of the changes I think you'll see, if you look at all of our evaluations, is some of the early ones, we were evaluating one of like 6 or 7 parts of a compact.

Whereas now what you'll see when you look at the compacts we've just signed and even the evaluations that have come out more recently, we tend to have either single sector or kind of adjacent sector compacts. And so, you know, unlike, fisheries and almonds. And I think there was a roads piece actually, or maybe an industrial platform, I may be confusing the second and the first, but like this hodgepodge of really great things, but they didn't necessarily hang together. In, say, Belize, a compact our board just approved, which we’ll sign in the coming months, it is in the power sector, and there's a really specific engagement there around the way Belize connects up to the regional power environment and how/where it takes on power and what it needs regulatory-wise internally.

And then there's a really deep investment in education, probably the deepest education investment that we're going to make, in that in terms of coverage of population, it includes curriculum. It includes thinking through how you incentivize teachers. It includes that middle level of, like middle school equivalent, so that kids can then choose to go into vocational or go into university and really kind of building in this, institutional infrastructure on that.

And that's it. There's a really deep one in education, and there's a very specific thing in power, and we know what it is we've set out to do. And the fact that we're going to evaluate it also even means when our team and the Belize country team are working together, we have discipline with them around being clear in the objective of the work, because we're going to have to evaluate it.

And so if we get mushy in our evaluation … or mushy on our objective, then we wind up with an evaluation we can't learn anything from. So I think the first was the shift to one or, either one sector of a lot of things that work together, or kind of two really clean things where you can tell with great depth on one what you're doing, and then you're really specific on the other.

I think the second piece is the satisfying, is that we also learn that it's you can't just build infrastructure, and anyone who's in that sector and is listening will roll their eyes and say, yes, duh, we've known that for forever, and now we know it in writing and with evidence and a lot of evaluations is behind us.

Which helps because everyone wants it to be true when they try something, it'll be different than when everyone else has tried it. And so actually, an evidence platform that we can point to saying, if you build a road from here to there, even if it is the most highly traveled road in the country, if you do not address highway maintenance and you do not address the funding system for highway maintenance, and you do not address trucking regulations, and you do not address, like, you know, pedestrian access or cross road things, it won't matter because in five more years you'll need to resurface that road, right? And so that's the easiest example, but I think those are the two pieces that, are satisfying to say that are lessons we've got.

On the side that is really honest, but less satisfying to people who haven't lived through it, I think, most of the lessons are not generalizable, and that, I think, is the hardest thing for people to accept as a reality. Because in the same way that, you know, let's use the road example again. We built roads on every continent at this point. And including in the middle of the South Pacific in Vanuatu. So, like truly the coverage of MCC and road building is, we have a lot of learning that we can do.

You can imagine, though, what it takes to put around building a sustainable roads investment in Vanuatu, which is a Pacific Islands state, in quite, not quite the middle of the Pacific, but very close versus what it takes in, say, Nicaragua versus, which is, you know, in Central America versus what it takes maybe in, Burkina Faso, which is an extremely arid desert state. So the demands on the road system are different. Or then throw in the. Yeah. So you in one place, you've got storms that you need to deal with, like giant cyclonic tropical storms. Right. In another place you've got arid desert conditions. In another you've got somewhat tropical humid conditions. And that's just the weather differences. You add in the politics of all of these places and who gets to decide whether the money goes into road maintenance, or it goes somewhere else, whose buy-in do you need to institutionalize that. Who even gets like, how do we help them to run the procurement in a clean way, so that whoever is doing the road building and the road maintenance is appropriately paid and compensated, and not excessively paid or compensated, or related to someone, all of those things vary widely.

So even when you're trying to do the same thing, the way you have to go about it is really different. I think people in international development learn that lesson again and again. I'd love at some point to have a conversation with folks who work domestically about how much we learned the difference between how, you know, when you do it in California versus Alabama versus Pennsylvania.

I assume that's pretty different, too, but I don't, I'm not in those conversations. So that's like, when you host that podcast that I'll tune and I'll tune in for that one.

[JEFF BERNSON]

We'll invite you back for sure. No. That's right. You mentioned something, critical about, how you work differently, which is, and it's almost like we take it for granted, but MCC spends a lot of time cultivating these partnerships with the country itself. You talked about, you know, your partnerships, with Belize and Lesotho and others.

And obviously this is just imbued in the DNA of MCC. But it all, I mean, in a world where one of the things that I feel like the global development sector is talking more about, is locally led development, which is really, making sure that the community is at the center of what we do. It's making sure that, government decision makers, not just at the top, but at all levels, are part of the, of the process of, design and developing these, these programs big and small, right? And I'm just wondering if there are things that you've learned about in terms of how to execute towards the goals of locally led development. And is there anything that the agency does today differently, you know, that was built on any of that earned wisdom, even though it's part of the DNA, right? But you've obviously learned a lot since then in terms of how to implement this work in such deep partnership.

[ALICIA PHILLIPS MANDAVILLE]

No, absolutely. And I think you have to keep learning on it and adjusting to keep that, keep the approach tailored to the time we live in. Right? And having said that, then maybe I'll explain. So I live around the corner from where the county is going to put in a new, vocational high school. And the amount of public consultation going on in my immediate neighborhood, the county as a whole, is really high.

And for folks listening, I live on the outskirts of Washington, D.C., in one of the suburbs that has an extreme amount of attention to itself. And so, the active participation of parents and families in this conversation, and neighbors who support the, you know, tree canopy fund are somehow also heavily involved in this. It's huge. There's a lot of discussion so that when they do it, we all have said our piece and understand what it is that they're doing. That feels really natural and everybody understands why you need to do that. And like living here, I understand it, too.

That same kind of conversation actually has to happen between the United States government and the other country government when we say we're going to work together and accomplish something, because, for all the same reasons the families, the communities, the infrastructure, all of that needs to be understood kind of by all sides. And, for folks who are newer to the international development side of the conversation, that's what people mean when they say country ownership or locally led development, is that you have that key decisions and key steps in the process and the implementation and kind of day-to-day management falls increasingly with the country that in which the investment is occurring, as opposed to the country that has the money for the investment.

And that sounds extremely logical. And, you know, on the evidence, like I assume most people listening be like, well, obviously we should do it that way, but it's actually really hard because most of Washington, D.C., which is the political center of the universe, as far as it’s concerned, thinks it has better ideas than everywhere else does. And so individual people, organizations, agencies, actors, technical folks, political folks, all have a good idea that they think every country should get.

And so if you're an education advocate, you think everyone needs an education system. If you are a roads engineer, you think everyone needs a roads network. And the politics of that play out in the same way. And so, it's actually more controversial, not controversial, but in the beginning, because as you said, country ownership is in our legislation. And we took really seriously implementing that in the beginning.

It meant we were trying something really new, like actually just asking a country, what did they want us to invest in. Just from clean. Not saying. All right, we want to do one of these six things. Not saying, okay, we've got money for power, but just what is it you need that's good for your economy was a huge experiment and everyone assumed we'd get terrible proposals, like, I'd like you to build a road from the capital city to my hometown, as the prime minister. And we did get some of those, to be clear. But so, part of the lessons learned and what we've built on, is that you do need guardrails around that conversation. So not what do you want? And the sky's the limit.

But what do you want that meets these criteria? It is going to generate economic growth. It is going to generate the kind of economic growth that we credibly believe is poverty reducing. So it's not investing in the mining sector, which is extractive, and it just doesn't generate the kind of jobs and economic benefits that people need. But investing in the road system that gets their agricultural products out, not in competition with the railroads where the mining sector is working.

Like that's the kind of investment we can make, right? So turning to a country and saying, what is it that you need and want? And then now that we understand there's a policy dimension, the question becomes, what is it that you need and want and are ready to work hard on? Because some of the kinds of policy reforms we ask countries to undertake when they're making an investment in the power sector is around the pay structure of of how do people pay for electricity? And that's deeply political and complicated. So we have learned a lot on that. And I think we define country ownership or local level leadership, however you call it, as the country is in the driver's seat for making critical decisions about the nature of the investment, the structure of the investment and during implementation, the day-to-day management of it.

So actually, MCC doesn't send a whole bunch of folks out to do the implementing. We don't hire them directly. It's different from a lot of the rest of the foreign assistance infrastructure. The country sets up its own unit of folks who live and work in the country. They report to their own government and they're in charge of—they work with us. We work closely with them on doing it within U.S. federal guidelines, but they do the procurements through a procurement agent. They oversee the engineering firms. So it really is locally done. And in the end, like we see countries willing to take on reforms that would otherwise be beyond what they're ready to do. So changes in the tariff structure for how people pay for electricity or changing the inheritance law so that women can inherit as equally as men, like things that are actually really significant in the world or in, in people's lives and in the world and doing it in part because we start with asking, well, what is it you need and want that meets these, you know, that has this effect in the world. And there's some more detail on on how we've kind of put that in more motion and really locked in some things over these last couple of years so that, you know, our technical experts and political experts’ tendency to believe that they know better, gets, you know, pushed to the corners a little bit more.

But that's a constant struggle, actually, I think, and I think it's one of the things we've got, we'll always have more room to grow on.

[JEFF BERNSON]

Yeah, no, that's that's really, really salient. I mean, we've talked a lot already about how MCC puts a premium on learning. You've given some great examples of that. I mean, and it makes me think that you must also have some thoughts or some, reflections on how the agency learns also has evolved over this period of time, you know, how you think about gathering the evidence, and then also reflecting on it within the investments. I can only imagine that that has not been a static process throughout. And I'm wondering if there are things that you can point to that, some of those evolutions that, maybe you're most proud of or, that you feel are most, have been most critical to this.

[ALICIA PHILLIPS MANDAVILLE]

Great, I think there's a technical one and there's an emotional one, for lack of a better word, to describe. And the technical one, it's actually quite specific, which is that, you know, I think in the beginning, we like many other folks who are thinking about evaluations, understood RCTs as the primary vehicle that we would use, in order to, to evaluate our work.

And I think, you know, I was there for nine in the agency's first ten years. And I remember kind of what was going on in the community as a whole and then what was going on inside of the agency as we felt our way through trying to operationalize some of the like, yes, you really can scientifically go at this and you really can learn, even if it's in the social sector, it doesn't have to just be medical trials. Anyway, we really took a lot of that to heart. And so, when you look, if you go to evidence dot mcc dot gov and you look at our evaluations, you'll see a lot of RCTs in the beginning. You will still see RCTs when they're, when they make sense. But the reality is there's not a counterfactual for, I'm going to put a road from the capital city to the coastline. Like that's you're either doing that or you're not doing it. And there's some other things around, you know, we want to assess the effect of policy regulation changing over time. And that's also you can't that, there's no RCT application for that. And so we've necessarily but appropriately needed to become more creative and practical about what are all the ways we think about evaluating the work and learning from what we've done.

And I think that you can see it. There's evidence of it when you look in the platform by date, what kinds of things are in there? And it's a space, frankly, where I think we're still really hungry to make progress. And I would say on that one in particular on policy reform, and how do we think about designing a MEL, like a MEL plan for when we know there will be pos—

Take, Solomon Islands, we're working with them on increasing investment in tourism. It's a beautiful, amazing country. It doesn't have much tourism infrastructure, so it's really niche tourism right now. It's good. It's a great investment. It's complicated. When we started, we couldn't say which piece of the regulatory environment was going to actually get addressed because there are infrastructure components, there are land-use components. There are like water and electricity components. There's so many moving pieces to build a viable tourism sector. And so we have this piece of a program called an iterative design-oriented piece of work and policy reform in the tourism sector. And we have, it is very complicated to figure out how are we going to say what we're going to measure, because we can tell you the range of things that might get better, but it's hard to get really specific on it. And this kind of blows our evaluators’ minds internally because we really want to define things down and say, okay, when we see X exactly we all know we have succeeded and we don't know which. There's like 20 possible Xs.

So I think there's a lot of space for continued growth here. And that's exciting to me. And that's the technical answer. We've gotten from mostly RCTs to this broader family of ways to think in a in an evaluative way.

The emotional one is that I think at the beginning, and I remember this, we assumed our evaluations would tell us we had done a good job, full stop. And a lot of people who had never thought about the function of an evaluation as learning, they saw it more as an accountability check to see, did you do what you said you were going to do? And let's prove I did a great job. And I'll be upfront. I've worked other places between. I was at MCC, before, I'm back now. In the window, I was other places, folks who worked with me there will hear me say you need to learn on the cheap how to be told you did it wrong because then you want to spend your money learning at a high quality how to do it right. And but, but you've got to break that emotional block to being told that it wasn't right, because one of our first sets of evaluations came in and showed in Ag that starter kits didn't work. Like this was like the most heartbreaking thing ever, because there's an entire sector built on the notion that this is actually how you should do agriculture extension work.

And we had three evaluations on three continents saying it didn't work, and that was just gutting. But you learn we learned fast then. Okay, now first we need to design to learn not just whether it worked, but why it worked or not. And we got to build these feedback loops. But I think that that's actually a really, really big deal, in particular, for places that are just starting down this path of learning to meaningfully use evaluations.

[JEFF BERNSON]

It also kind of brings me back to some of my first questions, which is, given that you have adopted this posture around, not just generating but also using data and evidence. I'm wondering, I, you know, I see it now with the Evidence Act, that more agencies are adopting similar postures. I can only imagine that, you're being contacted more around the lessons that you're learning and also ways that, they can adopt, what you've done, perhaps faster, right? Because of the basically building off the lessons that you've learned along the way. And I'm wondering if you can share any stories or common questions that you're getting from, any of those agencies or organizations.

[ALICIA PHILLIPS MANDAVILLE]

In the rankings that just came out, MCC has essentially been ranked now on multiple occasions as the most transparent aid agency on the planet, and we're super proud of that. And it does mean people turn to you and say, well, how did you do that? What can we do, too. And in fact, actually, this last time around, not only were we the most transparent bilateral aid agency, but in the top three of most transparent foreign assistance or aid, economic assistance agencies of any sort.

So the World Bank, like that of everyone who uses money in other countries to try to promote economic growth or economic outcomes, MCC is in the top three for most transparent. And that is a huge accomplishment for a small place. So we're really excited. And then people see it and they do call us up. And sometimes surprising actors as well, like the Small Business Administration in the U.S., is actually in a regular conversation right now with our director for results and learning about how we built our evidence platform and how we, because it's basically the evidence dot MCC dot gov platform that everyone listening can go and look at and search everything we've ever published, which is everything we've ever done in terms of evaluations and our data sets and underlying reports and all kinds of other work, is the same resource our own team uses to find the work we've done. So there's not like a hidden special vault of extra MCC information, anything that the PII is removed from. We even have a disclosure review board to publish that stuff, right?

So we're working with small business on how they build something similar so that people can think about how you encourage and invest in and promote small business growth all across the U.S. And that's not at all the same mission that we have, but it's the same kind of function of the same idea, and it's a really cool exercise to talk with them about what are the mechanical steps we went through and what are the questions we went through to get to some of the things we've done.

Conversely, then, I was just at an event in Indonesia with a large variety of bilateral donors, and MCC is not the United States’ biggest aid agency. We're actually quite small compared to USAID, but we're about the same size as, say, the Norwegian one or the one [in] South Korea. And so I had a great conversation with the two representatives from those agencies on how we think about accountability and how both evidence of our work like impact accountability and political accountability, because we're all, you know, part of democratic governments and systems, how do we manage those things together? Because bluntly, Congress expects us to be successful all the time, they haven't had that emotional learning around how evaluations actually work. And so, you know, the Koreans and the Norwegians have the same kind of concerns, right? So the ranking of being most transparent has been great because it like sets us up to have a community of actors that we can work with that are trying to do the same things, and we can talk about the real challenges and not the assumed challenges.

[JEFF BERNSON]

Really turning towards the future with the agency's overall mission, obviously to reduce global poverty through economic growth, as you mentioned early on, the strategic priorities will continue to get refined and I'm wondering, you know, right now you're working on climate change, you mentioned, inclusion and gender poverty reducing growth, health, education. Do you expect that the priorities list to, continue to grow? Are there other emerging areas that you're monitoring because you think that the effect, could even be greater in terms of what the mission and the work that you're doing directly with countries that you partner with?

[ALICIA PHILLIPS MANDAVILLE]

You know, this is this is an exciting piece of, I think MCC’s future, which is that because we don't start with a set of assertions about what sector we think we're going to work in, when we turn to countries, their actual needs and opportunities drive that conversation. And it means that, for example, MCC made a huge number of investments in agriculture before the U.S. launched Feed the Future, which is a large U.S. initiative to make sure that agricultural production meets, meets, needs and that people can eat.

We stopped doing as many of them once that was launched, because those needs were being met elsewhere. We still do some, but not as many as in the beginning. Similarly, we did in the beginning and continue to do a huge amount of estimates in the power sector in Africa in particular, before the United States launched Power Africa, which is kind of now the hallmark investment vehicle that the United States government understands as the way to focus on increasing access to electricity in Africa.

And we were doing that for years then before that launched as well. So there's a bit of the market leads MCC into the spaces that are actually emerging. And I think that is one of the coolest things about the agency actually, is, yes, it's great that we can be smart and forward-looking, but also we can just listen to market signal and country need and listen to our partners and the things that I think we see more of now, you know, you mentioned climate.

It is, you cannot look away from the effects of climate changes, regardless of what you think about the drivers of them and who should do what about them. You can separate everything political out of it, but you cannot look at Mozambique, which now has a 50-year cyclone like every three minutes, as far as I can tell. I mean, I'm exaggerating. I shouldn't be flippant about it, but the storm issues on that coast for Africa are so intense. You can't look away from the incursion of saltwater on the coast of Mauritania and the desertification that are pressing people into the capital city. Like, it's just true whether you whether we agree about what causes it or not. Addressing those kinds of things, building for resilience in infrastructure, thinking about migration, regardless of its drivers and what does that mean for a relatively nascent capital city?

Those kinds of things—urbanization, digital services, digital economy, the, climate adaptation and managing its effects—all of those things I think are all through our future. But I think we're actually really well calibrated to be able to step into that. And countries are asking us for that, like our investment in Mozambique that's getting up and running right now includes mangrove restoration, because that is a massive way that those coastal livelihoods are protected.

And it's an economic investment. It's not actually just a climate investment. And the bridge infrastructure so the water can go over it rather than push it out like things that we're doing are really directly affected by it, and we're able to flex in that way. I think we have some challenges around thinking about the digital economy is a complicated, it's not defined well, you know, and everyone's got a different sense in their heads. Countries want to think about their role in it differently. How do you order electricity, electrification and digital investments? There's things like that that we are in the midst of going to our toolkit, like our evidence based, our evidence production and use toolkit. How do we adjust our constraints, economic constraints analysis that we start everything with a country with to make sure we're asking about climate and digital and migration and all these other things and inequality and inclusion.

How do we when we do a cost-benefit analysis, like how do you think about climate effects? Because the countries we're working at are not the primary producers of greenhouse gases, but the effects are hitting them. So, when you do a cost benefit analysis and you talk about global effects, like how should you, should you scale it, like how should you think about that?

And these are like deeply in the weeds. And I'm really excited because I think it means we're in the place we can actually figure out how to address the issue and not just decide if we should or not. And I think that's that's such an exciting place to be for an evidence-based organization who's got a mission of reducing poverty.

It's really people-centered in the goal for the outcome. But we're really technical and evidence-centered in how we get there. I think our one challenge is going to be, because we are so evidence centered. There's a desire to say, “well show me where this has worked somewhere else.” And that's an appropriate and good reflex. But some of the things we're trying to address just haven't happened somewhere else, or they haven't been done somewhere else. And, you know, I think we're trying to be like proactive about making sure we stay evidence-focused and not evidence-addicted because there are places where it's just never been done. And so you can bring to bear the relevant evidence of other places and learn from them, but still be willing to take that risk and address the problem that just needs solving.

[JEFF BERNSON]

Alicia, I love it. Evidence-focused, but not evidence-addicted. I think it's a great note for us to wrap things up. I just want to thank you on behalf of On the Evidence for your time and spending it with us. This has been great. We have covered so much ground from, the creation of MCC, orienting our listeners to that, through all of the evidence-generation approaches that you all have taken, and, of course, how you learn and also looking towards the future. So we really appreciate it. I hope that we can have you back, and, and hear how the evolution continues at MCC. But for now, thank you so much.

[ALICIA PHILLIPS MANDAVILLE]

Thanks so much for having me. It's really been a pleasure.

[J.B. WOGAN]

Thanks again to our guest, Alicia Phillips Mandaville of the Millenium Challenge Corporation, and thanks to my Mathematica colleague, Jeff Bernson, for filling in as the guest host for this episode. And, of course, thank you for listening to another episode of On the Evidence, the Mathematica podcast. This episode was produced by my Mathematica colleague Rich Clement. If you’re fan of the show, help others discover our podcast by leaving us a rating and review. To catch future episodes of the show, subscribe by visiting us at mathematica.org/ontheevidence.