Cecilia Rouse on the Use of Evidence in Public Policy

[CECILIA ROUSE]

What we need in public policy is our problems are big, and we don't all experience them the same. The solutions are not going to be the same in every neighborhood in every part of the country. We need to have people who come with different perspectives in order to have the insight to make any policy we're contemplating more effective and more relevant to more people.

[J.B. WOGAN]

I’m J.B. Wogan from Mathematica and welcome back to On the Evidence.

We have a very special guest for this episode: Cecilia Rouse, who became the president of the Brookings Institution earlier this year. Cecilia is an economist who has led some of the most influential policy research institutions in the country, including Brookings, the Princeton School of Public and International Affairs, and the White House Council of Economic Advisors. Her career includes a one-year stint with the National Economic Council during the Clinton administration, two years as a member of the Council of Economic Advisors during the Obama administration, and two years as the chair of the Council of Economic Advisors during the Biden administration. In that last role, by the way, she became the first Black American to chair the council in its 75-year history.

For this episode, Mathematica’s president and chief executive officer, Paul Decker, returns as the guest host. During the interview, Cecilia talks about what drew her to economics, what she learned from her government posts about the role of research in influencing policy, how she thinks about the role of academic institutions in conducting policy research and training future public servants, what AI and other technology mean for public policy schools, how economics and related professions are doing at diversifying their workforces, and what she has learned about leadership from her experiences in academia, government, and the nonprofit sector.

If you’re new the podcast, please take a moment to subscribe. We’re on YouTube, Apple Podcasts, Spotify, and elsewhere. And if you like this episode, consider leaving us a rating and review. It helps others find our show.

With that, I’ll turn it over to Paul and Cecilia. I hope you enjoy the conversation.

[PAUL DECKER]

Cecilia Rouse, welcome to Mathematica. Welcome to On the Evidence, our podcast.

I thought -- well, first of all, I'm very excited for this conversation because I think the mission of Mathematica is basically that we bring the data and the evidence that bear it out, people make better decisions so that we can improve public well-being. In addition to that being our mission for us as an organization, it's kind a theme on your career as well. So I like that connection, and I'm interested to hear more detail about that.

[CECILIA ROUSE]

That's terrific! I'm excited to be here for exactly that reason.

[PAUL DECKER]

That's great. Before we get too heavily into that, I just to go back a little bit closer to the beginning of the story because we both have daughters in college now. My daughter's first year in college, I spent a lot of time observing how she's reacting to her first year of classes. I went so far as to buy the textbook for here Econ 101 class, so that was very interesting.

In getting ready for this session, it made me think about, well, what was Cecilia Rouse's response like in her first Economics class. So can you tell us about your first Economics class and how you reacted to the material?

[CECILIA ROUSE]

Sure, I'd be delighted to do so.

[PAUL DECKER]

Okay.

[CECILIA ROUSE]

So I like to think about how I got into Economics as it's the perfect combination of my parents. My dad was a theoretical astrophysicist, was doing nuclear physics for his money. That's how he paid the bills.

[PAUL DECKER]

Yes.

[CECILIA ROUSE]

My mom was a social worker, a school psychologist. So she was very interested in people and social issues, and he was more technical. I enjoyed math and science. I really wasn't sure I wanted to be an engineer, and it was my mom who suggested that I take an economics class. So I enrolled in Economics at 10 at Harvard. I will admit that I did not enjoy micro -- microeconomics, which is what we learned the fall semester.

Then in the spring semester, we were studying unemployment; but it was from the macro perspective. But there was analysis of what were the types of unemployment -- frictional, structural, cyclical. And the way it was analyzed and kind of broken down, I sort of went, "Aha, this is a field that allows you to break down social problems." I had been very interested in the unemployment. The unemployment rise at that time had just been coming down, and there were stories on the news about the unemployed; and I really felt for them.

I really enjoyed the way that economics allowed one to break down problems and analyze, which allowed them a way of thinking about how to address them. So the following year, I took a Labor Economics class, and I took micro again; and I just -- it clicked then.

[PAUL DECKER]

Mm-hm.

[CECILIA ROUSE]

And I just -- I was drawn to economics for exactly this reason, which is I thought it was a field, it was a way of thinking that allowed one to learn more about the world and potentially come up with ways of addressing problems.

[PAUL DECKER]

A phrase that you just used strikes me as a way of thinking. That's really what struck me with economics when I was first exposed to it. Hey, this is a great model, structure for how to think about things, including things that people wouldn't necessarily think were economics to begin with. And that's a lot of what we've seen over the last 20 or 30 years in the field.

[CECILIA ROUSE]

Mm-hm.

[PAUL DECKER]

So from Harvard, eventually you receive your Ph.D. and move to -- well, really from one legendary program to another in the economics field. So in joining Princeton, you're joining a team of Orley Ashenfelter and David Card. Was David still there?

[CECILIA ROUSE]

David was still there.

[PAUL DECKER]

Alan Krueger or Hank Farber.

[CECILIA ROUSE]

Correct.

[PAUL DECKER]

What was that transition like?

[CECILIA ROUSE]

I was starstruck. It was intimidating, but they were phenomenal mentors. I say this sincerely; they were just -- they worked hard, they were brilliant, they were inciteful, but they really in the industrial relations section in Princeton University, which is what this group was -- it was basically a labor economics program or center, and it's called the Industrial Relations Center. There was such a community of exchanging ideas, helping one work through problems, being critical but in a constructive way, that it was so vibrant and alive.

I had spent a lot of time as an undergraduate and as a graduate student at the National Bureau of Economic Research, which had that kind of similar environment as well but had different fields in economics. The section was just labor economics, and so this group of really -- these are luminaries in economics -- took me under their wing, but it was intimidating. I felt I had to work hard, and I had to prove -- I was a new assistant professor as well, so it was all about publish or perish. But it was a phenomenal community, and I really credit them with so much of what I've been able to do since then.

[PAUL DECKER]

Good. What were the key topics in your early scholarship that you really focused most of your attention on?

[CECILIA ROUSE]

Well, my dissertation had been on community colleges. So I continued to think about the economic benefits of community colleges and broaden that to economic benefits of college or education more generally. Then, thinking about inside the classroom, what's the role of a class size, of technology, school vouchers? So I really focused in the education and the economics of education.

My most favorite paper though was on the use of blind auditions in symphony orchestras, which was a bit of a one-off. It really started with my thesis advisor Claudia Golden at the end, the ER. It was playing in a graduate student orchestra; and she asked, "What do you know about blind auditions?" A student had a throwaway line in a paper which was, "The use of blind auditions increase the presence of women in orchestras."

She says, "What do you know about that?"

I said, "I don't know anything, but I'm happy to look."

That was while I was still at Harvard. When I was at Princeton at the end of my first year, I was casting around. I've got start thinking, "publish or perish." I asked some undergraduate students to start doing a little bit of work on that, and it was my most fun paper. I sat in on auditions, so I did the field work; collected data, so it was a little bit of a mash of history and econometrics. But that was a big paper that I focused on as well.

[PAUL DECKER]

That is a well-known paper. Just for the benefit of our listeners, can you explain a little bit how blind auditions work?

[CECILIA ROUSE]

Sure, so blind auditions are auditions where there is literally a screen. I hesitated there because often there's a round before the preliminary round, which they used to send in tapes. Now it's just some digital file. Those can be blinded too. So blinding is really just that the people listening don't know who they're listening to, and so they're judging the quality of the playing based on what they're hearing rather than knowing something about who's playing.

[PAUL DECKER]

Right.

[CECILIA ROUSE]

So in a live audition, it was literally that the musicians would be behind a screen. The challenge was that women are often wearing different type shoes than men. So they would either have a carpet or sometimes the manager would provide the compensating footsteps. They couldn't ask women to take off their shoes because then the absence of a heel also tips them off. But it's really the effort to hide the identity of the player.

[PAUL DECKER]

And it brings out potential bias.

[CECILIA ROUSE]

In theory.

[PAUL DECKER]

Yes.

[CECILIA ROUSE]        

Sometimes there was a, I think a French horn, or some wind instrument where it had a very distinctive way of playing, of breathing. So everybody who knew him could identify who that was, right? Sometimes--

[PAUL DECKER]

Or signature.

[CECILIA ROUSE]

Signature, exactly. Or if it's an instructor, they know their students well. But it cuts down a lot of bringing in that other kind of information.

[PAUL DECKER]

As you were conducting your search earlier in your career, how do you think about relating research to public policy? You mentioned community colleges as being one of the topics. How did you frame thinking about that connection between research and policy at that stage of your

[CECILIA ROUSE]

So I got into economics because, again, I liked the way one could use economics to -- it started with the theoretical framework, and then it turned into the empirical method. It could be used in answering questions in public policy.

So my first job out of graduate school was at Princeton, where I was jointly appointed in the school's Public Affairs in Economics. So the application of research and economics to public policy was always, for me, a great part of the motivation. That said (laughing)., it's not that I literally set out to say, "I'm going to estimate the economic return to community colleges because I know that I'm going to get this into hands of policymakers who are then going to do X, Y, Z with that information.

It was more that I tried to address problems. The problems that interest me I think are real-world problems that people face as they're making their own decisions or that I would think that we need to know in order to be guiding policy in a more abstract sense. So it's more that I've been guided in my own interests by questions I think that are relevant in the world at that time.

[PAUL DECKER]

Right, now you then got the opportunity to play a direct role in public policy with three different stints in the Federal Government, at least so far--

[CECILIA ROUSE]        

Mm-hm.

[PAUL DECKER]

...starting with the National Economic Council during the Clinton Administration in the 1990s. So I can kind of imagine that the background that you described was the motivation for doing it. But can you talk a little bit about how that transition happened and maybe how your research then influenced what became the issues that you engaged in on the National Economic Council?

[CECILIA ROUSE]

Sure, I'd be happy to do that. So I had been asked -- the Council of Economic Advisors, which I then went on to do two tours with--

[PAUL DECKER]

Right.

[CECILIA ROUSE]

...as part of their staffing, they have graduate students. So graduate students will take a year out. They have assistant professors who take a year out of people's early career. I'd been asked in previous years to spend a year at the CEA but had not taken up that opportunity because Princeton didn't stop tenure; and I very much wanted to get through that.

Larry Summers was the Assistant Secretary for Economic Policy, and he was looking for a new deputy assistant secretary to replace my colleague and friend Jon Gruber. So they called and asked if I might be interested to come down and spend a year or two at Treasury. That was just as I was putting in my tenure packet, which meant that even if Princeton didn't stop my clock I had done what I could for tenure. So I was more intrigued. I was much more interested because I did feel that as an academic who was interested in policy, I wanted to have some experience of what it was actually -- what it would mean to actually do some policy. I wanted to understand that real-world connection.

It became known that I was talking to Larry Summers in Treasury; so Gene Sperling, who was on the Economic Council, said, "Hey, what about us?" So I debated, and I went to the National Economic Council because I felt like it was more in the center of actual policymaking, that it was -- because the National Economic Council was designed to coordinate economic policymaking for the Administration, right? The president's going to decide if he's going to -- he or she -- is going to support, I don't know, an increase in the federal minimum wage.

Then the president wants to hear - or should be hearing -- from the news of different parts of the Cabinet around the Administration and then developing a view for the Administration, like they're going to support a bill, what action might be taken. The NEC is sort of that nexus, which means you're seeing everything. So it was a trial by fire. It means you're also the ones that are -- the lobbyists come and speak with you on any particular issue. At the NEC, I had the opportunity to negotiate on the Hill, which was also really fun, on a visa issue. So it was really exciting to me from that perspective.

The NEC also helped in the budget -- putting together the president's budget, spark new -- where are we going to put some emphases? So my own research has told me, or what I have learned through the data, was that if we look at college going that especially in community colleges that completion was a really big challenge. So many people have been focused on access, and I think people didn't realize that completion was another big problem. So I brought actually a very small bill. I think we called it the College Completion Challenge Grant. We actually got it into the president's budget, which I was pretty proud of; but that came directly from my own research.

I also did a lot of work on adult literacy, which hadn't so much been from my research but the community colleges are educating the young adults; but they're also educating older adults too -- so interest in job training. So I did a lot of work on adult literacy and actually increased the amount of funding that was given to adult literacy through some of my efforts. That also sprung from my research.

[PAUL DECKER]          

So in the process of this initial experience, what did you learn about the way in which research influences policy at the federal level in the U.S.? What lessons did you take away from that experience?

[CECILIA ROUSE]        

So the biggest lesson I took away, which I think has pulled through my -- the rest of my career, including where I am now at Brookings, is it's so important for that work to be translated. That as academics, our goal is to get our work into the academic journals -- the American Economic Association; and the AER, the American Economic Review; the Local Economy' the whatever, Labor Economics. But it is really to publish in the academic journals and that our academic papers are very dense because what we're trying to do is write estimated parameter and really ensure that that parameter is what we think it is and that we're not misinterpreting it in some way.

And that if you are actually a policymaker, you just want to know the results.

[PAUL DECKER]

Right.

[CECILIA ROUSE]        

And you also want to know is this a result I can trust or not. And they don't have the time to read through all of the different -- you know, I did all of these robustness checks. They don't have the time to read through all of that, and some of them don't have the expertise to read through it either.

[PAUL DECKER]

Right.

[CECILIA ROUSE]        

So the translation is so important, and that was a lot of the motivation. I was actually on the technical advisory group for the initial What Works Clearinghouse, which was an attempt to summarize a literature using empirical methods. It actually was based on that I joined with some colleagues at Princeton and at Brookings to start a journal -- or we took over a journal called The Future of Children.

[PAUL DECKER]

Yes.

[CECILIA ROUSE]

And we developed -- which were really just literature reviews done in a particular way, but our goal was to make them highly accessible because I felt like that was the piece that was so important for academics, who need to do the actual work. We need them to be doing the creative ideas, and we need them to be doing the robustness checks. We need all of that, but then there needs to be another step really for it to be useful for those who are making decisions.

[PAUL DECKER]

Yeah, I think that could be a tough transition for people to make. Not only clarity of communication in that setting but also being concise. Policymakers don't have a lot of time.

[CECILIA ROUSE]

We don't have a lot of time. We spent -- for The Future of Children, we had an editor who would take -- and the papers were written to be literature reviews, right? That was the intent, and we would have an editor that had to do a lot of work to turn them into something that was really accessible. I used to equipment -- and it wasn't a quip because it's actually true -- when I was a senior editor, for each of the volumes I worked on there was at least one paper I had to rewrite because it just wasn't written in a way that was going to be accessible to someone who wasn't already in the field.

[PAUL DECKER]

Right. So you later returned to the government after going back to Princeton. Then when that stint was done, you went back to the Government, as you mentioned, first as a member of the Council and then eventually as the Chair of the Council. How was that experience different than the NEC experience?

[CECILIA ROUSE]        

So the Council of Economic Advisors, the CEA, plays a different role. It is one of the inputs to the process that the NEC is playing. Some people describe the CEA as being a think tank within the White House. It is even structured like an academic institution, where we have research assistants and we have the junior faculty and the senior faculty and then we have the deans or something -- I don't recall the numbers, and the Chair. It is the place where -- I'm hesitating here.

I just gave a talk at the NBER in the name of Martin Feldstein, Marty Feldstein who was a Chair of the CEA, and who was willing to speak -- as I said in my talk -- speak truth to power. He has all of these cartoons. He has all of these cartoons at the time. No one ever did them of me, which I guess I'm very upset about. I guess I wasn't as contrarian. My favorite was they were in a skiff and they were all rowing and Feldstein was rowing backwards, and Reagan's yelling, "Feldstein!"

But I think that's the role of the CEA is to say, "Here's the economic perspective," on a particular issue. "We're not going to try to play the politics straight up." I'm not saying that politics aren't in the air. Let's be realistic; it's part of the White House. But if there's something that just -- the academic literature just does not support, it is the role of the CEA to say that to the president.

I would say to my staff, "You don't have to write or say something that would force you to give back your Ph.D."

That's what I interpreted the role of the CEA to be. So, day to day, we're interpreting the data. We got initial, the UI claims today, so the DEA would have gotten those data the day before. They would have written a memo for the president interpreting them and then have something to say about it the next day. So interpreting the data, helping to look around corners, and then obviously taking a role in the policy process as well.

[PAUL DECKER]          

Based on being involved in this process at multiple points, is there anything you can tell us about the trend of the influence of research in the government. Is that -- is research being more influential, or is it stuck? Ron Haskins, who both of us know, used to have this presentation where he had a pie chart of all of the different elements, all of the different contributors to policy. And he had this very thin slice of the pie for research, implying that somehow research was only a small part of the discussion. What's your take on that and the trend?

[CECILIA ROUSE]

So I think that trends are hard, depends on who’s in charge.

[PAUL DECKER]

Yeah.

[CECILIA ROUSE]

So I think it depends on whether the principal and the principal's team actually want to have evidence behind whatever they're doing.

You and I both know that if we can take any one issue, there's different ways to read the literature. So it's not to say that -- and if you summarize the literature, you can always end up with interpretations on each side. But this is, again, where I didn't want my team to give up their Ph.D.'s in order to interpret the literature. That is different, however, than when the decision is being made that the principal doesn't have to weigh many other factors; and I respect that.

I mean, you must have to do this in your role too. Like there are many other constraints or many other considerations. So CEA would often joke, "At the end of the bus, we get rolled to however you want to characterize it. But if the research and the evidence really flat out would say -- this is in my experience -- that something was absolutely the wrong thing to do, I was never had the experience that then that would be ignored.

[PAUL DECKER]`

No.

[CECILIA ROUSE]

If there was gray or incomplete -- and you and I both also know that our evidence often does not fully answer the question at hand, the data could be from 20 years ago. It's a case study, a very compelling case study; but we don't know how generalizable that is. Or it just doesn't quite answer the policy that's in front of us -- it's still a component, but it may not be the final determinate of the decision.

[PAUL DECKER]

Right, right. In cases where evidence is used, even in my experiences it can be a frustrating experience to see how evidence is used. Having been somebody who's been involved in a lot of program evaluations, a lot of what I would see is, "Hey, if you find positive impacts on outcomes of interest, that program survives. If you find negative impacts or zero impacts, maybe people try to do away with the program. That seems too simplistic a framing.

[CECILIA ROUSE]

Mm-hm.

[PAUL DECKER]

You had a piece with Ron Haskins, again, in The Future of Children that was really about the strategy of how the Federal Government would go about embedding evidence into policymaking and policy change framework.

[CECILIA ROUSE]

Mm-hm.

[PAUL DECKER]

I wonder if you could talk a little bit about that and tell me, are we making progress toward that kind of strategy? Really what I'm interested in is how do we get away from this kind of up or down on whether programs are working and more working on how do we use evidence to say we can do a little bit better here or we can continue improvement in serving a population that we've already committed to serve.

[CECILIA ROUSE]        

So you ask -- a little while ago, you asked me about what have I learned in policy. The other thing I was going to say is patience. I don't know that there's ever transfer; again, it depends on who's in charge and what the principle is and what the issues are.

But with the Obama Administration, especially in education where they tried to develop a structure of if there's -- especially in education -- so if there's an area where it's a brand-new idea, it's been used in only a few places but there hasn't really been any big experiments, they can have small dollars. But there was an encouragement to at least start collecting data, start trying to see what you're learning and then see what's working, and then to be able to pivot if those data and those results aren't panning out. But to learn from utilizing the evidence and then if that is looking promising, to try to scale up to do a slightly bigger project, maybe to do one randomized evaluation that's more of a case study. If one or two of those are looking more promising, you scale up to do the really big randomized evaluation that is much more generalized, where we have many different sites.

[PAUL DECKER]

Yeah.

[CECILIA ROUSE]

That is, I think, the kind of spirit of learning we want to be adopting in many places because it is the case that not every time any one study is going to be -- it's always going to be positive. But we have to be using data and evidence and learning from -- we can call them mistakes or what works well/what doesn't work well -- and using that to innovate. It's really the spirit of innovation and using data and evidence to do that.

Furthermore, there can be a chilling effect. If decisions are going to be made because you get a bad result where it's going to be killed, then just don't do the study in the first place; and then we never learn, and that I think is the bigger tragedy here.

[PAUL DECKER]

Oftentimes we use structured evaluations to assess the answer to a very specific policy question, but sometimes we're surrounded by observational data. So you have to kind of connect the dots, the research dots, if you will.

[CECILIA ROUSE]

Mm-hm.

[PAUL DECKER]

I wonder about kind of when you're in a setting like that and you see observational data, how do you decide when to take action based on that data versus saying, "We need more research to develop more confidence." At what point do we say, "We've got enough, given what we have in front of us, to take action?"

[CECILIA ROUSE]        

So I think often (laughing). The really well-implemented, convincing, generalizable, structured RCT or really convincing some kind of analysis that's really causal and you think that it's generating exactly and estimating exactly the right parameter is so rare that what -- so it's my view that we need that, but it's a broad church and it's a portfolio. We need to have the structure -- the structured RCTs, regression, discontinuity, which still takes a lot of data and doesn't exist everywhere; and then there are places where we've learned enough.

Like let's take return to schooling. As it turns out when people have come up with different ways of estimating the causal estimate of the return to schooling, it's generally generated an estimate of -- I don't know -- around 10% or so increase in earnings per year for each year of schooling. That's actually pretty close to what you genetic using observational data. So let's use observational data and call it good and move on.

[PAUL DECKER]          

Mm-hm.

[CECILIA ROUSE]        

Now, my moving on doesn't mean you move on forever. I actually think periodically you have to go back and revisit questions because of the society changes or economy changes. Our economy today is very different in many ways post pandemic than it was 20 years ago. So I think we have to pick up -- and I don't know that these are hard parameters like the speed of light, but I think that it's nice to have the causal estimates on some parts of the parameters but we often have to make decisions based on something else.

Medical doctors do too.

[PAUL DECKER]          

Yes.

[CECILIA ROUSE]        

We know that those little randomized experiments that they're basing on very selected samples may not generalize to different populations, and they have to be connecting dots in order to make decisions about how to treat a patient.

[PAUL DECKER]          

Yeah, in fact that example brings up that the stakes are important too.

[CECILIA ROUSE]        

Exactly.

[PAUL DECKER]          

So if we know smoking is harmful based on observational data, yeah, I can't map the exact causality; but the data are so suggestive and the harm is so substantial, then we have to think about taking action rather than just

[CECILIA ROUSE]        

Exactly, although the research does keep us in business.

[PAUL DECKER]          

Yeah (laughing).

[CECILIA ROUSE]        

The call for more research does keep us in business.

[PAUL DECKER]          

Right. Cecilia, as part of your experience at Princeton University, you served as the Dean of the School of Public and International Affairs there at the university. In your time that you spent at the Government, how did that influence how you think about the role of economic institutions -- like, for instance, in informing policy discussions?

[CECILIA ROUSE]        

Yeah, so one was that I think that we have to recognize that everything has a job to play. Everyone's got a role to play. I think it's very important that universities are there to be doing more basic research. Doesn't mean that it's not implied, but to be asking big questions longer term the faculty there have longer time to examine a question than the people who are, say, working at the Council of Economic Advisors or in Government often have in order to address a question. I think that plays a really important role in generating the evidence base and the research base on which the policymakers can make decisions.

They can also look around corners and are sometimes the ones to highlight that this is going to become -- this will be a problem or is a problem. You may not have looked at the data in this creative way, and so highlighting an issue that then gets to the attention of policymakers. So I think that kind of having the space and the creativity and the time to really think about the world in different ways, free of the demands of the budget cycle or whatever external pressures there might be for someone who's in Government, I think that's a really important role.

Obviously, they also educate future leaders and future followers too. But the future Americans and the Electorate and citizens of the world. From my perch, the value and the respect for evidence I think is an important role that universities play.

But one aspect that I really did get from my time at the National Economic Council in particular when I was on the Hill and was negotiating on this visa issue was also -- and the NEC more generally where I got to hear from different perspectives on issues -- was having a really healthy respect and actually I benefitted from hearing very diverse views. And that is something I did take with me into my deanship, where I -- not afraid of confrontation. I'm not afraid of somebody coming in and disagreeing with me because they have their perspective. Often, I learn something from their perspective. I may not agree with it, I may not implement it; but I'm a better person, I think more sharply for having heard it.

Also recognizing that our decision-makers come in many stripes, have many views, and that if our students don't reflect that, then we are not helping to train the whole diversity and heterogeneity, if you will, of policymakers. So at SPIA, we worked to have diverse students; and we had programs to help identify minority, students and first-generation students. We weren't doing as well on people who thought ideologically differently, conservative students.

But I think that that was -- that's an important role for us to play too because if we're really going to -- the kind of skills that we were imparting on our students at SPIA I think is important for every kind of decision-maker -- federal, state, local, not-for-profit, whatever it might be. I think it's important for people who think very differently to have those skills as well.

[PAUL DECKER]          

What about the role of technology in managing or implementing policy? How has that -- is that changing roles and feeding back on how you have to prepare students from other policy schools to be practitioners?

[CECILIA ROUSE]        

Yeah, I think it's actually a pretty critical role. At SPIA, students would learn economics, political science, econometrics, statistics. We were starting to weave in sociology, a little history. And I was trying to bring in more technological understanding as well. The role of the Affordable Care Act to me was an example of how technology cuts across all of the different policy areas, just like economics can or political science can. Not that we were going to train people to become the coders to develop the next version of AI or whatever software we're talking about, but they needed to be very savvy consumers of it. And they needed to be able to as what are the questions as they were using it.

We were all using it in how we did our business, right? So we were doing it in our research and even how we managed courses. We use it every day in writing papers. But they were also going to be using it for how it was going to come into their operations. It could come into -- have intersection with the policy areas that they cared about. So I thought that was an important skill. I used to say at some point I thought that having those kinds of skills would almost be as important as economics and political science and statistics. I don't know where SPIA is on that journey; but I think as we think about AI and how quickly it's all evolving, I think that's even becoming even more imperative.

[PAUL DECKER]          

There are a number of issues or a number of efforts underway in the social sciences that diversify the workforce in public policy, and Mathematica is an organization that participates in a number of those efforts. But in the past, you've chaired oner of those efforts for the American Economics Associations that had committee or has a committee on status of minorities in the economics profession.

[CECILIA ROUSE]        

Mm-hm.

[PAUL DECKER]          

If you were going to provide a progress report on how we're doing in terms of diversifying the workforce in the field of public policy, what would that progress report say?

[CECILIA ROUSE]        

I haven't looked at the data of late, but I would say there's been progress. It's not -- it's far from complete, but there has been progress. I think the Biden Administration without putting a super cheerleading hat on here really did make a very concerted effort to be diversifying his team from the top to the bottom. I think that's important. I think it was important to SPIA. I mentioned the ideological diversity. It was -- so CSMGEP, the Committee on the Status of Minority Groups, says about underrepresented minorities, which I think is very important and also incomplete in terms of progress.

What we need in public policy is our problems are big, and we don't all experience them the same. The solutions are not going to be the same in every neighborhood in every part of the country. We need to have people who come with different perspectives in order to have the insight to make any policy we're contemplating more effective and more relevant to more people. So I think it's very important. I think we have made progress. I think you can look at -- again, I haven't looked at the data. But you can go back, and there was really no diversity at all, especially in economics. But we've probably made more progress at the bachelor's degree level. We need to make far more progress in Ph.D.'s. I do not believe that everybody needs to get a doctorate in economics to be effective in public policy, but that is one measure of it too.

[PAUL DECKER]

One might argue that establishing greater diversity in the field is an element of rigor. When you think of rigor as you take the effort that's necessary to get as close to the truth as possible, bringing in those different perspectives, understanding the better element of the truth -- how people experience a program, what's necessary to prepare people to succeed in the world? Sometimes there end up being these arguments between rigor and qualitative methods or being informed by the population, and I just don't see them conflicting with each other. It seems like if you want to get closer to the truth, they're both important.

[CECILIA ROUSE]

I completely agree with you.

[PAUL DECKER]          

So now, now you're serving as the president of Brookings. So first of all, congratulations on that appointment.

[CECILIA ROUSE]

Thank you.

[PAUL DECKER]

You do that concurrent with maintaining your academic appointment at Princeton University as well. Brookings is a well-known brand in the public policy field, has been for many years; and you're still in your first year in this new role.

[CECILIA ROUSE]

Mm-hm.

[PAUL DECKER]

But understanding that you're still early in your experience, how do you think about Brookings' mission, and can you share some of your goals for Brookings going forward?

[CECILIA ROUSE]        

Sure, so I am new. I spent the first three months conducting 18 listen-and-learn sessions speaking to our staff -- senior scholars, junior scholars -- across the institution to get a feel for the place. It's a very diverse institution. It's got a wonderful community that are committed to public policy, and they're committed to bringing evidence and facts, whether that is using empirical methods or more qualitative methods, or interviews, or own experience. But they're really interested in bringing the expertise to the public policy realm whether it's proposing solutions, highlighting issues. So it's a wonderful community on that front.

But we know the world is changing; so we've got more polarization, political polarization in particular. We've got technological advances. So if you think about how we summarize our work, Cat-GBT can do that too. So what is the value add of a Brookings scholar or any scholar? Definitely this is a run for the money for universities too, right? I do think the human brain would do a better job, but is the value added of the human touch worth whatever it would take?

So I think how we do our work -- open AI, these large language models, small language models -- all of the technological advances are going to help us do our job can be helpful in how we conduct our research and learn about the world. So I think we need to be figuring out and using them more effectively. We need to use it even operationally. I know you're experiencing this yourself.

But we also need to be thinking so what is really the value add of a place like Brookings in that new environment? And I really do believe it is that the kind of research that we do can't be done with a large language model. As my brother who works in this field pointed out to me recently, we don't have a model of the human brain yet. So the human brain really is so much more creative, complex. It can put together new data in a way that technology cannot do and maybe one day will do, but it's not in the near term.

But it's incumbent upon us to take advantage of our brains and take advantage of what we do. So I really want to have a way for -- my goal is to maintain Brookings place as a premier place where people are a trusted source of information, a trusted think tank for people to come to for ideas to understand the world. And what does that look like? How do we even do that better in this polarized environment? How do we reach decision-makers and other people who want to be informed even more effectively going forward?

[PAUL DECKER]          

So you're in a rare position. You're leading Brookings now, and you've spent years at Princeton University conducting your own scholarly research; and you've been involved in the Federal Government applying that right in the direct public policy setting. As you think about those organizations that you've had experience with, what can they learn from each other in better carrying out their role in contributing to public policy?

[CECILIA ROUSE]        

Really good question! So I really do believe that everybody's got a role, right?

[PAUL DECKER]          

Yeah.

[CECILIA ROUSE]        

It's a team. It's a team sport, and I think that -- and I think that it's important for the universities that -- they want to be relevant. They want to be listened to as well, and they want their researchers -- their work that their faculty are doing to be relevant to the world.

As dean though, I also recognize -- so one of the things I lead by is that there are 24 hours in the day, which means that you can't do all things; and you have to prioritize. And that when we ask our researchers to do too many things, they will do them all less well. So I think while universities should be relevant and should be trying to play in that space, they can't be asking that only of the faculty to be doing that. They have to build other structures.

And I would say that that's true at a place like Brookings as well -- is that we ask a lot of our researchers. I really want them to have the space and the time and the resources to be generating that important research. When we start to ask them to do too many things, they're going to have less time to do that. So we need to really be thinking strategically about how they're doing that.

Then I think the governments -- what I would say in the government is we need the insights of both universities and the Brookings. The Brookings can translate what the researchers are doing as well. But the public sector really needs to be learning from the universities and the Brookings. So the translation -- we come back to the translation -- we have to make it accessible to the decision-makers because, as you pointed out, well they only have 24 hours in the day too. They have limited time.

[PAUL DECKER]          

Right. This is great. I've just got one more question to wrap up.

[CECILIA ROUSE]        

Okay.

[PAUL DECKER]          

This is a podcast where when I'm hosting, I often focus on leadership. You're a prominent leader in our field for many years now. Based on that experience, are there a couple of leadership lessons that you can share with us -- one or two leadership lessons based on your experience?

[CECILIA ROUSE]        

Look, one thing I've learned is you have to be your authentic self; so everybody leads differently.

[PAUL DECKER]          

Yeah.

[CECILIA ROUSE]        

I come from a university setting, so shared governance is very comfortable to me. I also -- through my different experiences, I'm comfortable listening to different perspectives and rolling it together, making a decision. So I tend to be rather consultative. I like to hear from different groups and different constituents because I think that people have to be heard in order to bring everybody along.

I was reading a book that Nan Cohain, who was the President of Duke and also Wellesley, wrote on leadership. She has a chapter which starts with an anecdote or a story which I will butcher, but it's great. It was from like the 1900s and the Admissions Office for the president at Wellesley went to the president and said, "President X, we don't know what to do with this one application. The letter of recommendation says that Sara is just not a leader, but she's a phenomenal follower." The president of Wellesley at that time said, "Admit her immediately because we've got a college full of leaders, and you can't be a leader if there are no followers."

And I don't take just as my team is followers. But you can see it. If a leader is just imposing top down and there's not buy-in and the community is not behind it, it will not endure. It will not be change that will stick. They will find workarounds. So I am very comfortable with the consultative, but you can't stay there. I've learned that through the White House as well. So you consult and decisions have to be made, but not everybody will be happy. But when people have been heard and you've explained why the decision was made, it's much easier for them to understand why a decision was made and to follow.

[PAUL DECKER]

So consulting yet decisive.

[CECILIA ROUSE]

Yeah, that's where I tend to be; and I like to talk to people.

[PAUL DECKER]

Appreciate that. This has been terrific, Cecilia.

[CECILIA ROUSE]

It's been fun.

[PAUL DECKER]

Thank you for joining me.

[CECILIA ROUSE]

It's been a pleasure.

[J.B. WOGAN]

Thanks again to our guest, Cecilia Rouse, and thanks to Paul Decker for stepping in as the guest host for this episode. This episode, by the way, was produced by my Mathematica colleague, Rich Clement. If you liked this episode, please consider leaving us a rating and review wherever you listen to podcasts. To catch future episodes of the show, subscribe at mathematica.org/ontheevidence.

New to Mathematica’s On the Evidence podcast? Subscribe for future episodes. If you like this episode, consider leaving us a rating and review on Apple Podcasts to help others discover our show.