As social scientists set out to investigate public policy questions, it is vital that they take into account the history of issues and the way people’s emotions will shape interpretations and responses to research findings. That’s the message that Mathematica’s Matt Stagner hopes to convey in his presidential address at a fall research conference hosted by the Association for Public Policy Analysis & Management (APPAM) in Denver.
On the Evidence interviewed Stagner about his upcoming address, his work on child welfare, and his reflections on the field of public policy research. This interview is part of a series of episodes produced by Mathematica in support of the APPAM conference in November. Follow Mathematica’s Facebook page for livestreamed interviews at APPAM, which will also appear as lightly edited episodes on Apple Podcasts, Stitcher, Spotify, Google Play, and other podcasting platforms.
Click here to listen to our full interview with Stagner. You can also read edited excerpts of the interview in the following transcript.
What is the topic of your address?
The title of my address is “Getting Closer: Embracing the Emotional Aspects of Our Craft to Help Policy Research Matter More.” I was thinking about the rationality we try to bring to policy research. The policy researchers and academics who are members of APPAM, weֹ’re all very excited about rigor. We equate rigor with distance from the thing that we’re studying. Yet, the people who are making public policy happen, making laws—but more importantly, doing things on the ground—are very emotionally invested in what they do. I want to challenge our APPAM audience to understand the emotion that people bring to interventions they operate and to think about how the work that we do, whether it’s evaluation or program improvement work, connects with that emotion.
What story are you going to tell?
I’m going to use a case study from my own experience and try to urge my audience think about how a [similar] story might play out in the areas where they work. My story will be from the child welfare arena, which is where much of my career has been. I’ll be talking about a set of services that people call family preservation services. Those are things that we do with families that we intervene with because of suspected child abuse or neglect. We try to keep the family together while we help them improve and prevent the child from being placed in out-of-home care or foster care.
For people who believe we take too many children out of their homes and place them in foster care, that’s a hugely emotional issue. People believe that the state should be very cautious in making that form of intervention and that it has costs that we maybe undervalue, such as traumatizing the child and family.
There’s also a lot of emotion around not removing children from potentially harmful situations. Child welfare agencies are beat up in the press almost daily. Somewhere in America, there’s a situation almost every week where a child is killed or severely injured, and the child welfare agency knew about the family, didn’t remove the child or returned the child back home, and something horrendous happened. The risk of not [intervening] is very high as well.
Evaluation efforts in this area, and impact evaluation in particular, are difficult as we try to bring our rational approaches of understanding the effects of programs into an emotionally fraught area. I’m going to tell a story about an evaluation from earlier in my career, focused on family preservation services, what the consequences of that evaluation were, and how we have a second chance on family preservation evaluation because of the Family First Prevention Services Act, which is bringing attention again to evidence-based policy around preventing the placement of children into foster care. And I’ll raise some ways that I think we might engage the emotional aspects of this more productively than we did in the past.
What’s one recommendation you have for APPAM audience attendees about how they can be more mindful of emotions in their research?
I’m still refining my take-home points, but I think the first one is to appreciate the history of an issue. As social scientists, we can come into the current situation, and we don’t think about the long history—decades, sometimes centuries—that exists around a policy issue. Understanding that, I think, puts us in a better position to understand the emotions about it: Where did this approach come from? Why does it look the way that it does? Why do people think it’s the right thing to do?
The second [take-home message] is one of closeness to the advocates for the issue or the people doing the work. We’ve often thought of ourselves as scientists. We’re social scientists. [Many of us seem to believe that] we need to have an arm’s-length distance from the things that we observe, and, if we get too close to these folks, we will somehow be distorted and influenced by them. But that closeness has an advantage, which is understanding what drives [people on the front lines], understanding what type of information they would use, how they might use it, and how we might best provide that. I’m not suggesting that we ourselves become overly emotionally invested [but that] we become close enough to those who are passionate about an issue that we understand them better and can adapt our craft to them.
You’ve been a member of APPAM, and the policy research community more broadly, since the 1980s and have watched it change over time. If you had to bet on how the field might change in the future, what is one thing that will be different years from now?
I would channel Mathematica’s President and CEO Paul Decker and say that the availability of data will change what policy research is and what APPAM is. Research won’t be so much the long-term, 10-year-study slog I’ve participated in but will be happening more in real time. The communications will be different, too. Nobody could have imagined when I first joined APPAM that there would be the Internet and Twitter and other new ways of getting research findings and thought-provoking ideas out to people.
I also hope that the diversity of the field continues to grow in every possible way—people from different scientific perspectives, people from different policy backgrounds, people with different racial and ethnic backgrounds. I’m hopeful that [the policy research community] will be an increasingly diverse and in some ways kind of messy place . . . to reflect the variety of things that policy research might be, in terms of the people and the methodologies and the disciplines reflected.
Want to hear more episodes of On the Evidence? Visit our podcast landing page or subscribe for future episodes on Apple Podcasts or SoundCloud.
