Improving Health Equity by Transforming Public Health Data Systems

[ALONZO PLOUGH]

What’s at stake is saving lives. Bad data, lack of timely data, lack of connected data, lives are lost. Lives are lost in cities where the early indicators of Covid happened in nursing homes – isolated populations, nursing homes and jails. But those data were not linked to other data systems, so you missed, in many cases, some of those early warning systems that should have put extra protections into those places. So, the lack of timely and appropriate distributed data around many different settings, the result is missing opportunities for early intervention that can save lives.

[J.B. WOGAN]

I’m J.B. Wogan from Mathematica and welcome back to On the Evidence.

Early in the COVID-19 pandemic, public health agencies in the United States struggled to respond to the disease’s inequitable impacts. One impediment to their response was inconsistent data collection and reporting, particularly for demographic and socioeconomic factors such as age, race, ethnicity, gender, income, and disability status. The same data issues would later hinder agencies’ ability to prioritize vaccinations for people most impacted by the pandemic.

As we record in late 2023, COVID-19 is no longer a global public health emergency, but these underlying problems with data? They remain.

In recognition of a need to modernize the nation’s public health data systems, the Robert Wood Johnson Foundation established a first-of-its-kind national commission “to reimagine how data are collected, shared, and used, and identify the investments needed to improve health equity.” That commission issued recommendations for federal, state, and local government, plus businesses, schools, nonprofits, health care systems, and foundations. For example, the commission called on health care systems to “collect social determinants of health data at every consumer encounter, using standardized questions and codes that allow data aggregation within communities.”

Or, to cite another example, the commission said state governments should “ensure that state policies for public health data collection, sharing, and analysis are equity driven and explicitly call out the influence and consequences of structural racism and other inequities on health.”

Following the commission’s recommendations, the foundation announced a $50 million investment in a range of projects across the country to accelerate progress in data systems and modernize public health data infrastructure.

Mathematica evaluated the initiative for the foundation.

On this two-part episode of Mathematica’s On the Evidence podcast, Alonzo Plough, George Hobor, Javier Robles, and Anita Chandra discuss the deficits of our public health data infrastructure, how these deficits affect health equity, and how public health agencies can respond better to future public health crises by transforming their data systems.

• Alonzo is the chief science officer and vice president of research, evaluation, and learning at the Robert Wood Johnson Foundation,
• George is a senior program officer at the Robert Wood Johnson Foundation,
• Javier is the director of the Center for Disability Sports, Health, and Wellness at Rutgers University and he was a member the foundation’s National Commission to Transform Public Health Data Systems, and
• Anita is a vice president and director of RAND Social and Economic Well-Being at the RAND Corporation.

Our guests spoke with On the Evidence while I was on parental leave, so I’m very grateful to my Mathematica colleagues Deric Joyner and Dave Roberts for filling in as interviewers. Deric is a senior communications specialist at Mathematica and Dave is the director of strategic communications at Mathematica.

In part 1, which you’re listening to right now, Deric speaks with Alonzo about the motivation behind Transforming Public Health Data Systems initiative, including why something so abstract as data systems matters so much for saving lives and avoiding serious illness.

In part 2, Dave moderates a conversation between George, Javier, and Anita about insights from the initiative and what changes need to happen next in the nation’s public health data systems.

I hope you enjoy Deric’s conversation with Alonzo.

[DERIC JOYNER]

The COVID-19 pandemic highlighted gaps in the gathering, analysis, and reporting of public health data specifically concerning race and ethnicity. So, the transforming public health data systems initiative had this ambitious goal of identifying the improvements needed to make sure public health data systems reflect the harms of racism and other forms of discrimination. With all that said, why was it important for the Robert Wood Johnson Foundation to create the initiative and take on this really big challenge?

[ALONZO PLOUGH]

Well, let me go back a little bit in history because I think the context of the decision to do this was very much set by that particular historical period.

So, there were two things that influenced us, and I think many people in the world, that happened around 2019-2020. One was the pandemic of COVID-19. But with the George Floyd murder later on, the – what I call the endemic of racism, which also had its influence on the differential outcomes in the pandemic, had us and other people realize even if we were doing health equity work, we had to dig deeper. And we had to – the way I like to put it, if you have bad information that you are monitoring to improve people’s health, if it’s not sufficiently granular by community, if it misclassifies people by race and ethnicity, if it is not asset oriented but only deficit oriented, you are going to end up making things worse. So we – we thought that to transform the information that we use, it has to be better information, more local, more granular, more community connected, more understandable, and more immediate, which is one of the lessons that we learned from Covid. You can’t have lags from the time it takes to get national data or state data disaggregated. All – all were lessons from Covid. All things, you know I was in public health practice for 30 years before this. All things we knew when we ran these big health departments, but I think this was a wakeup call for the field and we wanted to be kind of a catalyst in that.

[DERIC JOYNER]

You mentioned digging deeper, you mentioned transforming. Like this – that type of transformative change to public health data systems sounds difficult. Like how difficult is that kind of change?

[ALONZO PLOUGH]

It’s difficult and necessarily incremental, but you keep at it. The challenge to the data systems are that they are, you know, we call this public health systems, but think about it as public data systems. Or data – or information systems that are relevant to health, right? So, that includes formal public health governmental health at all levels, CDC, state, government. But it also includes other data that should be in the system that aren’t necessarily integrated. Like, oh, United Way’s 211 data. Social risk data that exist in many local contexts. And then, you know, I was a long time county health official with Los Angeles, Seattle, Boston. Sometimes you don’t have the right data you need from the other department in that mayor’s cabinet. I might not have the right data from the Social Service side. I might not have the right data from other parts of government that would help me understand how to manage a problem. So, data – the problem – the difficulty is data silos within organizations. A whole data system that is a little much on the measuring of problems that are manifest and not what to do upstream to keep them from happening. Those are just some of the challenges of what it means to transform into a community-oriented, prevention-oriented, asset-oriented approach to health – population health data.

[DERIC JOYNER]

It sounds to me like a holistic view of interoperability in every definition of that word, right?

[ALONZO PLOUGH]

Yeah, exactly. Exactly, Deric. It’s a holistic view of interoperability. It is kind of a holistic view of health being not just clinical outcomes, mortality, morbidity. But I keep talking about wellbeing, positive health, other dimensions of – of what we really want to see communities do under good times and try to get back to quickly in bad times which is wellbeing and thriving. Right. So, we worked with our national commission in the funding, and that funding is ongoing. We just – the Board just authorized another round of that funding. How can we be a catalyst to make these changes? People realize, at many sectors, many levels, that changes need to be made, and they are beginning to do that. We are just trying to be a catalyst and funder of innovation in that space.

[DERIC JOYNER]

So, you have alluded to some of this around like the wellbeing part. But what is really at stake when it comes to getting it right when it comes to public health data systems?

[ALONZO PLOUGH]

Well, I mean, just to put it most starkly, what’s – what’s at stake is saving lives. Bad data, lack of timely data, lack of connected data, interoperable data as you put it, lives are lost. Lives are lost in cities where the early indicators of Covid happened in nursing homes – isolated populations, nursing homes and jails. But those data were not linked to other data systems, so you missed, in many cases, some of those early warning systems that should have put extra protections into those places. So, the lack of timely and appropriate distributed data around many different settings, the result is missing – I’ll put it more positively – missing opportunities for early intervention that – that can save lives. And real understanding about what to do, like in the context of Covid.

[DERIC JOYNER]

You know, it’s interesting, you know, you hear a term like public health data system, and it doesn’t sound as impactful as like the life and death decision that you just described, but you’ve just described it so clearly. Like this isn’t just about numbers on the screen or on a page somewhere. This is about the information we need to really be able to make important, impactful, like knowledgeable decisions about how we take care of each other.

[ALONZO PLOUGH]

Yeah, exactly, Deric. That’s why we’ve been talking about it more like just usable information to help us both make the kind of changes we need in the health and healthcare systems and really actionable data that is better suited to addressing the fast-moving problems we have, like Covid, or the slow-moving problems we have, like fentanyl and increased drug deaths. We just are not having the information system that connects enough sectors that are – really lead to the kind of actions we have to do. I was in, you know, public health a long time. We have – we have like fast disasters. You know, we have high-profile disasters like Covid. And then we have chronic disasters like substance – like the disproportionate impact of substance abuse disorder, teenagers dying of fentanyl. We have no system to understand one, how that is happening except mortality reports, and what you do about it in terms of a prevention thing. So, we have another project – we have part of this project is working with the National Academy of Medicine around a different framing of the data and the information needed to really do a much more preventive work with substance use disorder. That’s another example of changing it. But you’re right. Data is not like a exciting term. That is why epidemiologists don’t title movies, you know, we – we – we don’t’ really have the best entry points for the – for the excitement and importance of what we do.

[DERIC JOYNER]

Well, this is why we are having these conversations so that the important work that you are doing can be lifted up and shared out with as many people as possible because think this is really important stuff to know.

So, you mentioned your long career in public health. You’ve mentioned like the work the Foundation has already done when it comes to public health and public health data specifically. So, how did the Foundation’s work specifically, you know, in their previous efforts, inform what happened with the initiative and how it works?

[ALONZO PLOUGH]

Okay. I’m going to go back a little bit more to when the Foundation really began to try to reconceptualize what we thought were the data for health and how to construct health. That goes back to about 2013-2014.

The Foundation played a major role in developing the evidence and some of the data and measures around what social determinants of health actually means. So, I think our story started with the work of a previous national commission on – on health disparities and beginning to do the work and norm the messaging around that our health and wellbeing, though medical care is important, that’s not the driver. And the social determinants of health, where you live, things you eat, the quality of your schools, all of those things, are the big drivers.

I think what has happened in the ten or 12 years since that has been just a deepening of what kind of data we need, what kind of transformed data systems we need, to understand how to address these social determinants, how to address the higher probability that the negative impacts of these social determinants happen in marginalized communities of color. You know, we’ve being do that – on that road for a long time. You may have seen the maps that we do, we fund, that shows life expectancy differences in urban cities where, and you know, it happens in any city, I’ll do LA since – if you drive from the airport LA, and you begin to drive east and south, you are losing – the communities that you are driving through are losing about two or three years of life expectancy at birth by the time you get from the west side in LA where to airport is to where you are over south LA, Compton, Watts, you know, you’ve lost 20, 25 years of life expectancy.

You know, those are the kind of data that you need and – and the understanding from those data. And then digging deeply into that data to understand what is going on. And that’s, going back to what I said earlier, that more granular data that say it’s kind of pretty easy and unfortunate to say here’s where the excess mortality is. But what – how do you get deeper? Much of what this project was about, the – is data that lets you get deeper. I call it more granular, more immediate. And then more understandable for policy change and – and understanding how to improve those conditions.

[DERIC JOYNER]

So, that – that deeper look into this sounds like the – the differentiator between like the initiative’s work and your prior efforts. So, I’m wondering, like how was the initiative positioned to have an impact with that different approach and really see the changes that are needed in the system?

[ALONZO PLOUGH]

Exactly. We’re doing what I think philanthropy does. We’re trying to set up partnerships and demonstration programs to show what can be done. And what came out of the first phase of transforming public health data, I’ll just mention a couple of – couple of projects which I think lands the changes we wanted to make.

One of them is work in collaboration with a group of historically black colleges in the Gulf area who have been doing some community-based work on environmental justice and climate justice. And – but those communities they’ve been working with just did not have the kinds of data that would drive action around the inequity of climate change on people’s health in all aspects, heat, inclement weather, all of that. So, that project is building partnerships in every Gulf state between the academic researchers, affected communities, kind of planning community-driven activity to bring this new generation of data that talks about climate change. Right, I’ve been in health departments for a long time. We would kind of look at heat when – in the summer when there were heat-related deaths that you could actually monitor. But we didn’t have data systems that looked at particulate matter in the air, or what is happening with sea level, or all the kinds of things you look at with climate change. So, that project is going to put together a whole new kind of data related to climate change and environmental justice and health with that community-based coalition or partners.

We’re doing a similar thing at five different places around the country to kind of transform what these local public health departments and their partners look at with the de Beaumont Foundation. And that is really what does it mean to explicitly put together data that helps you understand, in kind of a granular way, a data-driven way, the specific impacts of racism on poor health outcomes. And what – and then what you can do about it.

That’s what that project does. And those – those – those projects took a time to launch because they had – we had to find communities that were up for doing that work, and it wasn’t their first time doing this kind of community collaborative work. So, that’s another example of what we are doing out of this project.

A third thing I’ll mention is just – because it is more at a federal level – working with the Office of Management and Budget and the federal government to really refine the classifications of race and ethnicity so in the future we don’t lump groups together and call them things like Asian and Pacific Islanders when you are talking about 50 distinct groups, multiple languages, and a whole lot of different nuances in their particular health trajectories, right?

So, I don’t what to run on too much. I’m very excited about this project, I can talk all the – I can just kind of keep going, but, you know, those are three things that I think talk about getting it – I like those because it is getting it on the ground or serious engagement with federal officials who can make decisions to do their data categorizations better. And we’re engaged with that with them.

[DERIC JOYNER]

So, kind of looking back at the work at the initiative and looking forward, like what are the most important insights, I think, for philanthropy at large that they should apply to their efforts to address inequities in public health data systems?

[ALONZO PLOUGH]

Well, it’s – it’s a big problem, and it’s – it’s good that a number of foundations are working on this. You know, we are partnered with the de Beaumont Foundation, we’re doing a lot of this work on climate and health justice with the Rockefeller Foundation, Kresge Foundation. So there’s a lot of – of – of philanthropic interest in this one.

But these are big problems. Our – if we have a bias, it’s that some of these problems are going to be best addressed in a location, in a region, in a city. I’ll go back to what you said earlier. The impetus to kind of deal with interdependency and interoperability is kind of abstract at a national level. There’s too many hospital systems, too many people creating data. But once you get down to a particular area, right, a particular city, a particular – let’s just say city or county, you – all of the people who are producing those data that need to be connected are in the same place, have a stake in that community, and it is not kind of an abstract thing and you can begin to – you can begin to do the negotiating, the data agreements, and the data sharing agreements. We have projects funded by this project that have done that as well to strengthen those more local data sharing networks so you can get a fuller picture of – of the prevention and risk possibilities in an area.

So, that’s kind of our bias, and we think working – foundations working in those local contexts, national foundations like us working with local community foundations are interested in the same thing. It’s easy to do when the partners who need to determine what is important information, and how to share it, and to build the trust that it takes to share data, that happens locally in kind of our opinion.

[DERIC JOYNER]

Yeah. So, speaking of the community angle, what do communities bring to these efforts that wouldn’t be there if they were not involved?

[ALONZO PLOUGH]

Well, communities bring the experience of being the unfortunate recipients and their lives and their life trajectories of these problems we are trying to solve on – in terms of inequitable health outcomes, inequitable experiences with Covid or any quality in health outcomes in many other areas. So, they bring – they bring texture, they bring stories, they bring the reality beneath that population statistics can’t capture. You know. How is racism happening at the interface of health – of seeking care in a clinic where the cultural competency in the process is insufficient and therefore the care is not going to be what it should be? That can come out in narration and stories, it’s just not going to come out necessarily in the kind of statistics. So, what communities bring to the table is a understanding of what I would call the root causes of these problems are in community contexts. I’ll speak to our work also, some of this is funded in a way out of the same authorization. Our community part – we might – epidemiologists and researchers say there is – and there is – a huge problem with black infant mortality in the United States. We call it black infant mortality or black – better example – black maternal mortality. Two big problems. So we, working on the latter, black maternal mortality, that is kind of an epidemiologic construct, that’s a statistical construct. When you talk to community and the way the problem is framed, it is framed as birth justice. All right. The framing of birth justice says that we should be looking at a variety of other things than just clinical indicators. What do they mean by birth justice? Then it gets into the issues of I didn’t – I didn’t go – I tried to go to a care provider early in the trimester but – but, you know, I couldn’t – they didn’t make appointments, the intake was – they asked me questions I didn’t understand, there was no one that, you know, those kinds of things come out in narrative. And the challenge of the new information systems is how do you incorporate those? How do you incorporate narrative information, the historical information that leads to the current situation? The systems we are trying to build and transform are able to weave those things together.

[DERIC JOYNER]

Okay. And is there like one – you mentioned stories. Is there one story that comes to mind when you think about like a community being involved in an effort and the story that they told? Like shifting your understanding or the understanding of like what the initiative was doing in its approach?

[ALONZO PLOUGH]

Well, I think there – I think there are lots of stories. I mean, I think there are stories – one of our Commissioners, Abigail Echo-Hawk who is an indigenous epidemiologist working in Seattle, told many stories in our – in informing this work around just the inability for them to get information from the federal government in real time about what was happening to Native American populations in the greater Seattle area. Abigail’s stories and the stories from her community about the – all of what we were missing in addressing that, those are important stories about what needs to be connected and what needs to be – to be done. So, I mean, that would be one kind of story that came to us through the Commission discussions as well as deeper work that we are doing and funding for Abigail in Seattle.

[DERIC JOYNER]

So, that’s great. Well, Dr. Plough, thank you so much for sharing your insights and the stories you have told about the work that the initiative – it’s great work and looking forward to seeing what continues to come out of the work of the initiative.

[ALONZO PLOUGH]

We just started – we’re starting the second phase of it now, and I look – we’re evaluating all of that, as you know, because you are the evaluation partner. So, we – we will together be learning what we’ve been able to accomplish with this project.

[DERIC JOYNER]

Absolutely. Looking forward to it. Thank you so much.

[ALONZO PLOUGH]

Great. Thank you.

[J.B. WOGAN]

Thanks to our guest for this episode, Alonzo Plough, and thanks to my Mathematica colleague, Deric Joyner, for conducting the interview. This is the first in a two-part episode about improving health equity by transforming our public health data systems, with a focus on lessons from the Robert Wood Johnson Foundation’s Transforming Public Health Data Systems initiative. The second part, which is available now, features a roundtable conversation about the initiative and its implications, with guests George Hobor of the Robert Wood Johnson Foundation, Javier Robles of Rutgers University, and Anita Chandra of the RAND Corporation.

This episode was produced by the inimitable Rick Stoddard. If you like the show, please consider subscribing. We’re on YouTube, Apple Podcasts, Goodpods, Spotify, as well as other podcasting platforms. To learn more about the show, visit us at Mathematica.org/OnTheEvidence.

[ANITA CHANDRA]

Another aspect of modernization was being honest about the complexities of the health issues that we do have and the intersecting nature of those health issues. So it wasn’t just going to do to add a few measures on chronic disease or work around the edges anymore because the solutions to addressing those complex health issues were going to be structural; they were going to be systemic; they were going to sit in social determinates of health; they were going to implicate more sectors beyond the health sector. So an incremental change to the public health data system that didn’t address those intersections was just simply going to be not just subpar but was going to miss the mark completely.

[J.B. WOGAN]

I’m J.B. Wogan from Mathematica and welcome back to On the Evidence.

Early in the COVID-19 pandemic, public health agencies in the United States struggled to respond to the disease’s inequitable impacts. One impediment to their response was inconsistent data collection and reporting, particularly for demographic and socioeconomic factors such as age, race, ethnicity, gender, income, and disability status. The same data issues would later hinder agencies’ ability to prioritize vaccinations for people most impacted by the pandemic.

As we record in late 2023, COVID-19 is no longer a global public health emergency, but these underlying problems with data? They remain.

In recognition of a need to modernize the nation’s public health data systems, the Robert Wood Johnson Foundation established a first-of-its-kind national commission “to reimagine how data are collected, shared, and used, and identify the investments needed to improve health equity.” That commission issued recommendations for federal, state, and local government, plus businesses, schools, nonprofits, health care systems, and foundations. For example, the commission called on health care systems to “collect social determinants of health data at every consumer encounter, using standardized questions and codes that allow data aggregation within communities.”

Or, to cite another example, the commission said state governments should “ensure that state policies for public health data collection, sharing, and analysis are equity driven and explicitly call out the influence and consequences of structural racism and other inequities on health.”

Following the commission’s recommendations, the foundation announced a $50 million investment in a range of projects across the country to accelerate progress in data systems and modernize public

health data infrastructure. Mathematica evaluated the initiative for the foundation.

On this two-part episode of Mathematica’s On the Evidence podcast, Alonzo Plough, George Hobor, Javier Robles, and Anita Chandra discuss the deficits of our public health data infrastructure, how these deficits affect health equity, and how public health agencies can respond better to future public health crises by transforming their data systems.

 

• Alonzo is the chief science officer and vice president of research, evaluation, and learning at the Robert Wood Johnson Foundation • George is a senior program officer at the Robert Wood Johnson Foundation • Javier is a professor at Rutgers University where he is the director of the Center for Disability Sports, Health, and Wellness and he was a member the foundation’s National Commission to Transform Public Health Data Systems. • And Anita is a vice president and director of RAND Social and Economic Well-Being at the RAND Corporation.

Our guests spoke with On the Evidence while I was on parental leave, so I’m very grateful to my colleagues Deric Joyner and Dave Roberts for filling in as interviewers. Deric is a senior communications specialist at Mathematica and Dave is the director of strategic communications at Mathematica.

In part 1, Deric speaks with Alonzo about the motivation behind the Transforming Public Health Data Systems initiative. In part 2, which you’re listening to right now, Dave moderates a conversation between George, Javier, and Anita about insights from the initiative and what changes need to happen next in the nation’s public health data systems.

To start the conversation, Dave asked George Hobor of the Robert Wood Johnson Foundation why it was so important for the foundation during and after the pandemic to tackle problems with public health data infrastructure. We’ll start with George’s response.

[GEORGE HOBOR]

So for us, RWJF is the largest foundation in the nation with a focus on health. We also are really focusing on health equity and advancing health equity. So what we saw with COVID was many challenges obtaining the data that would help advance that goal of achieving health equity. There was a lot of problems around not getting disaggregated data. There was a lot of problems around getting data that would illuminate the contextual factors that were showing some of the differences we were seeing in outcomes, and there seemed to be a lot of other entities filling in the space of providing information that weren’t necessarily traditional. So it was clear there was a problem with the information around COVID 19; and that’s why we thought, given who we were, it was the right time to act.

[J.B. WOGAN]

You’ll now hear from Javier Robles from Rutgers University.

[JAVIER ROBLES]

Yeah, I was going to say with the organizations that I am working with, including the organization at Rutgers University, I think it was a similar issue. We just were not seeing any kind of data in terms of people with disabilities -- how many people with disabilities needed services, how many of them did not have things like personal care assistance services, or access to food – just regular things that everybody needs every day. When we asked Government, they had no idea. So obviously, COVID laid bare a lot of the things that, to some extent, a lot of us already knew. We don’t have enough information about the groups that need services in the places we live, work, and play.

[J.B. WOGAN]

You’ll now hear from Anita Chandra from the RAND Corporation.

[ANITA CHANDRA]

I’d just add I think COVID lit a spark on something that’s going to be even more challenging for us going forward in terms of our ability to make as best data-informed decisions as we can in a quicker fashion, given the nature of these kinds of urgencies like pandemics but also the kind of acute issues that we’re facing as a result of climate change. So we’re going to be in these cycles of disaster response in low-grade or high-grade for the foreseeable future. So in addition to addressing longstanding inequities in our data system to address health equity issues, we’re also going to be in a more acute phase where we’re going to need to make decisions quicker. COVID merely highlighted that.

So our ability to marshal resources, get investments to the people most in need, is only going to grow. So the urgency was underscored by COVID; but the Commission, I think, was responding to a broader urgency as well.

[DAVID ROBERTS]

George, when you look back at the start of this project, what made it unique in terms of how philanthropy approaches addressing health inequities?

[GEORGE HOBOR]

Well, I think the approach we took was pretty unique to this project. We’re making a big commitment fast, and I think a lot of philanthropies were doing that at the time. But the approach we took was really to see if we could establish a national conversation, and that was the purpose of the National Commission. So we pulled together a commission of national experts to debate some of the challenges that we were seeing and to come up with a set of recommendations that would set the stage for how we could work toward an ideal public health data system.

So I do think that point of establishing the Commission, getting feedback, and getting to some careful, well-thought-out recommendations that would guide a strategy moving forward – I thought that was pretty unique.

[DAVID ROBERTS]

Javier, what was it about the ability to engage in the data itself and around the narrative, where the narrative meets the numbers with this particular project, that stood out to you?

[JAVIER ROBLES]

Thank you. I think it’s important to note how much people just see numbers and just like their eyes glaze over. One of the things that at the Commission while we were working on this report that we just – everyone agreed on is that we have to make the information, the numbers, more real. We have to bring them to life in terms of making sure that regardless of which population we’re talking about, that the average person can pick up the report or pick up the data and say, “Oh, they’re talking about my grandmother,” “They’re talking about my sister or my brother or someone with a disability or an underserved community member."

So I think that that’s going to assist definitely in the future with – God forbid – other pandemics or other natural disasters or emergencies with ensuring that people get information in a digestible manner that they can understand at whatever level they are at. I think for too long we have thought of data as just an academic thing – that academics should have access to and researchers should have access to, but not your average person on the street.

[DAVID ROBERTS]

Building off of that, how important was it that people saw themselves in this work and in the data?

[JAVIER ROBLES]

I think it was hugely important. One thing that I know personally for me is that once you go through COVID, which I’m sure many people never expected that they would, and you turn on the television and you see all these people dying – a million-plus at this point in the United States – and you see real pictures, a lot of us wanted to know, “What aren’t we seeing?” I think that’s the question that we should all ask ourselves.

We see all these people dying, but what aren’t we seeing? What are the numbers? How many people with disabilities? How many people who are Latino or women or African American? What other diseases or disabilities were they dealing with? And more importantly, how can we address those once we have the information because, of course, gathering the data is only one portion; but what are we going to do with this data, and who’s going to be at the table to assist us with that?

I think another thing that the Commission really pointed out is that all these groups, all these people who we’re talking about, need to be at the table when decisions are made about them – which for too long has been an issue with many communities.

[DAVID ROBERTS]

So when we’re talking about transformational change, what does that mean in terms of the public health data system; and why was it important to take such a big swing instead of looking for smaller wins when it came to this work?

[GEORGE HOBOR]

I kind of want to answer this piggybacking off of the question to Javier around why it was important for people to see themselves in the data. One of the things we haven't said is that this work was also happening at a time where there was active efforts to undermine information and data and sow seeds of distrust in what people were hearing and reading. So getting to a point where you could have the data where people saw themselves in it and a part of it was critical for just having trust in institutions in general.

So why transformation?

I think this segues as to why you need a transformational change -- is clearly doing things the old way with that backdrop of active efforts to create distrust in data and information, that wasn’t going to work anymore. So that was an added – kind of an added problem to the fact that we had inadequate data to begin with that kind of allowed us to not see the inequities that were playing out with COVID, not being able to effectively respond. So all of that was coming together in a way that there needed to be significant change that altered the paradigm of how we were doing data work in general.

[ANITA CHANDRA]

I’d just add one of the through lines for the Commission and the work leading up to it was this idea of modernization. One of the dimensions of modernization is actually what George and Javier noted, which is thinking differently about how we center people’s experiences and how we link data to information and information that’s usable, that insightful, that’s impactful. But another aspect of modernization was being honest about the complexities of the health issues that we do have and the intersecting nature of those health issues.

So it wasn’t just going to do to add a few measures on chronic disease or work around the edges anymore because the solutions to addressing those complex health issues were going to be structural; they were going to be systemic; they were going to sit in social determinates of health; they were going to implicate more sectors beyond the health sector. So an incremental change to the public health data system that didn’t address those intersections was just simply going to be not just subpar but was going to miss the mark completely.

So you needed to be very realistic about the overhaul that was required in order to meet kind of the modern challenges in our nation. For too long we had kind of stumbled with public health 1.0, 2.0, 3.0 kind of thinking – which is important but still not meeting the call really in fullest terms. So there was both an urgency and an understanding of the need for something comprehensive that hit all of those points at the same time rather than trying to do this sequentially – which is what we kind of historically have been doing in piecemeal data modernization efforts.

So that was really a bold vision that the Commissioners had, that the Foundation had, which I think differentiated this effort from prior initiatives.

[JAVIER ROBLES]

To follow up on that, I don’t see up after this point how we can do it any other way. One of the things we’re now talking about is the backdrop of the Black Lives Matter movement along with COVID and all these things happening at the same time. We need to address these broad issues of structural systemic racism, ablism, sexism, and every other ism the way we haven't done it before; and that is at a broad level with the assistance of the Robert Wood Johnson Foundation and other foundations because that’s the only way to move forward.

I would venture to say that Washington is still broken in terms of fixing these broad policy issues. One of the things that we talked about at the Commission at that point was this may initially have to be a state-by-state/foundation-by-foundation effort just to move some of these issues along. But in the meantime, none of these isms have gone away. People with disabilities are still suffering the effects of COVID to some extent, and it wasn’t until just – I want to say like two weeks ago that they got designated a health disparity after a long time. This, to some extent, will make a difference; but it’s only a first step in terms of a lot of these things we’re talking about.

[DAVID ROBERTS]

Engaging different communities and making sure that their voices were a part of the conversation was a big focus of the work of the Commission. What benefits do you think organizing the work in that way had?

[JAVIER ROBLES]

I’ll start that one. So in the disability community, we have a saying that really should go for every community; and it’s, “Nothing about us without us,” right? I think every community that was represented there to some extent had that experience where decisions are made for them and about them all the time, and they have to deal with the policy issues. They have to deal with the economic issues of those decisions that are usually made by men somewhere who are saying, “Well, this is what we think is best for you.”

I think the Commission’s changed that by saying, “Well, why don’t we bring all the people to the table and ask them what’s best for them.” If you read the report, you’ll see that each individual panel had their own ideas about what was best for them. I think that’s how we should move forward to make this country better – to really ask people, “What is going to work for you?”

Again, Robert Wood Johnson Foundation does a great job at this. When we’re talking about equity, equity is all about what works for you – not what works for everybody.

[GEORGE HOBOR]

I would echo the things Javier said. I think additionally why involve the community, this was in the Mathematica evaluation actually, where there was a bit of it that said, “Hey, if you’re really looking for sources of innovation, a lot of your traditional partners are not where it’s at.” So it was a direct call out to philanthropy anyways to say, “Yes, you include community organizations,” for all the equity concerns that Javier pointed out. But in addition to that, these are -- community-based organizations are sources of new ideas. They’re closest to the problem.

So you have to start to welcome that thinking into the way a large philanthropy like RWJF is operating to really meet the goals you have and be effective. So that point on innovation was just another reason, in addition to, I think, the primary one that Javier pointed out of why you need to open up and be a little more democratic with data work – not a little, but completely democratic with data work.

[ANITA CHANDRA]

I would just add -- George and Javier, those were great points – two points. One is a lot of the data actually also resides in these places and spaces. So just practically, if we want to get an understanding of community experience, community actions, what’s really happening that’s driving some of these health inequities or the promotion of positive health, capturing information to the best we can from a broader range of sectors and community leaders and underrepresented populations is essential to actually having an objective “picture” of what’s going on that’s coming from multiple sectors and sources.

So we’ve only really uncovered half of the story at best in terms of our public health activities because we’ve not captured the insight and the information and the expertise and the data from those communities and those populations. Now, that means also thinking about data differently. It’s not the traditional formalized way that sometimes we think about data and broadening and widening that aperture of what is included – whether that’s a mix of qualitative and quantitative data, whether that’s data that’s coming from lived experience or understanding of tradition and ancestry. Capturing that as robust data as well is possible, but we don’t do that.

The second thing that was part of the run up to the Commission work, that informed the Commission work, were a set of expert panels that the Foundation convened, intentionally making sure that the voice of particularly historically underrepresented populations was centered. So it wasn’t a tack-on, but it was you have to be purposeful about capturing and leaving space for insights from populations that are from racial/ethnic minority backgrounds or populations that represent the disability community or others. That was baked in as opposed to now we’ve done this structure and let’s just add in a few perspectives on top of it.

That’s, unfortunately, often how we have pursued research and committees and commissions like this in the past; and we flipped the script a bit, if you will, in terms of how knowledge was being generated to even inform the Commission deliberations, much less how we value data from multiple places.

[JAVIER ROBLES]

If I could just say one quick thing about the research -- and this was brought up, I guess, by a few panels including us – is the fact that the researchers themselves that currently exist don’t look like a lot of the population that they’re studying. We don’t see that many researchers with disabilities. You don’t see that many researchers who are African American or LGBTQ or from any of these other populations. That’s problematic because they’re making up the questions; and the questions really lead to the dollars; and it leads to the answers, and it leads to what communities eventually get.

I think that is something else that we often don’t talk about but we really need to begin addressing. It’s not just the research and involving the community, but also making sure that we have enough researchers that represent everybody.

[DAVID ROBERTS]

So one of the main takeaways from a lot of the panels and the Commission was not just the importance of having community voice involved in identifying the problems but, as I said, being part of identifying the solutions as well and the work moving forward. Why do you think that was such a resonant and recurring theme across all the different populations involved?

[JAVIER ROBLES]

Because they have the lived experience. They’re the ones who can tell us the best why these solutions are going to work. A solution that’s going to work in north New Jersey might not work in some other place in California or Arkansas because things are different in those places. They’re dealing with different state governments, different laws, different cultural practices. Just because we’re in America doesn't mean that every state doesn't necessarily have their own cultural practices and different ideas about things like race and gender.

I think we should go into these places, like the panels recommended, and say, “What are your solutions?” Because again, too often – I’ve said this, and George said this and Anita said this as well – we just go in giving them the answers without asking them any questions. So I think that’s my (inaudible).

[ANITA CHANDRA]

I’d just add to Javier’s great points. We had this figure in some of the materials that informed the Commission, the sort of idea that data by itself is nothing, right? Data has to connect to a process that’s turning that into information, and then that information has to be turned into insight. Those processes have to be contextualized locally, whether by community or population or experience, to Javier’s point. So there’s no engine – there’s no sort of translation machine, if you will, if there is not community-centeredness in the design.

All those data, all those indicators, can kind of sit in the public health department or sit in the mayor’s office or sit in CDC; but it doesn't have any resonance until it actually hits ground. It’s in that connection between sort of data and community policies, expertise, experience, and history that it turns into something actually usable. Too often, we put those pieces very separate from each other.

There’s a data infrastructure that’s generating statistics and morbidity and mortality indicators, and then there’s a very long road to how it might get used or taken up or applied. That machine has to change going forward; and that’s, I think, a fundamental reason why the Commission focused on not just measures but also the systems in which these data sit, the narrative, the conversation, in which these data are translated – that there was a systemic way of thinking about modernization as opposed to just modernize the kind of data that you pull in. That’s really important going forward as we were talking about transformationalism versus incrementalism.

[GEORGE HOBOR]

I think the only thing I would add to what others have said on this – so why involve community in the solutions -- is historically they have not been involved; and for historically-marginalized populations, that hasn’t necessarily worked out very well. There are many instances in which that has worked out, well, bad.

But I also want to point out that in saying that we want to involve communities in the solution, I want to connect this back to the data work because I don’t want -- there’s a whole set of data-backed, evidence-backed interventions that supposedly were to improve X, Y, Z measures. We still want to do that work; but we want to recognize that, hey, the data that went into those, the design, all of that was done in a particular way that did not include community knowledge and different perspectives. So we have to work on both sides to refine the information that’s coming in and also how that’s used to create a solution in coming out. So the communities that are most affected have to be central to that process.

I just wanted to raise that. Not to say we’re going to dismiss that whole evidence-backed intervention field, no, we just want to change it.

[ANITA CHANDRA]

Yeah, and, George, I would just add to that. I think we’re having a good conversation in measurement and public health and research and practice that reimagining what is robustness and research doesn't mean reducing the quality. In fact, the argument is actually the contrary. So right now, I think it’s important for us to talk about this as important and critical, central to research and data and evidence as opposed to saying it’s this extra thing – and certainly, that’s not what you’re saying – this extra thing that we tack on. I think we have a real opportunity going forward to say, “No, this is rigorous and this is robust. Blending these voices and perspectives is actually the best of science.”

If you look at so many of the disciplines that inform health and public health specifically – sociology, anthropology, economics – they’re all taking different kinds of perspectives. So we’ve got a real opportunity to reimagine that sort of data conversation in a way that still holds up all the robustness of the scientific methods. So it’s really exciting.

The Commission, I think, really created some foundational space for that larger conversation so that people in academia or in the academies or all of the kinds of traditional research start to get disrupted a bit in terms of how they’re approaching data and measurement work going forward.

[JAVIER ROBLES]

Just to go off that, I think one of the best ways to do that is to make sure that our researchers also represent the people that we are researching because they’re going to ask questions that maybe we’re not asking and we haven't asked in the past. Those questions are going to be questions about culture; they’re going to be questions about language and disability and other issues that they may know a little bit more about than the researchers often do.

[DAVID ROBERTS]

What were some of the other themes or recommendations that came out of the Commission that resonated with you the most?

[GEORGE HOBOR]

Well for me, I know a set of recommendations that the Foundation are taking seriously were – there were about three of them that were connected to measurement. So one in particular is that we just don’t have measures for the things that we want to have measures for to advance health inequities – or health equity, excuse me, reduce health equities. So that in itself, we’re doubling down there and being like, okay, there’s quite a bit of measurement work that has to be done and data collection to fill the gaps. So that was one of the outputs.

Of course another response to that too is stuff we’ve been talking about – about how to center community knowledge in the creation of those measures. That was also a set of – that was also within the recommendations, and I think that’s where our efforts are focusing pretty squarely right now.

[ANITA CHANDRA]

I don't know, Javier, if you wanted to go; but I’ll just jump in and underscore those points from George. Just to put a little bit of a finer point on that, I think there are real areas where we don’t have measures. As we’ve been talking to health departments even since the Commission put out its recommendations, they want to do this; but they’re struggling. So areas of capturing positive health and community well-being in a more health-promoting kind of framework as opposed to always just tracking diseases and deficits. The ability to go upstream to capture drivers of health inequities – so systemic or structural measures that drive health inequities so we’re not so lagging when we’re measuring health disparities downstream. We’re actually looking at the source of health inequities. So I thought that was a pretty robust finding and a recommendation of the Commission.

I’d also say that recommendations about other sectors that are outside of traditional health can be part of the solution here and see themselves in what constitutes public health data. So some of that is what people might talk about – like housing authorities, understanding their role in providing data on the health of the community, or public safety and police – but also the role of the private sector. We’re still working on that. But the critical role of technology given how quickly data are moving right now – we’re constantly bombarded with data. Now, the question is, is it usable information for health? But we need more sectors to kind of put skin in the game in terms of the health and the well-being of the country going forward. A lot of those organizations are holding a lot of data that would be really valuable for some of our health work going forward.

[JAVIER ROBLES]

Yes, I didn’t mean – and I think one of the key points was to make sure that we put equity at the center of all of the collection of data and really look at how we’d be collecting data from the eye of making sure that we are not just being fair to the communities of research but actually looking at what some of the problem issues are. If it’s police brutality or living in a place that has no supermarkets or things like that. Really, I guess, (inaudible) most of the data that we get from all these places and really look at what is the bottom and why is this happening, and why is this happening in this particular ZIP Code and not happening two blocks away - things like that that give us real information as to some of the solutions.

Then I think having this data where we really go down deep and ask these questions is going to assist us in looking at structural racism and looking at these issues that are most systemic and longstanding. Unless we begin to address them in a systemic way, we’re not going to actually solve anything. But of course with that said, it’s not going to be easy because we’re really going to have to look more closely than we’ve been looking as opposed to saying, “Forty percent of the population is hungry.” Well, where are they hungry? Are they hungry in a certain city, or are they just hungry just generally speaking?

[DAVID ROBERTS]

George, the Foundation made a conscious choice to fund the Commission, to give them voice in what kind of investments were needed to turn ideas into action and to commit resources to evaluation throughout the entire process. Why was this so important, and what were you hoping to learn through the lens of evaluation?

[GEORGE HOBOR]

So your question is why it was important to have the Commission?

So that -- I think having a national commission is not a tool in a philanthropy toolbox we pull out often; but I think we’ve had a few commissions. Really, the purpose there is to, like I said earlier, establish the national conversation. Like how do you get legitimate voices that bring the credibility to issue and have the credibility to have a long and impressive communications campaign that sets the conversation for a while.

So then you asked about the evaluation, and what we’re looking for out of that. We put together that Commission and a lot of support into grantmaking, including developing information to inform the Commission in such a short amount of time with a lot of responses. So to do that, I think we wanted an evaluation to look at how it was implemented and say, “What could we learn in philanthropy if we operated in a different way to be flexible and to be fast, but also to be responsible and hopefully effective?” That effectiveness is the second part of the reason why we would want an evaluation.

In setting up a national commission, that’s one approach we could have taken. We did it quickly. We weren’t quite sure if that was the most effective approach. So getting an assessment of that, not just from our point of view but from the folks that were involved and even externally who might have just been viewing it from afar. Understanding if that approach was a good one was another thing we wanted the evaluation for. Then, my goodness, we made a lot of grants coming off that to tie to the Commission’s recommendations – so a lot of different data, a lot of different types of data.

So we’ve left this term, “public data system,” really broad. So in a way, the evaluation we were hoping would look at some of the grants we made and kind of point out where we actually had made some inroads, where we might have overlooked – give a little more detail to how we’re defining “public health data system,” not just to help our strategy with the Foundation moving forward, but to also externally communicate that to the field – that this is what we see as the key components when we say, “public health data system.” So those were what we were really looking for in evaluation.

[DAVID ROBERTS]

Do others have anything to add there about the impact or influence of having an evaluation and what it meant to the overall process?

[ANITA CHANDRA]

Well, I think it’s always terrific when you have kind of an evaluation that because it was moving so quickly, as George rightly notes, in terms of everything that was happening and all the innovations, what the Foundation moved to these quick grants, and then investment afterwards. There was a strong communication strategy. There was interest to get to multiple sectors with the information. There were many sort of prongs of dissemination and engagement. There were efforts to bring along the Commissioners to represent diverse backgrounds pretty quickly, which was no easy feat, and to have some people who haven't been part of traditional health-related commissions.

So evaluating how those pieces come together and whether those elements can work in concert was really helpful to see. I think on balance, from my understanding and reading of the evaluation, things worked pretty darn well in terms of meeting the core objectives of the Commission and the vision that the Foundation had. So that, I think, is terrific.

It also underscores kind of the infrastructure that’s needed if the Foundation wants to ramp up again – or any philanthropy wants to ramp up again. What could you keep in place? Not that you’re going to do a commission every year, but what could you kind of architect in a philanthropy in an organization such that the ramp up wouldn't be that steep.

So I think the evaluation provides some good nuggets on how you might think about that because what you don’t want is to build an apparatus and completely dismantle the apparatus such that you’ve got to start all over again if they decide to do something big like this again. So I think the evaluation was helpful in pointing those things out.

[JAVIER ROBLES]

I think it was great overall. Obviously, it was an amazing experience for me to see some of the amazing leaders throughout the U.S. I think there were issues with speed because when we’re working too fast, we forget things; and we have people in the room who are advocates. That’s what they do. They advocate for things. So you bring people in the room; and they’re going to tell you, “Hey, you got this,” “Hey, you should bring these people in.

I think relative to what we could do different next time, maybe to Anita’s point, maybe bring in more people – some of the recommendations were bring in more people from these communities or bring in people with disabilities that maybe are not represented here – so things like that, some logistical issues. I know there were some issues initially with some kind of access or whatever. But I think if we move a little bit slower, then we have time to think those things through and ask people, “What do we need?”

I think given the amount of time -- like George said -- that this whole thing went through, I think this is an amazing job. I honestly think that that’s because of Gail Christopher and her leadership and her pushing us to move this forward from the perspective of facility and racial and systemic justice which is, I think, at the end of the day what we want to see, like data that really gets people feeling that they are heard and that their numbers, their data, matter, right? Because a lot of people feel like, “Well, why should I tell you anything? Nothing’s going to happen. Nothing’s going to be different next year.”

So I think that gets back to the narrative again. I don't think it’s only important to make your numbers so that people understand them, but I think the narrative also has to get the message back to the community that, “You were heard. This stuff happened.” People have to see results -- which often you don’t see.

[DAVID ROBERTS]

Were there other parts of the evaluation or any of the things that came out of it that surprised you?

[JAVIER ROBLES]

After COVID, nothing really surprises me, I guess, to be honest. I tell that to my students. When I used to teach these things I would tell people – I would I tell my students, “Hey, we have the Americans with Disabilities Act, we have Section 504, we have all these laws and state laws and people with disabilities are doing so much better.” Then after COVID, I’m like, everything I said I feel like it was a lie. Like we were living in a different place, and I just didn’t know that it was a lie because I was part of a lie. Then COVID comes, and it just like rips this whole ceiling or whatever you want to call it off of the United States; and we see that there are injustices that we can just clearly identify. We don’t’ need ten years of research. Here are African Americans and people with disabilities and underserved communities, poor communities, dying at an exorbitant rate.

That’s sort of like the moment you say, “Well, this is where we are. It’s not where we thought we were, so we have a lot of work to do.” So, no, nothing – well I take that back. One thing surprised me. One of the leaders, one of the panel leaders -- I apologize, I don’t remember her name – told us a story that when her tribe asked for supplies from the U.S. Government, they instead sent like body bags. So that was somewhat surprising. But other than that, not much, I guess.

[GEORGE HOBOR]

So for me on the evaluation, it’s surprising things. One shouldn't be surprising at all. The evaluation pointed out the value of an impressive communications campaign, a very strategic communications approach. I’m a researcher by training, so of course am interested in the demonstration of research findings; but I’ve never really focused on all the different components of an aggressive communications campaign. We put together one for this, and the evaluation acknowledged it. So that was a pleasant surprise.

Then I’m not sure if there are things that are really surprising, but I do want to kind of thank Javier and Anita for pointing out some of the more detailed things about the Commission that I’m overlooking because going back to my initial comment of why this was unique is a foundation can move fast and with a lot of resources; but that doesn't necessarily mean they’re doing so responsibly. I’ve seen a lot of philanthropic efforts that put a lot of money in the field and then just want to sort out the details later. I think there’s mixed evidence on how well those efforts turned out.

I think what was surprising is kind of how well we were able to not do that. Instead what we’re going to do is say, no, we’re going to be careful; have the conversation; open this up’ try to get to what we see in collaboration with others as really important strategic next steps, and then we’re going to move. I think it showed that philanthropy has the ability to do that and not just move quickly with resources without thinking it through thoroughly.

[DAVID ROBERTS]

In many ways, the real work of transforming public health data is really just getting underway. But have you noticed anything that’s already changed in this space since this project got started?

[ANITA CHANDRA]

I’m happy to sort of jump in a little bit from what we’ve been observing. One of the things that I think is really positive as we’ve been talking to local and state health departments and just kind of out and about with communities that are working on their community health action plans and the like, I do think people are understanding the inadequacies of the data to address issues of health equity. So they have the ambitions of health equity often in their plans, but they realize there’s a big delta – a big gap – between that ambition and the data they actually have.

There have been increasing conversations about how do we think more holistically about health and capturing things like community and collective well-being, even subjective well-being – how are individuals in this community doing, some of the aspects of positive health. People are starting to have that conversation, which is great.

Second, there is a little bit more work even beyond where we were before COVID to try and bring sectors together to talk about what data they could serve up in terms of integrated plans. So that’s really promising.

Then certainly, some communities have been a little bit more active in terms of engaging other partners who might be able to provide data, whether in the private sector or in technology or the like. So I think that’s great. That’s fantastic. I think the areas where people are struggling and they’re looking for guidance, support, technical assistance, is, okay, we’ve recognized the issues of our inadequacies of data to support health equity conversations. How do we go back into our data systems and reimagine our measures, our indicators? Where are we going to get these data from? How are we going to think about getting information on systemic inequities or historical injustices or things that we know are drivers of health equity?

So now people are bought in, in some ways, but not necessarily with the supports or understanding of how to kind of operationalize it.

The thing that’s also happening is that as much as people have been talking about health equity, there has been more of a recent concern that after kind of the hubbub of conversation that happens after an emergency like COVID-19 and other related things that were happening, will there be the continued political will and the community will to press this conversation forward – and not just in health departments but in sort of a community wide way.

So people at this point of narrative that the Commission really pointed out, there’s the initial narrative of getting people to recognize the systemic and health inequities; but there’s sort of the second narrative that now needs to come through, which is maintaining that conversation, creating the stories so that it’s not a flash in the pan that we cared about health inequity three years ago but that it persists past 2023.

So I think people are now trying to move to that base too in terms of data, narrative, partnerships, and how to keep them sustainable. That’s the kind of opportunity, certainly, and where the Foundation and other people’s investments are so critical; but now in some ways the real work happens after a disaster like COVID or the events of 2020. So that’s what we’re seeing – a lot of energy, a lot of excitement, people trying to move the ball down the field but perhaps needing some supports to really take that next big step.

[GEORGE HOBOR]

I’ll add to that a little bit. Are we seeing change?

I think, for me, it’s been a real positive to – being introduced to a lot of different data work happening across the country. It is glaringly obvious when something is presented and there have been no equity considerations around what is being proposed. What’s been nice is there’s always someone in the audience, whether it be a conference or a webinar or whatever, that is going to point that out.

So I think the equity considerations around data – that conversation is stronger than I’ve ever heard it, and I don't think it’s going away because I also think the data world is kind of alienating for a lot of people. But I do feel that community-based organizations that may have felt alienated have also gotten a taste of working with data now and have found that to be empowering. So I think there’s – I don't think that’s going to go away. I think there is an active drum beat around the equity considerations in data.

So the Commission contributed to that somewhat. I see it cited a lot. I can’t say that it contributed – I can’t say at what level. Obviously, the pandemic was (laughing) -- really set the stage for that, but I do think the Commission played some role just because we did have that communications campaign. We do have vast networks, and we really worked on diversifying those networks too to get the messages around data equity to new audiences. So I think there is a change that’s exciting.

To Anita’s point, how long is it going to hold, I actually am pretty optimistic that it’s not going to be easy to roll any of this progress back.

[JAVIER ROBLES]

Yeah, I agree with both George and Anita. Some of this is hard to measure in terms of what has the report done in terms of moving some stuff along. I know in New Jersey, we still have issues just even getting an Office of Emergency Management to include people with disabilities. Even after all that New Jersey went through, it has been an uphill battle; and this started during COVID -- so things like that.

I hope that what we saw during COVID, when you are giving the funding to a community, you can eliminate child poverty. Who knew? But now, we see those numbers again going where they were. So to some extent, I think we have solutions to a lot of the problems if we’re willing to implement them and take the political hit. But I think we still are going to have issues.

For example, how do we make sure that we include disability across all data fields because disability is something that affects every data field. Whether you’re going to the hospital or getting transportation or getting housing, we have to ask these questions like...Are you disabled? What’s your disability? What are you going to need if you’re going out for employment? All these things are things we talked about.

It’s not just disability. It’s other populations as well. We have to do the hard work of figuring out how we’re going to make sure we ask these questions that are going to get us the answers we need so in the future we’re not like, “Well, we already have all these datasets; how do we make them better?” We should just ask the right questions from the beginning so in 20 years we have all the data we need.

To some extent, Google what’s on the Commission as well. I know they have a lot of data. A lot of these technology companies have huge amounts of data. I spoke to one person – I won’t repeat her name – but she called me because she thought that she could get a contract with our department. I said, “Well, what kind of data do you have?”

She said -- It’s like a pharmaceutical company, “We can get you any kind of data you want because we work with all the pharmacists.”

I was like, “Really, what kind of data?”

She’s like, “We can tell you who has whatever disability within a two-block radius.”

I’m like, “How can you do that?”

“Because we work with the pharmacists, so we know what pills people are taking,” which surprised me because I’m like, oh, I didn’t think we had access to any of this kind of data.

They’re like, “Yeah, if you’re willing to pay us, we’ll give it to you.”

So I’m like, “Yeah, okay, I get it.”

But maybe we need to be more creative and work with these people and say, “Well, if you have that data that we need, maybe we should try to figure out how to access it,” if at the end of the day, what we want to see is how do we eliminate disease or these things. I don't know, you guys are the researchers and stuff. I’m just thinking out loud.

[ANITA CHANDRA]

No, it’s a really good point, Javier.

I think one of the things that is a challenge, and we talk about this in a special issue that the Foundation put together with some of us in a publication called Big Data, which is how does the tech sector want to play in this conversation consistently going forward? So not just through proprietary payment mechanisms but the role of companies in social responsibility and the commitment to the national progress. They do have a role to play in the future health of the public of the U.S., and that ability – how do we want to harness it in a way that is sustainable, that can support some of the public health infrastructure but still is sort of honoring how public and private sector can work together in really integral and important ways. I think that’s still being sorted.

So it’s a big question that the Commission raised, that conversations that we had with Commissioners raised. It’s not going away either because more of that is going to be held by your Amazon’s, your CVS's, your Google’s. They have an interest, but shaping that interest that still serves the public’s health is probably a big question that needs to be answered over the next couple of years. But what was exciting was that the Commission had that representation and hit on these critical questions. That is part of modernization, right? You can’t leave them out anymore in the conversation; but how do that in a way that is fair and equitable and thoughtful, I think, that’s a big question.

[DAVID ROBERTS]

In this time that we have remaining, I want to go real narrow to George to talk about what, if any, changes you’ve seen sort of at the Foundation in terms of how you approach both public health data but also projects in general that might have come out of the work of the Commission and/or the evaluation of the project.

Then, I want to pose sort of a macro question to all three of you about what you think about – what does that transformative change look like. So a few years down the road, we’ve been able to solve the problems – the (inaudible) problems that you identified. We’ve been able to tackle and implement the solutions that folks identified as being effective. What does that transformative change look like? So that’s just sort of framing where we’re headed.

George, you mentioned a couple of ways that you wanted to evaluate, different ways that the Foundation could approach this work. Have you seen any differences come out of this project in terms of how the Robert Wood Johnson Foundation approaches public health data or sort of projects overall?

[GEORGE HOBOR]

So how we approach public health data is a big question. I think that earlier I said what would be nice out of the evaluation is to help narrow the scope of (inaudible) in so many different ways. I think we have effective programming in place around national dashboards. We’ve supported integrated data systems and have explored those. We’ve focused on how do you enhance access to federal administrative data and how could those data be used in either research on health inequities or public health practice. So we’ve really cast a wide net.

I do think – I’m not quite sure what direction we’re going to go into fully. We can’t do it all, but data work is big. So I would like to see some narrowing of the strategic scope. But I do think where we have seen a narrowing and what we’re really focusing on are the two things I mentioned earlier around measuring the gaps and how do you incorporate more community-based organizations into data and knowledge production. That is a strategy that I see coming out of the Commission that is being held.

The Commission – that work was interesting because it did challenge some of our operations. I would like to see what sticks out of operation’s changes. But I think because that work did show that a large foundation could be fast and flexible and could give away some of its strategy-making to its partners. So I’d like to see some of those things stick, and I think they will; but we’re still digesting a lot of the evaluation learnings.

[DAVID ROBERTS]

So as we zoom back out and we think about that idea of transformative change, what does that look like looking ahead? What does transformative change around public health data look like moving forward?

[ANITA CHANDRA]

I’m happy to start, and certainly Javier and George will have better things to say than me; but I think there are a few principles that I would argue. One is essentially that we have the kinds of measures that we’re tracking that actually tell us something more about the health and well-being of our country than our current measures. It doesn't mean that we don’t need – some agencies need really granular data on the number of people getting kidney dialysis or the number of people age-adjusted who have survived cancer. All that’s really critical information.

But one of the things that I think we’re losing in how our data systems are helping the country make decisions and how communities make decisions is we’re losing kind of what is the North Star of what we’re trying to get to. I would argue it’s both actively advancing health equity but also promoting positive health and well-being in a way that is culturally responsive but focused on holistic health.

If you actually look at the data and measures that we collect, both nationally and in many localities, we don’t have a lot of information in those two buckets. We have a lot of other information. So I would want a system that shifts the locus and the focus of data towards things that are going to give us a real clarity around our health, well-being, and health equity objectives. I just don’t think we have that yet.

The second is that it would be structured to be equity-centered, which means that it has all the features of what we talk about in equity so that it has access and inclusion and voice and agency of those most affected, what we might call “procedural equity,” but it also has all of the other aspects of equity, like contextual equity. It allows people to understand how the data about particular burdens of disease or burdens of health or opportunities for health situate in a community history and context. Again, we really don’t have that.

We have a lot of rapid data, and we’re going to be deluged with even more data as we were just talking about with tech. How does Javier, how do I, how does a practitioner on the ground make sense of that to actually make decisions?

Then the third thing we talked about a little bit in the Commission is how do we balance this need for kind of precision and granularity with the ability to move quickly with data so that communities don’t suffer. We probably have got to get a little bit out of historical academic public health way of thinking towards a robust evidence-informed way of public health, but that means that we’re bringing reason and common sense to how we’re thinking about data and the use of data. That was on full display during COVID.

So I think we’ve got a great example of that, and I hope for us something better because I think the American public is confused in some ways by like what are we doing in health and public health. I think we’re not serving the communities that we care about the most currently with the data systems.

So I would say those three elements, to me, are most critical for a transformed public health data system.

[JAVIER ROBLES]

Definitely what she said (laughing). I would like to see a transformed system that really takes into account all populations, including Puerto Rico and the territories, which are often forgotten. You don’t even want to talk about their health system, believe me, but also places like Appalachia and all these other places. I would like our citizens to realize that just because they live in one of these places it doesn't mean that they should be forgotten, and it doesn't mean that lawmakers should make decisions for them about health care and Medicaid and whether or not their state has these things. Data can help support that.

I think using data and the narrative in a way that we could explain to every individual, regardless of where they are, “This is your story, but we’re all in the same boat,” will help us.

I think to your point earlier, Gary, that there’s a lot of decision formation which really hasn’t helped us the last few years. It’s just made people feel like, “I can’t trust data; I can’t trust sciences; I can’t trust anything but my politician because my politician is not going to lie to me.” But the reality is that we have to turn that vote around. We have to get these people to see that this is your reality and the reality of millions of other people. But that’s obviously going to take a new rebuilding of trust from us to them, and I think that’s going to exist.

I also would like to see populations, like people with disabilities, included in all datasets because that’s going to help people with disabilities generally if we target a percent of this population and other populations. Because if we just keep excluding people from everything, then how do you know what they need? How do you know that they need transportation or anything else?

So I think if we can get to that point, it would be great.

[GEORGE HOBOR]

This is kind of a huge question to close on. I’m not quite sure what it would look like. The National Commission made recommendations. I think we can see those in data work that I can see progress is being made.

Also, a shout-out to one of our (inaudible), Actionable Intelligence for Social Policy. They have a racial equity toolkit for data. In that they have a nice data life cycle from data collection to data storage and sometimes to data analysis to data communication (inaudible). And they lay out some principles that you should be looking for this in your data systems. I think that’s a good report that also provides some – at least a structure to look and say, hey, we made progress. This is being achieved.

Similarly, CDC Foundation has an equity principles in data. They have guidelines for that as well. So if we could look at the data we’re using and the intent, how it’s being used, and if we can see these principles that are starting to proliferate in the field, we can see them clearly in the data, I’ll be pleased. Like I say, there’s been progress.

[DAVID ROBERTS]

Fantastic.

[J.B. WOGAN]

Thanks to our guests for this episode, George Hobor, Javier Robles, and Anita Chandra, and thanks to my Mathematica colleague, Dave Roberts, for conducting the interview. This is the second in a two-part episode about improving health equity by transforming our public health data systems, with a focus on lessons from the Robert Wood Johnson Foundation’s Transforming Public Health Data Systems initiative. The first part, which is available now, features an interview with Alonzo Plough of the Robert Wood Johnson Foundation about the motivation behind the initiative.

This episode was produced by the inimitable Rick Stoddard. If you like the show, please consider subscribing. We’re on YouTube, Apple Podcasts, Goodpods, Spotify, as well as other podcasting platforms. To learn more about the show, visit us at Mathematica.org/OnTheEvidence.