Improving Health Equity through AI and Data Innovation

[NGAN MACDONALD]

We always have to remember that the AI is predicated upon data of what has happened in the past. And what has happened in the past is that there are these gaps in care. And when you use the AI, what it does, if you just use it based upon that data that has been collected traditionally in the past, you amplify the disparity, because it’s basically projecting out the same types of biases that have existed. Which is why when you think about, “How do we implement AI?,” it’s gotta be hand in hand with humans, because we as humans, we’re really good at imagining a better world, right? Imagining a better way to do something.

[J.B. WOGAN]

I’m J.B. Wogan from Mathematica and welcome back to On the Evidence.

On this episode, I speak with Ngan MacDonald about the applications of artificial intelligence, or AI, for improving health care through data analytics and data innovation. Ngan recently joined Mathematica as the company’s director of health data innovations. She is also the chief of data operations for the Institute for Artificial Intelligence in Medicine at Northwestern University.

On the episode, we discuss the potential benefits of AI in health care, the equity implications of training AI on incomplete health care data, and what AI could mean for Mathematica’s work in data analytics.

I hope you enjoy the episode.

[J.B. WOGAN]

When we were talking before on a prep call, we were talking a little bit about how, hey, you're not an economist. We have a lot of economists at Mathematica, and a lot of them who appear on the show and have really cool research to share. I wondered, so I see you have a master's in information systems, I think.

[NGAN MACDONALD]

Yeah.

[J.B. WOGAN]

So first, what is that, and what is the perspective you bring to health policy, health policy research from that background?

[NGAN MACDONALD]

Yeah, so the master's in information systems is essentially like a, how do information systems work together, fit together? What is the difference between an information system that helps you manage a transaction and a process, which is very detailed, versus an information system that answers business questions, which rolls a lot of data into like a higher level and aggregates it? So what that master's degree gave me was kind of like this perspective of, you know, like in theory, how do these things fit together? In theory, how do we manage data for one purpose versus another? So it wasn't -- I mean, like there was things like, you know, database optimization that I kind of had to take for it, but like that, all of the technology that was a part of that degree is very dated, you know?

[J.B. WOGAN]

Did you know that you'd be working on artificial intelligence when you pursued that degree?

[NGAN MACDONALD]

I don't even think that I knew what that was when I pursued that degree. I think it was like one of those -- so I'll tell you this story. Like I was actually in HR back when I came out of college. I've always been a people person, and I did an internship with this company, didn't know what I wanted to do. And then I was graduating and they said, we'd like to give you a job.

And I said, great, like, what do you want me to do? And they're like, we'd like you to work in HR, like be a recruiter, interview people. And the long and short of it was I ended up being a, like, you know, being an HR department like generalist. And as part of that role, what I had to do was actually be on the steering committee for this implementation of this system that was like, you know, people soft, how do you manage people's salaries, all that stuff. And what I found was, wow, you know, like all these things that I was working to do manually by plugging them into spreadsheet, if we wanted to buy a software company, like what do their engineers make versus what our engineers make?

You know, like it's a data problem, but we didn't have any of that actually in any system. So we like literally would pull files and I would type in into a spreadsheet and then I would like average, you know, all of our engineers versus all their engineers. And so when we implemented the system, I was like, oh my gosh, this is amazing. I can literally just ask it a question. Just had to learn a few things about, you know, how to write crystal reports and all that. So that's when I ended up going back to graduate school because I wanted to be in this technology area. But even then, I'm not sure. I thought I would be in HRIS, but they didn't give me a job there.

[J.B. WOGAN]

So what is HRIS?

[NGAN MACDONALD]

It's the IT people who support all of the HR systems.

[J.B. WOGAN]

I see.

[NGAN MACDONALD]

Yeah.

[J.B. WOGAN]

That was the dream job at one point.

[NGAN MACDONALD]

Yeah. At one point, that was a dream job.

[J.B. WOGAN]

Well, I can understand the appeal of making things much more efficient and being able to get that information at people's fingertips.

[NGAN MACDONALD]

Yes.

[J.B. WOGAN]

Which seems to have a direct connection to what you're working on today.

[NGAN MACDONALD]

Yeah. The first class I took on data warehousing, data mining back -- I'm trying to think, I think it was like maybe 2000 or 1999, so I'm dating myself, was actually, that area was just starting to be like a field that people were thinking about. Because before that, we really thought about information systems as a way to manage discrete transactions, you know, like a checkbook. And then that field was like, okay, let's look at the totality of data and how do we structure it in order to answer questions. And, you know, all of the database administrators that were in that class with me, it blew their minds. They're like, why would you create redundant, you know, rows in order to answer questions? You know, like they were all about trying to create efficient transactions. And I, like, when I got into that class, I'm like, finally, this is the class that I wanted to be in. All of that other stuff was just background noise for, you know, how do you answer questions?

[J.B. WOGAN]

Last year, we did an episode for the 100th episode of On the Evidence, and one of our predictions was for the next 100 episodes that there'd be a common theme around artificial intelligence, it seems like. As a topic, it was already bubbling up, but I imagine it's going to be an even higher priority in terms of the evidence ecosystem in which Mathematica operates. And I hope our prediction is right. You're certainly one of our lead people on AI at Mathematica. I was wondering if you could ask kind of a very basic question for someone like me, which is, what do we mean when we talk about artificial intelligence in this space? Is it like, I think a lot of people are familiar now with ChatGPT and generative AI? Or what does it mean to you? What should listeners be thinking of? What should be top of mind for them?

[J.B. WOGAN]

Yeah, so I think what people don't realize is that we've been using AI in healthcare for a long time. And for me, AI means any kind of the use of, you know, computation and data in order to assist in answering questions or making decisions. And so if you take it down to that, we've been using it to collect data on people and help to predict the risk of getting a disease, for instance. In the health insurance field where I started, it was used to predict what their costs would be in the coming year so that we would be able to budget for it. And then you have like -- so that's kind of base data analysis, a lot of things that Mathematica is good at. And then sort of like that next level of what I would call pattern recognition computer vision. There are patterns within, you know, MRIs, x-rays that, you know, oftentimes it takes years of training for a human to kind of look at an x-ray and go, oh, yes, this like cloudy mass that you and I look at, like they see, you know, that it's a possible tumor.

And so what we've been able to do with AI for, you know, on the vision side is to be able to say, experts tag these things that are, you know, cancerous and then let the AI figure out, you know, how to recognize those patterns. And there's all kinds of funny stories about how it gets messed up. But, you know, we have a project at Northwestern where, you know, there is a disease that causes lesions all over your body. But that disease also leads some people, like 20 percent, to develop cancer. And, you know, like you're talking about people walk around in a lot of pain and you don't want to biopsy them for that 20 percent. So you want to be very certain. And because it is something where people have lesions all over, like it's a rare disease and few dermatologists have seen it. So in that case, what we can do is we can use AI to kind of recognize the patterns of what is cancerous or precancerous and what is not.

So that's kind of like the probably bread and butter of what we've been doing for a long time. And then on the scene comes things like ChatGPT and this whole idea of generative language models. And in that aspect, how we currently are thinking about using it as one, you know, just like, you know, the transcript for this podcast, it can be an ambient listening device for your conversation with your physician. And then at the end of that conversation, the end of that visit, it can summarize for the physician what happened. And the physician can look at it and go, yeah, you know, we also talked about this, but it didn't come through in the summary.

Today, a lot of doctors and other clinicians spend a lot of time documenting that visit with you. And it's time that they're not rendering care to you. So you know, like we can use it that way. Also, we can use it in terms of, you know, generating emails to patients because, you know, like there's this classic example of take two aspirins and call me in the morning. Like it's not that there's any physician who wants to say that. It comes off as cold and callous, but like they literally have a couple minutes here and there to be able to actually type out an email to you, read through your email.

But if we can augment that with some intelligence to say, this is the reason, hey, take two aspirins based on your symptoms, it should go down. And, you know, please don't think that this is, you know, healing you. Take it easy. And then let me know if you after, you know, 48 hours, you're still feeling like, you know, you need more intervention. So like adding more contacts helps people feel like their physicians have empathy. And actually, that's something ironically the machine can do.

[J.B. WOGAN]

Yeah. Yeah. As you're talking, I was thinking back to some of the pediatricians that I've come across during my kids' care who are just overwhelmed, big, huge caseloads, growing caseloads. And I'm always appreciative when they respond to our emails. We're on Kaiser, we're with Kaiser. Kaiser has this very nice portal where it's easy to send a direct message to your clinicians. But that is a burden that physicians 30 years ago wouldn't have had to have that kind of direct electronic communication with your patient caseload and have the expectation that you need to respond within a couple of days. And not only do you need to respond, but it should sound empathetic.

[NGAN MACDONALD]

Yeah.

[J.B. WOGAN]

Don't just provide the correct information, but do it in a way where you have thought of and clearly have thought about the patient's needs and stress levels and all that. I wanted to ask about healthcare policy research, because you've just been talking about potential applications in healthcare, which is important. But for a company like Mathematica, I'm also curious what AI can do in terms of augmenting the type of work that we do.

[NGAN MACDONALD]

Yeah, so the type of work we've done is look at policies, figure out how do we measure those policies, and do data collection and data analytics against those policies. This is all great. Sort of the next -- I view the work that we're doing around AI and data innovation as taking that one step further and creating a safe space for people to innovate and experiment with the data. Because what happens, I think about, if you were climbing Mount Everest, would you want to do it with the Sherpa who knows and understands the policy, knows and understands how hard it is to collect data, or would you want to just wing it on your own?

So I think that what we are bringing to the table, and with the help of AI and other technologies that are much easier to use nowadays, is trying to create that safe environment where people can experiment. And by the way, it also helps us internally in terms of creating a learning lab environment for our own people as they are coming up on their expertise.

[J.B. WOGAN]

I initially was going to ask you how you became interested in this work, but I feel like you've already sort of answered that a little bit. All right, I'm going to ask a slightly different question, which is, what is the problem that you're trying to solve in your new role at Mathematica?

[NGAN MACDONALD]

Wow, so the problem that I'm trying to solve at Mathematica is that AI is a team sport. Healthcare is a team sport. Healthcare is extremely local, and yet, you know, we have, what, 8 billion people in the world? And so chances are likely that there's somebody else who's experienced your condition. And how do we know that? We know that if we have the data that quantifies it, and we can be able to identify it in the data, identify what works with it and what doesn't work with it. But we can only really do this if we're able to bring all of the different stakeholders within the ecosystem together.

So like Mathematica with its policy front, Northwestern with its research side, industry with like, you know, like their business models around like how do they deliver care to individual patients? How do they create software that targets, you know, the right interventions? And so my role at Mathematica, the whole idea of data innovation is to bring different stakeholders and experts together to solve discrete problems. And that's what I think of as true data innovation. It's not that I created some fancy new algorithm. Somebody will do that, but they would do that with that, you know, cross-pollination of different people's expertise coming together.

[J.B. WOGAN]

And so you mentioned the safe space. Who would you hope will work with you in that safe space? Like what are your ideal partners? You mentioned Northwestern as a research organization. So yeah, but maybe could you put any more meat on the bones there in terms of the kinds of organizations or people that you would like to work with ideally?

[NGAN MACDONALD]

I have a whole laundry list. I think I could work with pretty much anyone. But let's start with our biggest customer, right, is the federal government. And one of the things that happens is a lot of these, you know, policies get written or these funding mechanisms get written somewhat in a vacuum. What if an agency, as it's planning out their budget, you know, comes in and we kind of facilitate and provide them data around like what are these things likely to cost? Who are the key constituents within them? Like kind of help them plan out their journey a little bit.

So that's kind of -- and also like as they're rolling out like measures, like is that measure feasible? Can we actually make it happen? Who do we need at the table in order to do that? On the foundation front, like foundations similarly to the government, they want certain outcomes to happen. And sometimes they put out an RFI out there that they, you know, just to gather information. But there's never really any solid information, because it's like drop it over the wall kind of process. What if we could create a safe space where people come together and they talk? And it's not just a dropping across the wall, but like let's talk and plan these things out. And then, you know, like I have worked in the past three years with a lot of startups and with the startups, they have awesome ideas, great ideas.

Sometimes they lack like the ability to actually like get their hands on the data because healthcare data is very restricted. And it's sort of like people who have a ton of money can buy data, but you know, your startup, they're not going to spend millions of dollars on buying the data. So you know, what if we could help them validate their model? I'm trying to think who else have I left out? Academic medical centers, same thing, right? Like they're trying to figure out, you know, they've created this great model that helps predict 30-day readmissions, for instance. And they're like, oh, this works so well for us.

We're going to go and we're going to try to like roll it out into the, you know, like to the rest of the academic medical centers in the US. But the problem is, is that their models get trained on their data. So they don't even really know if it's going to work on a national data set or not. And we can kind of help them do that. And then, you know, for payers, you know, which is where I kind of, you know, grew up in, there's all this data that they wish they had, but they don't even know who's the right partner to get it with. What would they do with the data if they had it? And it's like, do you want to spend a ton of money to get data that you don't know if it's going to be useful for you to connect with your own data or not? So these are all, I think, problems that we can help all of these different constituencies around like the healthcare ecosystem to figure out.

[J.B. WOGAN]

You mentioned something, I don't know if you actually used the word equity, but you were talking about how some organizations have the financial resources to pay for data or access data and others might not. And I did want to ask a little bit more about that. What are the equity implications as the use of AI becomes more common in this evidence ecosystem or evidence community?

[NGAN MACDONALD]

That is actually something I spend probably most of my time worrying and thinking about because healthcare, as it is in America, is inherently unequal. People, depending on where you live, have different ability to access healthcare services. And then there's also systematic racism that has created a fear of the healthcare system. And sometimes it's ethnic sort of bias in terms of like some -- I think about my mom, right? She thinks that going to the doctor is a last resort. So she doesn't think about it preemptively. And she would rather talk to somebody who, for want of a better word, is kind of like an apothecary, has all these herbs for her different ailments.

That's her go-to. Her data is unlikely to be fully fleshed out within the healthcare system because she just doesn't go. And if you live in rural Alaska, you don't have access to a clinic. Your data is not going to be in there. And then you have the fact that these institutions have varying resources for their ability to really collect or even analyze their data. So these are all problems that actually we have solutions for on the AI and technology front. And we're working through and trying to figure out some of those solutions right now. But we always have to remember that the AI is predicated upon data of what has happened in the past. And what has happened in the past is that there are these gaps in care.

And when you use the AI, what it does if you just use it based upon that data that has been collected traditionally in the past, you amplify the disparity because it's basically projecting out the same types of biases that have existed. Which is why when you think about how do we implement AI, it's got to be hand-in-hand with humans. Because we as humans, we're really good at imagining a better world, right? Imagining a better way to do something. And so I think the conversation around AI and data has to always be centered upon the fact that we know there are gaps in the data. We know it's part of our job to go and try to collect some of that data if possible, or at least understand that there are those gaps. And then that informs what kind of decisions we make going forward about whether or not we use the result of AI or not in certain situations.

[J.B. WOGAN]

So one of the questions I wanted to ask, you have this role at Mathematica, but prior to your coming to Mathematica and you continue to have this role as a chief of data operations for something that sounds very cool, the Institute for Artificial Intelligence in Medicine at Northwestern University. I was wondering if you could unpack that a little bit. What is your role there? And what is the Institute for Artificial Intelligence in Medicine at Northwestern?

[NGAN MACDONALD]

Yeah, so the Institute for Artificial Intelligence and Medicine at Northwestern, we launched that sort of at the beginning of the pandemic. Literally, I think in parallel, it was announced and then we all shut down. And it was the thinking around that is AI is a team sport. It is our engineering school and our medical school, as well as the hospital, all having different pieces that they would work on together within AI.

And sort of the reason why it was created was that very rarely would people come and say, oh, I've got this great idea and I have the data to action on it. And I have the resources, i.e. like the programmers, the AI scientists to be able to do something about it. So very rarely do those things exist together. And AI was essentially being done sort of in little pockets all over the university. And we wanted to bring and create a community together to bridge that, you know, like sort of what we call the three-legged stool. And so when we bring those people together, they have like really great ideas. And my job essentially in that role is one, to connect all these people together, sort of building that bridge like I love to do. And then two, kind of like how do they do it? What do they need to do it? Do they need data? Do they need like, you know, funds? You know, so whatever it was, like to actually make that a reality.

And also like I feel like in that role, there was a lot of event planning, you know, still a lot of event planning. We're hosting the International Population Data Linkage Network Conference, which happens every two years. It's a global conference. We raised our hands up and said, oh, we'd be willing to host it in Chicago in September. And so it's a little bit like a startup where whatever it takes to make these ideas a reality is part of my job description. And there's other people.

[J.B. WOGAN]

Okay, got it. Yeah. And so you mentioned event planning, I guess another way of describing that is convening that connects back to your idea of being a bridge builder. I noticed that on your bios, both for Mathematica and on LinkedIn, that you mentioned wanting to act as a bridge between academic research and the industry for healthcare data. I was curious about, so there's, I guess, the healthcare industry, but the industry for healthcare data, is that, how would you describe that? That's like just a niche within healthcare? Or what is that industry?

[NGAN MACDONALD]

There is a robust industry that is doing things like taking a bunch of data and creating risk scores on somebody, being able to identify a set of people with a certain disease so that clinical trials gets done. And I don't think like when I say bridge for like the healthcare data industry, like that's what I really meant. What I really mean is that the healthcare industry overall has all of these different constituents. And, you know, like within the US in particular, we have perverse incentives across all of these different stakeholders of healthcare.

So you've got your insurance company, which I worked for, and they are trying to quantify risk and be able to manage it from a financial standpoint. And then because of kind of all of these other factors, they end up like sort of partially tipping their way a little bit into the care piece of it. But then you've got like the hospital system, which has, you know, employees, physicians, and they get paid based upon, for the most part, what things they do to you.

So like, if you're sick, they're getting paid to manage, you know, and get you well. And then there's, you know, like the government as a single payer, there's the pharmaceutical. And I feel like part of my role as bridging between those is kind of being able to see it from both perspectives. Because if you put like a health system and an insurer in a room together, and they start to talk to each other, it's almost like, you know, like the conversation just bypasses each other. And it's pretty amazing.

So I think what I really love about Mathematica is that ability to be a neutral convener within the healthcare system, because we are neither a payer nor a provider. And we understand deeply the policy and sort of all of the processes that make healthcare happen. And so I feel like this, in particular, is a great role in kind of like that part of my mission, my personal mission to kind of bridge the healthcare system.

[J.B. WOGAN]

In your bio, you say you're passionate about creating a collaborative data ecosystem that liberates data for use to improve healthcare. The liberates data part caught my attention. Could you talk a little bit more about that? Like, in what ways is data not liberated now? And what's an example of how liberating data would improve healthcare?

[NGAN MACDONALD]

Yeah, so I think that data today, if you've ever had the experience of going to your doctor, well, you're with part of Kaiser, so all the data is together. But you go to your doctor, and then your doctor refers you to a specialist. And that specialist is unlikely to be like in the same data ecosystem that your doctor is in. So then you go to the specialist, and the specialist is like asking you the same questions.

There's sort of this lack of interoperability that we have today. And we've been striving for, frankly, since I think 2016, when the Office of the National Coordinator for Health Information IT was created. And we tried to solve it at first by, hey, let's just make all of these things digital. And then, you know, we put software companies in the mix. And they didn't have an incentive to actually make the records interoperable. And now, you know, we have legislation that basically says, your data belongs to you, and you should have access to it without special effort. Which means that for the first time ever, individuals get to own their data, and in theory, should be able to consent for their data to be used. And I think like the past history of why I talk about liberating data, it's like the people who own the data or think they own the data were either like, oh, you know, I'm a health system, this is all my treatment protocols, and therefore I own the data. But the treatment is upon individuals. And so like they're very incentivized to like kind of keep that close as a competitive edge.

Same with insurance companies. They have their data, and they're not going to share it for any reason. And so I think like we're getting to a better place where all of these different stakeholders are starting to realize, hey, you know, like this is a team sport. We do need to share this data. And if we share this data, we can make better decisions about people's care. When I say liberating data, I don't mean like everybody's data is just out there. What I think we need is we need to be able to understand the data and how to use it. But we also need to understand that there is a level of, you know, like a lack of real true transparency about our data and where it's being used.

And so in my ideal world, we would always damp like where did that data come from and whether or not it has consent, and like true consent, not just like the fact that you sign a sheet of paper when you went to your doctor's office and if you didn't sign it, your kid wasn't going to get seen. But like these are the intentional things I as a consumer want to be able to have my data used for. I want to be able to have my data used for research. I want to have my data used for clinical trials. And so we don't currently have any real mechanisms to do that. And I think part of our mission in the next, you know, 5, 10 years as, you know, in the healthcare space is to be able to really truly, you know, map out what that looks like.

[J.B. WOGAN]

Okay. So you had shared an article with some of my colleagues. It's been circulating around Mathematica from a publication I hadn't heard of before. NOEMA, N-O-E-M-A, about how AI could help rebuild the middle class. And I guess this is looking at the implications of AI for the workforce, not so much for healthcare. But it ends with the line, we should ask not what AI will do to us, but what we want it to do for us. So instead of to us, for us. So I wanted to put that question to you. Ideally, in your dream state, future state, what do you want it to do for us?

[NGAN MACDONALD]

So in healthcare, I wanted to create this hyper personalized experience in healthcare, in education, in our daily lives, because that's what the AI is capable of. It's capable of learning your unique history and being able to project back to you the experience that is going to most resonate with you. And so, you know, we know from adult education research, that the more you're able to ground learnings within like somebody's context, the more that those learnings stay with them. And so if you use AI to really hyper personalize that, you know, healthcare is a good example.

I tell this funny story about my mom, you know, like going to the doctor, and she has high blood pressure, and she has diabetes, and, you know, she has glaucoma. And so her doctor says to her, I want you to reduce sodium intake. And, you know, she's, you know, small little Asian lady, and she's like, okay. And she comes home, and she talks to -- like, imagine if she could come home and talk to an AI, right? She talked to me, and she said, oh, the doctor said, reduce sodium intake. I don't do any sodium. I'm like, mom, he means eat less salt. And she's like, oh, oh, is that what that means? Okay. And then, you know, like, she's like, I don't use salt. And I said, well, yes, you do. You cook with, you know, fish sauce, and you cook with soy sauce, and those things have salt in them. And so, like, he's telling you to use less of that in your cooking and in your food. And she's like, oh, okay. See?

And so, like, that sort of hyper personalization can happen if we use AI as kind of the way that we bridge some of the communication gap that currently exists. And then the other way that I would want AI, what I would want it to do for us is some of these things that you and I don't enjoy doing, like, it would give us back some of that time, like, for physicians having to summarize their actual interaction with the patient. If, you know, like, it takes them a minute to do as opposed to, like, 30 minutes to document an interaction, like, that's giving you back time. So these are all really great benefits of AI.

[J.B. WOGAN]

Thanks to our guest Ngan MacDonald and thanks to the inimitable Rick Stoddard, who produced this episode. Last month, we celebrated the five-year anniversary of Mathematica’s On the Evidence podcast. Whether this is the first time you’re hearing us, or you’ve been with us since 2019, thanks for listening. If you’re fan of the show, please leave us a rating and review wherever you listen to podcasts. It helps others discover our show. To catch future episodes, subscribe by visiting us at mathematica.org/ontheevidence.