Improving Health Through Responsible Use of AI

[ELLIE GRAEDEN]

What we're talking about here isn't just a one-stop roll-out AI to replace the humans. Where we see the most effective use cases are where we're dovetailing it to augment human capacity and human expertise. And so, what we're really seeing in the health care space, and have for a long time, is that it's an adjunct. It's a really useful tool for our experts to be taking advantage of, to make their life a little easier, to do some automated checks, to really be guiding the process.

[J.B. WOGAN]

I’m J.B. Wogan from Mathematica and welcome back to On the Evidence.

For this episode, we’re using a recording from a recent Mathematica webinar on how different parties within the health care ecosystem can responsibly use artificial intelligence, or AI, to improve patient health. The conversation comes amid increasing interest in the applications of AI in daily life and one month after Mathematica announced the launch of its Health Data Innovation Lab.

Noland Joiner, the chief technology officer of health care at Mathematica, guides the discussion with Ngan MacDonald and Ellie Graeden. Ngan, a recent guest on our podcast, is the director of health data innovations at Mathematica. Ellie is a partner and chief data scientist at Luminos.Law and she is an adjunct research professor at Georgetown University.

On the episode, Noland, Ngan, and Ellie explore AI’s implications for patient safety, data governance, and data-enabled workflow. They also discuss how AI can augment, rather than replace, the expertise of humans in health care. I hope you enjoy the conversation, which kicks off with brief introductory remarks from Noland Joiner.

[NOLAND JOINER]

Thank you all for joining. We're going to explore today with my panelists the question of, "Is it time to pump the brakes or hit the gas as it relates to AI?" I have spent my entire career with one singular goal...to improve patient outcomes. So I would like to take that into our discussion today with Ngan and Ellie. I'm going to turn to Ngan and Ellie to have them give you a brief introduction of themselves before we start our conversation.

[NGAN MACDONALD]

Thank you, Noland.

My name is Ngan MacDonald, and I am Director of the Health Data Innovation Lab for Mathematica. I also have a dual appointment, and I am Chief of Data Operations for Northwestern Institute for AI in Medicine. My entire career is really around data and how do we strategize, manage it, how do we use it to drive better outcomes. Thanks for having me.

 

Ellie?

 

[NOLAND JOINER]

Thank you, Ngan.

 

Ellie?

 

[ELLIE GRAEDEN]       

Yes, thank you so much for inviting me and for this conversation. I'm really looking forward to it.

 

I'm Ellie Graeden. I am the Technology Partner and Chief Data Scientist at Luminos.Law. We're a small boutique AI-focused law firm. I wear a second hat as Adjunct Professor at the Georgetown University Massive Data Institute with a secondary hat at the Center for Global Health Science and Security. A little like Ngan, I am a technologist at heart really focused on how data get used. That's meant that I've spent the last 15 years really looking at data and models writ large, which has very rapidly turned into AI now. I'm always thinking about how we apply it and use it to actually have impact in the world.

 

[NOLAND JOINER]

Excellent, excellent.

 

I would like to start to kind of frame up our conversation a bit. As we just -- if we think about all the things that are happening across the health care ecosystem, from provider/payer/consumer retail/health care/pharma/med tech, what is it -- how might we leverage AI to improve patient outcomes across the entire health care ecosystem?

 

Ngan, what do you think? What are some of the things that you think we might want to tackle in order to truly move the needle?

 

[NGAN MACDONALD]

Yeah, so I always kind of go back to the underlying data. We know that health care is inherently problematic today because we have people who don't have access to health care; we have systems that are not connected together; and we have just a whole host of complex incentives, which cause people to not always share data in the way that they should. So I think one area that we need to tackle is around interoperability and really creating that system in which the data is available when it's needed.

 

I think it's a misnomer to think that we haven't used AI in health care because we have. We do use AI every day for things like risk scoring to assess who's the most likely to get sick next.

 

[NOLAND JOINER]

That's right.

 

[NGAN MACDONALD]

Then, we have areas in which we use it. Right now at Northwestern, we're piloting the ability to use ambient listening to allow the AI to summarize a visit and then present that back to the physician for him to add his thoughts or approvals on it. It decreases the amount of time they have to spend actually trying to consolidate those notes.

 

Then, you think about other things that are more low-hanging fruit -- so like public health, for instance. We could use AI to do translations of educational materials much more easily. So sometimes it's not just that we do the fancy AI's, but some really basic blocking and tackling is necessary in order to advance health care across the ecosystem.

 

[NOLAND JOINER]

Ngan, I really appreciate that and underscore the whole interoperability piece. It's just critical and foundational to a lot of the issues that we have across health care. Ellie, what do you think? What are some of the things that we might tackle to advance care?

 

[ELLIE GRAEDEN]

I want to underscore what Ngan just said about the fact that we've actually been using elements of advanced analytics, machine learning, and AI for a lot of years both in health care and in facilities management is when we sort of think end to end on these systems and we think about the full sort of suite of how health care systems work.

 

So I think everything from using diagnostics, we have really good machine learning on image recognition. Now, I think one of the really important pieces here is that where we see the most effective use cases, we're dovetailing it to augment human capacity and human expertise.

 

[NOLAND JOINER]

Yes.

 

[ELLIE GRAEDEN]

What we're really seeing in the health care space, and have for a long time, is that it's an adjunct. It's a really useful tool for our experts to be taking advantage of, to make their life a little easier, to do some automated text, to really be guiding the process, and to be doing the things that machines are best at. Machines are very, very good at bulk ingest and actually integration.

 

A lot of the work that -- one of the places that LLMs are actually becoming really useful is in taxonomy and ontology development and structuring. That sounds very wonky. It's sort of a big data merge problem, but it turns out that it's actually a really effective use case because what you're figuring out is how to get data that come from very different places, described in very different language, to then merge and be able to drive or address some of the interoperability challenges that you were describing, Noland.

 

So I think a lot of this is in identifying what the right use cases are, where specific types of models are going to be most effective, and applying them in those specific contexts. What we're talking about here isn't just a one-stop roll-out AI to replace the humans.

 

[NOLAND JOINER]

Yeah, one of the things I really enjoy about what you both kind of discussed here is that everyone says that they tend to think about these really elaborate use cases. What you both kind of laid out is just some basic blocking and tackling that we can do to really advance care across the health care ecosystem. I think that would be a big takeaway, right? Yet there's some really far out, whiz bang things we could be doing. But if we just look at some of the basic blocking and tackling, some of those use cases, they would be pretty impactful.

 

As I think about that, I start thinking about some of the things we should consider as we think about what we can improve and how to do it across the health care ecosystem. I think it leads me to three things, right? What are some of the key things we have to consider as we select those use cases?

 

One would be around patient safety. I think it's critical that we start there. The second would be around data governance. How do we assure that the data that we're using to build our models are in fact good data? Then finally, it is data-enabled workflow. So how do we make sure that the data is landing at the right place at the right time so that a practitioner or an administrator could make the right decision regarding their patient or their member?

 

So let's start at the top. I would love to get your point of view about patient safety. Ellie, you want to jump in and start it off?

 

[ELLIE GRAEDEN]

Sure, I think there are a lot of ways to define patient safety first off. There are the sort of practical and tactical issues around just even moving through a hospital and making sure that the right information is with the right care providers at the right time. There are some really basic workflow information transformation systems. What we really need is information to follow the patient through their health care journey. So I think there are some basic things that we can do there that are mostly about making sure the right data are available to the right person at the right time.

 

So there's a security component. There's a privacy component. There's making sure that we have -- it sounds very, again, a little wonky here but it's about good data governance and provenance. And so making sure you know where the data came from, who has the authority to see those data at different levels of resolution, and making sure that that is transferring through the system well. So I think there's that piece of it.

 

There's then also the piece where a lot of those data can drive some automated decision-making and checkpoints that can actually highlight and improve, for example, which types of care are being offered by different providers. That's where we can actually start to get to identifying characteristics of the patient that we might be able to highlight specific types of care that would be most effective for them and really streamlining what is becoming a very information-dense environment for the health care providers. This is a way to start to prioritize some of that information and improve decision-making.

There's a great example. Mike Rayo's group at Ohio State University has been doing some really interesting work looking at decision support systems that integrate large amounts of data and surface it to dramatically reduce the amount of time it takes for a hospitalist to respond to a code blue. So code blue is where you're talking about sort of 90 seconds to three minutes to save somebody's life. If you can get them the right information really quickly, you can actually reduce that response time by 30 seconds or more, which is a huge differentiator for the ability to save lives. So I think there are a lot of these types of very specific applications that do a lot to meet patient safety, both in that sort of privacy and governance domain, where we're also taking good care of their data, but also very practically just improving the quality of care they receive.

 

[NOLAND JOINER]      

Yep, that's awesome! Ngan, could you weigh in here?

 

[NGAN MACDONALD]  

Yeah, absolutely. So one of the things Ellie mentioned is talking about providence of the data. I'm going to kind of swivel it towards patient safety in regards to do the models actually apply to the populations that they're intended to apply to. So because of some of the inherent irregularities in our data, what you see happening is we have an example, for instance, there's a sepsis model that when it rolled out, it was supposed to be like 80% accuracy predicting who was going to get sepsis. The reality is once it was rolled out at the local level, University of Michigan had this study that basically said I think it was 47% accuracy in terms of predicting who was going to get sepsis without prior clinical assessment by the team already.

 

So that's actually less than a coin toss. So we want to just be sure as we look at the patient safety aspect, we look at the data that is being used to train these models to determine whether that data population actually even applies to the population that you're triaging. So I think that there's just also this element of governance and trying to make sure that the results that you're getting out of the models that you are trying to deploy makes sense within your population or whether it's something where you may need to retrain it for your local population.

 

I keep telling people all the time it's like while there are -- what -- eight billion people in the world, and there's probably someone with a similar health condition to you, the reality though is health care is extremely local because the resources are different, the populations are different, and what you have access to is very different. So we have to think about patient safety not just in terms of is somebody going to fall but also in terms of are they getting the appropriate triaging, and are they being appropriately identified by the models.

 

[NOLAND JOINER]      

Yeah, and you speak of data governance. This is something I think, oh my god, all the clients that we work with -- they all have some varying levels of data governance issues, from not having any (chuckling) to not having the right kind of data governance. Ngan, could you continue to talk about that a bit? Tell me what do you think about data governance as it relates to this?

 

[NGAN MACDONALD]  

Well, the thing about AI is the data is the foundation for all AI models.

 

[NOLAND JOINER]

That's right.

 

[NGAN MACDONALD]  

So that means that if you don't manage your data and you're not proud of the data quality that you have, that means that you're not going to be proud of the AI model that comes out. So we have to learn how to do things like be able to quantify where the gaps are in the data and actually measure what bias we're seeing.

 

The problem is every single dataset has bias; but what we don't know is does that bias affect this particular use case or not. It may be okay to have bias in that data if it's something that doesn't actually affect that particular condition or that particular model, and you won't know that unless you've actually put in place processes that actually measure and evaluate what is coming out of your model and also what is the data that is going into your model and what is it that you expect. Measure all of that because the only way we can determine whether or not something is effective is if we measure it.

 

[NOLAND JOINER]

That's right. Ellie, what do you think?

 

[ELLIE GRAEDEN]

Yeah, building on exactly what Ngan was talking about, I think this is -- to really underscore, you have to know where your data come from; and you have to know what those data are, what restrictions are placed on the specific types of data for their different use cases. So if we're trying to get to patient models, we need to be checking for HIPAA compliance all the way through. We need to be checking for bias or doing bias checks all the way through.

 

One of the things that our firm does a fair bit of is actually performing those types of bias checks, not just on the models themselves but evaluating the entire sort of life cycle from data through models to products because it's not enough to just go in and do bias testing in one specific place. We sort of talk about a lot of the guidance issues used right now about being AI-specific. We have people come to us and say, "Oh, but none of these laws are in place yet."

 

It's sort of like, "No, actually we've been dealing with basic EEOC requirements since 1964." We are simply being held -- these models and the data that feed them are being held to the same basic requirements that we've had all along in terms of fair business practices, in terms of unbiased use cases.

 

So it's up to us to make sure that we really understand where the data are coming from, work those and follow those data through the model into their different use cases and evaluate them at each stage. That's where we can get to much better implementation of data governance and actually make it an end-to-end system that's really effective both for the business use cases and therefore, in this context, the patient use case and patient utility, but then also making sure that really we're doing our due diligence, if you will, all the way back to the source.

 

[NOLAND JOINER]      

Exactly, thank you for that. I want to step into the old data-enabled workflow. This is one that's kind of near and dear to my heart, I think, being a reengineering and change management person too. So what do you think, Ellie, about data-enabled workflow? What comes to mind for you there?

 

[ELLIE GRAEDEN]

I come back a lot to, again, what is the specific use case and where can we use very specific types of models for different types of applications. So if you're a hospitalist, it's going to be decision support tools on your phone. Those are the kinds of things that they want to be able to pull them up and actually be able to work through the information they need. This is a sort of code blue type app, is the emergency-level view; but there's basic patient management information that really needs to be tracked.

 

Also, I know this sounds funny, but it's the integration of datasets that really haven't talked to each other, like facilities data with patient-level data.

 

[NOLAND JOINER]      

Exactly.

 

[ELLIE GRAEDEN]

Some of the work that I did during COVID was looking at hospital impacts from hurricanes. When you get evacuation status or power outages, those data and facilities dramatically change the type of care that can be provided. Like if you're evacuating ICU patients, that's a whole level of care provisioning that's dramatically changing. But it turns out those databases, those data systems, don't actually typically integrate with health care data systems or patient-level data. Well, that meant that it was really hard to know if there were, for example, COVID patients or ICU COVID patients being evacuated versus your sort of -- well, your just non-infectious ICU patients being evacuated. Those require different levels of response, but we didn't have that kind of information because we weren't integrating those data.

 

This gets down to hospital operations as well. Again, I think it's about coming up with easily creative ways to do data integration that doesn't sort of impact the privacy and health compliance on the patient level side but that enables those data to be used for practical operational purposes as well. So I think this goes in both directions; but to me, it's about how do you get the right data. This is where we go back to the data governance and the provenance pieces. Pull the right data into the right decision frameworks for the right people, all of a sudden you can dramatically improve everyone's ability to do their job and to serve the patients in the end.

 

[NOLAND JOINER]

Yeah, one of the things you haven't mentioned is we've been talking about this from purely a provider or maybe even a payer point of view but not necessarily from a patient point of view.

 

Do you have any thoughts about that, Ngan, at all--

 

[NGAN MACDONALD]  

Yeah, I mean from --

 

[Noland Joiner]  ...as it relates to -- go ahead, I'm sorry.

 

[NGAN MACDONALD]  

I think from a patient point of view there are very simple things that we could do to data-enable some of those workflows. Just even think about how do you make appointments with your doctor's office. Today, it's very arduous. Like you call a 1-800 number and then you wait on hold. Maybe they call you back, maybe you get through. It feels very hit or miss. What we could be able to do is be able to data-enable that workflow so that there's a smaller set of choices for what is it that you're looking to schedule for and then have the information about what does that mean.

 

The patient shouldn't be expected to know whether am I doing a preventative-measure colonoscopy, or am I doing a diagnostic colonoscopy. That information exists in the EHR, and we just have to be able to integrate it in so that it helps the patient do the things that they're able to do. I always think about when you think about data-enabled workflow, think about how does it augment a human to be able to do their job better. Medication adherence is another one. Do we even know that the prescriptions are getting filled; and if they're not getting filled, then maybe it's a conversation like you serve it up at the right time to the physician to say, "You may want to ask your patient about this."

 

There's probably a way for us to look at labs data and see the levels that people have, and does that correlate to whether or not you're taking your prescriptions or not. I will tell you my mom is 84 years old, and her medication adherence is spotty at best (laughing). So those are ways that are very simple to be able to kind of create that data-enabled workflow from the patient standpoint.

 

Another thing that I want to kind of say is this AI thing -- it's a team sport. It can enable different people, but you have to be able to bring both the clinicians to the table, the technicians, the data scientists; and those are not always people who sit in a room together.

 

[NOLAND JOINER]      

That's true.

 

[NGAN MACDONALD]  

So like how to figure out what is the use case that is going to work for both the patient/for the provider. Maybe the provider isn't the doctor. Oftentimes from a technology company, people who write software think, "Well, I'm just going to interview a couple of actors,' and call that a day. That's not what data-enabled workflow is all about. It's about actually studying what exists today and who are all of the players within that workflow and then making a decision about what pieces of that do we augment versus what pieces of it we can automate.

 

A really crazy one that I -- a friend of mine works for GE Health, and her team uses machine learning algorithms to listen for sounds. When the equipment makes a certain pattern of sounds, they know to send the technician out. So they can proactively deploy somebody to be able to fix the machine before it actually even breaks. I mean, that's not even about privacy. These are cool uses of health care data that doesn't -- it's not a high-risk type of situation.

 

[NOLAND JOINER]

Yeah, that's awesome. As you were talking, I was thinking about other parts of the ecosystem maybe that might include state and local government and federal. Any thoughts about how they might participate in this if you think about it from a federal point of view, from a state and local government point of view?

 

[NGAN MACDONALD]

From the state point of view, obviously most of our public health infrastructure is regulated at the federal level and then implemented at the state level. At the local level, that ability to really collect data, that doesn't necessarily exist within the traditional health care system. I think about a partnership that Mathematica has around the collection of wastewater and monitoring of wastewater to determine where the opioid crisis is going. Like you can see it in the wastewater before you see it in like hospital admissions. So to be able to intervene early as you start to see it perking up and be able to take -- using AI, be able to translate to somebody's own context what it means, what opioids do to you and what are the effects, and maybe to the individual people who actually have sway in that community.

 

That's going to be different for every community. So I think that there's a huge impact that AI can have in the public health sphere.

 

[NOLAND JOINER]

Thank you for that. I tell you, patient safety, data governance, and the whole data-enabled workflow -- they're just the top of a whole list of things that we need to discuss. We could talk about this for hours. I know we already have, and we could continue doing that. But I want to give our listeners kind of some parting thoughts about how you would tell them to get started or what you would recommend them to do if they are in fact considering an AI use case.

 

What do you think, Ellie? Where would you tell our listeners to start?

 

[ELLIE GRAEDEN]

This builds actually a little bit on some of your question about the sort of state and local and federal uses of AI because I think the places I would start -- so first off at a baseline, if someone's thinking about implementing an AI system, the first question has to be, "What problem are you trying to solve?" There has to be this focus on the problem, the outcome, what we're actually trying to do, because that's actually what's going to tell us which type of model is effective and what's going to be the appropriate model or analytics to be applying in this particular case.

 

For example when we're talking about LLMs and these transformer-based system, these are fundamentally probabilistic outcomes and probabilistic models. They are designed to have some outliers in their results. So they can be really effective when we're trying to do something like improving basic communication person-to-person. The translation example is a really good one, whether we're talking about a relatively rare language or whether we're talking about sort of communicating or translating medicalese into human English. We can think about translation in both those contexts.

 

That's a specifically targeted use case for LLMs, but going to LLMs to ask for health care-type information is probably much less of a good idea. You really want question/answer pairs that, for example, have been vetted, validated, and are a more deterministic output. As we move backwards though, we can also start to think about the types of data that are being stored by states that, for example, maybe an early 2000 SQL database that is keyed on the individual, that tells you where everyone is vaccinated; but because it's keyed on the individual, you actually have to get the name to be able to find the neighborhood-level information.

 

But if we can extract that using a model, have neighborhood-level information, we can then marry that up certainly to data coming out of the wastewater system, for example, opioids or a specific disease type that we're looking for. But we can actually also do that now looking at the water sources, where there's actually some new legislation saying, hey, we need to be evaluating water sources for contaminants at the source before going into our water. We could pick things like lead contamination based on downstream impacts on neighborhoods.

 

So it allows us to go full circle, again, as we start getting really tight about what those use cases are. That's where we can say, "Okay, here's this use case. What data are going to be relevant? What types of models will be useful? Where do we have that full chain?" Then, we can start to apply that in very specific domains.

 

I think in what we're seeing, there was this really sort of big push toward broad applications. We're going to have "one AI to rule them all" sort of system. More and more what we're seeing is people applying AI in the context of the way that we've been doing modeling for a long time, which is specific models that are specific -- tailored to very defined use cases. That's actually where we get the most value.

 

[NOLAND JOINER]      

Yeah, a follow-up question is, Ellie, and one of the things I think about quite a bit is that I love the point that Ngan made about this is a team sport. I think a lot about the strategic partnerships that should be considered when building some of these use cases. I think about a lot of the misses, where our clients may not even be thinking about, "Hey, if we team with this organization to leverage these datasets, we could solve these problems together."

 

Any thoughts there? As you look across industry, any thoughts around partnerships and how those partnerships could be leveraged to solve some of those problems?

 

[ELLIE GRAEDEN]

So I'm sitting in a bit of a unique partnership already, right? I am this audible)

 

[NOLAND JOINER]      

There you go.

 

[ELLIE GRAEDEN]

I'm one of the only ones in the country -- I'm a data science partner in a law firm, and the firm was specifically set up and is predicated on the very idea that you need legal and engineering in the room at the same time solving these problems.

 

[NOLAND JOINER]

Yes.

 

[ELLIE GRAEDEN]

I think it certainly moves beyond that as well. It's making sure that you have the voices and the expertise all in the room to do the identification of the problem and then the sort of constructing of the solution. This is where -- one of the things that I've spent a lot of time thinking about actually through my work at Georgetown is how we take policy and turn it into data that can be applied as quantitative modeling parameters and ultimately as engineering specifications. There needs to be some sort of standardized mechanism to apply a law as an engineering spec and then conversely make sure that the engineering systems are communicated effectively to those crafting policy but also implementing legal requirements.

 

[NOLAND JOINER]

Yes.

 

[ELLIE GRAEDEN]

One of the things that I'm doing right now is actually working with a group to figure out what the appropriate data licensing strategies are and what the right contracting language is. It sounds, again, very pedantic and sort of deep in the weeds; but it turns out this is how we could do good that protection that still drives to really effective products. It's about getting tight on which data can we use in which contexts, for whom, and how do we actually then write the kinds of language that protects those data, protects the end users. Those drive towards our commercialization outcomes. So I think it's that ability to get multiple voices in the room, perspectives, and different types of applications that then really drive toward much more productive outcomes in the end.

 

[NOLAND JOINER]

Awesome, thank you for that. Ngan, what do you think?

 

[NGAN MACDONALD]

I think it's interesting because we are literally sitting in this webinar a partnership between a law firm -- Ellie and I both have like joint appointments at a university. I think what we have to realize is as much as there is tech, the underlying importance of trusted partnerships and relationships. If you think about universities and the academic setting, it is very much keyed upon answering these questions. It's not intended to be able to scale.

 

Then we look on the industry side, and a lot of times we don't really think about innovation. We get stymied by innovation because of our need to scale. I mean in my prior role, I was at Blue Cross/Blue Shield, and that's like one in three Americans. So when you're talking about a claim system that has to cover one in three Americans, you want to make sure that every change you make is well-tested. So what that does is it solidifies and creates kind of a brittleness to the system. So how do we kind of think in an innovative way as well as think in a scalable way?

 

The way we do that is we're all kind of coming at it from our different perspectives, which is why I keep saying AI is a team sport. That's why I'm so excited about this health data innovation lab. It's the idea of like we have access to all these different types of data that not all of our partners would have access to. Then, we also have trusted relationships that we build, that we can pull in depending on what the use case is, and be able to make sure that all the people in the room are people who have different viewpoints.

 

Innovation is really about kind of about kind of the collision of all of those different stakeholders. It isn't some lightbulb moment that Einstein thinks about in his garage. That's not realistic innovation. It's not scalable innovation.  

 

[NOLAND JOINER]

One of the things just to kind of pull on that point a little bit is that I think organizations really need to find a safe place to innovate. That's what really excites me about the innovation lab that we have is that some of the best innovation sometimes are done outside of the walls of an organization. Then secondly, when you can find a safe place to go and play with your data, we imagine some of the things you could do with your data. I think it makes the solution that much bigger and bolder, right?

 

So I think that's the big recommendation. Find a great partner, like Mathematica, that you can work with to solve some of these use cases, solve for some of these use cases. So I wanted to go around the horn real quick and ask the question that we asked at the top. Should we pump the brakes, or should we hit the gas?

 

Ellie, what do you think?

 

[ELLIE GRAEDEN]

I don't think we have much choice. I think that the train's moving. I'll start mixing my metaphors here.

 

[NOLAND JOINER]

All right.

 

[ELLIE GRAEDEN]

I think this is about keeping up. I think this is about getting ahead of it wherever we can. I think this about putting our standard practices in place. We've sort of been treating it as sort of this new thing that is one of monolith that we can pump the brakes or pump the gas, and I'm just not sure that that's actually quite the right framing. I think it's moving, and we have a lot that we can put in place to actually institutionalize and make this part of our standard governance practice. It can be part of our standard engineering practice, and we have some new tools that we can apply and some new risks to address as we apply those.

 

But I think a lot of the practice that we already have in place in health care to evaluate new tools, evaluate when and where they should be implemented, make sure that they're doing that well and that they have actually good patient outcomes before we deploy them at scale. I think a lot of those processes are where we need to focus the effort. So it's less on pumping the brakes so much as it is let's just make sure that we're actually doing it well as we deploy.

 

[NOLAND JOINER]

Yep.

 

Ngan, what do you think?

 

[NGAN MACDONALD]

I think if you think about the car analogy, that we do do concept cars first before we actually build a car that rolls out on the production line. I've found another metaphor for my lab. I used to use the pottery place as a place to be able to experiment and play around. But like we build a concept car; it's like one-off. We try it out; we see what works/what doesn't work. To Ellie's point, figure out what use cases are we trying to solve for, determine if that concept actually works. Because I will tell you, sometimes it doesn't work. Sometimes we're using the wrong technology.  

 

There are so many times when somebody says to me, "Well, can we do this with AI?" I'm like, "Huh, why weren't you able to just do that by generating a report and a visualization and being able to identify the outliers. Those are not fancy, but they do the job. So to Ellie's point, we do have tools. This is another tool in the toolbox. We need to take it out and run the test course with it and make sure that we have all the processes in place to determine whether the outcome is what we want and how do we implement it after we've determined that it does work.

 

[NOLAND JOINER]

Yeah, I think I am probably a hit-the-gas person, right? I do think about this quite a bit in terms of how do you hit the gas. Ellie, I love what you said about the bread and butter use cases, the baseline use cases that you go after first. How do we take some of these basic things that we can -- we have the data, and we know we have good data. We can tackle those things, and we can move on.

 

I'm also a big advocate of failing fast. Get out there, test it out, see what happens. I don't want us to put our patients' and members' lives at jeopardy, but let's get out there and let's fail fast, right? Let's make sure we document why we failed so we can reconfigure and keep moving. So I think I am kind of a hit-the-gas kind of person, but I would like for us to do it in those steps, right? Find the baseline use cases we can solve for, fail fast, make sure we document our learnings, and then move forward from there -- so yeah, definitely.

 

I want to take on a few questions from our listeners today and really have us kind of spend the last 10-15 minutes just really tackling or discussing some of those questions. Hold on one second here.

 

All right, this is a really good one here. I like this question a lot, and we've been kind of talking around it a little bit. The question is: "No data will ever be perfect. How do we reconcile that with developing effective AI tools and solutions?" We've been kind of talking around it during the afternoon. So tell me, how do you make sense of that? The data's not perfect, but we need to build an AI tool or solution that's actually going to meet the needs of our audience? How would we move forward there?

 

[NGAN MACDONALD]

I can start. One of the areas that we're really looking at is synthetic data and the ability to use AI in order to generate data that fills in some of the gaps. But we have to come back to the fact that if you don't know that you have those gaps, then you don't know how you're going to remediate for them. I mean within just plain old statistics, we have methods like oversampling of certain data populations so you start to get a set that is representative of what you want it to be, knowing that for some use cases we simply just don't have the data, and maybe those aren’t the use cases that we try.

 

So it goes back to everything we said about understand what your use case is, and that will drive what data you need. Then, you have to understand what's in your data, and you do that by provenance of the data, being able to measure what biases exist. Then, we have methods; and it cannot be understated too. We are working with foundations and state governments and local governments to actually collect additional data. Every day, I hear about additional data that is available, and it can be overwhelming.

 

So there is data out there.  How you don't overwhelm yourself is you ground yourself on the use cases and then look for what data can be used to enable that.

 

[NOLAND JOINER]

So you think in some cases our customers have too much data?

 

[NGAN MACDONALD]

Yes, yeah, because you think about health care, right? There is a ton of data, but a lot of it got collected in order to pay claims. So the things that you document when you're trying to substantiate whether or not you get paid are different things than you document when you're like trying to keep someone healthy or you just want to make sure that you have a chronological history of an individual. So we have systems that don't necessarily reward us for things that don't get paid for to document. So there's huge gaps that we see in the data.

 

(Multiple voices)

 

[NOLAND JOINER]

Go ahead, Ellie. I'm sorry.

 

[ELLIE GRAEDEN]

In terms of data quality and sort of sources of data that we're working with, I think certainly we can talk about it in the context of AI. But data quality issues have been at the heart of most of our health care challenges sort of since the beginning of time, right? Understanding, diagnosing, sharing information, tracking that information through making sure the right person has it at the right time is a major issue and always has been. What we're seeing now is the amplification of those issues. We're moving faster, we're pulling data in, we're analyzing more of it, we're integrating more data sources.

 

So I think what that really comes to is, one, getting back to really good basics on the data governance and making sure that the same data governance and AI governance strategies that you're sort of implementing within your engineering stack groups within operations or within patient care or within every element of your organization and that they're all following the same basic governance structures.

 

That sort of integration is institution-wide governance systems. Haven't necessarily been implement, and I think they go a long way toward making ensure that we're actually just doing the implementation well and that as we start pulling in particularly data from across the institutions and from external sources that we're actually able to do that really effective data governance. I think that's one of these pieces of addressing good governance to support the kinds of hit-the-gas scale that you're talking about, Noland.

 

If we do that, we just need to make sure that we're tracking what we're using and know where it's coming from; but again, to Ngan's point, that's good basic data science. That's how we make sure that the avatar-to-avatar is good, that they're solid, and that therefore the outputs of the model are going to be more reliable. Then, we can also evaluate the intermediate outputs and then final outputs of those models as well for their specific use case.

 

I think a lot of that is in place, and we can actually use some of these methods to evaluate the quality of the data, particularly when we're looking at a synthesis of small numbers. I think the other real value-adds here on the AI type is that we can do integration of data and find data from many more sources that might start to address the small numbers issue when we're getting to unique cases or subpopulations. That particularly if we're deidentifying, effectively verifying that data identification, then we can merge those data and start to look at patient outcomes for these smaller groups of patients.

 

So I think there are a lot of ways that we can actually use these tools to address those issues; but again, the way that we mitigate it is to make sure that we have good organization-wide industry practices that are being implemented and governance practices.

 

[NOLAND JOINER]

Thank you. I was just looking through some more questions, and this one kind of hit me because It's right at the heart, right? The question is: "The medical field tends to show skepticism for new technology, special fields that require a lot of specialized human knowledge. How do we as technologists help to bridge that gap and build trust?"

 

That is a really good question; and I'm wondering, Ngan, do you have any thoughts there about this? Because this is something that you and I kind of live and breathe all the time. How do we build trust? How do we make sure that we have equal partners and help them develop those solutions?

 

[NGAN MACDONALD]

Yeah, I think the biggest piece that technologists can do in order to build trust is realize that the community, the stakeholders, are a part of your journey. It is not a case -- I think in the past what we've done is we've built the tech; and then we've gone and we've tried to apply it to the work and said, "Oh, well I think based on a couple interviews with a couple of documents, I think this is how it might work." That is the reason why we have such a distrust.  

 

There's actually a study that shows the EHR, and one of the things I quoted was, “checkboxes are so crushing.” That was a direct quote from a survey. I thought, "Yeah, absolutely,” because these are people who have spent their entire careers trying to problem solve. So it's not just a checkbox to them; it is a workflow. We get back to that data-enabled workflow. In order to get to trust, you have to bring all the stakeholders together and work with them to figure out what is the actual full workflow, who's involved in it, and then what pieces do we want to automate versus what pieces do we want to augment.

 

Then, you apply the tech. You do all of this work that is not at all technology work first; and then you say, "Okay, well, I think I have something that can solve this particular problem," at the heart of it.

 

[NOLAND JOINER]

I remember in one of our previous assignments I had a quote on the wall behind me that said, "Respect the workflow." Respect workflow.

 

[NGAN MACDONALD]

That's funny.

 

[NOLAND JOINER]

I think when you work with health care professionals, particularly in a provider space, I think that is a very, very critical first step -- is to respect the workflow. Listen to what they're saying and respect it.

 

Ellie, any point of view you want to share?

 

[ELLIE GRAEDEN]

Yeah, to me this question is essentially, "How do you find product/market fit? Are you evaluating whether your end users actually want and need this product? Are you solving a real problem they have?" I think this is, again to Ngan's point, it's not technology first; it's problem set first, What is the problem that can be solved? Where would they like help? What is their priority, and then what is the right technology to help fill those gaps? Because again, these are tools being used by people. So we need to talk to those people. It's good UI/UX testing and design, and it's making sure that then that's aligned with the technology that's going to support it.

 

We can get a lot of that information from the end users. We can sit down with the people who are going to be using these different tools -- whether it's patients or whether it's the providers -- and ask them, "How accurate do you need this to be? How much do you want to be evaluating accuracy? How much do you need to make sure it's exactly right when it comes out, and how do we communicate that end uncertainty to you?"

 

There are some cases where what they need is qualitative uncertainty. Other places, they just want a qualitative sense for it. There are other times when actually they night not care too much if it's accurate if it points them in the right direction, right? Then, there's other times where there's just absolutely no room for error. So identifying really, again, which problem we're solving, getting tight on what therefore the specification of the system is allows us to build technology that will actually meet those requirements.

 

So I think that's how we build that trust, is that we're solving a real problem that they have; and then they're a whole lot more likely to use it, not least because it's useful. I think sometimes we lose oversight of that really. "It's a very cool technology; don't you want to use it?" It's like, well, no, these are not early adopters. They're adverse for a reason. There's a life on the line.

 

[NOLAND JOINER]      

That's exactly right. Speaking of risk, I'm wondering -- one of the questions I saw pop up is really what role should the government play in auditing or enforcing data workers to protect identity in this. Any thoughts about that -- I mean, the government's role about this? I would be curious.

 

Ngan, what do you think?

 

[NGAN MACDONALD]

Yeah, well, one of the things which I advocate for -- which I'm sure is not entirely feasible -- is we don't have a national privacy law. That makes it difficult for people to address privacies state by state. Any piece of software you roll out is you're basically trying to figure out whether or not it's going to violate a law in a specific state. So I think that there's a role for a comprehensive national privacy law. But to the point, I think I'll probably turn this over to Ellie to kind of talk about that we do have some things on the books already that are effective.

 

[NOLAND JOINER]      

Ellie?

 

[ELLIE GRAEDEN]

Yeah, and actually that question, Noland, dovetails really nicely with a question I'm seeing that are sort of presented to the panel from the audience as well which is asking about liability. This really gets us back to what Ngan was describing. So first off, we're talking about things like a national privacy law. I think it's also really important to expand that to include a broader data governance structure. There's very much this necessary privacy lens, and we can expand that to be more inclusive of this broader sort of scope of what data protections are needed.

 

I think as we start to expand that and think about how to implement that from a governance perspective or policy perspective, where we can focus is really how are these tools being used? What are the outcomes, and what is the appropriate regulatory framework and risk mitigation that aligns to those specific use cases? You have specific types of data, eye-tracking in a VR headset, for example, that as we're thinking eye tracking data, eye tracking data can be consumer data that are focused on how the headset is used and whether you're clicking on an app.

 

But it can also be used -- those exact same data can be used as health data or as health indicators. So we need to be thinking about regulating maybe not just the data themselves but actually those specific use cases of those data. That's where we can then get to applying our existing regulatory framework and just making sure that we're implementing those and applying those to these AI models and to these AI products as well. So this is where I think current liability framing and current liability regulations do apply and should apply here. That's a matter of going back through our regulatory domains and simply applying the existing laws to this new technology.

 

We're actually seeing that happen already. The FTC is doing a lot of saying that and saying, "Look, you can't have unfair business practices applied. Whether you're using an AI model or any other system, it's still an unfair business practice." They're simply applying that bar or threshold to these new types of products. So I think it's a "yes and..." We're going to need some new regulation that really does lay out requirements and standardizes those.

 

To your point, Ngan, it's really hard to do business in an environment where every state has their own details or differences. We also then just need broad application of the laws that are already in place.

 

[NGAN MACDONALD]  

Yeah, I mean I think we have like privacy laws and trademark laws and all of these that is currently being adjudicated. What we can't have is this weird exception that we've had in the past of, "Well, I'm a technology company; and I need to be able to innovate and grow. So therefore all of the regular common-sense regulations don't apply to me." That has been what has happened with technology so far, and so we have to be careful as practitioners in this space that we are accepting that there is a level of responsibility that technology has to work by the same rules of the road that someone who manufacturers a caterpillar makes or a semitruck.

 

[NOLAND JOINER]      

This has been an incredible discussion. I really enjoyed it. Ellie, I always enjoy talking to you and Ngan and getting your ideas and just hope that we can do this again. I want to close this out and really thank our participants for coming on -- thank our panelists for coming on, but thank our listeners that we’ve had join us. The one thing I would like to leave with is at Mathematica, we are data experts; and that's what we do. Data is foundational to building any AI use case. I think start with us first before starting with the use case. I think we can help you understand your data, contextualize your data, and really help you understand that this is something you should be doing in your organization.

 

So I want to just -- Ellie, thank you very much for joining us. I really appreciate it.

 

Ngan, thank you very much.

 

I would like to close out here and wish everyone a happy afternoon. Thank you.

 

[ELLIE GRAEDEN]

Thank you, all.

[J.B. WOGAN]

Thanks again to our guests for this episode, Noland Joiner, Ngan MacDonald, and Ellie Graeden. You can learn more about them and their work in the episode show notes. As always, thank you for listening to On the Evidence, the Mathematica podcast. If you liked this episode, please consider leaving us a rating and review wherever you listen to podcasts. To catch future episodes of the show, subscribe at Mathematica.org/ontheevidence.