Data-first: AI governance a fundamental pillar in the hospital pharmacy

Michael Palone: Regarding data and AI, we can draw a lot from clinical information more broadly. Over time in healthcare, we’ve built a good understanding of how to assess clinical tools; we rely on expert content, so in the pharmacy instance, we may have 10 or 15 of our most-vetted, best evidence-based tools that we use – whether it's dose adjustments or applicability for antibiotics to guide medication management.

I think the same principles apply to AI, which is what's the data source? We learned a lot during Covid that all medical information is not good medical information, and now, with it all at our fingertips, it's all the more important that we point these AI tools at vetted information, and the higher quality information that we have, the better our outcomes will be.

As it relates to data integrity, we need to have transparency with our partners, and to challenge vendors on what their pool of data is, and what are we feeding the AI to help give us good information and ensure it performs accurately.

Clinical information is not universal; we know that there's genomic differences for medications and that what works for one individual may not work for someone else within different culture, ethnicity or sex, so the sufficiency of the pool of data becomes a key question. And then, naturally, the backbone of everything today is security, and understanding whether data is being used well and ethically, and if it's being combined with other data, ensuring it's not identifying individuals on a personal level.

Michael Palone: As a caregiver and a patient, we want the data to be as vetted and as accurate as possible, and sometimes I worry about the speed of adoption and making sure that we're not putting the engine ahead of the dataset. There’s all this great work being done on the speed and the power of our engines, but I don't hear as much as I'd like to about the data source. And if we put those two things together, a really great data source with good quality and security, combined with all the incredible things being done with engines, then we have something massively powerful. But if either of those two things aren't working well together, then we have a mess.

Michael Palone: I think so. My colleagues would make the argument that we benefit from good reference data, and AI bringing that to us and not making us search. You might go through two or three different resources to get an answer to a clinical problem. It's wonderful if AI can help us sum up that good information, and then we can make the final determination on whether or not it's applied. So, we speed up the process by using AI, but as clinicians, we are ultimately responsible for making the decision, and we can’t deflect that onus to an AI tool.

Michael Palone: ​​​​​​​If we lean into what we've learned about clinical research studies, most organisations globally that are involved in these sorts of studies, whether drug studies or others, have well established protocols such as having good documentation and regular reviews.

I think we can adopt what we've learned about clinical research over the last 20-30 years to AI. It's not exactly the same, but if you think about some of the concerns, risks, and sensitivities around clinical research, they can readily be transposed to an approach towards AI’s implementation. I think you need to have the people on the ground, whomever is using the tool from the real world experience side, along with IT security teams and, CIOs, and vendor partners. That's how we'll make these tools really valuable.

Michael Palone: ​​​​​​​When we hear from our customers, the one thing that strikes me is that if I ask a pharmacy director anywhere in the world what their number one challenges are, it's drug shortages and staff.

Everybody's struggling with the same thing, and drug shortages require an awful lot of time to manage, so this is an example where AI and good algorithms can go a long way to being a good tool for the buyers in pharmacies.

There’s little regulatory guidance on these things, but organisations are getting better about reacting.  Being more proactive means that they may be able to pivot and ensure that they have some parallel clinical treatments for something that's not available. But if you can't get a bag of normal saline, that's hard to replace, and AI won't solve that problem for you.

We typically manage our inventory in a very old school manner in hospitals. We have expected stock levels and we replenish to that stock level, but care is much more dynamic than that today, and that's where AI can really make a difference.

Michael Palone: In areas like clinical research, there is less actual regulation, and more of a focus on best practices the industry adopts. I expect that AI will probably follow a similar course, especially since it’s difficult to get lawmakers to understand this area given it’s not their strength. Organisations within the pharmacy industry such as the Society for Hospital Pharmacy have already offered up guidance surrounding key considerations for AI governance, and over time, I think it will evolve into good standards of practice if we’re doing it right. If we have harm, then that is when lawmakers often respond and move towards legislation, but if we're doing it right, we can use our professional organisations in a positive way to handle such governance functions.