In endocrinology, artificial intelligence tools help predict glucose trends, automate insulin delivery, interpret retinal images, assess thyroid nodules, estimate fracture risk, draft clinical notes, and manage administrative tasks. As these technologies take on a greater role in patient care, they also raise important questions about how AI may reshape clinical judgment, physician oversight, and responsibility. 

Few physicians are as well positioned to examine these questions as Steven Petak, MD, JD, MACE, FACP. Dr. Petak is the retired chief of endocrinology at Houston Methodist Hospital and earned a law degree from the University of Houston Law Center. He has held academic appointments at Weill Cornell Medical School and Texas A&M Medical School, consulted for NASA’s Johnson Space Center, and is a past president of AACE.  

His decision to study law grew from his experience as a physician. “Medicine is practiced inside of the legal and regulatory environment, whether physicians recognize that or not,” says Dr. Petak. “I went to law school to become a better physician, not to stop being one.” 

In part one of his conversation with Johnson Thomas, MD, FSCE, FEAA, Editor-in-Chief of AACE Endocrine AI, Dr. Petak discusses how AI is already being used in endocrine practice and what clinicians should understand about the evolving medical-legal implications of AI. 

(Note: The information provided in this interview is for general informational and educational purposes only. It is not intended to constitute and should not be relied upon as legal advice. The following transcript has been edited for clarity and length.)  

Dr. Thomas: During my interactions with other endocrinologists, when I talk about AI, they sometimes say that they are not using AI these days. So, is this true? Are we using AI in our clinical practice, so people who don't think they're using AI are unknowingly using these algorithmic or AI tools? What do you think? 

Dr. Petak: Well, they're using AI whether they know it or not, even if the product doesn't say AI. I mean, continuous glucose monitors use prediction of trends. There are automated insulin delivery systems. There are retinopathy systems that interpret images. Ultrasound software scores nodules. There are molecular platforms that look at genomic signals on thyroid cancer risk. Fracture tools and opportunistic CT are all fracture risk estimation tools that most of us do use. And in addition to the clinic, we're using ambient scribes, inbox drafting, and prior authorization tools. It adds another layer of AI that we're oftentimes using and don't even know it. Not every algorithm is technically AI, but the question is, what role does it play in your practice? And it's probably there. If you have software that predicts, prioritizes, or recommends, it's part of the healthcare system, whether or not it says it's AI. 

Dr. Thomas: You've been a practicing endocrinologist for quite some time, and you've seen all these cases. So, across diabetes, thyroid nodules, and fracture risk, which one do you think carries the greatest near-term legal exposure for the practicing endocrinologist, because they are using it most frequently, or because of the risk associated with the decision? And why do you think so? 

Dr. Petak: Well, clearly, diabetes is first, because of potential severity, automated insulin delivery systems, the idea of delivering a high-dose drug with a delivery system that can under or overdose. Concerns over ketoacidosis or hypoglycemia are really major issues that are concerning in the diabetes world. There's major pump recalls that have already occurred. This doesn't mean device failures make the physician liable, but the exposure of the physician is still there, because we do control patient selection and education, backup plans, monitoring, and what we do when things don't go according to plan. So, I think that diabetes is clearly first. Because of the use of this high-risk medication, with these algorithms that are now controlling insulin use in many patients that are on pumps, for example. 

I think that the other areas of concern are the ambient scribes, the generative documentation. People have inaccurate notes sometimes, omitted findings, privacy issues, and HIPAA (Health Insurance Portability and Accountability Act).  And copied recommendations. Sometimes these notes are simply copied over from visit to visit. That is a recipe for disaster. So, I think that AI is oftentimes part of the problem with these notes that people are generating and repeating. 

Dr. Thomas: There are some AI tools that direct you to the right source or support a physician's judgment. Or there are tools like pumps that actually make that decision. So, where is the line in endocrinology right now? Do you worry that people are really paying attention to AI as an assistant, or AI making the actual change for the patient? 

Dr. Petak: The practical issue is whether clinicians can independently review the basis for the output and have an opportunity to change the plan before harm can occur. FDA has certain standards for clinical support software, for example, both in the diabetes world as well as fracture risk estimations; those are supportive roles. 

But a pump that adjusts insulin repeatedly is delegated action, and that's where the problems occur. So, once you have delegated action, AI is now working as an agent, and it's working independently of what a physician might be controlling. The same thing occurs with autonomous retinopathy screening, because oftentimes it'll return a clinical result without a specialist actually reviewing it. I think the position-in-the-loop part of this is becoming blurred. Having a rubber stamp where the position is simply signing off on a decision that AI is making is not going to work very well. When AI systems start to take over position judgment, then we're in trouble. 

These AI systems have been questioned in the insurance world as well. There's an important case that's in the courts right now in which an AI system was designed to supplant position decision-making without the physicians being aware of it. And this is something that the courts are going to find litigating more and more. So, you must have people in that loop, and AI can be supportive, but it cannot be an independent agent in that regard. 

Dr. Thomas: That's interesting, too, because I do a lot of thyroid biopsies, and I manage thyroid cancer patients, and many times when it comes back indeterminate, we use molecular markers. And it might come back as currently benign or X percentage of malignancy, and when we usually see it as benign, usually we sign off on that and go our merry way. What do you think about that? And that's also based on some algorithms. 

Dr. Petak: Yeah, the problem there is that you have to have a plan for follow-up. The patient needs to know that these tools have limitations, and if the patient has a relatively benign result from a tool that is evaluating this. The critical part is that you have a follow-up plan. You don't want to just say, you're good, don't need to come back. I don't think that is a good result. I think that you have to say, well this is based on what we know right now. We're going to monitor it. It looks like the risks are low, but you definitely have to work on that follow-up part, so don't sign off on those cases.