Introduction
Artificial intelligence (AI) is becoming one of the most influential technologies in modern healthcare. AI systems are now being developed to assist doctors with medical imaging, diagnosis, patient monitoring, clinical documentation, treatment decisions, and other healthcare activities. As these technologies become more capable, governments and regulatory agencies face a difficult challenge: how can they encourage innovation while ensuring that medical AI is safe, effective, reliable, and trustworthy?
In the United States, the U.S. Food and Drug Administration (FDA) plays a central role in regulating medical devices. The agency has already authorized a large and growing number of AI-enabled medical devices. However, the rapid development of generative artificial intelligence (GenAI) is creating new regulatory questions because these systems can produce different outputs in response to different inputs and may be more flexible, complex, and difficult to evaluate than traditional medical software. U.S. Food and Drug Administration+1
On August 18, 2026, the FDA released a discussion paper examining possible approaches for regulating generative-AI-enabled medical devices. The agency also opened a public docket to receive feedback from stakeholders. The paper explores whether some aspects of evaluating medical AI should be based on a competency-based approach, with comparisons to how medical professionals are assessed. U.S. Food and Drug Administration+1
This proposal represents an important development in medical technology regulation. It recognizes that AI systems are not always like traditional medical devices and that regulatory methods may need to evolve alongside technological capabilities.
What Is Medical Artificial Intelligence?
Artificial intelligence refers to computer systems capable of performing tasks that normally require human intelligence. These tasks can include recognizing patterns, interpreting information, generating text, making predictions, and supporting decision-making.
In healthcare, AI can be used in many different ways.
For example, an AI system may examine an X-ray or CT scan and highlight areas that could require a doctor’s attention. Another system may analyze an ECG and identify patterns associated with possible heart disease. AI can also assist with administrative work, summarize medical records, support clinical decision-making, or help researchers discover new medicines.
The FDA maintains an AI-enabled medical device list that provides information about AI-enabled devices authorized for marketing in the United States. The agency says the list can help healthcare providers and patients understand the growing landscape of AI-enabled medical technologies. U.S. Food and Drug Administration
Traditional medical AI systems have often been designed for relatively specific tasks. For example, an algorithm might analyze an image and identify a particular abnormality.
Generative AI is different.
What Is Generative AI?
Generative AI refers to systems that can create new content based on information they receive. This may include text, images, audio, recommendations, or other outputs.
Large language models are one example of generative AI. These systems can generate human-like responses to questions and can perform a wide variety of tasks.
In medicine, generative AI could potentially support activities such as summarizing patient records, assisting with clinical documentation, answering medical questions, generating recommendations, and interacting with patients.
However, the flexibility that makes generative AI powerful also creates regulatory challenges.
A traditional medical software program may produce a relatively predictable output for a specific input. A generative AI system can potentially produce different responses depending on context, wording, patient information, or other factors.
The FDA’s new discussion paper specifically recognizes that GenAI-enabled medical devices can have characteristics and risks that differ from traditional software and AI-enabled medical devices. U.S. Food and Drug Administration
Why Does the FDA Need a New Approach?
The main reason is that medical AI is evolving faster than many traditional regulatory frameworks were designed to handle.
Traditional medical-device evaluation generally focuses on demonstrating that a product is safe and effective for its intended purpose. For a fixed-function device, developers can conduct testing using defined performance measures.
Generative AI complicates this process because it may be capable of performing multiple tasks and producing variable outputs.
For example, consider an AI medical assistant that helps summarize a patient’s medical history. If the system consistently performs one narrowly defined task, regulators can establish specific tests for that function.
But if the same system can summarize records, answer questions, suggest possible diagnoses, generate treatment-related information, and interact with healthcare professionals, evaluating every possible output becomes much more complicated.
The FDA’s discussion paper therefore considers whether regulatory evaluation should focus more heavily on the competencies and risks associated with what the AI system is capable of doing. U.S. Food and Drug Administration+1
The Competency-Based Approach
One of the most interesting ideas in the FDA discussion paper is a possible competency-based evaluation approach.
The basic concept is that instead of evaluating a generative AI system only as a conventional piece of software, regulators could assess what the system is capable of doing and how well it performs those activities.
This idea has been compared to the way human clinicians are assessed.
A medical student, for example, is not judged solely on whether they can memorize medical information. Their competency may be assessed through examinations, practical demonstrations, clinical scenarios, and other forms of evaluation.
Similarly, a generative AI medical device could potentially be evaluated according to specific competencies relevant to its intended medical function.
This does not mean that AI would literally receive a medical license like a doctor. Rather, the comparison provides a framework for thinking about how complex AI systems could be tested.
The FDA discussion paper notes that some researchers and experts have advocated approaches inspired by the evaluation and credentialing of human clinicians because GenAI can be broad, adaptive, and difficult to evaluate using conventional methods. U.S. Food and Drug Administration
Risk Should Determine the Level of Evaluation
Another important part of the proposed framework is the idea that regulatory requirements should be related to risk.
Not every AI medical application creates the same potential harm.
For example, an AI tool that helps organize administrative information may present relatively low medical risk.
An AI system that helps interpret a medical scan could have greater consequences if it makes an incorrect recommendation.
A system that directly influences a critical treatment decision could potentially present even greater risk.
The FDA’s discussion paper explores a framework that considers both the capabilities of the device and the potential risks associated with incorrect outputs. Axios+1
This approach could allow regulators to apply stronger evaluation requirements to higher-risk applications while avoiding unnecessary barriers for lower-risk technologies.
Pre-Market Evaluation
One major area of concern is how generative AI should be evaluated before it reaches the market.
Traditional medical devices generally undergo premarket evaluation designed to demonstrate that they meet applicable safety and effectiveness requirements.
For generative AI, developers may need to demonstrate performance across a wide variety of scenarios.
Testing could potentially include:
- Accuracy of medical information
- Ability to recognize dangerous situations
- Consistency of responses
- Performance across different patient populations
- Ability to handle unusual or unexpected cases
- Resistance to misleading inputs
- Appropriate communication of uncertainty
- Protection against harmful recommendations
The goal would be to ensure that a medical AI system does not simply perform well in ideal conditions but remains reliable when exposed to the complexity of real-world healthcare.
Post-Market Monitoring
Testing before a product is released is only one part of the regulatory challenge.
AI systems can change over time. Some may be updated, retrained, or modified after deployment. The way users interact with them may also change.
This means that regulators may need to monitor AI systems after they enter clinical practice.
The FDA has already been examining methods for measuring and evaluating the real-world performance of AI-enabled medical devices, including approaches for detecting performance changes over time and mitigating potential problems throughout the device’s life cycle. U.S. Food and Drug Administration
Post-market monitoring could include collecting information about:
- Unexpected errors
- Patient safety incidents
- Changes in performance
- Differences between patient populations
- Software updates
- New use cases
- User complaints
- Bias or disparities
- Cybersecurity problems
This is particularly important for AI because a system’s performance in the real world may differ from its performance during development and testing.
The Problem of AI Hallucinations
One major concern with generative AI is the possibility of hallucinations.
In AI terminology, a hallucination occurs when an AI system generates information that appears convincing but is incorrect or unsupported.
In healthcare, this can be particularly dangerous.
If an ordinary chatbot produces an incorrect answer to a general question, the consequences may be relatively limited. But if a medical AI system provides incorrect information about a patient’s condition, medication, or treatment, the consequences could be much more serious.
For this reason, medical AI requires a higher standard of reliability than many consumer applications.
A regulatory system must therefore consider not only whether an AI system is usually accurate but also how it behaves when it is uncertain or wrong.
The Importance of Transparency
Another major issue is transparency.
Healthcare professionals need to understand how and why AI is being used in clinical settings.
If an AI system provides a recommendation, doctors may need information about the system’s intended use, known limitations, validation, and performance.
Transparency can also help patients make informed decisions.
The FDA’s broader work on AI-enabled medical devices reflects the need for developers to provide information that supports safety and effectiveness throughout the product lifecycle. U.S. Food and Drug Administration
However, transparency is challenging because some advanced AI systems can be difficult to interpret even for their developers.
This creates an important question for regulators: How transparent does an AI system need to be before it can safely be used in healthcare?
Bias and Fairness
AI systems learn patterns from data. If the data used to develop an AI system does not adequately represent different populations, the resulting technology may perform differently for different groups.
This creates concerns about fairness.
For example, an AI system trained primarily using data from one demographic population might not perform equally well when used with another population.
Medical AI therefore needs to be evaluated across diverse groups of patients.
Regulators and developers need to consider factors such as age, sex, geographic region, medical history, and other relevant characteristics when determining whether a system performs reliably.
The objective is not simply to create an AI system that is accurate on average. It should also perform safely and effectively for the patients who are actually going to use it.
Benefits of Stronger AI Regulation
Although regulation can create additional requirements for technology developers, effective regulation can also encourage responsible innovation.
Patients are more likely to trust medical AI when they know that it has been evaluated according to established safety standards.
Doctors may also be more willing to adopt AI when there is reliable evidence showing that the technology works in real clinical environments.
Strong regulation can therefore help create a balance between innovation and patient safety.
If regulatory requirements are too weak, unsafe systems could reach patients.
If requirements are unnecessarily strict, useful technologies could take too long to reach healthcare professionals.
The FDA is attempting to find a balance that allows innovation while addressing the unique risks associated with advanced AI. U.S. Food and Drug Administration
Challenges for Technology Developers
The proposed approach could also create challenges for companies and researchers developing medical AI.
Developers may need to conduct more extensive testing and provide stronger evidence of performance.
They may also need systems for continuous monitoring after deployment.
For generative AI, developers must consider a large range of possible outputs rather than testing only a small number of fixed responses.
This could increase development costs and make regulatory preparation more complicated.
At the same time, clearer regulatory expectations could help companies understand what evidence they need to provide before launching new products.
The Importance of Human Oversight
Even as AI becomes more capable, human oversight remains an important part of safe medical practice.
AI can process enormous quantities of information quickly, recognize patterns, and provide recommendations. However, healthcare decisions involve more than information processing.
Doctors must consider patient preferences, medical history, social circumstances, symptoms, physical examinations, and other factors.
AI systems may therefore work best as tools that support healthcare professionals.
A doctor might use AI to identify a potential abnormality, summarize a large patient record, or suggest possible considerations. The healthcare professional can then evaluate that information and decide what action is appropriate.
The FDA’s regulatory discussion is taking place against a broader debate about how increasingly capable AI should be incorporated into healthcare without compromising patient safety.
The Future of Medical AI Regulation
The FDA’s August 2026 discussion paper suggests that regulation itself will need to evolve as AI technology changes.
Rather than creating a single set of rules that applies equally to every AI system, regulators may increasingly consider:
- What the AI is capable of doing
- How it is intended to be used
- How much harm an incorrect output could cause
- How predictable its behavior is
- How it performs across different populations
- How it will be monitored after deployment
- How frequently the system changes
This could result in a more flexible and risk-based regulatory framework.
The FDA is currently seeking feedback rather than announcing a final regulatory framework. The discussion paper is therefore best understood as an exploration of possible future approaches. U.S. Food and Drug Administration
Global Importance
The FDA’s decisions could influence medical AI regulation beyond the United States.
AI medical technologies are increasingly developed and marketed internationally. Different countries and regulatory agencies are therefore facing similar questions about safety, effectiveness, transparency, and accountability.
A regulatory model developed in the United States could potentially influence international discussions and encourage other countries to develop comparable approaches.
At the same time, international cooperation may become increasingly important because AI systems can be developed in one country and deployed in another.
Common principles could make it easier to evaluate technologies while maintaining strong patient protections.
Conclusion
Artificial intelligence has the potential to transform healthcare, but its rapid development also creates difficult regulatory challenges.
The FDA’s August 18, 2026 discussion paper on generative-AI-enabled medical devices represents an important step toward addressing those challenges. The agency recognizes that generative AI can be more flexible, adaptive, and unpredictable than traditional medical software and that these characteristics may require new approaches to evaluation. U.S. Food and Drug Administration+1
One of the most notable ideas is a possible competency-based approach, which would focus on what an AI system can do and whether it can perform those functions safely and effectively. The FDA is also considering how risk should influence regulatory requirements and how AI systems should be monitored after they enter the market. Axios
The future of medical AI will depend not only on technological progress but also on responsible regulation. AI systems must be accurate, reliable, secure, fair, and appropriate for their intended medical uses.
The challenge for regulators is to avoid two extremes: allowing potentially unsafe technologies to reach patients without adequate evaluation, or creating such restrictive requirements that beneficial innovations cannot reach healthcare professionals in a reasonable amount of time.
Ultimately, the goal should be to create a healthcare environment in which AI can provide its benefits while patients remain protected from unnecessary risks.
The FDA’s current initiative is still under discussion, but it highlights an important reality: as medical AI becomes more advanced, the way we evaluate and regulate it must become more advanced as well.
The success of future medical AI will therefore depend on cooperation among regulators, doctors, researchers, technology companies, and patients. By combining innovation with evidence-based oversight, healthcare systems can work toward a future in which artificial intelligence supports better medical decisions while maintaining the safety, trust, and human responsibility that patients deserve.