As artificial intelligence (AI) rapidly transforms healthcare, a new report is urging the U.S. Food and Drug Administration (FDA) to bolster its oversight of AI-powered medical tools to ensure patient safety and ethical use. The report, published this week in the open-access journal PLOS Medicine, argues that the current regulatory framework struggles to keep pace with the unique challenges posed by these evolving technologies.
Unlike traditional medical devices, many AI tools continuously learn and adapt after they are approved for use. This constant evolution, while potentially beneficial, can also lead to unpredictable changes in their performance and potential biases, according to the report’s authors, led by Leo Celi of the Massachusetts Institute of Technology.
“AI is becoming increasingly powerful in healthcare, aiding doctors in diagnosis, monitoring, and even treatment recommendations,” Celi explained. “However, because these tools can change over time, we need a more agile and transparent regulatory system to ensure they remain safe and effective for all patients.”
The researchers highlight specific areas where the FDA’s current system falls short, including:
- Post-Market Surveillance: The FDA’s existing framework isn’t designed to effectively monitor the ongoing performance and potential biases of AI tools after they’ve been released into clinical practice.
- Transparency and Bias: Algorithms trained primarily on data from specific populations may not perform accurately for other groups, potentially exacerbating existing healthcare disparities.
- Ethical Considerations: The report emphasizes the need for ethical guidelines and practices in the development and deployment of AI in healthcare.
To address these concerns, the authors propose a series of practical solutions, including:
- Mandatory Transparency: Requiring developers to disclose detailed information about how their AI models were trained and tested.
- Patient and Community Involvement: Involving patients and community advocates directly in the FDA’s decision-making processes.
- Public Data Repositories: Creating centralized databases to track the real-world performance of AI tools.
- Incentivizing Ethical Practices: Offering tax incentives to companies that adhere to ethical guidelines in AI development.
- Training the Next Generation: Educating medical students on how to critically evaluate and use AI tools responsibly.
“We need a patient-centered, risk-aware, and continuously adaptive regulatory approach,” the authors state in their report. “This ensures AI remains an asset to clinical practice without compromising safety or worsening inequalities in healthcare.”
The authors hope their analysis will prompt the FDA to re-evaluate its existing oversight mechanisms and adopt a more proactive approach to regulating AI-enabled medical technologies. This includes continuously monitoring approved AI tools and quickly adapting the regulations as needed.
The full report, “Oversight of artificial intelligence in medicine,” is freely available in PLOS Medicine and can be accessed here: https://plos.io/3HgQkja