American College of Physicians Releases Recommendations for AI Tools in Healthcare

Posted by Carolann Curry on June 24th, 2024 Posted in: Blog, Data Science, Technology
Tags: artificial intelligence, Data science

---

On June 4, the American College of Physicians (ACP) released recommendations for use of Artificial Intelligence (AI) and and Machine Learning (ML) tools in clinical and healthcare settings. The article, published as a position statement online ahead of print in the journal Annals of Internal Medicine, outlines 10 recommendations for using these tools in clinical settings and ultimately calls for more research on clinical and ethical implications of these technologies on patient health and wellbeing. Their policy recommendations emphasize the potential benefits of AI while also addressing the challenges and risks associated with its implementation.

ACP Recommendations

1. ACP firmly believes that AI-enabled technologies should complement and not supplant the logic and decision making of physicians and other clinicians.
2. ACP believes that the development, testing, and use of AI in health care must be aligned with principles of medical ethics, serving to enhance patient care, clinical decision making, the patient–physician relationship, and health care equity and justice.
3. ACP reaffirms its call for transparency in the development, testing, and use of AI for patient care to promote trust in the patient–physician relationship. ACP recommends that patients, physicians, and other clinicians be made aware, when possible, that AI tools are likely being used in medical treatment and decision making.
4. ACP reaffirms that AI developers, implementers, and researchers should prioritize the privacy and confidentiality of patient and clinician data collected and used for AI model development and deployment.
5. ACP recommends that clinical safety and effectiveness, as well as health equity, must be a top priority for developers, implementers, researchers, and regulators of AI-enabled medical technology and that the use of AI in the provision of health care should be approached by using a continuous improvement process that includes a feedback mechanism. This necessarily includes end-user testing in diverse real-world clinical contexts, using real patient demographics, and peer-reviewed research. Special attention must be given to known and evolving risks that are associated with the use of AI in medicine.
6. ACP reaffirms that the use of AI and other emerging technologies in health care should reduce rather than exacerbate disparities in health and health care. To facilitate this effort:
   1. ACP calls for AI model development data to include data from diverse populations for which resulting models may be used.
   2. ACP calls on Congress, HHS, and other key entities to support and invest in research and analysis of data in AI systems to identify any disparate or discriminatory effects.
   3. ACP recommends that multisector collaborations occur between the federal government, industry, nonprofit organizations, academia, and others that prioritize research and development of ways to mitigate biases in any established or future algorithmic technology.
7. ACP recommends that developers of AI must be accountable for the performance of their models. There should be a coordinated federal AI strategy, built upon a unified governance framework. This strategy should involve governmental and nongovernmental regulatory entities to ensure:
   1. the oversight of the development, deployment, and use of AI-enabled medical tools
   2. the enforcement of existing and future AI-related policies and guidance; and
   3. mechanisms to enable and ensure the reporting of adverse events resulting from the use of AI.
8. ACP recommends that in all stages of development and use, AI tools should be designed to reduce physician and other clinician burden in support of patient care.
9. ACP recommends that training be provided at all levels of medical education to ensure that physicians have the knowledge and understanding necessary to practice in AI-enabled health care systems.
10. ACP recommends that the environmental impacts of AI and their mitigation should be studied and considered throughout the AI cycle.

Reference
Daneshvar N, Pandita D, Erickson S, Sulmasy LS, DeCamp M; ACP Medical Informatics Committee and the Ethics, Professionalism and Human Rights Committee. Artificial Intelligence in the Provision of Health Care: An American College of Physicians Policy Position Paper. Ann Intern Med. 2024 Jun 4. doi: 10.7326/M24-0146. Epub ahead of print. PMID: 38830215.