'Artificial intelligence for health insurance: A proposed framework for FDA oversight' by Renee Sirbu, Jessica Morley and Luciano Floridi comments
Despite mounting enthusiasm regarding the introduction of artificial intelligence (AI) software as a medical device (SaMD) to clinical care and, consequently, the development of a new regulatory proposal for the federal oversight of AI/ML medical devices, little attention has been paid to the oversight of AI tools used by large insurers. The U.S. Food and Drug Administration (FDA) has advanced an “Action Plan” for clinical AI (CAI) governance. However, the U.S. healthcare system remains threatened by the unregulated application of insurance AI (IAI). In this article, we use IAI tools in the Medicare Advantage (MA) prior authorization pathway as an illustrative case to argue that these technologies require further regulatory attention by the FDA. Specifically, we propose a redefinition of “medical device” under the 21st Century Cures Act as necessarily inclusive of IAI and advance an actionable framework for FDA oversight in the approval of IAI tools for deployment by large healthcare insurers.
'AI as Agency Without Intelligence: On ChatGPT, Large Language Models, and Other Generative Models' by Luciano Floridi in (2023) Philosophy and Technology comments
The article discusses the recent advancements in artificial intelligence (AI) and the development of large language models (LLMs) such as ChatGPT. The article argues that these LLMs can process texts with extraordinary success and often in a way that is indistinguishable from human output, while lacking any intelligence, understanding or cognitive ability. It also highlights the limitations of these LLMs, such as their brittleness (susceptibility to catastrophic failure), unreliability (false or made-up information), and the occasional inability to make elementary logical inferences or deal with simple mathematics. The article concludes that LLMs, represent a decoupling of agency and intelligence. While extremely powerful and potentially very useful, they should not be relied upon for complex reasoning or crucial information, but could be used to gain a deeper understanding of a text’s content and context, rather than as a replacement for human input. The best author is neither an LLM nor a human being, but a human being using an LLM proficiently and insightfully.
