The TGA report Clarifying and strengthening the regulation of Medical Device Software including Artificial Intelligence states
In the 2024-25 federal Budget, the Australian Government provided $39.9 million over 5 years for the development of policy and capability across government to support Safe and Responsible AI. The measure includes work to clarify and strengthen existing laws and address risks and harms from Artificial Intelligence (AI) through an immediate review of priority areas, including health and aged care sector regulation, Australian consumer law, and copyright law.
As part of the Australian Government’s Department of Health, Disability and Ageing (the Department), the Therapeutic Goods Administration (TGA) regulates therapeutic goods, including software and AI models and systems when they meet the definition of a medical device under the Therapeutic Goods Act 1989. Software-based medical devices (including AI models and systems) have been regulated by the TGA for many years. In 2021, we clarified the classification levels of software to account for the potential and emerging risks of harm associated with software, and introduced a number of “carve-outs” for very low risk products or products that had oversight from other regulators. With input from relevant industry stakeholders, we published guidance about our refined regulatory framework, setting out how regulatory requirements apply to these kinds of devices. Since that time, the TGA has monitored the refinements to identify when further review and adjustment was required, including to address emerging risks as technology like AI is rapidly adopted and deployed in healthcare settings.
In 2024, the TGA conducted a review in tandem with the Department’s broader review of health and aged care legislation, to: • determine whether our existing legislation, regulations and guidance are appropriate to meet the challenges associated with an increasing use of medical software and AI across the healthcare sector, and • identify measures to clarify and strengthen existing regulation to mitigate risks and leverage opportunities associated with medical software and AI use in the therapeutic goods sector.
Extensive targeted engagement with stakeholders from cohorts including the medical device industry, consumers and clinicians has been conducted, followed by a public consultation process seeking more information and feedback about strengths of the system, opportunities for improvements and identified issues and areas of concern. Our review also included mapping the existing medical device legislative framework against the mandatory guardrails for use in high-risk settings proposed by the Department of Industry, Science and Resources (DISR) in their consultation: Introducing mandatory guardrails for AI in high-risk settings: proposals paper.
The TGA goes on to comment
It is likely that the time and costs associated with regulatory requirements appear to developers to be disproportionate when compared to the time and costs associated with the development of a software product. A further cultural issue is the pervading belief among some developers that software products don’t present a meaningful risk to consumers and users, particularly when they are integrated with the provision of healthcare, where a human is in the loop, or where outputs are information only.
Stakeholders, including clinicians and consumers who use these kinds of products, have identified that the absence of humans, lack of transparency and failure to engage with existing regulatory requirements represent a combination of circumstances that may lead to patient harm. In many instances, users are not aware that AI or machine learning has been used in the development of software, or is used operationally within the clinical workflow.
Further
Regulatory requirements for medical devices, including software, are principles-based and apply regardless of whether the product incorporates components like AI, chatbots, cloud, mobile apps or other technologies. As such, software that incorporates generative AI such as large language models (LLMs), text generators, and multimodal generative AI are all regulated as a medical device if they meet the definition under the Act. As a component of the review, we mapped the existing legislative framework, including regulations and guidance, against the mandatory guardrails proposed for use in high-risk settings under the proposal put forward by DISR in their consultation: Introducing mandatory guardrails for AI in high-risk settings: proposals paper. A summary is at Attachment A.
This section documents key features of the existing framework for the regulation of medical devices including: • Technology agnostic regulation • Risk based classification • Principles based regulation • International harmonisation
Technology-agnostic regulation
Australia’s regulatory approach to medical devices is technology-agnostic, with legislative requirements centred on risk and principles rather than linking specific requirements to explicit features or technologies. A technology-agnostic approach requires those responsible for manufacturing a medical device to: • identify the specific and potential risks associated with the device throughout its lifecycle • institute measures to mitigate both identified and residual risks • have measures in place for ongoing review and monitoring of the device’s performance after it has been deployed, and • engage in ongoing review and refinement of the device once deployed.
This approach provides flexibility and responsiveness to emerging technologies, allowing lower risk devices to enter the market expeditiously while subjecting higher risk devices to greater regulatory scrutiny to ensure quality, safety, and performance throughout the device life cycle. The continuation of a technology-agnostic approach will provide flexibility to ensure appropriate regulation is capable of being applied to emerging technologies without the need for continual review and refinement of legislation. Moving away from a technology-agnostic approach where the onus for demonstrating safety, quality and performance rests with the manufacturer or deployer may lead to the introduction of risks as developers adopt a “tick-box” mentality to regulation rather than a proactive engagement and assessment of the risks posed by their products.
Development of specific regulatory requirements for individual technologies is also likely to become a limiting factor with respect to the development of innovative devices in the long term, as devices that don’t easily fit within specified parameters struggle to meet requirements that were never intended for devices of their nature.
Risk based classification
In Australia, devices are classified using classification rules set out in Schedule 2 of the Therapeutic Goods (Medical Devices) Regulations 2002.
The classification of a medical device is determined by factors including how long the device will be continuously used for and how invasive the device is. For software-based medical devices, classification may also be impacted by whether the device is intended for use by a clinician or a consumer, and the seriousness of the illness or condition for which it is intended to be used. The classification of a device will determine the level of scrutiny and pre-market assessment applied to the device before it can be deployed/supplied.
Principles based regulation
In Australia, manufacturers are required to demonstrate that medical devices comply with the essential principles. These are legislative requirements that are further set out in Schedule 1 of the Regulations, and relate to specific characteristics of medical devices including design, construction, evidence supporting the use of the device and information to be provided with the device.
Manufacturers must ensure their devices meet all relevant principles and sponsors must either hold or be able to obtain this evidence from their manufacturer on request. Principles-based regulation, as opposed to prescriptive or rules-based regulation, provides flexibility. This approach accommodates the broad complexity and diversity of medical devices regulated, including as new technologies like AI emerge. A rules-based approach may, for example, require compliance with prescribed requirements including international standards such as ISO or IEC standards.
Demonstrating compliance with the essential principles may include compliance with relevant international standards, but for emerging technologies where an appropriate standard may not yet exist, other approaches may be used. The flexibility to adapt the principles to the unique circumstances of a medical device, particularly those incorporating emerging technologies, allows approaches to evolve over time without continuous review and updating of legislative frameworks.
International harmonisation
Our current approach and commitment to international harmonisation allows sponsors of medicines and medical devices to use international assessment and approvals from comparable overseas regulators to support applications for inclusion of their therapeutic goods on the ARTG.
The TGA is also a member of the IMDRF, which seeks to “strategically accelerate international medical device regulatory convergence to promote an efficient and effective regulatory model for medical devices that is responsive to emerging challenges while protecting and maximizing public health and safety.”
The IMDRF has published a significant number of regulatory guidance documents for adoption by jurisdictions globally. Guidance documents are developed through specialised Working Groups and involve global public consultation processes. The TGA is an active member of both the IMDRF Software as a Medical Device (SaMD) Working Group and the IMDRF Artificial Intelligence/Machine Learning Working Group, which have both published a range of guidance documents. The AI Working Group is currently focused on finalising additional guidance on good machine learning practices and new guidance on AI lifecycle management, while the SaMD Working Group is developing an approach to pre-approved change control plans (PCCPs).
Software regulation and reforms
The TGA regulates AI when it meets the legislative definition of a medical device in Section 41BD of the Act. AI products likely to meet this definition include those intended to be used for the diagnosis, prevention, monitoring, prediction, prognosis, treatment, or alleviation of a disease, injury or disability.
In recent years, software has become increasingly important in medical devices and digital adoption more broadly. It is also becoming more important as a medical device in its own right. Rapid innovation in technology has driven significant changes to software function and adoption, giving rise to a larger number of devices able to inform, drive or replace clinical decisions, or directly provide therapy to an individual.
Advances in computing technology and software production have led to a large increase in the number of software-based medical devices available on the market, requiring the implementation of reforms to ensure patient safety. Software-based medical devices are medical devices that incorporate software or are software, including software as a medical device, or software that relies on hardware to function as intended, and are regulated in Australia by the TGA.
These kinds of devices may be integrated within electronic health records systems, used by clinicians or health professionals in the provision of care, or used to determine how or when patients will receive care. Their increasing use, integration in healthcare systems, and complexity have given rise to new regulatory challenges. In 2021, the TGA introduced a number of regulatory refinements aimed at ensuring the regulation of software-based medical devices, including software that functions as a medical device, remains appropriate and targets the risks associated with these kinds of devices appropriately. Refinements included: • amendments to the essential principles include the addition of Essential Principle 12.1, which details specific requirements for programmed or programmable medical devices or software that is a medical device • new classification rules for software based medical devices used for diagnostic or screening purposes to capture their potential to cause harm through the provision of incorrect information • introduction of an exemption from TGA regulation for certain clinical decision support software, and • exclusion of certain software products for the sake of clarity, or where existing oversight measures were available through other regulatory frameworks to ensure these products were safe and fit for their intended purpose.