An Official Warning on Accountability
A federal health department advisory group in Australia has sounded the alarm over the rapidly growing use of AI tools in medicine, particularly concerning AI scribes. These tools listen to, transcribe, and summarise doctor-patient consultations, and their
use by Australian doctors nearly doubled in just over a year. In a recent briefing, the AI advisory group highlighted that while these tools can reduce doctor burnout, their limitations in quality and accuracy have serious implications for “patient safety, clinical accountability, and the integrity of data.” This flags a crucial gap: when a doctor relies on an AI-generated summary that contains an error, determining who is responsible for any resulting harm becomes incredibly complex.
A 'Patchwork' of Regulations
The problem is compounded by a regulatory environment described as a “patchwork” with “little oversight.” Currently, responsibility is spread across several bodies, including the Therapeutic Goods Administration (TGA) and the Australian Health Practitioner Regulation Agency (Ahpra). However, officials have noted that some AI developers market their products as being outside the scope of existing medical device regulations, creating a grey area where tools are used in clinical settings without clear safety and quality checks. This lack of a unified framework means that crucial aspects like patient consent are handled inconsistently, with some patients reportedly being told to find a new provider if they don't consent to the use of AI.
More Than Just Scribes
While AI scribes are the immediate focus, the issue points to a much larger transformation in medicine. A recent report from Australia's national science agency, CSIRO, confirms that AI has moved from a background technology to a central force in healthcare. The report, “AI Trends for Healthcare 2026,” details how AI is already embedded in clinical decision support, medical imaging analysis, and personalised care. Dr David Hansen of CSIRO’s Australian e-Health Research Centre noted that the rapid rise of generative AI has sharpened the focus on safety and responsible use. This shift from niche to mainstream underscores the urgency of establishing robust rules before the technology becomes even more deeply integrated into the health system.
The 'Black Box' Problem
At the heart of the accountability challenge is the “black box” nature of many AI systems. Often, the complex algorithms that produce a diagnosis or recommendation operate in ways that are not fully transparent, even to their developers. This creates a legal and ethical minefield. If a patient is harmed, is the doctor who trusted the AI liable? Or is it the hospital that procured the software? Or the company that developed the algorithm? As it stands, legal precedent in Australia has not yet been tested, but the prevailing guidance is that medical practitioners remain ultimately accountable for the care they provide, even when using AI assistance. This puts clinicians in a difficult position, caught between the benefits of new technology and the immense responsibility for its outputs.
Forging a Path to Trust
Australian authorities are now moving to address these gaps. The TGA is undertaking a review to determine if AI scribes should be formally classified as medical devices, which would subject them to much stricter oversight. The CSIRO report strongly advocates for embedding quality management and regulatory strategy into the development of AI tools from the very beginning, rather than as an afterthought. Furthermore, new bodies like the National Clinical Governance Committee for Digital Health have been formed to specifically advise on the safe implementation of AI. As nations like India also accelerate their adoption of digital health technologies, Australia's struggle to create a framework for safe and accountable AI offers a crucial and timely case study for the world.
