Australia’s healthcare sector increasingly relies on AI-driven innovations, which hold promise for advancing diagnostics, refining treatment strategies, and enhancing patient outcomes. Yet, the accelerated adoption of these technologies introduces complex legal, regulatory, and ethical considerations that must be carefully managed as they enter clinical workflows.
In Australian healthcare settings, AI tools are being used to listen to patient consultations, generate notes and care plans, and assist surgeons in making decisions. Other emerging technologies include OpenAI’s ‘ChatGPT Health’, which allows patients to upload medical records and connect to health apps, and DeepSUVR, an AI model designed to improve the accuracy of PET scans.
AI governance in Australian healthcare currently operates through a mix of federal, state, and territorial laws. To help providers meet their professional duties when incorporating AI, the Australian Health Practitioner Regulation Agency (AHPRA) has issued official guidance.
AHPRA’s approach centers on key values such as responsibility, comprehension, openness, and patient agreement. Providers are expected to exercise professional judgment on AI-generated recommendations, understand how their AI tools work, and disclose AI involvement to patients.
Regulating AI Healthcare Tools
The Therapeutic Goods Administration (TGA) oversees AI applications in medical devices and therapeutic products. Tools like chatbots and clinical decision aids fall under the Therapeutic Goods Act 1989 when designed to guide or support clinical choices.
The TGA uses a four-tier risk-based classification system, considering the device’s intended purpose and level of harm it may pose. Examples of AI-enabled devices include the Ibex platform, which assesses and interprets pathology slides, and the Merlin.Net Patient Care Network, which allows practitioners to remotely monitor implanted heart devices.
As AI tools continue to develop, regulators will need to keep pace with these emerging technologies to ensure appropriate use and patient safety. International developments, such as the US Food and Drug Administration’s (FDA) regulation of AI in healthcare, provide a glimpse into potential changes to the legal and regulatory system in Australia.
In the US, tech companies have come under scrutiny for AI-assisted technology, including a lawsuit against Character.AI for providing false medical licence information. Similarly, Sharp HealthCare is facing a class-action suit for allegedly violating state consumer privacy laws by using AI to record patient-clinician conversations without notice or consent.
Global AI Governance Efforts
The European Union has introduced the Artificial Intelligence Act, which sets out specific obligations for providers and deployers of AI systems, including transparency and AI governance requirements. Liability in relation to AI medical devices is governed by the Product Liability Directive, which has undergone changes related to AI.
AHPRA has confirmed that regardless of what technology is used in providing healthcare, the practitioner remains responsible for delivering safe and quality care. However, advancements in technology will lead to the emergence of autonomous AI systems that can assess patients and make recommendations for treatment without practitioner input.
Medical professionals must preserve records that are accurate, current, and thorough. With more than 40% of Australian clinicians now using AI scribes to ease paperwork, this practice is growing rapidly.
However, AI scribes are not without risk, including the tendency of clinicians to overly rely on AI-generated documentation without sufficient verification. There have been cases where AI scribes have generated completely fabricated or incorrect information, such as inventing diagnoses or recording examinations that never happened.
For example, a well-publicised recent news story concerning a urologist who was required to apologise to his patient after an AI scribe utilised during their consultation hallucinated an error into her medical records, stating that the patient had taken psychedelic mushrooms when she had not. Conversely, regulators are pushing the adoption of AI in healthcare settings, with the Tribunal imposing conditions on a medical practitioner to complete education on AI scribe programmes.
Managing Patient Data Risks
There are many questions that come with the rise of PGHD. These include the extent to which a practitioner should rely on the PGHD, whether a practitioner is required to respond to PGHD, whether a contributory negligence argument may arise against a patient or device manufacturer for inaccurate PGHD, and how PGHD ought to be recorded in the practitioners’ medical records. The US-based Society for Participatory Medicine sets out strategies for managing risk, including working with patients to achieve a common understanding of the types of data patients will share.
AI Applications and Regulatory Guidance
AI tools can analyse images to detect skin, cervical, breast, and prostate cancer. They can also assess risks to patients and predict healthcare system usage.
The Queensland Adult Deterioration Detection System uses a scoring system to identify patients at risk of clinical decline. GuideStream and the Oncology Search Clinician Assistant Registry are other tools that provide relevant clinical information.
Regulatory agencies have issued guidance on AI use in healthcare. The Australian Health Practitioner Regulation Agency requires practitioners to understand AI tools and obtain informed consent from patients. The Australian Department of Health has published an AI Clinical Use Guide to support safe and responsible AI use.
