Welcome to The Responsible AI Audit™ Brief. Skills4Good AI's weekly audit of a real AI failure and the questions it should have raised before anyone relied on it.

The Quick Facts

After the 2022 floods in the Northern Rivers region of New South Wales, Australia, residents applied to a $920 million government program to buy back, retrofit, or rebuild damaged homes. To apply, they provided the NSW Reconstruction Authority (RA) with their names, addresses, and dates of birth, and, in some cases, sensitive personal information and limited financial details.

In March 2025, a government RA contractor uploaded a Microsoft Excel spreadsheet containing applicants’ personal data to ChatGPT, a generative AI tool. The spreadsheet contained more than 12,000 rows of personal data. Here's what the RA said:

"The breach occurred when a former temporary staff member of the RA uploaded data containing personal information to an unsecured Artificial Intelligence (AI) tool which was not authorised by RA. We understand this news is concerning and we are deeply sorry for the distress it may cause for those involved in the program.”

The flood victims lost control of their data the moment the government contractor uploaded it into ChatGPT. Mandy Turner, adjunct lecturer in cyber criminology at The University of Queensland, described what the unauthorized upload means for the flood victims:

“It is heartbreaking that these people, who have already gone through so much with flooding, now have the added stress that their information was uploaded to a platform where they have no control of where their personal and sensitive data may go or be used.”

“The flood victims trusted a government recovery program with their data.
One upload to ChatGPT took it out of their control.” 

The AI Risk

This is an AI privacy breach: personal information ending up where it does not belong. It does not require a third-party bad actor to commit a privacy breach. It takes one professional doing their work and carelessly uploading someone else's information into an AI tool without that person's consent.

Each of the 2,031 flood victims trusted the RA with their data. None of them consented to share it with OpenAI, the creator of ChatGPT.

Three privacy principles clearly show where the RA went wrong:

  • Prior Consent. Consent does not follow the data. The flood victims agreed to share their information with the NSW RA recovery program, not with an AI company. The breach occurred at the moment of upload, even before the AI generated any output.
  • Data Minimization. Even with prior consent, use only the personal information the task requires, and strip out the rest before you prompt. In this case, the entire spreadsheet went in, including birth dates and sensitive details.
  • Storage Limitation. Personal data should be retained only as long as necessary and by only those who need it. Once uploaded to an AI tool, such data resides on systems you do not control. Turner warned that “no online platform is immune from the threats of data exposure”.

Lawyers face the same risk with client files. In a decision reported this year, the UK Upper Tribunal warned:

"Uploading confidential documents into an open-source AI tool, such as ChatGPT, is to place this information on the internet in the public domain, and thus to breach client confidentiality and waive legal privilege."

Both regulators that accredited our AI Risk Detection Series for Lawyers agree.

The Law Society of Ontario advises lawyers to "redact any identifying, sensitive, or confidential information" before using generative AI”. 

The State Bar of California states that “a lawyer must not input any confidential information of the client into a generative AI solution that may present material risks to confidentiality or security, absent informed client consent as to the underlying risks".

The Responsible AI Audit

This week's human skill: Creativity

Creativity is the second of the Five Human Skills That AI Won’t Replace™ in our Human Skills Flywheel™: the ability to connect ideas in new ways to solve real problems. Here, the problem is getting AI assistance without breaching the data of the people who trusted you with it. It asks two questions.

  • Need: Can I get the same help from the AI without including anyone's personal information?
  • Audience: Who will read this output, and does every personal detail belong in front of them?

AI tools will use the information you give them, whether in your prompt or in uploaded files in a project. Deciding what to leave out takes creativity. And it sharpens each time you rebuild a prompt without using someone else's personal data.

Before you paste any information about a client, patient, employee, or customer into an AI tool, ask three questions.

  • Did they agree to their data being used there?
  • Does the work task require their data?
  • Am I certain that the AI tool will not use their data for training?

If any answer is no, remove names and identifying details first, or leave the entire file of personal data out. Then write down what you removed and why.

A privacy breach cannot be undone. Once inputted into a prompt, an individual's personal data cannot be called back.

Our Privacy Breach Detection course teaches the complete method for detecting AI privacy breaches before and after you prompt, and ends with a Responsible AI Audit™ Record, the documented proof that a human performed the verification. It is available in two series:

For Professionals: Privacy Breach Detection for Professionals, in The AI Risk Detection Series™ for Professionals, accredited by OACETT.For Lawyers: Privacy Breach Detection for Lawyers, in the Responsible AI Audit™: AI Risk Detection Series for Lawyers, accredited by the Law Society of Ontario and approved by the State Bar of California.

If there's something you want this newsletter to cover, reply and let me know. I'd love to cover it.

And if you know someone who uses AI at work, forward this to them.

I'll see you at next week's Responsible AI Audit.

Josephine

Josephine Yam, JD, LLM, MA Phil (AI Ethics)
CEO & Co-Founder, Skills4Good AI
AI Lawyer | AI Ethicist | TEDx Speaker

Explore the Responsible AI Audit™: AI Risk Detection Series by profession