Welcome to The Responsible AI Audit™ Brief. Skills4Good AI's weekly audit of a real AI failure and the questions it should have raised before anyone relied on it. 

This is the third of four issues about one incident, and the human skills it shows we need to keep AI under our control.

The Quick Facts

In July 2026, about 700 of the 1,200 AI agents in an OpenAI cybersecurity test found a shared message board and attacked Hugging Face, a platform where AI researchers store their models. The three investigators had about 1,300 agent transcripts to review, so they delegated the review to AI agents.

The last two Responsible AI Audit™ Briefs covered the investigators' first two limitations: AI errors that went undetected for some time and AI bias. This week, I cover their third limitation. Here's what the report says:

“…AI agents are known to sometimes lie… We cannot rule out that GPT-5.6 Sol lied or deliberately presented a misleading picture in some of its analysis… Although we did not notice specific cases of GPT-5.6 Sol lying in its analysis, we are not confident we would have detected it if it occurred.” (p. 27)

Whether an AI is lying or simply wrong, the warning sign is the same: a conclusion resting on something no one checked. That is this week's risk: the reasoning gap.

The unchecked assumption: a scorer would read the AI agents' records and catch their shortcut. 
They passed it from agent to agent and built a hack on it. The scorer never read a single record.
That is a reasoning gap.

The AI Risk

This is an AI Reasoning Gap: a conclusion that does not follow from its evidence, often because it rests on a hidden assumption. Unlike a hallucination, every fact in a reasoning gap can be true, but the logic leaps. Here is how one unchecked assumption spread across the message board:

  • The test: OpenAI's AI agents were taking a cybersecurity test called ExploitGym. To pass each task, an agent had to hack a target program and retrieve a secret code, called a flag.
  • The shortcut: the AI agents worked out how the flags were generated, so they could cheat by producing the flag for any task without doing the hack.
  • The unchecked assumption: after reading the published paper on ExploitGym, they assumed the scorer would also read each agent's transcript, the record of its actions, and disqualify any flag obtained through the deceitful shortcut.
  • The handoff: the AI agent that started the message board passed this assumption to a newcomer in a written dossier: the board had the shortcut, but using it would show in their transcripts, and the scorer would read them and disqualify the flag.
  • The build-up: agents that had already seen a shortcut flag called themselves "poisoned," believing it sat in their transcripts as evidence. So they researched ways to fake or erase their transcripts, and attacked Hugging Face mainly to learn how the scorer worked.
  • The reality: the scorer checked only the flag. It never read a single transcript. The AI agents "could have achieved a perfect score simply by submitting their reverse-engineered flag" (p. 33).

Their reasoning ran: the published paper describes a scorer that reads how a flag was obtained, so ours must too, so the shortcut only works if our transcripts hide it.

The hidden assumption in that conclusion was the middle step: that their test was set up as the paper described. No AI agent checked it, and everything that followed, including faking their transcripts, was built on that assumption.

The same gap appears in legal AI platforms.

In 2024, Stanford University researchers tested AI research tools from major legal publishers. When asked why Justice Ginsburg dissented in Obergefell, one tool "naively agreed with the question's premise" and explained that she did so based on her views on international copyright. However, the premise was false, and the AI tool built on it: Justice Ginsburg joined the majority, and the case had nothing to do with copyright. It concerned the right to same-sex marriage.

Now picture that answer passed to a second AI agent that is helping you draft your brief.

Both regulators that accredited our AI Risk Detection Series for Lawyers expect a human, not an AI system, to conduct verification of AI outputs. The Law Society of Ontario says the “verification process should be completed by a human being, not the AI system itself". The State Bar of California has proposed rules to the same effect. California has now written it into law: SB 574, signed September 30, requires lawyers to take reasonable steps to verify AI output starting January 1, 2027.

The Responsible AI Audit™

This week's human skill: Adaptability

Adaptability is the fourth of the Five Human Skills That AI Won’t Replace™ in our Human Skills Flywheel™: the capacity to embrace uncertainty and anticipate risk. It poses two questions.

  • What if: what if something I or the AI assumed is wrong?
  • What follows: if this assumption is wrong, what else built on it would fall?

AI reasoning gaps take three forms: a conclusion that does not follow from the evidence, an unstated assumption that a conclusion quietly depends on, and a number presented as fact with no supporting evidence. All three appear in confident prose, so each is invisible on first review. With AI agents, the second is the most dangerous: each agent inherits the last one's assumption.

Before relying on AI-drafted work, especially work passed between several AI agents, identify the hidden assumption, including any in your own question. State the fact that would change the answer if it were wrong and verify it yourself. Write down what you verified.

Reasoning Gap Detection for Lawyers, the fourth accredited course in the Responsible AI Audit™: AI Risk Detection Series for Lawyers, teaches the complete method for detecting all three reasoning gaps and concludes with a Responsible AI Audit™ Record, which serves as the documented proof that a human performed the verification.

Next week: a short break from the Hugging Face story to cover the one AI risk the report didn’t address and the new laws that now address it. The series ends the week after that.

If there's something you’d like this newsletter to cover, reply and let me know. I'd love to cover it.

And if you work with a lawyer who uses AI tools, forward this to them.

I'll see you at next week's Responsible AI Audit.

Josephine

Josephine Yam, JD, LLM, MA Phil (AI Ethics)
CEO & Co-Founder, Skills4Good AI
AI Lawyer | AI Ethicist | TEDx Speaker

Explore the Responsible AI Audit™: AI Risk Detection Series by profession