‘Human intelligence problem’: Deloitte to refund Australian govt over AI-generated report errors

The review, first published in July, flagged structural problems with the compliance system — including mismatches between the system’s rules and the underlying legislation and software defects that appeared to assume non-compliance by benefit recipients. But independent researchers and media outlets subsequently identified several erroneous or nonexistent references in the document.

University of Sydney academic Dr Christopher Rudge, who highlighted the problems, described the mistakes as AI “hallucinations,” where generative systems invent or misattribute sources. He said the report’s corrections appeared to replace one dubious reference with multiple others, calling into question the evidentiary basis for some claims in the text.

DEWR said the corrected version fixes a “small number of references and footnotes” while leaving the review’s core findings and recommendations unchanged. The department confirmed Deloitte would refund the final instalment of its contract; details of the repayment will be published once the transaction is completed.

“Deloitte conducted the independent assurance review and has confirmed some footnotes and references were incorrect,” a department spokesperson said. “The substance of the independent review is retained, and there are no changes to the recommendations.”

What did Deloitte say?

In a revised appendix, Deloitte disclosed that parts of the report were produced with the assistance of a generative AI toolchain (Azure OpenAI GPT–4o) that ran within DEWR’s own cloud tenancy. The firm has insisted the adjustments do not alter the substance of the review and has defended the accuracy of its principal conclusions.

“The updates made in no way impact or affect the substantive content, findings and recommendations in the report,” it said in the amended version.

A Deloitte spokesperson also added that “the matter has been resolved directly with the client”.

Labor senator Deborah O’Neill sharply criticised the episode, saying it appeared firms were letting AI do “the heavy lifting” and urging stricter oversight. “Deloitte has a human intelligence problem,” she said, calling the partial refund a “partial apology for substandard work” and urging future contractors to be transparent about who — or what — is doing the analysis.

The Australian Financial Review reported the original document included fabricated citations, such as phantom reports attributed to academics and an incorrect reference to a court case. Deloitte acknowledged errors in its summary of the cited legal proceeding and amended that section.