AI can fool voice recognition used to verify identity by Centrelink and Australian tax office

A voice identification system used by the Australian government for millions of people has a serious security flaw, a Guardian Australia investigation has found.

But following reports that an AI-generated voice trained to sound like a specific person could be used to access phone-banking services overseas, Guardian Australia has confirmed that the voiceprint system can also be fooled by an AI-generated voice.

Using just four minutes of audio, a Guardian Australia journalist was able to generate a clone of their own voice and was then able to use this, combined with their customer reference number, to gain access to their own Centrelink self-service account.

The self-service phone system allows people to access sensitive material such as information on their payment of benefits and to request documents to be sent by mail, including replacement concession or healthcare cards.

When Guardian Australia contacted Services Australia with details of the security vulnerability, it declined to say if the voiceprint technology would be changed or removed from Centrelink.

Ed Santow, a former human rights commissioner and now director of policy at the Human Technology Institute at the University of Technology Sydney, said government agencies using biometrics as a form of verification needed to ensure they had the best systems in place, and that there was legislation underpinning those systems.

A spokesperson for the ATO said the agency had robust measures in place to protect the system from threats including AI voice cloning.

In the blog post, the company outlined its efforts to detect synthetic voices, and claimed its latest technology could accurately detect and flag the use of cloned voices in 86% to 99% of cases, depending on the technology used.

Voice cloning, a relatively new technology using machine learning, is offered by a number of apps and websites either free or for a small fee, and a voice model can be created with only a handful of recordings of a person.

While the voice generated is better with high-quality recordings, anyone with public recordings of themselves on social media, or who has been recorded elsewhere, could be vulnerable to having their voice reproduced.