Should We Fear Voice Clones?
Should We Fear Voice Clones?
Evidence of Misuse is Uncommon
Remember when the dominant narrative around voice clones was the risk they posed? Someone is going to impersonate you and get your relatives to do something on your behalf that is illegal. Someone else is going to impersonate your boss and get you to transfer funds to a bank account which will be promptly drained by a criminal. Well, that happened. In 2019, The Washington Post reported:
Thieves used voice-mimicking software to imitate a company executive’s speech and dupe his subordinate into sending hundreds of thousands of dollars to a secret account, the company’s insurer said, in a remarkable case that some researchers are calling one of the world’s first publicly reported artificial-intelligence heists.
The managing director of a British energy company, believing his boss was on the phone, followed orders one Friday afternoon in March to wire more than $240,000 to an account in Hungary, said representatives from the French insurance giant Euler Hermes, which declined to name the company.This happened again in 2020, and the stakes were much higher. According to reporting by Forbes:
In early 2020, a bank manager in the Hong Kong received a call from a man whose voice he recognized—a director at a company with whom he’d spoken before. The director had good news: His company was about to make an acquisition, so he needed the bank to authorize some transfers to the tune of $35 million. A lawyer named Martin Zelner had been hired to coordinate the procedures and the bank manager could see in his inbox emails from the director and Zelner, confirming what money needed to move where. The bank manager, believing everything appeared legitimate, began making the transfers.
What he didn’t know was that he’d been duped as part of an elaborate swindle, one in which fraudsters had used “deep voice” technology to clone the director’s speech...These are spectacular stories. There are surely more of them. Granted, the $35 million swindle used a voice clone, but it was one of the fraudulent tools used in the heist. A voice clone makes a more interesting story angle because fraud via email or humans impersonating a boss, relative, or friend have been common for years.
Voice Clone Fraud is Uncommon
Email phishing and spear phishing scams that lead to money transfers, proprietary information theft, and credential compromise are the biggest risks by far. However, similar situations have been playing out via email for years and at a much larger scale. The U.S. FBI estimates that $43 billion has been stolen since 2016 through business email compromise (BEC).
A quick Google search brings back two examples of voice clones used in scams. It’s a big world. There are surely more of these. However, you will find many more examples of the “boss impersonation fraud” being used over email and phone calls from people not using any voice modification software.
Technology is neither good nor bad. The one thing you can be sure of is that it will be used for both purposes. Forbes reporting says there were two cases as of October 2021, but some experts are convinced the voice clone attacks will intensify. A year later and it appears the number of voice clones used for thefts remains at two.
It’s only the second known case of fraudsters allegedly using voice-shaping tools to carry out a heist...Jake Moore, a former police officer with the Dorset Police Department in the U.K. and now a cybersecurity expert at security company ESET [said]“Manipulating audio, which is easier to orchestrate than making deep fake videos, is only going to increase in volume and without the education and awareness of this new type of attack vector, along with better authentication methods, more businesses are likely to fall victim to very convincing conversations.”
October 2021I find it remarkable that so few incidents have occurred. Maybe there will be a big uptick in this type of misuse as the technology improves and becomes less expensive and more accessible. However, the many hyperbolic claims around voice clones have not materialized.
Not-So-Nefarious Use Cases
What has emerged are a lot of popular use cases. Voice clones are used for film and television and for voice prostheses for people that can no longer speak as they once did.
It is worthwhile to point out that voice clones and synthetic voices are not the same things. Voice clones are a particular type of synthetic voice that replicates that of a real person with the intent of representing that person in media or some interactive experience [or fraud in rare cases]. Amazon Alexa’s voice was based on a real person, but the intent was not to represent that person. It was just about having a voice that was then assigned to a newly created synthetic personality.