The latest high-profile case involves a physical education teacher and former athletic director of a Baltimore County high school arrested for using an AI voice cloning service to frame the school principal. But over the past year, interest in AI-powered voice cloning technology has grown as services get better at sounding more human. And cases of improper, if not downright criminal, use of apps and software for voice cloning are also growing.
The Baltimore case stems from a recording that circulated on social media in January of alleged fake audio of Pikesville High School Principal Eric Eiswert making racist and anti-Semitic comments. Experts told the Baltimore Banner and police that the recording, which led to Eiswert’s brief suspension, has a “flat tone, unusually clean background sounds and lack of coherent breathing sounds or pauses.”
Baltimore County police tracked down the recording to Dazhon Darien, a former athletic director at the school whose name was also mentioned in the audio clip. He allegedly used the school’s computers “to access OpenAI tools and Microsoft Bing Chat services,” as reported by WBAL-TV 11 and NBC News. He was also linked to the release of the audio via an email address and associated recovery phone number.
It is unclear which AI voice platform Darien used. Police arrested the man Thursday at the airport and said Pikesville High School’s athletic director made the recording to get back at Eiswert, who was pursuing an investigation into potential mismanagement of school funds at the time. He was released on bail and faces charges including theft (for the problem with school funds), disrupting the operations of a school, retaliation against a witness and stalking.
Other notable cases also occurred in Pakistan, where the political party of Imran Khan, the jailed former Pakistani prime minister, used ElevenLabs, a popular commercially available text-to-voice generation platform, to replicate his voice during the electoral campaign. In Texas, two companies were linked to a fake robocall pretending to be President Joe Biden telling people not to vote. The Federal Communications Commission banned the use of AI robocalls in February. And of course, the fake Drake used AI to create the song “Heart on My Sleeve.”
In this tense climate, OpenAI decided in March to hold back its AI text-to-speech generation platform, Voice Engine, for public use. The service, which requires only a 15-minute audio clip to clone someone’s voice, is only available to a limited number of researchers due to a lack of guardrails around the technology.
US lawmakers have introduced, but not yet passed, several bills such as the No Fakes Act and the No AI Fraud Act that seek to prevent tech companies from using an individual’s face, voice, or name without their permission. The Baltimore Banner reports that Baltimore County police believe a recording.