How to identify an AI imposter in video, audio and text as deepfake technology goes mainstream

Carl Froggett worked as one of Citibank's chief information security officers for over two decades, protecting the bank's infrastructure from increasingly sophisticated cyberattacks. And while criminal trickery, from low-tech paper forgery to rudimentary email scams, has long plagued banking and the wider world of business, deepfake technology powered by generative AI is something previously unseen.

"I am very worried about deepfakes being used in business," said Froggett, who is now the CIO of Deep Instinct, a company that uses AI to combat cybercrime.

Industry experts say that boardrooms and office cubicles are quickly becoming a battlefield where cybercriminals will routinely deploy deepfake technology in the attempt to steal millions from companies, and as a result, they are a good testing ground for efforts to spot AI imposters before they are successful in scams.

"The challenge we have is that generative AI is so realistic," Froggett said.

Generative AI video and audio tools are getting deployed, and getting better, quickly. OpenAI released its video generation tool Sora in February and, at the end of March, it introduced an audio tool called Voice Engine that can realistically recreate an individual speaking from a 15-second soundbite. OpenAI said it launched Voice Engine to a small group of users given the dangers that the technology it poses.

A native of the United Kingdom, Froggett uses his regional British accent as an example.

"I use nuances and words you have never heard of, but generative AI consumes things I have made public; I am sure there is a speech I have given posted somewhere, and from it, it generates hyper-realistic voicemail, email, and video," he said.

Experts cite a widely reported case in Hong Kong last year in which an employee of a multinational corporation was duped into transferring $25 million to a sham account run by cyber criminals after attending a Zoom call populated by her coworkers including the company's CFO — except all the colleagues were convincing deepfakes. Experts believe that the case is illustrative of what is to come.

Even as OpenAI limits access to audio and video tools, dark websites have exploded in number in the past few months selling GPT knockoff products. "The bad guys have literally just gotten their hands on these tools. ... they are just getting started," Froggett said.

It only takes a 30-second or less snippet of someone talking to make a flawless deepfake, said Check Point Software president Rupal Hollenbeck, and cybercriminals are now able to gain access to AI-driven deepfake tools for dollars, if not pennies. "But that is just on the audio side. The same is now true for video, and that changes the game," Hollenbeck said.

Steps that corporations are beginning to take to prevent successful deepfakes are instructive for how all individuals should conduct their lives in a gen AI world and interact with friends, family and coworkers.

How to identify an AI video imposter

There are many ways to spot an AI imposter, some relatively simple.

For starters, Hollenbeck says if there is any doubt about a person's video veracity, ask them to turn their head to the right or the left, or look backward. If the person complies but their head disappears on the video screen, end the call immediately, Hollenbeck said.

"Right now, I am going to teach this to everyone I know, to have them look right or left. AI does not have the capacity to go beyond what you can see. AI is flat today, and that is very powerful," she said.

But there's no telling how long that will last.

Chris Pierson, CEO of Blackcloak, a company specializing in digital executive protection, thinks it is only a matter of time before deepfakes have 3D capability. "The models are improving so quickly that those tricks will fall by the wayside," Pierson said.

He also says don't be afraid to ask for old-fashioned "proof of life" video evidence of authenticity, like requesting the person on the conference to show a company report or even a newspaper.  If they can't follow those basic commands, that is a red flag.

How use of code words and QR codes can help

Old-fashioned code words can be effective too, but only transmit them through a separate medium and keep them in unwritten form. Both Hollenbeck and Pierson are recommending to the executive teams at companies the generation of a code word for each month, stored in encrypted password vaults. If there is any doubt about the person you are talking to, you can ask for the code word to be sent via text. And set a threshold to deploy the code word. For example, if anyone asks you to conduct a transaction over $100,000, then the code word tactic should be deployed.

For businesses, having corporate calls only on approved company channels also greatly reduces the risk of being duped by a deepfake.

"Where we are getting in trouble is going outside the network," Pierson said. 

Real-world examples of business deepfakes are increasing, Nirupam Roy, an assistant professor of computer science at the University of Maryland said, and it's not just about the criminal bank account transfers. "It is not difficult to imagine how such deepfakes can be used for targeted defamation to tarnish the reputation of a product or a company," he said.

Roy and his team have developed a system called TalkLock that works to identify both deepfakes and shallowfakes — which he describes as relying "less on complex editing techniques and more on connecting partial truths to small lies."

It may not be the answer to highly personalized, AI-generated scams, but it is designed for individuals (who can access an app) and corporations (given access to a verification module) to be able to spot AI manipulation. It works by embedding a QR code into audiovisual media such as live public appearances by politicians and celebrities, as well as social media posts, advertisements and news, that he says can prove authenticity. It combats a rising problem associated with unofficial recordings — for example, video and audio taken by audience members at events, which unlike official media, cannot be identified by metadata.

How to live a multi-factor authentication life offline

Even with more protections techniques, experts predict a spiraling arms race of deepfakes vs. deepfake tools. For companies, there are certain procedures that can be put in place to prevent the worst consequences of deepfakes that are less readily adaptable to individual life.

Eyal Benishti, CEO of Ironscales, an email security software company, said organizations will increasingly adopt segregation of duties so that no single person can be duped enough to harm a company. In particular, this means a division of labor processes for handling sensitive data and assets. For example, changes to bank account information used to pay invoices or payroll should require two people to change and (ideally) a third person for notification. "This way, even if an employee falls for a social engineering attack that requests they redirect an invoice payment, there will be stop-gaps as different stakeholders are brought in to fulfill their roles in the chain of command," Benishti said.

At the most basic level, organizations and their people must start living their lives in a multi-factor authentication way, according to Hollenbeck, having multiple ways to verify reality. In the end, old-school still works, such as walking down the hall to see the boss in person. So far, that can't be deepfaked.

"It used to be that seeing was believing, but not so much anymore," Hollenbeck said.

It's also wise to remember that deepfakes are simply the latest in a long line of scams, from three-card monte to the pigeon drop, that prey upon human vulnerabilities by creating a false sense of urgency. That means the best antidote to a deepfake, according to Pierson, may be the simplest: slow down. This is one tactic that is arguably easier for individuals to use in their personal life than for employees in work life.

"Slowing down almost always yields a definitive answer. Every company should have a safe harbor policy that if you ever feel like you are being rushed into making a decision, an employee should feel the right to decline, contact security, and be held harmless," Pierson said. Often corporate culture doesn't give employees much deference.

"We have to give people the advantage of being able to stop and say no. If they don't feel like they can say no — and no one feels like they can say no — that is when mistakes happen," Pierson said.