Modern Fraud Groups Utilize Generative Artificial Intelligence (GenAI) and Deepfakes to Steal Millions of Dollars

Question: How are modern fraud groups using generative artificial intelligence (GenAI) and deepfakes to steal millions of dollars?

Answer: If you imagine a fake identity document, what springs to mind? Is it a faded World War II-era identity card with a false name written in neat script next to a black and white photo? Is it a driver’s license from a state you had never actually been to with your photo and a fake name, address, and birthday that you showed to the bars in your college town so you could drink when you were underage? Sometimes those documents worked. Sometimes they didn’t.

The forgeries created by modern fraud groups use cutting-edge AI-driven deepfake technology to create fake identities, avatars, and documents that even a trained eye would have trouble spotting, says Ofer Freidman, chief business development officer of identity verification and ID management automation company AU10TIX.

The Role of GenAI in Modern Fraud

These groups are using GenAI to steal personal data, craft convincing fake identities, and execute fraud schemes. Fraudsters acquire personal data from individuals through a combination of methods, including phishing, social engineering, and hacking corporate databases. They can also buy the information from cybercrime marketplaces. They can then use an AI system to randomize the information, such as names, addresses, and document numbers, to generate new fake identities. AI is better at avoiding repetitive patterns than humans, making it more likely that these fake identities will evade detection systems and monitors, Friedman says.

The Impact of Deepfakes on Traditional Forgeries

Traditional forgeries could be identified by spotting inconsistencies, such as mismatched shadows or areas that were low resolution compared to the rest. While there are ways to identify deepfakes (such as counting the fingers on the person’s hands), each model is getting better at producing uniform, high-quality content.

The Industrialized Nature of Modern Fraud

For fraudsters, every payday doesn’t need to be huge because they are operating in an "industrialized, systematic way," Friedman says. "You can go for $1 million or you can go 100 times for $10,000. It’s the same effort because it’s a machine doing it for you, unlike the good old days when you actually had to do it one by one."