Propaganda has long been a factor in global conflicts, with examples dating back to 515 BC, as seen in the Behistun Inscription, which chronicles Persian King Darius’ rise to power. More recently, the way different newspapers report on wars has led to the phrase, “The first casualty is the truth.” While these forms of communication had the power to shape people’s beliefs, they also had limitations in terms of scalability, with messaging and propaganda often losing their impact over long distances.

However, with the advent of social media and the online world, these limitations no longer apply, as information can now reach a global audience with ease. The rise of AI has further amplified this, allowing for unparalleled scalability. This article delves into the implications of AI-powered information manipulation and deception for societies and organizations.

This article explores the consequences of AI-powered information manipulation and deception on societies and organizations, including the use of AI-powered phishing simulation to educate employees and individuals about the dangers of AI-powered deception.

The emergence of the echo chamber

A significant shift in how people consume news has been observed, with around one-in-five Americans now relying on social media for news, according to the Pew Research Center. In Europe, there has been an 11% increase in people using social media platforms to access news. AI algorithms are driving this behavioral change, but they are not bound by the same journalistic standards as traditional media, often prioritizing content that engages users over fact-based reporting.

This has led to the creation of digital echo chambers, where people are only exposed to information that reinforces their existing views, potentially resulting in polarized perspectives. Furthermore, AI-generated content, including over 1,150 unreliable news websites identified by NewsGuard, is increasingly prevalent, making it challenging to distinguish fact from fiction.

AI’s role in deception

Human unpredictability and biases can be exploited by malicious actors using AI. This has led to various forms of deception, including:

• Deepfake videos: AI tools can create convincing fake footage using text prompts, as seen during the US election, where Microsoft reported that threat actors from China and Russia integrated generative AI into their influence efforts.
• Voice cloning: Attackers can mimic anyone’s voice using AI, as demonstrated in the case of a Slovakian politician in 2023, where a fake audio recording spread online, potentially influencing voters before an election.
• LLMs faking public sentiment: Adversaries can use Large Language Models (LLMs) to communicate in multiple languages and scales, as seen in 2020 when an early LLM, GPT-3, was trained to write thousands of emails to US state legislators, blurring the lines between human and AI-generated content.

AI’s impact on democratic processes

While it is still possible to identify many AI-powered deceptions, the technology is rapidly evolving, making it increasingly difficult to distinguish fact from fiction. Fact-checkers and websites like Snopes can debunk conspiracy theories, but it is challenging to ensure that corrections reach everyone who saw the original posts.

Evolution and adaptation

Human senses have evolved over millions of years to trust what we see and hear. However, AI has developed at an unprecedented pace, with ChatGPT being released publicly in November 2022. Our brains cannot adapt at the same speed, making it essential to educate people to be skeptical and aware of AI-powered deception.

Organizations must equip their workforces with knowledge and awareness to recognize and resist AI-generated content engineered to manipulate. This includes simulating AI-powered attacks to provide firsthand experience and teaching individuals to pause, reflect, and challenge what they see online. By doing so, humans can shape society and defend against AI-powered deception.