In 2016, Russian operatives used fabricated identities, targeted advertisements, and coordinated social media accounts to influence political discourse surrounding the United States presidential election. Nearly a decade later, Zohran Mamdani’s underdog campaign for mayor of New York City demonstrated the legitimate side of the same transformation. Through a highly effective social media strategy, Mamdani went from a relatively unknown state assembly member to winning both the 2025 Democratic primary and the general election. His Instagram audience has since grown to more than 10 million followers.
These two cases could not differ more in intent. One was foreign sabotage; the other, a democratic campaign. Yet in both, power was built out of popularity. In Mamdani’s case, that popularity was beautifully democratic. In the case of Russian interference, that popularity was artificial and potentially democracy-destroying. The trouble is that, in the digital world, the two can be almost impossible to tell apart.
Consider a post claiming:
“A candidate and their associates received more than $20 million from foreign entities.”
Is it true? Most people do not have the time, evidence, or expertise to investigate every political claim placed in front of them. Instead, we rely on signals of credibility. Who posted it? How many people shared it? Does it look real? Do people we trust seem to believe it?
Today all of those signals can be artificially created.
When Consensus Can Be Manufactured
Misinformation has always existed. What is changing is the speed, scale, and realism with which it can now be produced. Imagine a swarm of millions of AI-operated accounts generating posts, images, audio clips, and videos around the clock. Each post on its own might draw almost no attention. Together, they can manufacture the appearance of an entire political movement.
The danger is not simply that people will believe one fake image or video, but that synthetic media can distort our perception of what everyone else believes.
A genuine recording can be dismissed as a deepfake. A fabricated video can be defended as authentic. Evidence no longer settles an argument because the authenticity of the evidence becomes an argument itself. This is sometimes called the “liar’s dividend”: the existence of convincing synthetic media gives people a plausible excuse to deny authentic evidence. In that environment, AI does not merely create more misinformation; it weakens the idea that anything can be verified.
Why Simply Banning AI Content Will Not Work
It is tempting to argue that social media platforms should simply ban AI-generated political content. But such a rule would immediately confront a difficult question of how to determine what is generated by AI. AI can be used to create an entire video, but it can also be used to remove background noise, generate captions, translate speech, correct lighting, or edit a few frames. A blanket ban could therefore capture legitimate creative and accessibility tools alongside intentionally deceptive content.
The more practical target is not all AI-assisted media, but deceptive synthetic media presented as authentic. Platforms could require political advertisements and widely distributed political media to disclose substantial AI generation or manipulation. Content impersonating real candidates, officials, journalists, or events could face stricter rules. Repeated efforts to distribute unlabeled synthetic media could result in removal or account penalties. But enforcement still depends on one missing capability, reliable verification.
The Technical Problem of Detecting AI
Frontier AI companies have made progress on provenance, though the approaches fall into a few distinct categories that each carry their own weakness. Standards like C2PA attach metadata to a file describing where it came from and how it was edited, however stripping the metadata or screenshotting the image removes it entirely. Watermarks are more durable than metadata because they survive cropping and compression reasonably well, but still is far from a perfect solution. Even invisible watermarks can be reconstructed away by generative models (arXiv 2023), and the attacks are only growing more sophisticated (NeurIPS 2025). The third approach abandons embedded signals altogether and trains a detection model to recognize AI-generated content from its artifacts. These run into their own wall, since a detector trained on today’s generators tends to perform well on familiar content while degrading sharply on a new model, an unfamiliar editing pipeline, or media that has simply been resized. Generalizing to unseen generators remains a central open challenge (arXiv).
This does not mean detection is hopeless, only that no single permanent test will ever cleanly separate human and AI-generated content. The solution must be a system rather than a classifier.
The Opportunity to Build Trust Infrastructure
There is an enormous opportunity for a company focused on verifying digital media across text, images, audio, and video.
Such a company should not simply produce an “AI” or “not AI” label. That kind of binary judgment would be too fragile, especially in high-stakes settings. Instead, it could combine multiple forms of evidence: watermark and provenance verification, metadata and chain-of-custody analysis, forensic analysis of the image, audio, video, or text itself, detection of coordinated account behavior, and comparison against known generation models, each producing not a verdict but a confidence score and an explanation, with human review reserved for the most consequential decisions.
Its customers could include social media platforms, newsrooms, schools, courts, political campaigns, government agencies, insurers, and online marketplaces.
Most importantly, the company would not need to solve AI detection perfectly to be valuable. Spam filters, fraud detection systems, and cybersecurity software are all imperfect, yet remain economically useful because the losses they prevent and the review costs they reduce exceed the cost of using them. Conveniently (most importantly), AI detection would also benefit society at large.
A Future in Which Nothing Is Real
While the political consequences of AI-generated content are urgent and far-reaching, the implications extend far beyond elections.
Imagine opening a social media feed filled with people who are not real, describing experiences they never had, promoting products they never used, and expressing emotions they never felt. Then, when you arrive at the hotel, taste the restaurant’s food, or open the dress you ordered, reality bears little resemblance to what you were shown.
AI-generated media can be creative and useful, it should be celebrated. However, people need to know what they are seeing, for without that context, synthetic media will erode our ability to establish reality.
The future of social media should remain social; human to human. The content we consume should preserve a meaningful connection to real people, real experiences, and real emotions. AI will inevitably become part of how media is produced. As it does, being able to determine what is real will become essential both for protecting democracy and for preserving the fundamental trust that scaffolds societies.