Deepfakes and Misinformation

Generative AI made convincing fake voices, pictures and videos cheap. This page looks at what has actually happened, what the evidence says about the damage, the subtler harm of doubting everything, and the defences that work — from content credentials to Malaysia's new online safety law.

Two days before the New Hampshire presidential primary in January 2024, thousands of voters picked up the phone and heard what sounded like President Joe Biden telling them to "save" their vote for November and stay home. It was not Biden. A political consultant had paid a magician US$150 to make the recording with an AI voice tool. The call was traced within days, the consultant was fined US$6 million by the Federal Communications Commission — and then, in June 2025, a jury acquitted him of the criminal charges. One cheap fake; a fast detection; a messy legal ending. That pattern recurs throughout this story.

What generative AI changed

Fake photos and doctored quotes are as old as photography and newspapers. Misleading content does not need AI: a real photo with a false caption ("cheapfake") still fools more people than any deepfake. What generative AI changed is the economics. Follow the steps of a typical deepfake investment scam to see where.

Cost, scale and realism all moved together. So did voice cloning, which powers "it's me, I'm in trouble, send money" calls — covered, with practical defences, on Staying Safe with AI.

Documented cases

These cases are well documented by courts, regulators or major news organisations. Filter by type As of October 2026.

  1. Sep 2023Slovakia: fake audio before polling dayDays before the 30 September parliamentary election, an audio clip spread in which the leader of the liberal Progressive Slovakia party appeared to discuss rigging the vote with a journalist. Both said it was fake, and fact-checkers found signs of AI manipulation. It circulated during the legal pre-election silence period, when media could not easily rebut it. His party came second; whether the clip changed any votes is unknown.
  2. Jan 2024New Hampshire: the fake Biden robocallThe call described above. In February 2024 the US FCC ruled that AI-generated voices in robocalls count as "artificial" under existing law. The telecoms carrier settled for US$1 million; the consultant was acquitted by a jury in June 2025 and told reporters he would not pay the FCC fine.
  3. Jan 2024Sexual deepfakes of a pop starExplicit AI images of Taylor Swift spread widely on X before removal; the platform temporarily blocked searches for her name. Researchers have long found that most deepfakes online are non-consensual sexual images, overwhelmingly of women.
  4. Early 2024Hong Kong: the deepfake video meetingAn employee of the engineering firm Arup transferred about HK$200 million after a video call in which the "finance director" and colleagues were all deepfakes, as Hong Kong police described.
  5. 2023–2025Malaysia: deepfake adverts of national figuresScam adverts have repeatedly used AI-generated likenesses of Prime Minister Anwar Ibrahim, the King and well-known business people to promote non-existent investment schemes. In November 2025 MCMC removed clips featuring the King and the Prime Minister; similar adverts reappeared under other faces.
  6. Jan 2026Malaysia and Indonesia restrict GrokAfter xAI's Grok chatbot on X was widely misused to create sexualised edited images of real women and minors, Indonesia blocked it and MCMC restricted access in Malaysia from 11 January 2026, saying X's response relied too much on user reporting. Access was restored later that month after the platform added safeguards.

The scale in Malaysia is visible in the regulator's own takedown figures. These count all fraudulent and scam content removed at MCMC's request, not only AI-made material, and the government itself says they are approximate.

The liar's dividend

The most corrosive effect may not be that people believe fakes, but that they stop believing anything. Law professors Robert Chesney and Danielle Citron named this the liar's dividend in work published in 2019: once everyone knows realistic fakes are possible, a person caught on a genuine recording can simply claim "that's AI". The more the public hears about deepfakes, the more credible the denial becomes.

This has already happened. Politicians in several countries have dismissed authentic audio as fabricated, and courts now face defendants arguing that real video evidence might be synthetic. In other words, deepfakes tax the truth twice: once by spreading lies, and again by giving cover to people who want true things disbelieved.

Key point

The right response to deepfakes is not "trust nothing" — that hands liars the dividend. It is "check the source", using the same habits fact-checkers use for any surprising claim.

How much harm, really?

Debated 2024 was billed as the year of AI election chaos, with national elections in dozens of countries. Researchers then went looking for the damage — and disagree about what they found.

Several careful studies found less than predicted. The Alan Turing Institute's Centre for Emerging Technology and Security (CETaS) reported in late 2024 that of 112 national elections since January 2023, 19 showed AI-enabled interference, and that there was no conclusive evidence AI disinformation had changed any result. Meta reported in December 2024 that AI-generated content made up less than 1% of the election, political and social misinformation its fact-checkers rated. In 2023 researchers Felix Simon, Sacha Altay and Hugo Mercier argued that fears were "overblown": the limit on misinformation is not supply but demand and attention, and those who consume it are mostly already convinced.

Other views
  • "Smaller than feared" (Simon, Altay and Mercier; CETaS on election outcomes; Meta): deepfakes mostly reinforce existing beliefs; panic about them can itself erode trust in genuine news.
  • "Harm is real but indirect" (CETaS; many fact-checkers and journalists): even without swinging votes, AI content amplifies conspiracy theories, harasses candidates — women in particular — and wears down trust in the whole information environment, effects that are hard to measure.
  • "We are measuring the wrong place": researchers in Asia, Africa and Latin America point out that most studies cover rich democracies and large platforms, while much manipulation happens in closed messaging groups, local languages and countries with weaker fact-checking — and that the biggest everyday harms are fraud and sexual abuse, not elections.
  • Platform self-reports such as Meta's are useful but cover only content fact-checkers reviewed; critics ask for independent data access.

Speculative Whether the balance shifts as tools become more realistic, interactive (deepfake video calls) and personalised is an open question; most researchers expect fraud and abuse to grow faster than election effects.

Defences that work

No single fix exists. Defences work in layers, from the technology to the individual.

C2PAContent provenanceProve what is real

Content credentials following the C2PA standard attach a signed record of which camera, app or AI tool made or edited a file. The idea flips detection around: rather than spotting fakes, it lets genuine material prove its origin. The weakness: credentials can be stripped, and most content still carries none As of 2026. Details on Making Images and Media.

WMWatermarkingMark what is synthetic

An AI watermark such as Google DeepMind's SynthID hides a signal in generated images, audio, video or text that a matching detector can find. It only works for tools that add it, and open-source tools need not. The EU AI Act requires providers to mark AI-generated content in machine-readable form and deployers to label deepfakes; these transparency duties began applying in August 2026, with some deadlines for existing systems extended As of 2026.

PLATPlatform policiesLabels and takedowns

Major platforms require creators to disclose realistic AI-altered content, add "AI info" labels, ban deceptive election deepfakes and non-consensual intimate imagery, and remove scam adverts when reported. Enforcement is uneven, and paid scam adverts keep reappearing — which is why regulators increasingly target the platforms themselves.

SIFTMedia literacyThe habit that scales

Detection tools are unreliable for ordinary users, so the most robust defence is behaviour: pause before sharing, check who is behind a claim, look for the same story from independent reliable sources (lateral reading), and trace the original. The Toolkit page Cutting Through the Noise turns this into five quick questions.

LAWLawMalaysia's framework

General information, not legal advice.

The Communications and Multimedia (Amendment) Act 2025 came into force on 11 February 2025 (with minor exceptions). It raised the maximum fine for improper use of network facilities under section 233 — the provision often used against harmful or false online content — from RM50,000 to RM500,000. The Online Safety Act 2025 (Act 866) came into force on 1 January 2026. It places duties on licensed platform and network providers — not individual users — to reduce harmful content such as financial fraud and child sexual abuse material, and gives MCMC powers to order removal As of 2026. Critics, including civil-society and press-freedom groups, warn that broad powers over "false" or "offensive" content can also be used against legitimate speech. See Regulating AI for the wider picture.

Test yourself: realistic scenarios

Each situation is the kind of thing that turns up in Malaysian family chat groups. Pick the best response.

Scenarios

A video ad on social media shows the Prime Minister recommending an investment platform that "guarantees" 30% monthly returns. What should you do?

Leaders do not endorse private investment schemes, and guaranteed high returns are a classic scam sign. Sharing spreads it further.

Your aunt receives a voice message from "your cousin" crying and asking for an urgent transfer. What is the best first step?

Calling back through a known channel defeats voice cloning. Personal details can be found or guessed by scammers.

Two days before an election, an audio clip of a candidate "admitting" fraud goes viral. Which response fits the evidence best?

Late-campaign fakes (as in Slovakia, 2023) rely on there being no time to check. Refusing to believe anything hands liars the dividend.

A politician caught on a genuine recording says, "That's obviously a deepfake." What is this an example of?

The existence of deepfakes makes it easier to deny real evidence. Provenance records are one way to push back.

An "AI detector" website says a photo is 92% likely to be real. How much should that settle it?

Detection tools give false results in both directions. Source checking and provenance are more reliable.

// Pause and reflect

  1. When did you last forward something in a group chat without checking where it came from? What would a 30-second check have looked like?
  2. Does knowing about deepfakes make you more sceptical of everything you see? Is that healthy, or does it hand liars the dividend?
  3. Who should carry most of the burden — the AI companies, the platforms, the government, or each of us? Why?

Your notes are saved privately in this browser.

Key terms on this page

Sources & further reading