01
Why this, why now
The July edition of this dossier carried a section called the regulation calendar. It described a set of dates approaching. The most important of them has now passed.
Since 2 August 2026, Article 50 of the EU AI Act is enforceable. Deepfakes, AI-generated text on matters of public interest, and chatbots must disclose that the content was created or manipulated by AI, and the disclosure has to be machine-readable. That is not a proposal or a consultation. It applies to organisations operating in the European market, which for a Dutch business means it applies now.
Two things about it are less widely understood than the deadline itself, and both are in section 9.
The Act is written to be technology-neutral and never names a standard. In practice there is one widely deployed open standard that fits the description, and the associated Code of Practice recommends pairing it with invisible watermarking because either alone struggles to meet the legal test. So the law is neutral and the compliance path is not. We think that distinction matters when you are choosing a vendor.
And adoption is uneven in a way that will produce the first enforcement stories. More than 6,000 organisations have adopted the provenance standard, including Adobe, Google, Microsoft and OpenAI. At least one prominent image generator had not, as of the deadline.
There is a second reason to rewrite rather than patch, and it is uncomfortable. Working through the loss figures for this edition, I found that the numbers circulating in this field are substantially worse-sourced than the numbers in any other dossier in this series, and that several of them contradict each other. Section 4 is about that, and it changed what this dossier is willing to claim.
Timeline
- Feb 2024 · A worker at Arup authorises 15 wire transfers worth $25.6 million after a video call populated entirely by synthetic colleagues.
- 2025 · The FBI opens its first standalone AI-fraud category.
- 2025-H1 2026 · The large majority of all documented deepfake fraud losses are recorded.
- 22 Jul 2026 · Signatory deadline for the EU Code of Practice on transparency.
- 2 Aug 2026 · Article 50 of the EU AI Act becomes enforceable.
02
Contents
1. The collapse of the faking cost
2. The psychology: the fakes do not just pass, they win
3. The business blast radius
4. The numbers problem in this field
5. Why detection loses, structurally
6. The feed problem: slop economics
7. The liar's dividend
8. What provenance can fix, and where it leaks
9. The law that arrived
10. The new literacy: verification as workflow
11. The scarce asset: a human who vouches
12. What we are watching
13. Verification and sources
03
1. The collapse of the faking cost
The foundation of this dossier is not a capability, it is a price.
Producing a convincing fake video used to require a studio, a budget and a team. It now requires a prompt and a few minutes. Every defence built over the last century against forged evidence, from photographic verification to voice recognition to the simple heuristic that a video of someone saying something is evidence they said it, rested on the assumption that faking was expensive. That assumption has gone, and it went quickly enough that institutions have not adjusted.
The consequence is not primarily that people will be fooled by specific fakes, though they will. It is that the entire category of media evidence has been devalued at once, and everything downstream of that category, journalism, courts, insurance claims, recruitment, corporate authorisation, has to find a new basis.
04
2. The psychology: the fakes do not just pass, they win
The uncomfortable finding in this literature is not that synthetic media fools people. It is that it can outperform the real thing.
Generated faces are frequently rated as more trustworthy than photographs of actual people, because the generation process smooths towards an average that human perception reads as honest. Synthetic voices can be rated as more authoritative. The fake is not a degraded copy trying to pass as genuine; it is optimised on exactly the dimensions people use to judge authenticity, and it beats the unoptimised original.
That inverts the intuitive defence. "Look closely and you will spot it" assumes the fake is worse. Increasingly the fake is better, in the specific sense of better-matching what people expect a trustworthy thing to look like, and closer looking makes it more convincing rather than less.
05
3. The business blast radius
For an organisation the exposure is broader than reputational.
The canonical case remains Arup, where in February 2024 a finance worker in Hong Kong authorised fifteen wire transfers totalling about $25.6 million after a video call in which every other participant was synthetic. It is still, as far as we can establish, the largest publicly documented single loss, which is itself worth noticing two and a half years later.
The general shape is that every process which authenticates a person by seeing or hearing them is now unreliable. Payment authorisation by video call. Identity verification by selfie. Password resets by voice. Recruitment interviews. Any approval that depends on recognising a face or a voice was built on an assumption that no longer holds, and most such processes were designed before anyone considered the question.
06
4. The numbers problem in this field
This section did not exist in July and it should have, because it changes how the rest of the dossier should be read.
Assembling the loss figures for this edition, I ran into something I have not encountered to the same degree in any other dossier in this series. The widely quoted statistics on deepfake fraud come overwhelmingly from companies that sell deepfake detection, they are frequently inconsistent with one another, and the inconsistencies are of a kind that suggests the numbers are not measuring the same thing.
Some examples from a single afternoon's reading. One figure puts AI-driven deepfakes behind roughly 11 percent of fraud worldwide. Another, circulating just as widely, puts it at 6.5 percent. One source reports at least $3.7 billion in documented global losses; another reports $1.1 billion in US losses, which may or may not be a subset. Investment fraud is described as 57 percent of analysed deepfake losses in one place and social media as 47 percent of losses in another, using bases that are not stated and probably not comparable.
The one figure with an unimpeachable provenance is the weakest-sounding: the FBI's first standalone AI-fraud category logged about $893.3 million in adjusted losses for 2025 across 22,364 complaints. That is a law-enforcement body counting reports made to it, with all the undercounting that implies, and it is an order of magnitude below the largest circulating claims.
Two conclusions follow, and they pull in opposite directions, which is why both belong here.
The threat is real and the FBI number is a floor, not a ceiling. Fraud is systematically underreported, victims of authorisation fraud are often institutionally embarrassed, and a category opened in 2025 has no history to compare against.
And a board presentation quoting a billion-dollar figure from a detection vendor is quoting marketing. This is not an accusation of dishonesty. It is the observation that the incentive runs one direction, the methodology is usually unstated, and the numbers behave accordingly. Where this dossier previously used such figures with a medium confidence rating, it now names them and their source, or does not use them.
The practical implication for a reader is narrower than it sounds: the case for acting does not depend on the size of the aggregate. It depends on whether your own authorisation processes can be defeated by a video call, and that is a question you can answer about yourself without any market statistic at all.
07
5. Why detection loses, structurally
Before the practical sections we have one argument to clear away, because it is the first thing most organisations reach for and we think it leads nowhere.
The instinct is to buy a detector. If synthetic media is the problem, find the tool that spots it. We think that instinct is wrong, and we would rather show why than assert it, because the reason generalises to several other problems in this series.
Detection is an adversarial contest with asymmetric costs. A detector is trained on the artefacts that current generators leave behind: inconsistent lighting, implausible hands, statistical fingerprints in the noise. Every one of those is a defect, and defects get fixed. Worse, a published detector is a training signal: it tells the next generation of generators precisely what to eliminate. The defender must catch everything and must publish enough about the method for anyone to trust it. The attacker needs to get through once and learns from every failure.
There is a second asymmetry that is less discussed. Detection accuracy is quoted on benchmarks of known fakes, and the operational question is different. If one in a thousand items crossing your desk is synthetic, a detector with 99 percent accuracy on both classes will flag roughly eleven items for every genuine one it catches, because the false positives are drawn from a pool a thousand times larger. That is the base-rate problem, it is arithmetic rather than a limitation of any particular product, and it means a detector good enough to demonstrate impressively is often useless enough to be switched off within a month.
None of that makes detection worthless. We treat it as a filter that reduces volume, not a gate that establishes truth, and we expect an organisation that treats it as the second to be surprised. The two things we have seen work are in sections 8 and 10: authenticate the origin rather than interrogating the artefact, and design processes that do not require the artefact to be trusted at all.
08
6. The feed problem: slop economics
Alongside deliberate fraud sits the larger-volume, lower-intent problem: synthetic content produced because producing it is nearly free.
The economics are straightforward. If content generates revenue per view and costs almost nothing to make, the equilibrium quantity is enormous. The result is a rising share of what circulates being generated rather than reported, observed or experienced, and the effect on any given piece is less important than the effect on the base rate. When most of what you encounter might be synthetic, the cost of verifying each item exceeds its value, and the rational response is to stop verifying and start discounting everything.
We locate the damage there. The reasonable person stops extending belief at all, and that is harder to repair than any single false story.
09
7. The liar's dividend
The mirror image of the fake that fools you is the real thing that no longer convinces.
Once everyone knows that video can be generated, genuine video acquires a defence: it can be dismissed. A recording of someone doing something becomes deniable, and the denial is now plausible in a way it was not five years ago. This is called the liar's dividend, and it accrues to whoever has the most to deny.
For institutions we regard this as the quieter half of the problem. Fraud is expensive and detectable. The erosion of the ability to prove anything with a recording is neither, and it degrades processes, evidence and accountability without any specific incident to point at.
10
8. What provenance can fix, and where it leaks
The main technical answer is provenance: cryptographically signing content at the point of capture or creation, so that a chain of custody travels with the file.
It is a real answer to a real part of the problem, and we want to be precise about which part. Provenance does not detect fakes. It authenticates originals. A signed file can be shown to come from a particular camera or tool with a particular edit history; an unsigned file is simply unsigned, which is the state of essentially all content produced before now and much produced since.
The leaks are structural rather than technical. Metadata is stripped by most platforms as a matter of routine processing. A screenshot of a signed image is an unsigned image. The chain requires every step, capture, editing, publication, redistribution, to preserve it, and the weakest step governs. And provenance says nothing about truth: a genuinely captured video of a staged event carries a perfect chain of custody.
What provenance does deliver, and we do not think it is nothing, is a floor. In a world where most content is unsigned, a signed item is distinguishable. That is worth more inside an organisation, where you control the pipeline, than out in the open feed, where you do not.
11
9. The law that arrived
Article 50 of the EU AI Act became enforceable on 2 August 2026, and the July edition's countdown can now be replaced with a description.
What it requires. Deepfakes must be disclosed as artificially generated or manipulated. AI-generated text published on matters of public interest must be disclosed. Chatbots must make clear that the user is interacting with a machine. The disclosure must be machine-readable, which rules out a visible caption alone.
What it does not say. It does not name C2PA, or any other standard. The Act is technology-neutral by drafting policy.
What that means in practice. There is currently one widely deployed open standard that meets the description, and the associated Code of Practice recommends using cryptographically signed metadata together with invisible watermarking, on the reasoning that either alone struggles to satisfy the legal requirements of being effective, interoperable, robust and reliable. So an organisation choosing how to comply is formally free and practically steered. That gap between the letter and the path is where the early compliance advice will be sold. We would want to understand it before buying any of that advice.
Where the exposure sits. More than 6,000 organisations have adopted the provenance standard, including Adobe, Google, Microsoft and OpenAI. At least one prominent image generator had not deployed it as of the deadline, which is the kind of situation that produces the first enforcement case and a great deal of coverage.
For a Dutch organisation the practical question is not whether the law applies. It is which of your outputs count. Marketing imagery that has been AI-enhanced, a customer-service chatbot, a synthetic voice in a training video, an AI-drafted opinion piece under an executive's byline: some of those are clearly in scope, some are arguable, and the arguable ones are where the work is.
We will walk through why the arguable ones are arguable, because the pattern repeats across almost every organisation we have looked at.
The obligation on deepfakes attaches to content that resembles real people, objects or events and would falsely appear authentic. A wholly generated photograph of a person who does not exist, used as a stock image, is squarely inside. A photograph of a real product with the background replaced is somewhere else entirely, and reasonable advisers currently disagree about where. The line the drafting seems to intend is between manipulation that changes what a viewer would conclude about reality and manipulation that does not, which is a sensible principle and a poor operational test, because it asks a marketing team to predict how a regulator will read a viewer's mind.
The obligation on text is narrower than most summaries suggest. It attaches to text published to inform the public on matters of public interest, which is a category built for something closer to journalism than for a product description. A company blog about a new feature is probably outside it. A company blog arguing a position on regulation is closer to the line than the people writing it usually realise.
And the obligation on chatbots is the clearest of the three and the one most often already satisfied by accident, since most deployed assistants announce themselves anyway.
The practical advice that follows is dull. Inventory first, in three buckets: clearly in, clearly out, arguable. Label everything in the first bucket now, because it is cheap and it removes the easy findings. Keep a written record of the reasoning on the second and third, because the value of that record is not that it is right but that it demonstrates the question was asked, and the early enforcement in regimes like this one has historically fallen hardest on organisations that could not show they had considered the question at all.
I would add one caution about the advice market that is forming around this. The gap described earlier, between a technology-neutral law and a single practical compliance path, is exactly the shape that produces confident vendor claims about what the law requires. The law requires disclosure that is effective, interoperable, robust and reliable. It does not require any particular product, and anyone saying otherwise is describing their own roadmap.
12
10. The new literacy: verification as workflow
The individual-level answer, learn to spot fakes, does not scale and section 2 explains why: the fakes are optimised on the cues people use.
The answer we would build is procedural, and it belongs in process design rather than in training. The minimum viable version has three parts.
Authenticate through a second channel that the attacker does not control. A video call requesting a payment is verified by a callback on a known number, not by looking harder at the video. This single change would have prevented the Arup loss.
Set thresholds that require more than one human. Any authorisation above a stated value requires a second approver reached independently. Our point is not that the second person is harder to fool. It is that fooling two people through two channels is a materially harder attack.
And we would decide in advance what your organisation treats as evidence. Most organisations have never written this down, which means the answer defaults to whatever the most senior person in the room finds convincing, and section 2 is about how that intuition now fails.
13
11. The scarce asset: a human who vouches
The pattern this dossier ends on is the same one our what-stays-human dossier reaches from an entirely different direction.
When content is free to produce and impossible to authenticate by inspection, the scarce thing is a person or institution willing to stake something on a claim being true. Not a detection algorithm, which is in an arms race it structurally loses, and not a watermark, which section 8 shows is a floor rather than a guarantee. A named human with a reputation at risk.
That is why we think provenance matters more inside institutions than outside them, why trusted intermediaries become more valuable as content becomes more abundant, and why we read the organisational response to synthetic media as a governance question rather than a technology purchase.
14
12. What we are watching
The first Article 50 enforcement action. Who, for what, and how severe. The answer will set the practical compliance bar far more than the text does.
Whether platforms stop stripping provenance metadata. This is the single change that would make section 8 work at internet scale, and there is now a regulatory reason for it to happen.
Whether the loss statistics improve. If independent bodies, insurers, regulators or law enforcement start publishing methodology alongside numbers, section 4 becomes unnecessary. That would be good news.
Authorisation fraud against mid-sized companies. The reported cases are large organisations. The exposure is worse at companies with fewer controls and no dedicated security function, and those cases are less likely to be reported.
Whether the liar's dividend shows up in court. The first serious dispute in which genuine video evidence is successfully challenged on the grounds that it could have been generated will be a landmark, and it is a matter of when.
15
13. Verification and sources
This dossier draws on live web research and a personal archive of more than 15,000 sources. Section 4 explains why this appendix is doing more work than usual: the sourcing quality in this field is materially worse than in the rest of the series, and several widely circulated figures have been dropped from this edition rather than carried with a caveat.
| Claim | Confidence | Note |
|---|---|---|
| Article 50 of the EU AI Act enforceable from 2 August 2026; deepfakes, AI text on public-interest matters and chatbots must disclose; disclosure must be machine-readable | High | The Act and contemporaneous compliance analysis. Scope questions at the margin, particularly for AI-assisted rather than AI-generated material, are genuinely unsettled and we do not resolve them here. |
| The Act is technology-neutral and does not name C2PA; the associated Code of Practice recommends signed metadata plus invisible watermarking | Medium-high | Compliance analyses, 2026. The reading that this makes one standard a de facto path is an inference we endorse, not a legal finding. |
| More than 6,000 organisations have adopted C2PA, including Adobe, Google, Microsoft and OpenAI | Medium-high | Standard-body reporting. "Adopted" covers a wide range from membership to full deployment. |
| At least one prominent image generator had not deployed provenance as of the deadline | Medium | Contemporaneous reporting. A situation that can change quickly, and may have by the time this is read. |
| 22 July 2026 signatory deadline for the transparency Code of Practice | Medium-high | Contemporaneous reporting. |
| Arup: about $25.6 million lost via 15 transfers after a video call of synthetic participants, Hong Kong, February 2024 | High | Widely reported, company-confirmed. Still the largest publicly documented single case as far as we can establish, which is a statement about disclosure as much as about frequency. |
| FBI's first standalone AI-fraud category: ~$893.3 million adjusted losses in 2025 across 22,364 complaints | Medium-high | Law-enforcement reporting of complaints received. The best-sourced figure in this dossier and a floor rather than an estimate of the total, since fraud is systematically underreported and authorisation fraud especially so. |
| At least $3.7 billion in documented global deepfake-fraud losses, with the large majority recorded in 2025 and the first half of 2026 | Low-medium | A commercial study, 2026. Counts only incidents with publicly reported losses, which is both an undercount and a selection. Reported here with its provenance because section 4 argues the provenance is the story. |
| Deepfakes behind ~11% of fraud worldwide | Dropped | Contradicted by an equally widely circulated figure of 6.5%, with neither base stated. We could not establish which, if either, is right, and a percentage of "all fraud" without a defined denominator is not a usable number. |
| Investment fraud at 57% of analysed deepfake losses; social media at 47% of losses; 8 million deepfakes online; synthetic identity fraud at ~$6 billion in annual credit losses | Low | Vendor-published figures with unstated methodology and non-comparable bases. Listed so a reader who encounters them elsewhere knows we saw them and declined to build on them. |
| Generated faces frequently rated more trustworthy than photographs of real people | Medium-high | Peer-reviewed perception research. Effect sizes vary by study and population; the direction is robust. |
| Provenance authenticates originals rather than detecting fakes; metadata is routinely stripped by platforms; a screenshot of a signed image is unsigned | High | Properties of the standard rather than empirical claims. |
| The liar's dividend | Framing | An established concept in the literature on synthetic media, not a measured effect. |
Charts labelled "BFF" are our own, drawn from the sources named beneath them.
Dossier as pdf
Download this dossier
The full dossier as a pdf, with every figure and the source list. Fill in your details and the download starts right away.
Ruben Horbach
Co-founder · Back From the Future
Ruben researches how organisations adopt AI meaningfully — not as technology, but as a change in work and people. He builds the agent infrastructure behind BFF and speaks about the near future of work.
