# This Wasn't AI Generated — Principles for Breaking Generative Watermarks

- URL: https://bryancowan.com/this-wasnt-ai-generated-principles-for-breaking-generative-watermarks/
- Published: August 12, 2026
- Series: Conference notes
- Tags: defcon34
- Part of: [DEF CON 34 Notes: 15 Talks on AI Security, Agentic Systems, and Red Teaming](https://bryancowan.com/def-con-34-notes-15-talks-on-ai-security-agentic-systems-and-red-teaming/)

**Speakers:** [Thomas Mason](https://info.defcon.org/defcon34/people/67936), [Tahseen Rabbani](https://info.defcon.org/defcon34/people/67916) · **GitHub:** [WAVES (synthid-regen-kit)](https://github.com/rabbanitw/WAVES/tree/synthid-regen-kit)

📄 [Download this note as Markdown](https://raw.githubusercontent.com/bryancowan/defcon34-notes/main/This%20Wasn%27t%20AI%20Generated%20-%20Principles%20for%20Breaking%20Generative%20Watermarks.md)

## Overview

Google's SynthID watermark invisibly tags content generated by its models (Nano Banana, Gemini, etc.) — when asked whether an image is AI-generated, Gemini invokes a "Verify AI" tool to scan for it. This talk demonstrates two attack strategies to remove it: (1) a lesser-known regeneration attack (Zhao et al. 2023) that removes SynthID identification with 100% success on a held-out set of 104 photorealistic images, and (2) a surrogate detector built on Apple's Pico-Banana-400K dataset, fine-tuning a ResNet-18 to detect the watermark offline in ~27.5ms on a CPU — fast enough to iterate an attack without ever querying the real deployed detector. Removing the watermark can also induce hallucinations, with Gemini confidently making false assertions about a now-unmarked simulated image as if it were real, with implications for confused-deputy attacks, retrieval pipeline poisoning, and facial recognition bypasses.

## Notes

### Watermarks 101
Watermarks indicate provenance — a stamp for real vs. generated. The visible badge on an AI image isn't the watermark itself, just an indicator one is present. "In-processing" watermarks add a key during generation; "post-hoc" methods apply it afterward. SynthID (Google DeepMind) is a post-hoc, cross-modality family of watermarks, and Gemini ships a SynthID checker.

### Experiment
Simple transformations — color/B&W, saturation, bordering, resizing, rotating, cropping — weren't able to reliably remove a SynthID watermark from Nano Banana output.

### Technique 1: Screenshotting
Taking a screenshot was a reliable, very simple bypass — and screenshots are common enough to evade suspicion.

### Technique 2: Regeneration
Run the image through an off-the-shelf diffusion model: add noise, then de-noise — the opposite step of normal generation, which removes noise to arrive at a clear image. Using Stable Diffusion v1.4 at N=10 steps, detection dropped to 20%; more steps drop detection further but introduce artifacts a human can spot, so there's a real tradeoff between machine and human detection.

### Hallucination as a Byproduct
Once the SynthID is removed, the model no longer knows an image is probably not real — it changes how it treats and reasons about the image, confidently trying to determine details as if it were genuine.

### Building a SynthID Detector
Google's real detector is a black box, so the researchers built pairs of watermarked/clean images from Apple's Pico-Banana-400K dataset and trained a CNN discriminator — 90% final test accuracy, more conservative than the real detector, and useful offline without tipping off the system you're trying to fool.

### Security Implications
Google, OpenAI, ElevenLabs, and Kakao all use SynthID, but proprietary watermarks can still be removed with off-the-shelf and older models. Recommendations: standardize, open-source more components for battle-testing, and use defense in depth — pairing watermark checks with metadata standards like C2PA and LLM reasoning over the whole image for obvious inconsistencies.

### From the Repo
Ships 104 SynthID-watermarked images with generation prompts, pre-rendered attacks at N=10/20/40/80 steps, and 15 real/synthetic pairs from Pico-Banana-400K. The `regen/` module implements the regeneration attack with a custom resumable pipeline; `detector/` is the ResNet-18 surrogate (89.8% accuracy). At N=80 steps, detection drops to 37.5% from an 89.4% baseline — deeper regeneration cracks the watermark further at the cost of more visible artifacts.

_Filed Under:_ #defcon34