Speakers: Thomas Mason, Tahseen Rabbani · GitHub: WAVES (synthid-regen-kit)

📄 Download this note as Markdown

Overview

Google’s SynthID watermark invisibly tags content generated by its models (Nano Banana, Gemini, etc.) — when asked whether an image is AI-generated, Gemini invokes a “Verify AI” tool to scan for it. This talk demonstrates two attack strategies to remove it: (1) a lesser-known regeneration attack (Zhao et al. 2023) that removes SynthID identification with 100% success on a held-out set of 104 photorealistic images, and (2) a surrogate detector built on Apple’s Pico-Banana-400K dataset, fine-tuning a ResNet-18 to detect the watermark offline in ~27.5ms on a CPU — fast enough to iterate an attack without ever querying the real deployed detector. Removing the watermark can also induce hallucinations, with Gemini confidently making false assertions about a now-unmarked simulated image as if it were real, with implications for confused-deputy attacks, retrieval pipeline poisoning, and facial recognition bypasses.

Notes

Watermarks 101

Watermarks indicate provenance — a stamp for real vs. generated. The visible badge on an AI image isn’t the watermark itself, just an indicator one is present. “In-processing” watermarks add a key during generation; “post-hoc” methods apply it afterward. SynthID (Google DeepMind) is a post-hoc, cross-modality family of watermarks, and Gemini ships a SynthID checker.

Experiment

Simple transformations — color/B&W, saturation, bordering, resizing, rotating, cropping — weren’t able to reliably remove a SynthID watermark from Nano Banana output.

Technique 1: Screenshotting

Taking a screenshot was a reliable, very simple bypass — and screenshots are common enough to evade suspicion.

Technique 2: Regeneration

Run the image through an off-the-shelf diffusion model: add noise, then de-noise — the opposite step of normal generation, which removes noise to arrive at a clear image. Using Stable Diffusion v1.4 at N=10 steps, detection dropped to 20%; more steps drop detection further but introduce artifacts a human can spot, so there’s a real tradeoff between machine and human detection.

Hallucination as a Byproduct

Once the SynthID is removed, the model no longer knows an image is probably not real — it changes how it treats and reasons about the image, confidently trying to determine details as if it were genuine.

Building a SynthID Detector

Google’s real detector is a black box, so the researchers built pairs of watermarked/clean images from Apple’s Pico-Banana-400K dataset and trained a CNN discriminator — 90% final test accuracy, more conservative than the real detector, and useful offline without tipping off the system you’re trying to fool.

Security Implications

Google, OpenAI, ElevenLabs, and Kakao all use SynthID, but proprietary watermarks can still be removed with off-the-shelf and older models. Recommendations: standardize, open-source more components for battle-testing, and use defense in depth — pairing watermark checks with metadata standards like C2PA and LLM reasoning over the whole image for obvious inconsistencies.

From the Repo

Ships 104 SynthID-watermarked images with generation prompts, pre-rendered attacks at N=10/20/40/80 steps, and 15 real/synthetic pairs from Pico-Banana-400K. The regen/ module implements the regeneration attack with a custom resumable pipeline; detector/ is the ResNet-18 surrogate (89.8% accuracy). At N=80 steps, detection drops to 37.5% from an 89.4% baseline — deeper regeneration cracks the watermark further at the cost of more visible artifacts.

Filed Under: #defcon34