# Evading LLM Detection

- URL: https://bryancowan.com/evading-llm-detection/
- Published: August 12, 2026
- Series: Conference notes
- Tags: defcon34
- Part of: [DEF CON 34 Notes: 15 Talks on AI Security, Agentic Systems, and Red Teaming](https://bryancowan.com/def-con-34-notes-15-talks-on-ai-security-agentic-systems-and-red-teaming/)

**Speakers:** [Hanley Shun](https://info.defcon.org/defcon34/people/67593), [Cong Zhang](https://info.defcon.org/defcon34/people/67569), [Oscar Skjerven](https://info.defcon.org/defcon34/people/67637) · **Source:** [DEF CON 34 talk page](https://info.defcon.org/defcon34/content/66769)

📄 [Download this note as Markdown](https://raw.githubusercontent.com/bryancowan/defcon34-notes/main/Evading%20LLM%20Detection.md)

## Overview

From OKX (fintech and crypto company). Software supply chain attacks are no longer a distant threat — they're happening at scale and are extremely dangerous from a crypto exchange's POV. As build pipelines grow more complex and dependencies multiply across npm, PyPI, SBOM, and internal registries, a single scanning layer is no longer enough to detect malicious code. This talk covers an AI journey — from deploying a single AI agent to gate code merges, to architecting a full multi-agent system hardened through structured simulated exercises — along the way catching real-world attacks, including the coordinated compromise of a highly popular npm package spanning over 2 billion weekly downloads.

## Notes

### Using an LLM to Detect Malicious Code
- Example: `simple-swizzle@0.2.3` on npm was compromised to add a crypto stealer — a package with 2 billion downloads. Their pipeline's LLM detection was one of the first to flag it, returning `is_malicious`, `probability`, `reasoning`, and `identified_threats` for the security team to confirm.
- Prompt: *"You are a security expert analyzing packages for malicious code for the package at [location]. Output as JSON."*
- Pipeline: merge request → scan → warn → continue or block. Testing found it can spot encryption and obfuscation well.

### Failure Modes
- **Context exhaustion** — malicious code gets skipped because the model never saw it. Solution: read metadata first, flag unreasonable scenarios, scan incrementally/in batches, sample files and follow only the execution chain.
- **Steganography** — the model sees an innocent-looking image but can't inspect the raw JPEG data, and no eval/exec call is visible, so the payload stays invisible. Solution: prohibit dynamic file reading/generation/modification, disallow unexpected file types.
- **Prompt injection** — text embedded in a file tells the agent what conclusion to reach about that file's own content, poisoning the analysis with an authoritative-sounding false conclusion. Solution: run an isolated verification agent with isolated context in a separate run, requiring verdict agreement between the two — injection in one run doesn't affect the other.
- **Git submodule** — stage the payload in an older commit; the submodule points to the same repo so the LLM says it looks fine, and the payload never shows up in a normal git diff. Solution: harden the build environment.

### Multi-Agent Strategy
An orchestrator spawns agents (dependency, branch, verification, Semgrep) and collects results.

### LLM Detection Tradeoffs
**Weaknesses:** indeterminism/inconsistency, binary blindness, content rot, hallucination, false positives.
**Strengths:** intent reasoning without signatures, no rule updates required, cross-file reasoning, low development effort, minimal maintenance.

_Filed Under:_ #defcon34