Speakers: Hanley Shun, Cong Zhang, Oscar Skjerven · Source: DEF CON 34 talk page

📄 Download this note as Markdown

Overview

From OKX (fintech and crypto company). Software supply chain attacks are no longer a distant threat — they’re happening at scale and are extremely dangerous from a crypto exchange’s POV. As build pipelines grow more complex and dependencies multiply across npm, PyPI, SBOM, and internal registries, a single scanning layer is no longer enough to detect malicious code. This talk covers an AI journey — from deploying a single AI agent to gate code merges, to architecting a full multi-agent system hardened through structured simulated exercises — along the way catching real-world attacks, including the coordinated compromise of a highly popular npm package spanning over 2 billion weekly downloads.

Notes

Using an LLM to Detect Malicious Code

• Example: [email protected] on npm was compromised to add a crypto stealer — a package with 2 billion downloads. Their pipeline’s LLM detection was one of the first to flag it, returning is_malicious, probability, reasoning, and identified_threats for the security team to confirm.
• Prompt: “You are a security expert analyzing packages for malicious code for the package at [location]. Output as JSON.”
• Pipeline: merge request → scan → warn → continue or block. Testing found it can spot encryption and obfuscation well.

Failure Modes

• Context exhaustion — malicious code gets skipped because the model never saw it. Solution: read metadata first, flag unreasonable scenarios, scan incrementally/in batches, sample files and follow only the execution chain.
• Steganography — the model sees an innocent-looking image but can’t inspect the raw JPEG data, and no eval/exec call is visible, so the payload stays invisible. Solution: prohibit dynamic file reading/generation/modification, disallow unexpected file types.
• Prompt injection — text embedded in a file tells the agent what conclusion to reach about that file’s own content, poisoning the analysis with an authoritative-sounding false conclusion. Solution: run an isolated verification agent with isolated context in a separate run, requiring verdict agreement between the two — injection in one run doesn’t affect the other.
• Git submodule — stage the payload in an older commit; the submodule points to the same repo so the LLM says it looks fine, and the payload never shows up in a normal git diff. Solution: harden the build environment.

Multi-Agent Strategy

An orchestrator spawns agents (dependency, branch, verification, Semgrep) and collects results.

LLM Detection Tradeoffs

Weaknesses: indeterminism/inconsistency, binary blindness, content rot, hallucination, false positives.
Strengths: intent reasoning without signatures, no rule updates required, cross-file reasoning, low development effort, minimal maintenance.

Filed Under: #defcon34