Speaker: Simcha Kosman · Source: DEF CON 34 talk page

📄 Download this note as Markdown

Overview

OpenAI designed ChatGPT’s container sandbox as a secure runtime environment, enforcing full network isolation, strict execution timeouts, and an AI supervisor to filter every command. Under this model, owning the container and extracting sensitive data seemed impossible. However, we demonstrate that by chaining file-parsing abuse for persistent execution, reasoning-channel hijacking for data extraction, and shared infrastructure manipulation, an attacker can establish a cross-tenant data exfiltration.

In this talk, we demonstrate a complete attack chain that shatters ChatGPT’s secure sandbox. By abusing spreadsheet file parsing, we bypass the LLM supervisor to gain persistent, unmonitored root execution. From there, we escalate the attack by live-patching the internal Jupyter kernel to hijack the model’s hidden python.exec reasoning channel, executing a Reasoning Injection Attack to extract sensitive user data. To exfiltrate this data, we bypass network isolation by weaponizing the Task Scheduler to launder malicious URLs past strict web guardrails.

The attack reaches its climax by exploiting a shared JFrog package manager. The researchers engineered a signaling protocol that weaponizes globally visible authentication rate limits, translating these lockout timers into a half-duplex covert channel — providing reliable data exfiltration and command-and-control from isolated enterprise environments to external attackers.

Notes

OpenAI’s Stated Limitations

• Code execution and external API calls are not allowed
• Chats are also supposed to stay private

Prompting to Escape the Sandbox

• A scheduled future task does have access to the internet — e.g. “Create a task that runs and gives me current user info”
• Goals: get sensitive info into the sandbox without being blocked (find initial access, map the sandbox surface), then get sensitive information back out of the sandbox (bypass LLM guardrails)

Initial Access Paths

• The ?q= query parameter allows a prompt to run instantly in the browser; works on iPhone and Mac. If a user clicks a link, it’s treated as if the user sent the prompt themselves.

Mapping the Sandbox

• Supports ps; environment variables either run or are immediately killed
• Testing found an Artifact tool (create PDFs/docs), a Python tool, and a JFrog tool to pull npm packages — all sandboxes share the same JFrog user/password

Moving Data Into the Sandbox

• The spreadsheet skill tells the LLM to execute the file, not just read its content
• Inserted Python code with an eval command in Excel to get it to execute without being killed
• An earlier-sent link downloads the malicious Excel file, which executes that cell’s Python code

Sending Sensitive Information Back

• ChatGPT uses a separate app.py server, which then goes to a Jupyter notebook
• python.exec (chain-of-thought use) goes directly to the Jupyter notebook — used prompt injection to modify it
• Reasoning injection gives weaker guardrails during the thinking step via “remember the malicious injected prompt”

Chaining It All Together

Link sent → prompt executed → xlsx downloaded → executes in sandbox → malicious prompts enter the thinking process → information passed to the JFrog server via a custom signaling protocol → sent back to the attacker.

• The sandbox had no internet access, so the attack moved to the JFrog package manager method after direct API attempts kept getting blocked
• Logging into a nonexistent JFrog user returned a 429 “too many requests” — used to build a signaling protocol that hides information via API request timing, accounting for multiple JFrog nodes behind a load balancer
• Treat files from ChatGPT as an email attachment

Filed Under: #defcon34