In July 2026 autonomous AI agents worked their way into parts of Hugging Face’s production systems. Each step relied on code running somewhere that also held, or could reach, credentials. Here’s how Cengrid keeps every command and CLI call its agent runs on a separate machine that holds as little as possible, and where that design stops.
What happened
On 16 July Hugging Face disclosed an intrusion it said was “driven, end to end, by an autonomous AI agent system.”1
OpenAI says agents it was testing in an isolated cyber-evaluation environment exploited a previously unknown vulnerability in an internal package mirror to reach the public internet from 8 July. There they found publicly exposed credentials and used some of them against third-party services. The activity “culminated in the compromise of parts of Hugging Face’s production infrastructure between July 11 and July 13.”3
Hugging Face’s technical timeline traces about 17,600 attacker actions between 9 and 13 July.2 In its words, the chain went like this:
A malicious dataset abused the dataset loader on a production worker. One read “returned the worker pod’s full environment, including some secrets and credentials”, and a template injection gave the agent code execution.
From inside the pod, the agent read the EC2 node role credentials from the instance metadata service.
It “created a bare privileged pod with the host filesystem mounted and escaped to root on the node”, because there was “no admission policy rejecting privileged or hostPath pods.”
A connector credential “shared across clusters” gave it cluster-admin, and a mesh-VPN key from the stolen secrets let it enrol the node into the corporate network.
Hugging Face’s own list of lessons starts with “strict isolation around evaluations, narrow trust boundaries, short-lived credentials, blocked metadata access”, alongside better detection.2 OpenAI’s report puts it more bluntly: “no single control should ever be assumed to be fully robust.”3
The pattern: code ran next to things worth taking
None of these weaknesses were new. What made them expensive was what sat beside the code. The process handling untrusted input could read its environment’s secrets, reach the cloud’s metadata service and borrow a credential shared with other clusters. In the evaluation environment, a package mirror the agents shared became their route to the internet.
An AI agent that runs shell commands for your team sets up the same situation on purpose. A model decides what to run, often after reading documents, web pages or messages nobody has vetted. So the question that matters is what that command can reach when it runs.
How Cengrid splits the agent from its commands
Cengrid runs in two places.
Cengrid core is trusted. It runs the agent loop, makes the model calls, checks identity and permissions, and holds the secrets: model API keys, the keychain and signing keys. Durable state lives in Postgres beside it.
Agent computers are untrusted and disposable. There’s one for each Slack thread, and one for each person or room everywhere else. A computer runs the commands core sends it and nothing else.
Scroll sideways to see the agent computers.Only the tool gateway sends commands to an agent computer. Model API keys, database access and signing keys stay in core. The dashed boundary is untrusted because it runs model-written commands.Open full size
Every command passes a gate first
The model never talks to a computer directly. Each command goes through core’s tool gateway, where command policy and the approval gate run before the computer is even started. A computer can’t approve its own commands, because those decisions are made outside it.
The machine holds only this turn’s logins
Model API keys, database access, signing keys and the conversation transcript never reach a computer. It receives the logins and connector credentials for the current turn, plus an agent API token that expires after an hour. Core checks every call that token makes back into it.
One computer per conversation
A computer belongs to one Slack thread, person or room, and only core routes work between computers. A compromised machine holds one scope’s credentials for one turn, not the keys to a fleet.
No route to the metadata service
When a computer’s traffic is forced through Cengrid’s egress proxy, every connection is decided against that scope’s egress policy. The cloud metadata service, link-local and loopback addresses are always refused. The check runs on the resolved address, so a hostname can’t point around it. Private network ranges can be denied by policy too. If the decision service is down, nothing leaves, and allowed and refused connections are both recorded against the scope and person.
Computers don’t stay up
A computer starts on the first command of a turn and suspends when the turn ends, unless a background process is still running. After three idle days it’s removed. There’s no long-lived worker quietly accumulating access.
The same chain, step by step
In the incident
In Cengrid
Untrusted input led to code execution on a production worker.
Commands shaped by untrusted input run only on that conversation’s computer, after policy and approval checks in core.
The worker’s environment held secrets and credentials.
Model API keys, database access and signing keys never reach a computer.
Node credentials came from the instance metadata service.
Where traffic is forced through the egress proxy, metadata and link-local addresses are always refused.
A credential shared across clusters gave cluster-admin.
Credentials are issued per scope and per turn, and the agent API token expires after an hour.
About 17,600 actions over four and a half days.
Every command passes core’s tool gateway, and every egress decision is recorded.
What this doesn’t do
Isolation narrows what one bad command can reach. It doesn’t make an agent safe on its own, and you should know where the edges are.
Logins on a computer are readable while in use. A compromised command can use or copy that turn’s credentials. Scoping and short lifetimes limit the damage; they don’t prevent it.
Command policy is a speed bump, not the boundary. It reads shell text, so obfuscated or scripted commands can get past it.
Forced egress depends on the computer backend. Where a backend can’t enforce it at the network level, the proxy is only as strong as the software that honours it.
Cengrid protects the agent’s side. It doesn’t harden the systems your credentials reach, and it wouldn’t have fixed a bug in someone else’s dataset pipeline.
Cengrid is early software. Isolation by scope is a design goal, not a guarantee or a certification.
OpenAI’s report makes the same point from the other side: each security objective needs “multiple, overlapping, and independent security controls”.3 Where an agent’s commands run, and what sits next to them, is one of those layers. It’s the one Cengrid is built around.
Want to see how this fits the way your team works?
