One agent is a deployment. A platform full of them is a different problem. Run as ordinary pods, every agent holds its memory while it waits for the next message, carries a provider key it can read, calls the provider directly, and shares the node's kernel with every other team's agents.
This open source solution shows you how to run agents on kagent and Agent Substrate, so that each session is its own sandbox that holds no memory between turns, no agent ever holds a key, and every model and tool call goes through agentgateway. Every component is open source, and the whole stack installs with one command.
Common challenges
• Idle agents that still hold a pod
An agent waits for a person, thinks for thirty seconds, calls a few tools, and goes quiet again. Run as a pod per session it holds its memory the whole time, whether or not anything is happening inside it.
• The key in the agent's environment
A provider key in an environment variable is readable by anything that can read the process, including the code the model asks the agent to run. Every agent that holds one is a place for it to leak from.
• One kernel, every team
Pods on a node share its kernel. An agent that runs code it was handed by a model is one container escape away from every other team's agents on that node.
• No place to stand
Each agent calls its provider directly, so there is no single place to authenticate agents, stop a runaway job, decide where an agent may send data, or say which team spent what.
What you build
- One command to a running stack.
kagent installbrings up kagent, Agent Substrate and PostgreSQL together, and the guide then shows what it built and what to harden.
- One sandbox per session. Each session is an Agent Substrate actor in its own gVisor sandbox. It is restored from a snapshot when a message arrives and suspended when the turn ends.
- Memory follows turns, not sessions. A suspended session holds no memory, only a snapshot of its volumes. Workers are sized for the turns in flight at your busiest moment, not for every session open.
- No agent holds a key. The agent is given a placeholder. Substrate's egress gateway puts the real credential on the request after it leaves the sandbox, and the provider key itself is stored once, in the gateway.
- One way out, with a cap. Every model and tool call goes through agentgateway, each agent may reach only the destinations its egress policy allows, and each team's tokens are capped per hour.
- Sign-in and session ownership. The UI and API sit behind single sign-on with your identity provider, and a session belongs to the user who started it.
- A trace for every turn. Each turn is traced from the controller to the provider and kept in your cluster, with the session, the user and the tokens on it, and access to traces governed like everything else.
Common questions
Q. How do I run many agents on Kubernetes without a pod per agent?
Run them as actors on a shared worker pool. With Agent Substrate each session is restored onto a warm worker when a message arrives and suspended to an object store when the turn ends, so only sessions that are working occupy memory. A suspended session holds no memory, only a snapshot of its volumes.
Q. Does Agent Substrate use less memory than a pod per agent?
No, and the guide says so. A pod per agent uses less memory, but it shares the node's kernel and keeps every session of that agent in one process. Against a pod per session, which isolates sessions the way Substrate does, Substrate holds memory only for the sessions that are working.
Q. How do I keep provider API keys out of agent containers?
Give the agent a placeholder and inject the credential outside it. kagent hands the agent a placeholder value, Substrate's egress gateway swaps it for the agent's gateway key on the way out, and the gateway holds the provider key. In the validated deployment the agent's environment held only the placeholder, and a search of its sandbox and of its suspended snapshot found neither key.
