The agentic AI space is moving incredibly fast. Not long ago, I learned about a cool project called agent-sandbox, which provides a sandboxed environment for AI agents by leveraging many of the building blocks we have already developed for Kubernetes pods, such as identities, storage and networking.
If you've ever read the horror stories about AI coding agents or followed projects like OpenClaw & NemoClaw, you know how important it is to provide a secure and isolated environment for your agents. Without proper isolation, agents can surprise you by doing things you never intended, such as deleting family photos or modifying critical files.
Just a few weeks ago at Open Source Summit North America in Minneapolis, while chatting with Bob Killen in the hallway track, I learned about a new project called agent-substrate. What immediately caught my attention was its ability to dynamically wake up agents based on invocation thus allowing more agents on the same infrastructure resources while still providing the security benefits of sandboxed execution.
Naturally, the first thing I did was discuss with our team (Idit Levine, Eitan Yarmush, Yuval Kohavi, John Howard, and others) to explore how we could integrate it with kagent and agentgateway .
What Are the Differences Between the Two Projects?
Agent Sandbox
The agent-sandbox project provides a Sandbox Custom Resource Definition (CRD) and controller for Kubernetes under the umbrella of Kubernetes SIG Apps.
Its primary focus is on providing:
- Strong identities for agents
- Persistent storage that survives restarts
- Lifecycle management of sandboxed pods
- Security and isolation through the Sandbox controller
In short, agent-sandbox focuses on making agent execution secure, manageable, and Kubernetes-native.
Agent Substrate
The agent-substrate project is currently a standalone project and is not part of any Kubernetes SIG or other cloud native foundation project, though that may change in the future.
Built on top of Kubernetes, agent-substrate aims to go beyond sandboxing by focusing on:
- Higher scale
- Better resource efficiency
- Lower latency execution
- More dynamic lifecycle management for agents
My understanding is that agent-substrate provides the runtime building blocks needed to run AI agents securely at very high scale.
Instead of keeping agents running continuously as pods, agents execute in secure worker pods for short bursts, suspend when idle, and resume later on any available worker. The worker pod lifecycle is decoupled from the agent “actor,” which is managed by the agent-substrate control plane.
In this model, agents behave more like on-demand serverless workloads: they can be scheduled, paused, and resumed with minimal overhead, while still benefiting from Kubernetes-based sandbox isolation using lightweight runtimes such as gVisor or Kata Containers.

Do We Need Agent Substrate When We Already Have Agent Sandbox?
I believe the answer is yes.
Sandboxing your agents is necessary, but not sufficient.
In most Kubernetes environments, resources are constrained. You have to be selective about which agents run continuously. Many agents are only useful occasionally, and keeping them always on is inefficient.
This creates an awkward tradeoff:
- Keep agents running idle and waste resources
- Or constantly spin them up and down, adding overhead and latency
Neither option scales well.
This is where agent-substrate becomes interesting. While agent-sandbox focuses on security, isolation, and lifecycle management, agent-substrate focuses on density, efficiency, and operational scalability, while still preserving a secure execution model.
You can think of it as making large-scale agent fleets practical: not just safe, but economically viable.
Agent Substrate Integration with kagent
One of the things I appreciate about kagent is its simplicity and declarative YAML-based workflow. You always know what is running in your cluster, and you can recreate environments easily from source-controlled manifests.
With the efficiency introduced by agent-substrate, we can support many more agents using a shared pool of worker resources. Instead of assigning a dedicated pod per agent, we can use shared worker pools and templates that dynamically execute agents on demand.
Agents can now appear and disappear based on invocation, while still running on the same underlying infrastructure.
I’ve updated my AIRE agent to use agent-substrate as its runtime. This allows me to invoke agents only when needed, without keeping them running continuously.
It also enables multiple AIRE agents (each with different skills, echoing the idea of “Don’t put agents! build skills instead”) to share the same worker pool or even same pod.
The result is a more efficient system: agents remain available on demand, idle resource consumption drops significantly, and there is far less need to constantly scale pods up and down.
For example, my six AIRE agents map to six actor templates but only require a single worker pod to execute them, as long as they are not running concurrently. If concurrency increases, I can simply scale the worker pool (kagent-default) horizontally to increase the number of worker replicas.

Final Thoughts
Agent-sandbox and agent-substrate solve related but different problems.
Agent-sandbox asks: How do we run agents securely?
Agent-substrate asks: How do we run agents not only securely but also efficiently at scale?
As AI agents become more common in Kubernetes environments and AI cost remains one of the biggest concerns in adoption, we need to be more innovative and avoid tying an agent’s lifecycle too closely to Kubernetes pods.
Security, identity, isolation, and policy controls remain essential. At the same time, we need a runtime model that allows hundreds or thousands of agents to exist in a dormant state, waiting to be invoked, without requiring hundreds or thousands of pods for workloads that are idle most of the time.
The future is not just secure agents, it’s scalable, efficient, and ephemeral agents.
Additional Resources
- Get started with kagent and agent-substrate: https://kagent.dev/docs/kagent/examples/agent-substrate
- Deep dive on agent-substrate from Christian Posta
- Check out the agent-substrate project’s roadmap

%20(1).png)



























%20a%20Bad%20Idea.png)










