For the three years I lived in San Francisco, I owned a car and paid for a parking space, and I drove that car maybe twice a month. Why? Two reasons: Habit (I'd always owned a car) and convenience (when I decide I'm going somewhere, I don't like to wait).
Yes, my need for immediate gratification exceeded my ability to realize that owning a car you don't use is a HUGE waste of money...
In my defense, I didn't know I wasn't going to drive it. It's just that parking in San Francisco both sucks and is expensive, so it makes more sense to use an ephemeral means of transport... ride share.
Everything I just shared relates directly to how we should be running agents. Most of us run them like my car and its parking space - draining resources while providing no service. Instead, we should be running them like ride share.
Agents on plain Kubernetes are parked cars
The usual way to run an agent on Kubernetes is a pod that stays up, holding memory on a node whether anything is happening or not. Same two reasons I kept the car. Habit: Kubernetes grew up running stateless microservices, and a long-running Deployment is what we reach for. Convenience: a pod that's already running answers right away.
Here's the problem though. Agents bring state back, and an agent conversation spends most of its life waiting. The user reads the answer, goes to a meeting, comes back tomorrow. The turn where the agent actually works is short, and the pod holds its memory the whole time in between. That memory is the parking space, and you pay for it every hour the car doesn't move.
There are two ways to lay this out on pods, and neither is great:
- One pod per agent. Every conversation with that agent shares one process. It's cheap, but it's a school bus: noisy, and nothing isolates one snot-nosed unwell child (or agentic session) from the others.
- One pod per session. Each conversation gets its own sandbox, which is the isolation you want. Now you're paying for a parking space per conversation, active or not.
kagent on Agent Substrate runs each session like a ride
Agent Substrate runs each conversation with an agent as an isolated actor. An actor is a small private workspace for one conversation: its own process, in its own sandbox, holding only that conversation's files and history. It only holds memory and CPU on a worker while there's work to do. When the turn ends, those resources go back to the cluster for other sessions to use, the same way the car drives off to pick up its next passenger. (The conversation's state is saved to storage for next time, like the ride-share app keeping your account and preferences.)
That's a single ride. You get the car to yourself for one trip, so there's no kid in the back sneezing on your while shouting "6, 7, I'm eleven." One conversation gets one actor, and nobody else's session rides along.
kagent is the app you book the ride with. You tell kagent what an agent is, and kagent handles the rides. It asks Agent Substrate for an actor when a conversation starts, packs it away when a turn ends, brings it back on the next message, and cleans everything up when you delete the agent.
There's no idle timeout to configure. kagent suspends a session's actor the moment its last turn finishes, so idle lasts about a second. If a turn ends waiting on something (input, an approval, an auth prompt), the actor pauses on its worker instead, because that session is still mid-turn. That's asking the driver to wait while you run into the store.
And there's no long-lived actor per agent. Ten sessions with one agent are ten actors, each started from the same golden snapshot.
Same rule as ride share: don't leave anything in the car (on the k8s pod). A session picks up where it left off because the runtime keeps it in a database on its volume, and that volume is what the snapshot holds.
The wait I was avoiding by owning a car? Each resume boots the runtime image again, which is mere milliseconds and imperceptible in terms of agentic loop times.
Agent Substrate frees a session's memory while it waits
With ride share, you only pay for the trips you take. Agent Substrate works the same way. A session uses memory while a turn is running and gives it back when the turn ends, so a conversation waiting on its next message holds no memory at all.
That changes how you size the cluster. It only needs room for the turns running at any moment. A hundred open conversations with five of them mid-turn needs room for five.
There's one fixed cost, Agent Substrate's own control plane, which runs once per cluster. Someone has to run dispatch.
And you keep the isolation. Every session still runs in its own sandbox, the same as giving each conversation its own pod, without paying for the parking space.
Try it with one command
kagent install stands the whole thing up in about two minutes: seven Helm releases on the cluster your kubectl context points at. That's kagent, Agent Substrate underneath it, and agentgateway as the path every model call leaves through, so the model credential never sits on the agent.
The install starts small (one worker, a single PostgreSQL pod, getting-started defaults), which is enough to watch sessions suspend and resume on your own cluster. The deployment guide walks through it end to end: https://www.solo.io/solutions/operating-isolating-and-scaling-agent-workloads-on-kubernetes-w-agent-substrate-oss
%20(1).png)
%20a%20Bad%20Idea.png)
