Scaling Agentic Workloads with Multi‑Cluster Ambient Mesh

Ambient Mesh for agentic AI workloads: Istio Ambient Mesh secures and connects AI agents, models, and tools without injecting a sidecar into every pod. A node-level ztunnel proxy gives every workload mTLS encryption and identity at Layer 4, and optional waypoint proxies add Layer 7 policy only where it is needed. Because agents scale up and down quickly, removing per-pod sidecars cuts resource overhead and restarts while keeping zero-trust security and observability across clusters.

AI Workloads and Agentic Architectures are Scaling. Fast.

‍By 2026, over 75% of large enterprises will run containerized AI workloads in production according to Gartner. Yet the complexities of managing secure, observable and scalable microservices remains a top barrier to digital innovation. New mesh patterns, like Ambient Mesh, are critical in enabling enterprises to manage AI workloads today and into the future.

Ambient Mesh: A Primer

Ambient Mesh is the latest evolution in service mesh technology within the Istio project. Unlike sidecar models which require a proxy alongside every pod, Ambient Mesh introduces a sidecarless approach that splits between a shared L4 proxy and optional L7 proxies:

• Node-level (ztunnel) proxies offer always-on mTLS and baseline security. The secure ztunnel proxy can be shared across all workloads on a node.
• Layer 7 (L7) waypoint proxies are deployed only where advanced routing or policy is needed, avoiding latency overhead.

For AI and agentic workloads, Ambient mesh eliminates redundant proxy overhead, reduces operational friction and supports rapid onboarding. 

If you’re new to Istio, start with Istio’s documentation.

AI & Agentic Workloads Have Unique Demands

Agentic systems and AI microservices pose unique challenges:

• Agile scaling: Agents and models spin up and down rapidly. The operational burdens of managing static sidecars can limit agility.
• Observability at scale: AI pipelines and agentic interactions generate vast, complex traffic.
• Secure autonomy: Each agent must communicate securely, but not at the cost of developer velocity.

Ambient Mesh is built to support those demands, providing IT organizations with:

• Cost savings: Reduces infrastructure costs by up to 90%+. Calculate your potential savings using our Cost Savings Calculator.
• Operational simplicity: No app code changes or pod restarts - simply label a namespace and workloads join the mesh.
• Zero trust by default: Always-on encryption and fine-grained access controls to bolster security.
• Improved performance: A streamlined data plane provides better network performance and reduced latency, especially for workloads that don’t require Layer 7 processing.

5 Steps to Deploy Ambient Mesh for AI & Agentic Workloads

1. Assess and Plan Migration 
   • Inventory your workloads: Identify which workloads require L7 features (e.g. AI inference and API gateways) and which workloads can operate at L4.
   • Review production deployment best practices to anticipate mesh stability, upgrade and monitoring needs.
2. Install Istio Ambient Mesh
   • Deploy ztunnel proxies at the node level. This secures all L4 traffic via mTLS without sidecars.
   • For AI-specific workloads, consider deploying Solo Enterprise for Istio for centralized control and multi-cluster management.
3. ‍Incrementally Onboard Services
   • Label namespaces to join the mesh—no restarts or refactoring needed for most agentic workloads.
   • For advanced L7 routing (e.g., model selection, traffic splitting), deploy waypoint proxies only where required.
4. ‍Secure, Monitor and Observe
   • Leverage open source tools for policy enforcement and telemetry.
   • Use kgateway health checks and configuration validation for operational assurance.
5. Scale and Automate
   • Manage policies, upgrades, and mesh operations centrally to supportmulti-cloud, multi-region AI and agentic workloads.
   • GitOps workflows and CI/CD tools can streamline deployment and mesh governance.

For more information, see the Quick Start Guide for Ambient Mesh.

Advanced Tips: Integration and Scaling for AI and Agentic Ecosystems

• ‍Smart Routing for AI: Integrate the Gateway API Inference Extension for dynamic LLM or agent routing.‍
• Policy Automation: Implement Open Policy Agent for fleet-wide AI policy governance.‍
• Observability at Scale: Use distributed tracing and telemetry pipelines for agentic workload visibility.‍
• Hybrid/Multicloud: Deploy Solo Enterprise for Istio to manage distributed clusters and multi-cloud AI services from a single pane.

Conclusion

Ambient Mesh unlocks secure, agile and cost-effective AI and agentic workloads for the enterprise while eliminating operational complexity, enabling IT organizations to focus on creating valuable business outcomes. With its proven cost savings, operational simplicity and zero-trust security, Ambient Mesh is the clear choice for enterprises seeking to future-proof their platform strategy.

Frequently asked questions

What is an agentic AI workload?

An agentic AI workload is software in which AI agents pursue goals autonomously: they plan steps, call LLMs, use tools and APIs (often through MCP), and communicate with other agents. On Kubernetes these workloads are bursty and highly connected, generating many short-lived service-to-service and egress calls that need identity, encryption, and policy.

Why is Ambient Mesh better for AI agents than sidecars?

Sidecars add a proxy container to every pod, which raises CPU and memory cost, slows pod startup, and requires restarts to onboard or upgrade. Ambient Mesh moves Layer 4 security to a shared ztunnel on each node and adds waypoint proxies only for workloads that need Layer 7 features. For fleets of agents that scale rapidly, that means lower overhead, faster scaling, and simpler operations with the same mTLS and policy guarantees.

How do kagent and agentgateway relate to Ambient Mesh?

kagent is an open source, Kubernetes-native framework for building and running AI agents, and agentgateway is an open source proxy for agent traffic to LLMs, MCP tools, and other agents. Ambient Mesh secures the network underneath them: agents deployed with kagent join the mesh through a namespace label and get mTLS and workload identity automatically, while agentgateway applies AI-specific policies such as authentication, rate limiting, and token usage tracking.