kgateway as Ingress for Ambient Service Mesh

The Kubernetes ecosystem continues its rapid evolution, introducing innovations that fundamentally alter how applications are deployed and managed. Among the most significant advancements is Istio's Ambient Mesh, a transformative approach to service mesh. Ambient Mesh promises the full power of a service mesh, but notably, without the architectural complexities of sidecar proxies. However, integrating external traffic into this streamlined mesh, particularly at Layer 7, demands a robust and intelligent ingress solution capable of seamlessly bridging the traditional network edge with the mesh's inherent security and advanced capabilities.

This post will detail how kgateway, an open-source gateway solution aligned with the Kubernetes Gateway API, can serve as the ideal ingress for Ambient Service Mesh deployments. This combination delivers efficient traffic management and the powerful security features of Istio's latest architecture.

But Wait!  Doesn’t Istio Have a Built-in Ingress Gateway?

It does!  And for many use cases, the default Istio Ingress Gateway is sufficient.  In many ways, the Istio Ingress Gateway and kgateway are similar.  Both are built on Envoy, a lightweight, highly performant distributed proxy designed for services and applications.  Both can leverage the universal Kubernetes Gateway API, providing a consistent API interface for deploying and managing gateway deployments.  The goal of kgateway is to build on the robust features of the Istio Ingress Gateway and provide a rich set of API management features designed for the full API lifecycle.

The standard Ingress Gateway provides powerful features such as:

• Traffic Routing: Directing traffic to different services based on headers, URI paths, and methods
• Load Balancing: Distributing traffic across service instances using various algorithms
• Security: Providing features like TLS termination and identity-based authentication
• Observability: Generating detailed metrics, logs, and traces for monitoring and troubleshooting
• Resiliency: Implementing advanced resiliency patterns such as circuit breaking, timeouts, and retries
• Deployment Strategies: Enabling modern deployment strategies like canary releases and weighted traffic splitting.

Kgateway offers all the features of Istio Ingress Gateway, plus additional capabilities tailored for modern application architectures.

• Advanced Security Capabilities: Enabling advanced security tools to help protect your APIs, such as External auth to integrate with your existing authorization services, built-in rate limiting, and native support for CORS and CSRF.
• Data Transformation: Modify requests and responses on the fly, for example, by converting between different data formats like XML and JSON. This is crucial for integrating disparate services without requiring backend applications to handle data format conversions.
• Extended Traffic Control: Kgateway goes beyond basic load balancing with sophisticated traffic management capabilities like canary releases, A/B testing, and circuit breaking. While some of these are available in Istio, Kgateway’s implementation is more streamlined and geared towards a pure API gateway use case.
• Route Delegation: Large, complex routing configurations can be broken down into smaller, more manageable units. Ownership of these simplified configurations can then be delegated to the respective application or domain teams.
• External Processing Servers: Extend kgateway by developing and integrating an external gRPC processing server capable of reading and modifying all aspects of an HTTP request or response.

…and more!  For robust and complex ingress needs, Gloo Gateway extends the open-source kgateway project to provide a comprehensive enterprise experience.

Ingress within Ambient Mesh

Even with the inherent elegance and efficiency of Ambient Mesh, the fundamental problem of ingress persists. External traffic must gain entry into the mesh, be routed accurately to its intended destination, and rigorously adhere to defined security and traffic management policies. Traditional ingress controllers primarily focus on routing plaintext or TLS-terminated HTTP traffic directly to Kubernetes Services. However, an Ambient Mesh workload, secured at Layer 4 by ztunnel, fundamentally expects mTLS traffic encapsulated within HBONE (HTTP-Based Overlay Network Encapsulation).  For a deeper dive on the fundamentals of Ambient Mesh and a quick primer to get you started, check this blog post out.

An effective L7 ingress for Ambient Mesh must therefore perform several critical functions:

1. Terminate External TLS: It must securely terminate the TLS connections initiated by external clients.
2. Apply Initial L7 Routing and Policies: It should be capable of applying initial Layer 7 routing rules (e.g., path-based, host-based) and preliminary policies (e.g., rate limiting, basic authentication).
3. mTLS Connection to Mesh: Communication must be mTLS from the gateway into the service mesh, specifically targeting the ztunnel on the node hosting the destination workload. This ensures secure communication from the very edge of the mesh all the way to the workload.
4. Ensure Mesh Policy Adherence: It must ensure that L7 policies defined deeper within the mesh are respected and, where necessary, enhanced or chained from the ingress point.

It is precisely here that kgateway, when integrated with Ambient Mesh, truly shines.

Getting Started

Integrating kgateway with Ambient Mesh is a straightforward process. For a guided, hands-on experience, a free lab, Integrating kgateway with Istio at Ingress, is available. Additionally, this video provides a complete walkthrough of the integration.

1. Prerequisites:

• A Kubernetes cluster.
• Istioctl: The command-line utility for Istio

2. Deploy Istio Ambient.  This will deploy Istio in ambient mode.

istioctl install --set profile=ambient

‍

3. Deploy the Kubernetes Gateway API CRDs.  This is a prerequisite for kgateway.

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.3.0/standard-install.yaml

‍

4. Deploy kgateway: Install kgateway into your cluster. This process will deploy the kgateway CRDs, create a GatewayClass resource, and deploy the necessary controller and associated pods.

helm upgrade -i --create-namespace --namespace kgateway-system --version v2.0.4 kgateway-crds oci://cr.kgateway.dev/kgateway-dev/charts/kgateway-crds

helm upgrade -i --namespace kgateway-system --version v2.0.4 kgateway oci://cr.kgateway.dev/kgateway-dev/charts/kgateway

After creating the CRDs and deploying kgateway, ensure the kgateway pod has been created and is running:

kubectl get pods -n kgateway-system

Example output:

NAME                        READY   STATUS    RESTARTS   AGE
kgateway-5495d98459-46dpk   1/1     Running   0          19s

5. Enroll the default namespace in the Ambient Mesh.  This will tell Istio that any applications deployed to this namespace should be part of the Ambient Service Mesh.

kubectl label namespace default istio.io/dataplane-mode=ambient

6. Deploy the httpbin sample application from the Istio Github repository

kubectl create -f https://github.com/istio/istio/raw/master/samples/httpbin/httpbin.yaml

7. Define a Gateway Resource: Create a Gateway object in the default namespace. This resource represents the ingress's network listener, defining which ports (e.g., 80/443) and hosts it will expose to external traffic.

apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: ambient-ingress-gateway
  namespace: default
spec:
  gatewayClassName: kgateway
  listeners:
  - name: http
    protocol: HTTP
    port: 80

8. Define HTTPRoutes: Create HTTPRoute resources to define how specific external hostnames and URL paths map to your services that are running within the Ambient Mesh.

apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
  name: httpbin-route
  namespace: default
spec:
  parentRefs:
  - name: ambient-ingress-gateway # This links the route to your Gateway
  rules:
  - matches:
    - path:
        type: PathPrefix
        value: /
    backendRefs:
    - name: httpbin
      port: 8000

9. Verify Ambient Workloads: Verify that your gateway and httpbin are correctly enrolled in Ambient Mesh (e.g., its namespace is in Ambient mode, and ztunnel has successfully picked it up). Traffic originating from kgateway will then flow securely and efficiently through ztunnel.

istioctl ztunnel-config workload

You should see output similar to this.  Note that the PROTOCOL is HBONE (vs TCP for other workloads).  This indicates that the kgateway POD and httpbin are both using mTLS through the Ambient Mesh.

NAMESPACE          POD NAME                                 IP           NODE                WAYPOINT PROTOCOL
default            ambient-ingress-gateway-5d59765d48-92xdd 10.104.93.74 kind4-control-plane None     HBONE
default            httpbin-787cdcc9df-v8npz                 10.104.93.73 kind4-control-plane None     HBONE

10. Note the service IP address of your kgateway POD and send traffic to it

kubectl get svc|grep ambient-ingress-gateway

NAME                      TYPE           CLUSTER-IP     EXTERNAL-IP    PORT(S)        AGE
ambient-ingress-gateway   LoadBalancer   10.4.208.214   172.19.104.1   80:32757/TCP   140m

11. Curl the IP address and port of the service to test connectivity

curl 172.19.104.1:80/ip

You should see output similar to the following

{
  "origin": "10.104.93.74"
}

In a few short steps, you’ve deployed Istio in Ambient mode, deployed an application, onboarded it to the service mesh, deployed kgateway, configured kgateway to route traffic securely, and verified everything.  Pretty cool!

Advanced Considerations

• TLS Termination: kgateway proficiently handles external TLS termination, allowing your HTTPRoutes to define secure ingress while offloading TLS processing from your application.
• Authentication & Authorization: kgateway can integrate with external authentication systems or apply fundamental authorization policies before traffic enters the mesh. For more complex, workload-specific authorization requirements, Istio's comprehensive authorization policies applied on waypoints (or directly on ztunnel for L4 policies) take over.
• Observability Deep Dive: Leverage powerful observability tools such as Grafana for metrics visualization, Kiali for mesh topology and traffic flow, and Jaeger for distributed tracing. These tools, integrated with Istio's telemetry, provide an unparalleled view of the request lifecycle from kgateway through ztunnel and any waypoints to your final application.

Conclusion

The combination of kgateway with Istio's Ambient Mesh represents a significant and decisive leap forward in Kubernetes networking. This synergy offers a powerful, flexible, and exceptionally efficient solution for ingress, extending the profound benefits of Ambient's sidecar-less architecture directly to your cluster's edge. By providing a unified API for all traffic management, ensuring robust security from the very ingress point, and delivering comprehensive observability across the entire data path, this architecture empowers organizations to simplify complex operations, substantially reduce resource consumption, and build more resilient, performant, and secure microservices applications. Kgateway and Ambient Mesh are not merely tools; they are shaping the very future of cloud-native infrastructure.

To learn more or get involved with the kgateway and Ambient Mesh projects, check these out:

• Free, hands-on kgateway labs‍
• Join the kgateway community on Slack
• Ambient Mesh quickstart
• Cost savings estimator for Ambient Mesh