The evolution of VM support in Istio 1.8 (with video)

Istio releases a new minor version every quarter, and most recently the community released 1.8.0. VM support for Istio has been progressing along across the last few releases. For example, in  Istio 1.6 the WorkloadEntry resource was introduced. This allowed the mesh operator to specify VM instances and their IPs as part of the mesh. In Istio 1.7, a safer way to bootstrap the VM identity was introduced. I did an in-depth two-part video on this capability if you’re interested. In Istio 1.8 two new features were introduced:

  • VM instance auto-registration
  • Sidecar DNS support

These features greatly improve the user experience and overall mesh capabilities. In this blog post, we take a look at these features include show a quick video demonstrating the capabilities.

VM Auto Registration

With the VM auto registration capabilities, Istio can automatically add a WorkloadEntry for a VM instance that connects with  a valid identity token. This is especially useful when VM instances are auto-scaled and need to be included in the mesh. For example, let’s say a service named python-http is deployed as three replicas on  three different VMs. With Istio 1.8, all we have to do is specify the new WorkloadGroup custom resource and any time one of the python-http instances connect, a new WorkloadEntry  will be created. Here’s an example of the WorkloadGroup  we might use:

apiVersion: networking.istio.io/v1alpha3
kind: WorkloadGroup
metadata:
  name: python-http
  namespace: vm-services
spec:
  metadata:
    annotations: {}
    labels:
      app: python-http
  template:
    ports: {}
    serviceAccount: my-vm

When these WorkloadEntrys get created, they will have the IP address and other metadata about that specific VM instance. We still need some way for services within the  mesh to address the services running in the VM. For that we can create a ServiceEntry which uses workload selectors to select the specific WorkloadEntrys:

apiVersion: networking.istio.io/v1beta1
kind: ServiceEntry
metadata:
  name: vm-workload-svc
  namespace: vm-services
spec:
  hosts:
  - vmservice.example.com
  location: MESH_INTERNAL
  ports:
  - number: 80
    name: http
    protocol: HTTP
    targetPort: 9090
  resolution: STATIC
  workloadSelector:
    labels:
      app: python-http

Let’s take a look at this image to recap how this all comes together:

A WorkloadGroup resource specifies a template that will be used to create WorkloadEntry resources. These WorkloadEntry resources are used to describe IP address, locality, and other metadata of the instance. You can kind of squint and think of the relationship between WorkloadGroup and WorkloadEntry similarly to how a Deployment specifies the template and properties of a Pod. Once these entries are created, we need to group them by using selectors and labels so that they can be addressed. That’s where the ServiceEntry comes into play. Again, if you squint, this plays a similar role to the Kubernetes Service. Then a service in the mesh uses the hostname specified in the ServiceEntry and  will be able to route traffic to a VM (one of the WorkloadEntry endpoints).

Sidecar DNS

One of the problems with ServiceEntrys for services that either do not really exist, or whos hostname needs to be stubbed out, is that services that try to use the hostnames will fail resolving DNS. For example, in the previous ServiceEntry we used a hostname of vmservice.example.com which would not resolve to any IP. If a service within the mesh tried to use this hostname, it would fail. On the other end of the spectrum, if we deploy an Istio sidecar proxy on a VM, we would not be able to address the services running within the proxy with their hostnames such as httpbin.default.svc.cluster.local. Those hostnames for the service are only available within the mesh and Kubernetes cluster.

In Istio 1.8, the sidecar now has a DNS proxy which caches endpoints for those in the mesh and those created by ServiceEntry resources. For other hostnames, it delegates to the system DNS as a normal DNS proxy. So from a VM, a local service can address a service that’s in the mesh by using it’s FQDN such as httpbin.default.svc.cluster.local. The sidecar DNS would know how to resolve this correctly and the proxy would then know how to route correctly. This means operators can simplify how they set up (or in some cases, hack) their DNS setup to get VMs (and multi-cluster Istio) to  work. See our blog post on the Evolution of multi-cluster Istio in 1.8 for more information on the multi-cluster usecase.

The current state of VM support in Istio

The following diagram illustrates how VMs are supported in Istio in 1.8

When a VM connects up to the Istio control plane, it does so by going through an “east-west gateway”. This gateway is really just an Istio Gateway specifically designated for mesh-internal traffic. Separating out the gateway used for untrusted user traffic from the internal mesh traffic is a pattern we’ve suggested as part of Gloo Mesh for some time now. Now we see it as a recommended deployment in Istio 1.8.  Once the connection is made from the VM sidecar to the Istio control plane, the appropriate WorkloadEntry resources are created and the VM sidecar is made aware of all services in the cluster. From here a service can address the httpbin.default.svc.cluster.local directly. The DNS name gets resolved by the proxy and gets routed through the “east-west gateway” to the appropriate service in the mesh.

Deep dive video on how this works

In this video, we see all of this in action:

 

Where to get commercial help with Istio?

At Solo.io, we are the Envoy and Service Mesh experts. With our Gloo Mesh product built on Istio, we solve a lot of the difficult enterprise problems with running Istio in production including team processes and APIs, multi-cluster federation for routing and failover, and of course Istio break-fix support. Reach out to us to discuss how we can help with your Istio and service mesh phased deployments and adoption.