A Local Development Kubernetes Environment for Gloo Edge

Daniel Wessels | July 28, 2021

This is yet another blog on an option for that mythical beast: the developer- or laptop-based Kubernetes instance to access Kubernetes clusters and Gloo Edge. In the last year many projects in this space have matured and are feature rich, ones that spring to mind are KinD, Minikube and K3d. There are many more like K0s and Microk8s.

Connecting to in-cluster resources

I recently joined Solo.io and wanted to learn about Gloo Edge. Solo.io documentation generally recommends KinD, which like many of these projects, leverages Docker Desktop on a macOS. One limitation is that with Docker networking on a macOS, it isn’t easy to connect to in-cluster resources of services of type LoadBalancer or Ingress type. I needed a Kubernetes cluster (single node) that runs on my laptop with stable seamless connectivity to the in-cluster resources mentioned above.

Microk8s is an exception in the bunch. It uses Multipass to provision the cluster node and configures a network bridge to allow seamless connectivity between your macOS terminal and the cluster’s nodes and more importantly the service type LoadBalancer or Ingress resources.

Getting to a routable IP address

Kubernetes LoadBalancer services and Ingress resources require an external-to-cluster routable IP address. Kubernetes hosted in cloud providers such as AWS, GCP, and Azure will leverage the provider’s loadbalancer service to provision this connectivity, but on-premises (or on your laptop) you don’t have these services. Metallb is a popular solution in this space. It’s the IPAM for Kubernetes service type LoadBalancer and Ingress resources when cloud providers aren’t available.

So what do we need?

macoS (BigSur)
Microk8s (with Multipass)

Optional:

Kubectl

The installation process

Microk8s – You can follow the quickstart guide for your particular laptop OS, in this case macOS.

Metallb – Microk8s comes with ready to install addons, including Metallb (yes!) Once you have completed step 4 “Wait for MicroK8s to start” in the quick start, run:

microk8s enable dns rbac storage metallb:192.168.64.50-192.168.64.100

This command installs not only mettallb, but also a few other addons that you’ll likely need. Wait for the installations to complete.

❯ kubectl get po -A
NAMESPACE        NAME                                      READY   STATUS    RESTARTS   AGE
kube-system      calico-kube-controllers-f7868dd95-k674l   1/1     Running   0          5m48s
kube-system      calico-node-7f8k4                         1/1     Running   0          5m43s
kube-system      coredns-7f9c69c78c-sdfxw                  1/1     Running   0          2m29s
metallb-system   speaker-t8pbj                             1/1     Running   0          35s
kube-system      hostpath-provisioner-5c65fbdb4f-7g9c9     1/1     Running   0          35s
metallb-system   controller-559b68bfd8-jd2cb               1/1     Running   0          35s

That’s it, you’re now ready to go.

TIP: If you use kubectl, then you can extract the required cluster context by running:

microk8s config >admin.conf

Verifying your networking

Let’s take a quick look at the Multipass networking. We find looking at the route tables the easiest way to see the “network magic”. Run:

netstat -rnf inet

Specifically the bridge1 interface:

192.168.64.2 ee.5.eb.f5.84.9f UHLWIi bridge1 1112

This is the IP address of the kubernetes node server and because you can ping it, you can ssh to it.

TIP: The private key can be found at /var/root/Library/Application Support/multipassd/ssh-keys/id_rsa

Trying out Gloo Edge

Now that we have a Kubernetes API, lets install Gloo Edge, as described in the Getting Started Solo.io documentation:

# Install glooctl
curl -sL https://run.solo.io/gloo/install | sh
export PATH=$HOME/.gloo/bin:$PATH

# Install Gloo Edge
glooctl install gateway

❯ glooctl install gateway
Creating namespace gloo-system... Done.
Starting Gloo Edge installation...
.......
Warning messages
.......
Gloo Edge was successfully installed!

❯ kubectl get po -A
NAMESPACE        NAME                                      READY   STATUS    RESTARTS   AGE
kube-system      hostpath-provisioner-5c65fbdb4f-m26fw     1/1     Running   0          17h
kube-system      coredns-7f9c69c78c-gfsgc                  1/1     Running   0          17h
kube-system      calico-kube-controllers-f7868dd95-xphrx   1/1     Running   0          17h
kube-system      calico-node-wrwsl                         1/1     Running   0          17h
gloo-system      discovery-897f8c8cb-68qpn                 1/1     Running   0          9m6s
gloo-system      gateway-proxy-7c8bf6fcb5-mlmrb            1/1     Running   0          9m6s
gloo-system      gloo-5ff796587c-lw8hx                     1/1     Running   0          9m6s
gloo-system      gateway-99bd84596-sgjxn                   1/1     Running   0          9m6s
metallb-system   speaker-fsz5j                             1/1     Running   0          6m44s
metallb-system   controller-559b68bfd8-2dk7p               1/1     Running   0          6m44s
❯

Gloo Edge’s Gateway Proxy is a service type LoadBalancer resource and will have an IP assigned by metallb’s address pool “192.168.64.50 – 192.168.64.100” as seen in the “EXTERNAL-IP” column below.

❯ kubectl get svc -A
NAMESPACE     NAME            TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)                               AGE
default       kubernetes      ClusterIP      10.152.183.1               443/TCP                               18h
kube-system   kube-dns        ClusterIP      10.152.183.10              53/UDP,53/TCP,9153/TCP                18h
gloo-system   gloo            ClusterIP      10.152.183.163             9977/TCP,9976/TCP,9988/TCP,9979/TCP   11m
gloo-system   gateway         ClusterIP      10.152.183.32              443/TCP                               11m
gloo-system   gateway-proxy   LoadBalancer   10.152.183.24    192.168.64.50   80:32382/TCP,443:30349/TCP            11m

The first IP, 192.168.64.50, from the range has been assigned.

OK, Gloo Edge is ready, now we can start looking at Gloo Edge’s many capabilities:

Authentication
Authorization
Traffic Routing
Rate Limiting
Service discovery
Load balancing
Timeouts
Retry / Budgets
Circuit breaking
Tracing, observability
Secure transport
Extension

You can continue using the Solo.io documentation to try all these and more features. A good place to start is the “Hello World” example application which illustrates Traffic Routing.

Learn more

There are many ways to set up your own laptop for access to Kubernetes clusters and Gloo Edge, but the Microk8s option may be particularly useful since all manner of projects’ “How to” guides are written with the assumption you have seamless network connectivity to ingress resources.

You can continue your learning about API Gateways and Service Meshes with our many “getting started” blogs. Also see our Events where we provide hands-on, instructor-led workshops and more.