How to configure zero trust Authn/Authz with Istio

In this blog, we will configure your workloads to properly verify the requesting source by implementing zero trust Authn/Authz with Istio. While the concept of zero trust networks has been around for over a decade now, the move to cloud computing and service-oriented architectures has pushed the idea to the forefront. Security breaches involving massive amounts of online data being stolen or exposed have become commonplace. While not all of these security compromises are due to a lack of network security it has become ever more important to operate from a sense of distrust wherever possible. Authentication (Authn) and authorization (Authz) are two tools you should be using consistently in your service mesh.

In this example, we used Istio 1.10, but you may find the resources work in earlier versions. This example will use the bookinfo application (modified to propagate crucial headers) as well as sleep to show you how to allow or deny requests within your mesh.

Getting started on zero trust Authn/Authz with Istio

We will install Istio with the demo profile and the bookinfo application by following the instructions at https://istio.io/latest/docs/setup/getting-started/. Verify that everything works by bringing up the bookinfo application in your browser:

Bookinfo screenshot

Note that this application is made up of several services that provide information to display on this page. The stars come from the ratings service and we will focus our efforts on whether those should be shown based on the requestor’s identity.

Next we want to ensure that we have encryption enabled at the edge. Follow the steps at https://istio.io/latest/docs/tasks/traffic-management/ingress/secure-ingress/ but substitute bookinfo for the httpbin application. We want the resulting host to be served at bookinfo.example.com and the corresponding certificates in place. Note the curl command on that page that will test that everything is working properly. Of course, you should be able to visually inspect this through the browser.

For the purposes of this demonstration, we are using Keycloak and Gloo Edge to route to https://bookinfo.example.com/productpage in our cluster. We will skip this setup for now as you could easily generate a valid JSON Web Token (JWT) through a tool like jwt.io to use in your request. However, we will have two users in this setup – user1@solo.io and user2@solo.io, and we only want to allow user1 into the ratings application.  

The last part of the basic setup is to turn on strict mutual Transport Layer Security (mTLS) throughout the Istio service mesh. This is done with the following PeerAuthentication setting:

apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
 name: "default"
 namespace: "istio-system"
spec:
 mtls:
   mode: STRICT

Trust Nothing by default in your service mesh

In a zero-trust model, we should not allow any services to talk to another service unless we explicitly allow it. Luckily, this setting is easy with Istio’s AuthorizationPolicy:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
 name: allow-nothing
 namespace: default
spec:
{}

Now any attempts to curl from one pod to another will fail with the Error 403: Forbidden. Note that you can still hit the /productpage route, but it will not be very informative. To get the rest of the page visible again follow the steps at https://istio.io/latest/docs/tasks/security/authorization/authz-http/#configure-access-control-for-workloads-using-http-traffic to install the bookinfo application AuthorizationPolicies.

Enabling Trust for Specific Users

Let’s inspect the current AuthorizationPolicy for the ratings service:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
 name: ratings-viewer
 namespace: default
spec:
 selector:
   matchLabels:
     app: ratings
 action: ALLOW
 rules:
 - from:
   - source:
       principals:
       - cluster.local/ns/default/sa/bookinfo-reviews
   to:
   - operation:
       methods:
       - GET

Currently, we are only allowing the service account for the bookinfo reviews application access to ratings. What if we wanted to let in only the user with email “user1@solo.io”? Let’s see how to do that with JWT claims validation.  

To set this up, let’s first create a rule to deny traffic that does not have a JWT. To do this, apply the following policy:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
 name: deny-no-jwt
spec:
 selector:
   matchLabels:
     app: ratings
 action: DENY
 rules:
 - from:
   - source:
       notRequestPrincipals: ["*"]

Hitting refresh on the browser should show that the reviews no longer have any star ratings. 

Now we will add a rule to ensure that user1 has access.  But, first, we need to tell Istio how to extract the claim since we are putting the id_token in a header named Jwt.  For this we need to apply a RequestAuthentication resource.

apiVersion: security.istio.io/v1beta1
kind: RequestAuthentication
metadata:
  name: ratings-jwt
  namespace: default
spec:
  selector:
     matchLabels:
       app: ratings
  jwtRules:
  - issuer: http://keycloak/auth/realms/bookinfo
     jwksUri: http://keycloak/auth/realms/bookinfo/protocol/openid-connect/certs
     fromHeaders:
     - name: Jwt

In the RequestAuthentication resource, we tell Istio where to find the public key for the token (jwksUri) and what header the token will be found in.  By doing this, Istio will now make the claim available in the AuthorizationPolicy via request.auth.claims.

Therefore, we can apply the following policy to make sure that only user1 is allowed in:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: ratings-viewer
  namespace: default
spec:
  action: ALLOW
  rules:
  - from:
    - source:
        principals:
        - cluster.local/ns/default/sa/bookinfo-reviews
    to:
    - operation:
        methods:
        - GET
    when:
    - key: request.auth.claims[email]
      values:
      - user1@solo.io
  selector:
    matchLabels:
      app: ratings

It is not necessary here to inspect the requestPrincipal coming from Keycloak for this client because the RequestAuthentication resource applied ensures that the token was there and came from the authorized issuer.  

The “when” clause provides us the ability to inspect the claim for a match. This is a simplistic example and would not scale. It would make more sense to inspect the role of the user or groups in production. However, we only have two users here.

Let’s test to make sure this works:

Keycloak

Bookinfo2

So, logging in as user1 works! And we can inspect proxy logs just to ensure that all is good.

Proxy logs

Let’s try user2.

Keycloak2

Bookinfo3

The user was denied: no ratings!  Let’s look at the proxy logs.

 

Proxy logs2

We can clearly see that the policy was not matched and the request was denied.

Enabling trust for specific applications using Authn/Authz with Istio

We have seen this in some respect for the bookinfo application. To make sure this works across applications in different namespaces with different deployment lifecycles, let’s take a further look by deploying the sleep application.

First, we will create two namespaces – foo and bar:

❯ kubectl create ns foo
❯ kubectl create ns bar

Let’s deploy the sleep app to both namespaces:

❯ kubectl apply -f <(istioctl kube-inject -f samples/sleep/sleep.yaml) -n foo
❯ kubectl apply -f <(istioctl kube-inject -f samples/sleep/sleep.yaml) -n bar

Now, let’s amend the AuthorizationPolicy for the ratings service so that it accepts requests from the sleep app living in the foo namespace only:

apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: ratings-viewer
  namespace: default
spec:
  action: ALLOW
  rules:
  - from:
    - source:
        principals:
        - cluster.local/ns/default/sa/bookinfo-reviews
    to:
    - operation:
        methods:
        - GET
    when:
    - key: request.auth.claims[email]
      values:
      - user1@solo.io
  - from:
    - source:
        principals:
        - cluster.local/ns/foo/sa/sleep
    to:
    - operation:
        methods:
        - GET

By adding the source principal of the sleep service account in foo we can now verify that foo can successfully send a request to the ratings app by using curl.

$ kubectl exec "$(kubectl get pod -l app=sleep -n foo -o jsonpath={.items..metadata.name})" -c sleep -n foo -- curl -kv http://ratings.default:9080/ratings/0

Proxy logs3

The request is denied because we do not have a token.  Let’s take the id_token for user1 and pass it in the Jwt header.

$ kubectl exec "$(kubectl get pod -l app=sleep -n foo -o jsonpath={.items..metadata.name})" -c sleep -n foo -- curl -v http://ratings.default:9080/ratings/0 -HJwt:${TOKEN}
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 10.1.74.15:9080...
* Connected to ratings.default (10.1.74.15) port 9080 (#0)
> GET /ratings/0 HTTP/1.1
> Host: ratings.default:9080
> User-Agent: curl/7.78.0-DEV
> Accept: */*
> Jwt: REDACTED
> 
{"id":0,"ratings":{"Reviewer1":5,"Reviewer2":4}}* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< content-type: application/json
< date: Wed, 11 Aug 2021 13:09:59 GMT
< x-envoy-upstream-service-time: 5
< server: envoy
< transfer-encoding: chunked
< 
{ [59 bytes data]
100    48    0    48    0     0   2944      0 --:--:-- --:--:-- --:--:--  3000
* Connection #0 to host ratings.default left intact

Sending the same request from the sleep app in the namespace bar should be denied.

$ kubectl exec "$(kubectl get pod -l app=sleep -n bar -o jsonpath={.items..metadata.name})" -c sleep -n bar -- curl -kv http://ratings.default:9080/ratings/0 -HJwt:${TOKEN}
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 10.1.74.15:9080...
* Connected to ratings.default (10.1.74.15) port 9080 (#0)
> GET /ratings/0 HTTP/1.1
> Host: ratings.default:9080
> User-Agent: curl/7.78.0-DEV
> Accept: */*
> 
RBAC: access denied* Mark bundle as not supporting multiuse
< HTTP/1.1 403 Forbidden
< content-length: 19
< content-type: text/plain
< date: Wed, 11 Aug 2021 13:08:10 GMT
< server: envoy
< x-envoy-upstream-service-time: 10
< 
{ [19 bytes data]
100    19  100    19    0     0    956      0 --:--:-- --:--:-- --:--:--  1000
* Connection #0 to host ratings.default left intact

Using Istio’s custom resources allows us to design a system that is intentional where trust zones are declarative all while maintaining strict security protocols.

Reach out to us at Solo.io on Slack to find out more how Istio, Gloo Mesh and Gloo Edge can assist with your zero trust security goals. You can always talk to an expert too!