Configuring CORS and JWT in Istio

As more and more organizations leveraging Istio service mesh turn to Solo.io for production support, FIPS compliance, and architecture/operations best practices, we start to see patterns emerge and common questions arise. When we see enough of those questions, we try to share when we have a few moments to write. In this blog post, I'll show how to configure CORS and JWT to secure traffic when requests are part of cross-origin web application requests. CORS (Cross Origin Resource Sharing) is a well-explained model for allowing browsers to read the responses from requests made to backend APIs that don't originate on the same domain as the web page making the request. As long as the response from for an API call includes the appropriate Access-Control-Allow-Origin header, the browser will understand this cross-origin request was expected and is allowed. There are situations where a browser may send a "preflight" request to find what CORS policy exist for a particular resource. This is sent as an HTTP OPTIONS request with  various Access-Control-Request-x headers and response headers which give the allowable methods, duration, headers, and origins.

An Example

For example, to configure CORS in Istio, we configure a VirtualService resource with the appropriate CORS settings:

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: web-api-gw-vs
  namespace: istioinaction
spec:
  hosts:
  - "istioinaction.io"
  gateways:
  - web-api-gateway
  http:
  - route:
    - destination:
        host: web-api
        port:
          number: 8080
    corsPolicy:
      maxAge: 1m
      allowCredentials: true
      allowHeaders:
      - foo
      - bar
      allowMethods:
      - GET
      allowOrigin:
      - http://istio.io
      allowOrigins:
      - exact: http://istio.io

If we try call this with a preflight request, we may see a response like this:

$  curl -i -X OPTIONS -H "Host: istioinaction.io" -H "Origin: http://istio.io" \
-H "Access-Control-Request-Method: GET" $(istioctl-ip)

HTTP/1.1 200 OK
access-control-allow-origin: http://istio.io
access-control-allow-credentials: true
access-control-allow-methods: GET
access-control-allow-headers: foo,bar
access-control-max-age: 60
date: Tue, 02 Feb 2021 22:29:04 GMT
server: istio-envoy
content-length: 0

Knowing this preflight response, we should be able to make the real call and expect the CORS headers to be returned:

$ curl -I -H "Host: istioinaction.io" -H "Origin: http://istio.io" \
-H "Access-Control-Request-Method: GET" $(istioctl-ip)

HTTP/1.1 200 OK
access-control-allow-origin: http://istio.io
vary: Origin
date: Tue, 02 Feb 2021 22:31:58 GMT
content-length: 1105
content-type: text/plain; charset=utf-8
x-envoy-upstream-service-time: 130
server: istio-envoy
access-control-allow-credentials: true

So far this has been pretty straight forward. But what happens when the cross-origin request needs to identify the caller with some kind of access token?  As we know CORS itself doesn't provide any security guarantees (the browser does this on behalf of the user), but we should probably validate the identity of the caller. We can do that with JWT tokens and configure Istio to require them.

Using JWT tokens in CORS request

To configure JWT expectations for requests coming into Istio, we use a RequestAuthentication resource:

apiVersion: security.istio.io/v1beta1
kind: RequestAuthentication
metadata:
  name: jwt-soloio-reqauth
  namespace: istio-system
spec:
  jwtRules:
  - issuer: solo.io
    jwks: |
      { "keys":[ {"kty":"RSA","e":"AQAB","kid":"858d304f-6df9-463e-a763-b735a4449857",
"n":"x7s5_6vgGPoW1PKkSMkau-Xr9JVZJNbqKXJ7RWKCqPxL5Vtj8II7lPi8d-x60f54oJTnyL_0zMVVsSq3msDhqzBSMpCUCR2q-
pHjS_29rrFBlBHy1ty8kiMo-qXZn3SmSqGRo53bdWoiQ6ZWVZ2mkgYpdlNzNaSUe8AOQKnxBC3rBwx3-
0g9RJvd4MM46YkN6Epr_NUKN___osqzfVfNoG9YLp1pbatjpqvO1XlxX4qCNLScY5FOFrkTf95O3a3Y195o89
D9XwULWT1baVuO_z7Ueug68b0t51mGXuEC572DmEbc8xHLnVfLM18QADvOnSzqY6vtIQcuGE_V_c7ATQ"}]}
  selector:
    matchLabels:
      app: istio-ingressgateway

We configure Istio's ingress gateway to expect a valid JWT token when the request comes in. This is true except for preflight requests -- those won't need the JWT as we can bypass the validation in order to understand the CORS semantics before we send the real request. Now if we send a real request with a token, we should see it works:

curl -I -H "Host: istioinaction.io" -H "Origin: http://istio.io" -H "foo: bar" \
http://$(istioctl-ip)  -H "Authorization: Bearer $TOKEN"

HTTP/1.1 200 OK
access-control-allow-origin: http://istio.io
vary: Origin
date: Tue, 02 Feb 2021 22:37:41 GMT
content-length: 1104
content-type: text/plain; charset=utf-8
x-envoy-upstream-service-time: 47
server: istio-envoy
access-control-allow-credentials: true

Great! What if we send in a request with a bad token (expired, doesn't exist, etc)?

curl -I -H "Host: istioinaction.io" -H "Origin: http://istio.io" \
-H "Access-Control-Request-Method: GET" $(istioctl-ip)  -H "Authorization: Bearer 12345"

HTTP/1.1 401 Unauthorized
content-length: 79
content-type: text/plain
date: Tue, 02 Feb 2021 22:39:01 GMT
server: istio-envoy

So we do expect this response, right? We should be blocked if the identity token (JWT) was missing or is incorrect. However, what about the CORS headers? We notice they don't exist in the response here. This is a problem if this request was sent from the browser since it would be blocked for not satisfying the same-origin policy.  This response indicates that the server does not understand CORS (which it did for preflight!) and moreover makes it difficult to understand why there would be an error here: is it because of CORS or bad identity token? To understand how best to fix this, we can check the ordering of the Envoy HTTP filters in the Istio ingress gateway and we would notice that the JWT filter comes before the CORS filter. We can use an EnvoyFilter resource to correct this (see this issue for more):

apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
  name: cors-bypass-jwt
  namespace: istio-system
spec:
  workloadSelector:
    labels:
      app: istio-ingressgateway
  configPatches:
  - applyTo: HTTP_FILTER
    match:
      context: GATEWAY
      listener:
        filterChain:
          filter:
            name: "envoy.filters.network.http_connection_manager"
            subFilter:
              name: "envoy.filters.http.jwt_authn"
    patch:
      operation: MERGE
      value:
        name: "envoy.filters.http.jwt_authn"
        typed_config:
          "@type": "type.googleapis.com/envoy.extensions.filters.http.jwt_authn.v3.JwtAuthentication"
          bypass_cors_preflight: true

This EnvoyFilter resource changes the order of the CORS filter and now we get the right response even when we send a bad token (note: use of EnvoyFilters is an advanced topic, reach out to us and we can help with this and any other Istio questions)

$ curl -I -H "Host: istioinaction.io" -H "Origin: http://istio.io" \
-H "Access-Control-Request-Method: GET" $(istioctl-ip)  -H "Authorization: Bearer 12345"

HTTP/1.1 401 Unauthorized
content-length: 79
content-type: text/plain
access-control-allow-origin: http://istio.io
access-control-allow-credentials: true
date: Tue, 02 Feb 2021 22:45:34 GMT
server: istio-envoy

What next?

Istio service mesh is a powerful cloud-native technology but guidance from experts with many years of experiences (and the scars to prove it!) can benefit your journey. Solo.io provides LTS (long-term support) for Istio, with FIPS compliance, ARM builds, enterprise SLAs and operational guidance / best practices. We are also offering Istio training and workshops around Day-2 operations and especially multi-cluster operations. Please reach out to us or join our Slack and chat directly with our engineers!