[Tutorial] External Authorization of Service Requests in Istio Service Mesh
Istio is an open source and platform-independent service mesh that provides functionality for traffic management, policy enforcement and telemetry collection in Kubernetes application environments. As a service mesh, Istio solves the service-to-service communication for the applications deployed within the cluster.
Istio has a robust feature set to address these east-west traffic concerns. In fact, we are huge Istio fans at Solo.io. But we’re also experts in building Envoy-based edge gateways like our Gloo product. So we’re aware that while enterprises might like to leverage the native Istio Ingress Gateway as a full gateway solution, it is not full-featured out of the box.
So consider this question: What if there were a way to enhance the Istio ingress gateway to add easy-to-use, north-south features like rate limiting and external authorization?
That’s the question we’ll explore in this tutorial. Specifically, we’ll walk through how you can use the Developer Portal for Istio by Solo.io to configure an External Authorization server to manage the publication of APIs, API policies, and client identity.
How Does It Work?
The Developer Portal is a Kubernetes-native solution using native Custom Resource Definition objects (CRDs). They are specified in a familiar Kubernetes declarative style, and can be managed using both native tools and modern CD platforms like Flux and Argo. No external databases or other outside resources are required to operate the Developer Portal.
Both the general Developer Portal configuration and request authorization are driven by three types of CRD objects: ApiDoc, ApiProduct, and Portal. ApiDocs are typically created by consuming an OpenApi / Swagger spec or a gRPC schema from an upstream application. ApiProducts are then assembled from the components of one or more ApiDocs. These ApiProducts are where Usage Plans are created. These Usage Plans specify rate limiting policies that are then applied to specific Portal Users and Groups for a given API.
The Developer Portal ships with an ext-auth server that plugs into Istio’s ingress gateway component. Authorization decisions for all service requests are delegated to this ext-auth server. The Developer Portal uses Istio’s EnvoyFilter resource to wire all of this up, which alleviates the need for a custom proxy. You can use the native Istio Ingress Gateway as a lightweight API Gateway. For a more full-featured, powerful API Gateway see the Gloo API Gateway. Like the Developer Portal, Gloo is also built on Envoy and plugs into Istio.
An ApiProduct can be exposed either to internal or external developers via a Portal. Portal publishers select the ApiProducts to be exposed, add RBAC User and Group specifications, plus add custom branding and documentation content to support Portal consumers. Portal consumers gain access through a secured web UI, browse published APIs, and produce API keys via a self-service API.
See a more complete technical deep dive of the Istio Developer Portal here.
External Authorization Tutorial
This tutorial walks you through configuring the Developer Portal ext-auth service for the ubiquitous Petstore sample API. It assumes as a starting point that you have followed the Getting Started guide. This tutorial includes two parts:
- Part 1 to set up the Petstore service, ingest its OpenAPI spec into an ApiDoc, then use that component to publish an ApiProduct and connect to it via the Istio Gateway.
- Part 2 then establishes a basic portal configuration that dynamically produces a web interface for developers to consume.
This tutorial then layers an ext-auth example over top of this configuration.
First, let’s review where we are. We have deployed the Petstore service with an Istio sidecar. An Istio Ingress Gateway is deployed to manage request traffic into the cluster. Using Developer Portal, we have now configured this Gateway to allow outside access to the Petstore service. The Gateway has Envoy filters installed that allow us to attach authorization and rate limiting policies. We have also established a Developer Portal admin server to provide an internal web interface for managing this configuration. Finally, an external Portal web interface allows users to browse published APIs and provision API tokens.
Below is a k9s view of the relevant Kubernetes pods running in a GKE cluster. At request time, the istio-ingressgateway receives the request. Where there are Usage Plans in place, the gateway delegates those requests to the ext-auth service to confirm eligibility of the client request based on the Users and Groups assigned for the given plan. The rate-limiter interrogates its Redis store and responds whether or not the request is acceptable based on its configured policies.
In this initial state, we can access the API endpoints anonymously and with no rate limiting.
$ curl http://api.example.com/api/pets
[{"id":1,"name":"Dog","status":"available"},{"id":2,"name":"Cat","status":"pending"}]
Establish a Usage Plan
To apply external auth policies via the Developer Portal, we first need to establish a Usage Plan that specifies a request limit for the managed API. We will do this via the internal administrative web interface, although since the rate limits are an element of the ApiProduct CRD, we could just as easily change the specification via kubectl or from a code repo using a CI/CD framework.
In the API Product editor on the admin interface, we establish a usage plan that effectively allows unlimited requests and authorizes requests using an API Key.
Finally, we need to create a Portal User object and associate that User with the Usage Plan. We navigate to the Access Control tab of the admin interface and Create a User.
We make the Petstore Product API available to our new User.
We select the Petstore Portal to make available to our new user, and then we Create the User.
Now that the user has been created and associated with the Petstore API, we can edit it and assign our unlimited-plan to the User.
Once we update the User with its Usage Plan, we return to the API Product viewer and note that the new user is assigned to this API and that the Usage Plan is bound to that User.
Now that a Usage Plan has been applied to the Petstore API product, anonymous access fails with a 401 Unauthorized error.
% curl http://api.example.com/api/pets -v * Trying 34.75.131.4... * TCP_NODELAY set * Connected to api.example.com (34.75.131.4) port 80 (#0) > GET /api/pets HTTP/1.1 > Host: api.example.com > User-Agent: curl/7.64.1 > Accept: */* > < HTTP/1.1 401 Unauthorized < www-authenticate: API key is missing or invalid < date: Wed, 30 Sep 2020 22:01:31 GMT < server: istio-envoy < content-length: 0 < * Connection #0 to host api.example.com left intact * Closing connection 0
Test the “Unlimited” Policy
At this point, we have secured our externally facing API by injecting a single-user authorization and rate limiting Usage Plan into the Istio Ingress Gateway using our Developer Portal.
Our final task is to “switch hats” and assume the role of the developer coming in from outside to use our Portal. For that, we will navigate to the user-branded Portal’s external web interface and login as the external user1 we provisioned through the admin server.
Now we can browse the APIs allocated to us, generate API Keys if necessary, and then test them out.
From the Portal’s home page, we navigate to API Keys and follow the steps to Add an API Key.
We can now test out that API Key either from the Portal itself or from a curl CLI. Starting with the Portal, we navigate to the Petstore Portal API and click Authorize, specifying the API key we just generated for the Unlimited Usage Plan.
Now that we are authorized, we’ll use the Swagger interface to expand the GET /api/pets endpoint and Execute the operation with the Portal supplying our API Key. Note the 200 OK response and the body containing the same two pets as before.
Build a “Limited” Policy for a Group
Now let’s assume that a couple of problems arise. First, the load on our system is unsustainable. Some of our external developers like user1 are irresponsibly banging on expensive endpoints like this /api/pets dump-the-whole-database interface. So we want to restrict basic Users like user1 to lower request levels.
Second, we want to limit the administrative burden associated with on-boarding new users. We don’t want to be forced to provision detailed permissions for every new user. We’d like to establish user Groups instead and assign permissions at that level. Then for a new user, we would simply add them to the appropriate Group. We could then delegate authorization decisions to the ext-auth server, which understands how to make decisions based on either individual User permissions or Group permissions.
So to address both of these problems, we’re going to establish a new Limited Usage Plan that limits the request volume to our API product. Then we’ll establish a User with no direct privileges. Finally we’ll create a Group and assign the privileges to it.
We will carry out all of these policy building steps on the administrative web interface. To create the Limited Usage Plan, return to the API Product we created. Follow our earlier creation of the Unlimited plan, but create a Usage Plan with name limited-plan, display name Limited, and a maximum rate of 3 requests per minute. When you save that, the plans for the petstore-product should look like this:
To establish the new user, return to Access Control and create a new User user2. But unlike user1, do not assign any privileges directly to the user. Instead, create a new Group group1 with user2 as its only user, plus the Petstore API, the Limited Usage Plan, and access to the Petstore portal. Following these operations, your collection of portal uses should look like below, with user1 having direct access to the API and Portal with the Unlimited plan, and with user2 having access only to the rights granted through membership in group1.
Test the “Limited” Policy
To test ext-auth using only group privileges, we will first logout of the active user1 session in the portal, then reconnect as user2. Generate an API key for user2 as we did earlier for user1. Then test that the new API key works in the portal sandbox.
Finally, we’ll switch to curl in a terminal interface to demonstrate that the group1 limits on request volume are indeed being enforced. At first, the endpoint responds exactly as we see in the portal. But after the fourth consecutive request in less than a minute, we hit the tripwire and begin to see 429 Too Many Requests errors. So our rate limit on this API for all users in group1 is being enforced by the ext-auth and rate-limit servers just as expected.
$ curl -v -X GET "http://api.example.com/api/pets" -H "accept: application/json" -H "api-key: YTgxN2JiMjgtZWYxZi1iMWQzLWI5MWMtNWJhOGNkM2Q3NjVh" * Trying 34.75.214.139... * TCP_NODELAY set * Connected to api.example.com (34.75.214.139) port 80 (#0) > GET /api/pets HTTP/1.1 > Host: api.example.com > User-Agent: curl/7.64.1 > accept: application/json > api-key: YTgxN2JiMjgtZWYxZi1iMWQzLWI5MWMtNWJhOGNkM2Q3NjVh > < HTTP/1.1 429 Too Many Requests < x-envoy-ratelimited: true < date: Wed, 19 Aug 2020 21:53:55 GMT < server: istio-envoy < content-length: 0 < * Connection #0 to host api.example.com left intact * Closing connection 0
What Have We Accomplished?
First, we used the Dev Portal to configure the Istio ingress gateway to delegate authorization decisions to our ext-auth server. Significantly, we did this using standard Kubernetes objects and storing them using standard Kubernetes persistence mechanisms. We were not required to configure an external database.
Second, we configured these authorization and rate limiting policies external to the application itself. Our target application doesn’t need to know anything about the policies we applied in this exercise.
Third, we used the Dev Portal User and Group abstractions to make it easier for administrators to efficiently manage authorization policies.
Finally, we produced a user-branded, dynamically generated Portal that allows external developers to provision their own API keys and also provides a sandbox to test them out.
Watch the Demo
Learn More
Istio is a powerful solution for solving service-to-service communication challenges and helping to operate microservices at scale. However, once you’ve deployed a large graph of microservices, not only do you need to operate it, but you need to enable others to harness the power of the services and APIs you’ve deployed. That can be a challenge.
The Istio Developer Portal leverages your Istio deployment while allowing you to specify policies such as rate limiting and external auth in a declarative fashion on a cloud-native platform.
