[Tutorial] Rate Limiting of Service Requests in Istio Service Mesh
Istio is an open source and platform-independent service mesh that provides functionality for traffic management, policy enforcement and telemetry collection in Kubernetes application environments. As a service mesh, Istio solves the service-to-service communication for the applications deployed within the cluster.
While Istio has a robust feature set, end users are currently unable to set rate limits between the services on the service mesh. Rate limiting is a policy used to control the number of requests a service is allowed to handle within a given time period. Rate limiting policies can be effective in thwarting DoS attacks, metering service usage, and reducing the impact of rogue clients.
Rate limiting policies are commonly applied at the edge of an application network. API Gateways such as Gloo (and others) provide rate limiting services that allow separation of policy enforcement from applications.
Rate Limiting and Istio?
Could you use the service mesh to deliver an externally facing rate limiting facility? Should you already have an Istio deployment in place, you may be asking yourself this exact question.
Istio Mixer has historically provided rate limiting policies, however it is now deprecated and does not represent the best path forward. API/Edge gateways are an option to protect the network but perhaps you want to explore another option.
The Developer Portal for Istio by Solo.io allows you to leverage your Istio investment to support service rate limiting, for your API consumers both inside and outside your application network.
This blog post will explain how rate limiting works in Developer Portal along with a brief tutorial on how to configure it.
Rate Limiting through a Developer Portal
The Developer Portal for Istio makes it easy for an application team to catalog the APIs running within a service mesh. Plus, it provides a great user experience to browse, consume, and document the APIs running in that service mesh. Once APIs are defined, they can be exposed through custom portals that allow developers to browse and consume the published APIs. These portals provide self-service facilities for users to signup, obtain security keys, and test the APIs. Additionally, the Developer Portal enhances security for Istio with integration to authentication systems and Single Sign-On capabilities.
How Does It Work?
The Developer Portal is a Kubernetes-native solution using native Custom Resource Definition objects (CRDs). They are specified in a familiar Kubernetes declarative style, and can be managed using both native tools and modern CD platforms like Flux and Argo. No external databases or other outside resources are required to operate the Developer Portal.
Both the general Developer Portal configuration, and rate limiting are driven by three types of CRD objects: ApiDoc, ApiProduct, and Portal. ApiDocs are typically created by consuming an OpenApi / Swagger spec or a gRPC schema from an upstream application. ApiProducts are then assembled from the components of one or more ApiDocs. These ApiProducts are where Usage Plans are created. These Usage Plans specify the rate limiting policies that are then applied to specific APIs.
The Developer Portal ships with a rate limit server that plugs into Istio’s ingress-gateway component. The Developer Portal uses Istio’s EnvoyFilter resource to wire all of this up, which alleviates the need for a custom proxy. You can use the native Istio Ingress Gateway as a lightweight API Gateway. For a more full-featured, powerful API Gateway see the Gloo API Gateway. Like the Developer Portal, Gloo is also built on Envoy and plugs into Istio.
An ApiProduct can be exposed either to internal or external developers via a Portal. Portal publishers select the ApiProducts to be exposed, add RBAC user and group specifications, plus add custom branding and documentation content to support Portal consumers. Portal consumers gain access through a secured web UI, browse published APIs, and produce API keys via a self-service API.
See a more complete technical deep dive of the Istio Developer Portal here.
Rate Limiting Tutorial
This tutorial walks you through configuring the Developer Portal rate limiting service for the ubiquitous Petstore sample API. It assumes as a starting point that you have followed the Getting Started guide. This tutorial includes two parts:
- Part 1 to set up the Petstore service, ingest its OpenAPI spec into an ApiDoc, then use that component to publish an ApiProduct and connect to it via the Istio Gateway.
- Part 2 then establishes a basic portal configuration that dynamically produces a web interface for developers to consume. This tutorial layers a rate limiting example over top of this configuration.
First, let’s review where we are. We have deployed the Petstore service with an Istio sidecar. An Istio Ingress Gateway is deployed to manage request traffic into the cluster. Using Developer Portal, we have now configured this Gateway to allow outside access to the Petstore service. The Gateway has Envoy filters installed that allow us to attach rate limiting and external auth policies. We have also established a Developer Portal admin server to provide an internal web interface for managing this configuration. Finally, an external Portal web interface allows users to browse published APIs and provision API tokens.
Below is a k9s view of the relevant Kubernetes pods running in a GKE cluster. At request time, the istio-ingressgateway receives the request. If there are rate limiting Usage Plans in place, the gateway delegates those to the rate-limiter service. The rate-limiter interrogates its Redis store and responds whether or not the request is acceptable based on its configured policies.
In this initial state, we can access the API endpoints anonymously and with no rate limiting.
$ curl http://api.example.com/api/pets
[{"id":1,"name":"Dog","status":"available"},{"id":2,"name":"Cat","status":"pending"}]
Establish a Rate Limiting Policy
To apply rate limiting via the Developer Portal, we first need to establish a Usage Plan that specifies a request limit for the managed API. We will do this via the internal administrative web interface, although since the rate limits are an element of the ApiProduct CRD, we could change the specification just as easily via kubectl or manage it from a code repo using a CI/CD framework.
In the API Product editor on the admin interface, we establish a basic usage plan that allows 3 requests per minute and authorizes requests using an API Key.
Finally, we need to create a Portal User object and associate that User with the Usage Plan. We navigate to the Access Control tab of the admin interface and Create a User.
We make the Petstore Product API available to our new User.
We select the Petstore Portal to make available to our new user, and then we Create the User.
Now that the user has been created and associated with the Petstore API, we can edit it and assign our basic-usage-plan to the User.
Once we update the User with its Usage Plan, we return to the API Product viewer and note that the new user is assigned to this API and that the Usage Plan is bound to that User.
Since the APIProduct is a first-class Kubernetes object, we can also confirm from kubectl that our 3-request-per-minute Usage Plan has been successfully deployed with apiKey authorization.
$ kubectl get apiproduct petstore-product -o yaml
apiVersion: devportal.solo.io/v1alpha1
kind: APIProduct
metadata:
<snip>
plans:
- authPolicy:
apiKey: {}
displayName: Basic Usage Plan
name: basic-usage-plan
rateLimit:
requestsPerUnit: 3
unit: MINUTE
publishRoutes: true
status:
<snip>
publishedPortals:
- domains:
- portal.example.com
name: petstore-portal
namespace: default
state: Succeeded
usagePlans:
- name: basic-usage-plan
Now that a Usage Plan has been applied to the Petstore API product, anonymous access fails with a 401 Unauthorized error.
$ curl -v http://api.example.com/api/pets * Trying 34.75.214.139... * TCP_NODELAY set * Connected to api.example.com (34.75.214.139) port 80 (#0) > GET /api/pets HTTP/1.1 > Host: api.example.com > User-Agent: curl/7.64.1 > Accept: */* > < HTTP/1.1 401 Unauthorized < www-authenticate: API key is missing or invalid < date: Wed, 19 Aug 2020 21:08:48 GMT < server: istio-envoy < content-length: 0 < * Connection #0 to host api.example.com left intact * Closing connection 0
Test the Policy
So at this point, we have secured our externally facing API by injecting an authorization and rate limiting Usage Plan into the Istio Ingress Gateway using our Developer Portal.
Our final task is to “switch hats” and assume the role of the developer coming in from outside to use our Portal. For that, we will navigate to the Portal’s external web interface and login as the external user we provisioned through the admin server.
Now we can browse the APIs allocated to us, generate API Keys if necessary, and then test them out.
From the Portal’s home page, we navigate to API Keys and follow the steps to Add an API Key.
We can now test out that API Key either from the Portal itself or from a curl CLI. Starting with the Portal, we navigate to the Petstore Portal API and click Authorize, specifying the API key we just generated for the Basic Usage Plan.
Now that we are authorized, we’ll use the Swagger interface to expand the GET /api/pets endpoint and Execute the operation with the Portal supplying our API Key. Note the 200 OK response and the body containing the same two pets as before.
We’ll switch to curl to demonstrate what happens when the Usage Plan rate limits are exceeded. At first, the endpoint responds exactly as we see in the portal. But after the fourth consecutive request in less than a minute, we hit the tripwire and begin to see 429 Too Many Requests errors. So our rate limit on this API for this user is being enforced just as expected.
$ curl -v -X GET "http://api.example.com/api/pets" -H "accept: application/json" -H "api-key: YTgxN2JiMjgtZWYxZi1iMWQzLWI5MWMtNWJhOGNkM2Q3NjVh" Note: Unnecessary use of -X or --request, GET is already inferred. * Trying 34.75.214.139... * TCP_NODELAY set * Connected to api.example.com (34.75.214.139) port 80 (#0) > GET /api/pets HTTP/1.1 > Host: api.example.com > User-Agent: curl/7.64.1 > accept: application/json > api-key: YTgxN2JiMjgtZWYxZi1iMWQzLWI5MWMtNWJhOGNkM2Q3NjVh > < HTTP/1.1 429 Too Many Requests < x-envoy-ratelimited: true < date: Wed, 19 Aug 2020 21:53:55 GMT < server: istio-envoy < content-length: 0 < * Connection #0 to host api.example.com left intact * Closing connection 0
Watch the Demo
Learn More
Istio is a powerful solution for solving service-to-service communication challenges and helping to operate microservices at scale. However, once you’ve deployed a large graph of microservices, not only do you need to operate it, but you need to enable others to harness the power of the services and APIs you’ve deployed. That can be a challenge.
The Istio Developer Portal leverages your Istio development while allowing you to specify policies such as rate limiting and external auth in a declarative, cloud-native fashion.
