Web Assembly at Scale with Gloo Edge

WebAssembly (abbreviated Wasm) is a binary instruction format for a stack-based virtual machine. Wasm is designed as a portable compilation target for programming languages, enabling deployment on the web for client and server applications. At Solo.io, we are very excited about Web Assembly as a way to extend our Envoy Proxy-based API Gateway (Gloo Edge) and the most popular Service Mesh (Istio). We recently made it very easy to use WebAssembly with Gloo Edge and Gloo Mesh.

This is the last blog post of a series about Gloo Edge scalability. In the first blog post, Envoy at Scale with Gloo Edge, we performed benchmarks to determine how Gloo Edge was able to scale in term of Requests Per Second (RPS.) We also provided information about the throughput one can expect based on the number of CPUs allocated to the gateway-proxy (Envoy) pod.

In the second blog post, Security at Scale with Gloo Edge, we measured the impact of enabling different security features, including HTTPS, JWT, API keys, and WAF.

In this blog post, we’ll measure the impact of deploying a Web Assembly (Wasm) filter to modify the request or response headers or body. We know based on my previous tests that we could get close to 90,000 RPS with standard HTTP requests when we don’t set any CPU limits on the gateway-proxy pod.

In the following tests, we’ll compare the impact with a limit configured to eight CPUs and without a limit. With a limit of eight CPUs, we could get more than 16,000 RPS with standard HTTP requests.

The regex Wasm filter

To be able to create a regex filter, we need to pick a language which includes a regex package and can be compiled in Wasm format.

When we started to work on this filter, we wanted to use AssemblyScript, but there was no regex support at that time. (There is now a community module.) So, we decided to use TinyGo, but when we started to test my filter, we discovered that it was always crashing due to out of memory issues. We opened an issue on the TinyGo repository which has now been fixed. Long story short, the filter is now ready and stable.

We can easily deploy this filter on Gloo Edge or Istio (using Gloo Mesh) and we can pass some information about the operation we want to execute using the format `Action&&Match&&Replacement`. For example, if we want to replace the value foo by the value bar in the response body, we’ll use the option ResponseBody&&foo&&bar. The tag of the Wasm filter is `webassemblyhub.io/djannot/tinygo-regexp:0.1`.

Deploying the Wasm filter

The echoenv Docker image we use as a backend in our test is based on the Gin Golang Web Framework, so the response body contains the word GIN. Here is the response body we get before we deploy the filter:

{
  "env": [
    "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
    "HOSTNAME=echoenv-deployment-1-56479646dc-rbd2d",
    "GIN_MODE=release",
    "PORT=8080",
...

It’s easy to deploy a Wasm filter on Gloo Edge. We simply need to modify the gateway-proxy gateway as follows:

apiVersion: gateway.solo.io/v1
kind: Gateway
metadata:
  annotations:
    meta.helm.sh/release-name: gloo
    meta.helm.sh/release-namespace: gloo-system
  labels:
    app: gloo
    app.kubernetes.io/managed-by: Helm
  name: gateway-proxy
  namespace: gloo-system
spec:
  bindAddress: '::'
  bindPort: 8080
  httpGateway:
    options:
      wasm:
        filters:
        - config:
            '@type': type.googleapis.com/google.protobuf.StringValue
            value: ResponseBody&&GIN&&TONIC
          image: webassemblyhub.io/djannot/tinygo-regexp:0.1
          name: myfilter
  proxyNames:
  - gateway-proxy
  useProxyProto: false

We set my filter to replace GIN by TONIC. Here is the response body we get after deploying the filter:

{
  "env": [
    "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
    "HOSTNAME=echoenv-deployment-1-56479646dc-rbd2d",
    "TONIC_MODE=release",
    "PORT=8080",
...

Response Body

We can now launch the benchmark and check the results on Grafana. Here is the result without a limit.

As we can see, we still get really good performance with my WASM filter. The throughput is more 50,000 RPS. Here is the result with a limit of eight CPUs:

The impact is a little bit higher when we set a CPU limit, but the throughput is more than 7,000 RPS.

In the remaining tests, we’ll always set a limit of eight CPUs.

Response Headers

Here are the headers we currently get:

HTTP/1.1 200 OK
content-type: application/json; charset=utf-8
date: Tue, 06 Apr 2021 13:16:07 GMT
x-envoy-upstream-service-time: 1
server: envoy
transfer-encoding: chunked

We’re now configuring my Wasm filter to replace utf-8 by ascii in the response headers. Here is what we get after we apply the new parameters:

HTTP/1.1 200 OK
content-type: application/json; charset=ascii
date: Tue, 06 Apr 2021 13:16:07 GMT
x-envoy-upstream-service-time: 1
server: envoy
transfer-encoding: chunked

We can now launch the benchmark and check the results on Grafana. Here is the result with a limit of eight CPUs:

The impact is higher, but the performance is still good. The throughput is higher than 3,500 RPS.

Request Body

We modify the test plan to send POST requests with the data username=foo:

          <elementProp name="HTTPsampler.Arguments" elementType="Arguments" guiclass="HTTPArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
            <collectionProp name="Arguments.arguments">
              <elementProp name="username" elementType="HTTPArgument">
                <boolProp name="HTTPArgument.always_encode">false</boolProp>
                <stringProp name="Argument.value">foo</stringProp>
                <stringProp name="Argument.metadata">=</stringProp>
                <boolProp name="HTTPArgument.use_equals">true</boolProp>
                <stringProp name="Argument.name">username</stringProp>
              </elementProp>
            </collectionProp>
          </elementProp>

And we’re now configuring the Wasm filter to replace foo by bar in the request body. We can now launch the benchmark and check the results on Grafana. Here is the result with a limit of eight CPUs:

The throughput is more than 7,000 RPS, similar to the throughput we was getting when modifying the response body.

Request Headers

We add the following section to the test plan:

        <hashTree>
          <HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="HTTP Header Manager" enabled="true">
            <collectionProp name="HeaderManager.headers">
              <elementProp name="" elementType="Header">
                <stringProp name="Header.name">Key</stringProp>
                <stringProp name="Header.value">foo</stringProp>
              </elementProp>
            </collectionProp>
          </HeaderManager>
          <hashTree/>
        </hashTree>

This way, we tell JMeter to add a Key header with the value foo. And we’re now configuring the Wasm filter to replace foo by bar in the request headers. We can now launch the benchmark and check the results on Grafana. Here is the result with a limit of eight CPUs:

The impact is higher, but we still get around 2,000 RPS.

What if we want to use the Wasm filter only on some routes ?

Then we probably don’t want to have the performance hit for other routes. There’s a simple workaround for that. First, we create a Gloo Edge Gateway object where the Wasm filter will be applied:

apiVersion: gateway.solo.io/v1
kind: Gateway
metadata:
  labels:
    app: gloo
  name: gateway-proxy-wasm
  namespace: gloo-system
spec:
  bindAddress: '127.0.0.1'
  bindPort: 8082
  httpGateway:
    virtualServices:
    - name: wasm
      namespace: gloo-system
    options:
      wasm:
        filters:
        - config:
            '@type': type.googleapis.com/google.protobuf.StringValue
            value: ResponseBody&&GIN&&TONIC
          image: webassemblyhub.io/djannot/tinygo-regexp:0.1
          name: myfilter
  proxyNames:
  - gateway-proxy
  useProxyProto: false

Note that this gateway is bound to localhost and to a different port. We can see that this Gateway reference a wasm VirtualService. Here is how we defined it:

apiVersion: gateway.solo.io/v1
kind: VirtualService
metadata:
  name: wasm
  namespace: gloo-system
spec:
  virtualHost:
    domains:
      - "*"
    routes:
    - matchers:
      - prefix: /myroutewithwasm
      routeAction:
        single:
          upstream:
            name: default-echoenv-service-1-8080
            namespace: gloo-system

Next, we need an Upstream which corresponds to localhost:8082:

apiVersion: gloo.solo.io/v1
kind: Upstream
metadata:
  name: wasm-upstream
  namespace: gloo-system
spec:
  static:
    hosts:
      - addr: localhost
        port: 8082

Finally, we can create a VirtualService referenced by the gateway-proxy gateway with some routes configured to use the Wasm Upstream:

apiVersion: gateway.solo.io/v1
kind: VirtualService
metadata:
  name: default
  namespace: gloo-system
spec:
  virtualHost:
    domains:
      - "*"
    routes:
    - matchers:
      - prefix: /myroutewithwasm
      routeAction:
        single:
          upstream:
            name: wasm-upstream
            namespace: gloo-system
    - matchers:
      - prefix: /myroutewithoutwasm
      routeAction:
        single:
          upstream:
            name: another-upstream
            namespace: gloo-system

Now, only the routes configured to send the traffic to the Wasm Upstream will have the Wasm filter applied. Note that there is a Pull Request (http: add composite filter allowing dynamic filter selection) in Envoy which will make it possible for us to improve the user experience with Wasm and Gloo Edge in the future.

What if we want to use the Wasm filter on Istio ?

Gloo Mesh makes it easy to deploy Wasm filters on Istio. Many familiar with Istio know about the Bookinfo demo application. If we send a request from the productpage service to the reviews service, we get the following response headers:

{'x-powered-by': 'Servlet/3.1', 'content-type': 'application/json', 'date': 'Tue, 06 Apr 2021 16:22:31 GMT', 'content-language': 'en-US', 'content-length': '379', 'x-envoy-upstream-service-time': '859', 'server': 'envoy'}

To deploy my Wasm filter on the v1 version of the reviews service in cluster1, we can simply create the custom resource (CRD) below:

apiVersion: networking.enterprise.mesh.gloo.solo.io/v1beta1
kind: WasmDeployment
metadata:
  name: reviews-wasm
  namespace: gloo-mesh
spec:
  filters:
  - filterContext: SIDECAR_INBOUND
    wasmImageSource:
      wasmImageTag: webassemblyhub.io/djannot/tinygo-regexp:0.1
    staticFilterConfig:
      '@type': type.googleapis.com/google.protobuf.StringValue
      value: "ResponseHeaders&&en-US&&fr-FR"
  workloadSelector:
  - kubeWorkloadMatcher:
      clusters:
      - cluster1
      labels:
        app: reviews
        version: v1
      namespaces:
      - default

Now, if we send a request from the productpage service to the reviews service, we get the following response headers:

{'x-powered-by': 'Servlet/3.1', 'content-type': 'application/json', 'date': 'Tue, 06 Apr 2021 16:22:33 GMT', 'content-language': 'fr-FR', 'content-length': '295', 'x-envoy-upstream-service-time': '19', 'server': 'envoy'}

Easy.

Conclusion

The data we obtained when limiting the amount of CPU used by the gateway-proxy (Envoy) pod is the most useful. It can be used to determine the throughput we’ll get with eight CPUs depending on the transformation we want to perform with a Wasm filter.

Here is a table that summarizes the results:

Keep in mind that 1,000 RPS means more than 86 millions requests a day!