Skip to content

Benchmark and Validate gRPC with SLOs

The load-test-grpc experiment generates call requests for gRPC services, collects latency and error-related metrics, and validates service-level objectives (SLOs).

This experiment is designed for the following use-cases.

  • Load test
  • Benchmark
  • Validate service level objectives (SLOs)
  • Safe rollout
  • Continuous integration and delivery (CI/CD)

Before you begin

Run the gRPC sample service from a separate terminal.

docker run -p 50051:50051
You can also use Podman or other alternatives to Docker in the above command.

Basic example

Benchmark a gRPC service by specifying its host, its fully-qualified call (method) name, and the URL of Protocol Buffer file (protoURL) that defines the service.

Launch load-test-grpc experiment
iter8 launch -c load-test-grpc \
--set host="" \
--set call="helloworld.Greeter.SayHello" \
--set protoURL=""

Metrics and SLOs

The following metrics are collected by default by this experiment:

  • grpc/request-count: total number of requests sent
  • grpc/error-count: number of error responses
  • grpc/error-rate: fraction of error responses

The following latency metrics are also supported.

  • grpc/latency/mean: Mean latency
  • grpc/latency/stddev: Standard deviation of latency
  • grpc/latency/min: Min latency
  • grpc/latency/max: Max latency
  • grpc/latency/pX: X-th percentile latency, for any X in the range 0.0 to 100.0

Latency metrics have msec units. Any latency metric that is specified as part of SLOs is also collected.

For example, set the following parameter values in the iter8 launch command above.

--set SLOs.grpc/error-rate=0 \
--set SLOs.grpc/latency/mean=50 \
--set SLOs.grpc/latency/p90=100 \
--set SLOs.grpc/latency/p'97\.5'=200

In the above setting, the following SLOs are validated.

  • error rate is 0
  • mean latency is under 50 msec
  • 90th percentile latency is under 100 msec
  • 97.5th percentile latency is under 200 msec

View experiment report

iter8 report
The text report looks like this
Experiment summary:

  Experiment completed: true
  No task failures: true
  Total number of tasks: 2
  Number of completed tasks: 2

Whether or not service level objectives (SLOs) are satisfied:

  SLO Conditions                   |Satisfied
  --------------                   |---------
  grpc/error-rate <= 0             |true
  grpc/latency/mean (msec) <= 50   |true
  grpc/latency/p90 (msec) <= 100   |true
  grpc/latency/p97.5 (msec) <= 200 |true

Latest observed values for metrics:

  Metric                    |value
  -------                   |-----
  grpc/error-count          |0.00
  grpc/error-rate           |0.00
  grpc/latency/mean (msec)  |21.48
  grpc/latency/p90 (msec)   |34.00
  grpc/latency/p97.5 (msec) |37.00
  grpc/request-count        |200.00
iter8 report -o html > report.html # view in a browser
The HTML report looks like this

HTML report


Assert that the experiment completed without failures, and all SLOs are satisfied.

iter8 assert -c completed -c nofailure -c slos

The iter8 assert subcommand asserts if the experiment result satisfies conditions that are specified. If assert conditions are satisfied, it exits with code 0; else, it exits with code 1. Assertions are especially useful inside CI/CD/GitOps pipelines.

Sample output from assert
INFO[2021-11-10 09:33:12] experiment completed
INFO[2021-11-10 09:33:12] experiment has no failure                    
INFO[2021-11-10 09:33:12] SLOs are satisfied                           
INFO[2021-11-10 09:33:12] all conditions were satisfied

Load profile

Control the characteristics of the generated load generated by setting the number of requests (total), the number of requests per second (rps), number of connections to use (connections), and the number of concurrent request workers to use which will be distributed across the connections (concurrency).

Sample load profile
--set total=500 \
--set rps=25 \
--set concurrency=50 \
--set connections=10

Refer to the chart's values.yaml file for additional parameters related to the load profile such as duration, maxDuration, connectTimeout, and keepalive.

Call data

gRPC calls may include data serialized as Protocol Buffer messages.

Specify call data as values.

--set"frodo" \
--set data.realm.planet="earth" \
--set data.realm.location="middle" 

Use JSON data from a local file.

--set dataFile="/the/path/to/data.json" # "./data.json" also works

Supply a URL that hosts JSON data. Iter8 will download the data from this URL and use it in the requests.

--set dataURL="https://location.of/data.json"

Use binary data from a local file serialized as a single binary message or multiple count-prefixed messages.

--set binaryDataFile="/the/path/to/data.bin" # "./data.bin" also works

Supply a URL that hosts binary data serialized as a single binary message or multiple count-prefixed messages. Iter8 will download the data from this URL and use it in the requests.

--set binaryDataURL="https://location.of/data.bin"

For client streaming or bi-directional calls, this experiment accepts an array of messages, each element representing a single message within the stream call. If a single object is given for data, then it is automatically converted to an array with single element.

--set data[0].name="Joe" \
--set data[1].name="Kate" \
--set data[2].name="Sara"

In case of client streaming, this experiment sends all the data in the input array, and then closes and receives.

Call metadata

gRPC calls may include metadata which is information about a particular call.

Supply metadata as values.

--set metadata.darth="vader" \
--set metadata.lord="sauron" \
--set metadata.volde="mort"

Use JSON metadata from a local file.

--set metadataFile="/the/path/to/metadata.json" # "./metadata.json" also works

Supply a URL that hosts JSON metadata. Iter8 will download the metadata from this URL and use it in the requests.

--set metadataURL="https://location.of/metadata.json"

Proto and reflection

The gRPC server method signatures and message formats are defined in a .proto source file, which may also be compiled to a .protoset file.

Use a local .proto source file.

--set protoFile="/path/to/helloworld.proto" # "./helloworld.proto" also works

Use a URL that hosts a .proto source file. Iter8 will download the Protocol Buffer file and use it in the experiment.

--set protoURL=""

Supply the name of a .protoset file that is compiled from .proto source files.

--set protosetFile="./myservice.protoset"

Supply a URL that hosts a .protoset file.

--set protosetURL=""

In the absence of .proto and .protoset information, the experiment will attempt to use server reflection. You can supply reflect metadata.

--set reflectMetadata.clientId="5hL64dd0" \
--set reflectMetadata.clientMood="delightful"

Streaming gRPC

Refer to the values.yaml file which documents additional parameters related to streaming gRPC.

Back to top