Skip to content

Benchmark and Validate HTTP services

The load-test-http experiment generates requests for HTTP services, collects latency and error-related metrics, and validates service-level objectives (SLOs).

This experiment is designed for the following use-cases.

  • Load test
  • Benchmark
  • Validate service level objectives (SLOs)
  • Safe rollout
  • Continuous integration and delivery (CI/CD)
Before you begin

Run the httpbin sample service from a separate terminal. 

docker run -p 80:80 kennethreitz/httpbin
You can also use Podman or other alternatives to Docker in the above command.

Basic example

Benchmark an HTTP service with a GET endpoint by specifying the url.

iter8 launch -c load-test-http --set url=http://127.0.0.1/get

Specify metrics and SLOs

The following metrics are collected by default by this experiment:

  • http/request-count: total number of requests sent
  • http/error-count: number of error responses
  • http/error-rate: fraction of error responses
  • http/latency-mean: mean of observed latency values
  • http/latency-stddev: standard deviation of observed latency values
  • http/latency-min: min of observed latency values
  • http/latency-max: max of observed latency values
  • http/latency-pX: Xth percentile latency, for X in [50.0, 75.0, 90.0, 95.0, 99.0, 99.9]

All latency metrics have msec units.

Launch the following experiment. The --noDownload flag reuses the Iter8 experiment charts folder downloaded during the previous iter8 launch invocation.

iter8 launch -c load-test-http --noDownload \
--set url=http://127.0.0.1/get \
--set SLOs.http/error-rate=0 \
--set SLOs.http/latency-mean=50 \
--set SLOs.http/latency-p90=100 \
--set SLOs.http/latency-p95=200

In the experiment above, the following SLOs are validated.

  • error rate is 0
  • mean latency is under 50 msec
  • 90th percentile latency is under 100 msec
  • 95th percentile latency is under 200 msec

View experiment report

iter8 report
The text report looks like this 
Experiment summary:
*******************

  Experiment completed: true
  No task failures: true
  Total number of tasks: 2
  Number of completed tasks: 2

Whether or not service level objectives (SLOs) are satisfied:
*************************************************************

  SLO Conditions                   |Satisfied
  --------------                   |---------
  http/error-rate <= 0             |true
  http/latency-mean (msec) <= 50   |true
  http/latency-p90 (msec) <= 100   |true
  http/latency-p97.5 (msec) <= 200 |true


Latest observed values for metrics:
***********************************

  Metric                     |value
  -------                    |-----
  http/error-count           |0.00
  http/error-rate            |0.00
  http/latency-max (msec)    |8.93
  http/latency-mean (msec)   |5.49
  http/latency-min (msec)    |2.71
  http/latency-p50 (msec)    |5.38
  http/latency-p75 (msec)    |6.71
  http/latency-p90 (msec)    |7.77
  http/latency-p95 (msec)    |8.27
  http/latency-p97.5 (msec)  |8.60
  http/latency-p99 (msec)    |8.80
  http/latency-p99.9 (msec)  |8.92
  http/latency-stddev (msec) |1.57
  http/request-count         |100.00

iter8 report -o html > report.html # view in a browser
The HTML report looks like this

HTML report

Assert experiment outcomes

Assert that the experiment completed without failures, and all SLOs are satisfied. 

iter8 assert -c completed -c nofailure -c slos

The iter8 assert command asserts if the experiment result satisfies conditions that are specified. If assert conditions are satisfied, it exits with code 0; else, it exits with code 1. Assertions are especially useful inside CI/CD/GitOps pipelines.

Sample output from assert 
INFO[2021-11-10 09:33:12] experiment completed
INFO[2021-11-10 09:33:12] experiment has no failure                    
INFO[2021-11-10 09:33:12] SLOs are satisfied                           
INFO[2021-11-10 09:33:12] all conditions were satisfied

Specify load profile

Control the characteristics of the load generated by the load-test-http experiment by setting the number of queries (numQueries), the number of queries sent per second (qps), and the number of parallel connections used to send queries (connections).

iter8 launch -c load-test-http --noDownload \
--set url=http://127.0.0.1/get \
--set numQueries=200 \
--set qps=10 \
--set connections=5

Specify payload

Send any type of content as payload and specify the HTTP Content Type header. These options switch the HTTP method to POST.

Specify payload as a string. By default, its content type is set to application/octet-stream. 

iter8 launch -c load-test-http --noDownload \
--set url=http://127.0.0.1/post \
--set payloadStr="abc123"


iter8 launch -c load-test-http --noDownload \
--set url=http://127.0.0.1/post \
--set payloadStr="abc123" \
--set contentType="text/plain"

Fetch JSON content from a URL. Use this JSON as payload. Set content type to application/json. 

iter8 launch -c load-test-http --noDownload \
--set url=http://127.0.0.1/post \
--set payloadURL=https://data.police.uk/api/crimes-street-dates \
--set contentType="application/json"

Fetch jpeg image from a URL. Use this image as payload. Set content type to image/jpeg. 

iter8 launch -c load-test-http --noDownload \
--set url=http://127.0.0.1/post \
--set payloadURL=https://cdn.pixabay.com/photo/2021/09/08/17/58/poppy-6607526_1280.jpg \
--set contentType="image/jpeg"

To learn more about all the parameters of the load-test-http chart and their default values, please refer to the chart's values.yaml file.

Back to top