Benchmark and Validate HTTP services¶
load-test-http experiment generates requests for HTTP services, collects latency and error-related metrics, and validates service-level objectives (SLOs).
This experiment is designed for the following use-cases.
- Load test
- Validate service level objectives (SLOs)
- Safe rollout
- Continuous integration and delivery (CI/CD)
Before you begin
Run the httpbin sample service from a separate terminal.
docker run -p 80:80 kennethreitz/httpbin
Benchmark an HTTP service with a GET endpoint by specifying the
iter8 launch -c load-test-http --set url=http://127.0.0.1/get
Specify metrics and SLOs¶
The following metrics are collected by default by this experiment:
http/request-count: total number of requests sent
http/error-count: number of error responses
http/error-rate: fraction of error responses
http/latency-mean: mean of observed latency values
http/latency-stddev: standard deviation of observed latency values
http/latency-min: min of observed latency values
http/latency-max: max of observed latency values
http/latency-pX: Xth percentile latency, for X in
[50.0, 75.0, 90.0, 95.0, 99.0, 99.9]
All latency metrics have
Launch the following experiment. The
--noDownload flag reuses the Iter8 experiment
charts folder downloaded during the previous
iter8 launch invocation.
iter8 launch -c load-test-http --noDownload \ --set url=http://127.0.0.1/get \ --set SLOs.http/error-rate=0 \ --set SLOs.http/latency-mean=50 \ --set SLOs.http/latency-p90=100 \ --set SLOs.http/latency-p95=200
In the experiment above, the following SLOs are validated.
- error rate is 0
- mean latency is under 50 msec
- 90th percentile latency is under 100 msec
- 95th percentile latency is under 200 msec
View experiment report¶
The text report looks like this
Experiment summary: ******************* Experiment completed: true No task failures: true Total number of tasks: 2 Number of completed tasks: 2 Whether or not service level objectives (SLOs) are satisfied: ************************************************************* SLO Conditions |Satisfied -------------- |--------- http/error-rate <= 0 |true http/latency-mean (msec) <= 50 |true http/latency-p90 (msec) <= 100 |true http/latency-p97.5 (msec) <= 200 |true Latest observed values for metrics: *********************************** Metric |value ------- |----- http/error-count |0.00 http/error-rate |0.00 http/latency-max (msec) |8.93 http/latency-mean (msec) |5.49 http/latency-min (msec) |2.71 http/latency-p50 (msec) |5.38 http/latency-p75 (msec) |6.71 http/latency-p90 (msec) |7.77 http/latency-p95 (msec) |8.27 http/latency-p97.5 (msec) |8.60 http/latency-p99 (msec) |8.80 http/latency-p99.9 (msec) |8.92 http/latency-stddev (msec) |1.57 http/request-count |100.00
iter8 report -o html > report.html # view in a browser
The HTML report looks like this
Assert experiment outcomes¶
Assert that the experiment completed without failures, and all SLOs are satisfied.
iter8 assert -c completed -c nofailure -c slos
iter8 assert command asserts if the experiment result satisfies conditions that are specified. If assert conditions are satisfied, it exits with code
0; else, it exits with code
1. Assertions are especially useful inside CI/CD/GitOps pipelines.
Sample output from assert
INFO[2021-11-10 09:33:12] experiment completed INFO[2021-11-10 09:33:12] experiment has no failure INFO[2021-11-10 09:33:12] SLOs are satisfied INFO[2021-11-10 09:33:12] all conditions were satisfied
Specify load profile¶
Control the characteristics of the load generated by the
load-test-http experiment by setting the number of queries (
numQueries), the number of queries sent per second (
qps), and the number of parallel connections used to send queries (
iter8 launch -c load-test-http --noDownload \ --set url=http://127.0.0.1/get \ --set numQueries=200 \ --set qps=10 \ --set connections=5
Send any type of content as payload and specify the HTTP Content Type header. These options switch the HTTP method to POST.
Specify payload as a string. By default, its content type is set to
iter8 launch -c load-test-http --noDownload \ --set url=http://127.0.0.1/post \ --set payloadStr="abc123"
iter8 launch -c load-test-http --noDownload \ --set url=http://127.0.0.1/post \ --set payloadStr="abc123" \ --set contentType="text/plain"
Fetch JSON content from a URL. Use this JSON as payload. Set content type to
iter8 launch -c load-test-http --noDownload \ --set url=http://127.0.0.1/post \ --set payloadURL=https://data.police.uk/api/crimes-street-dates \ --set contentType="application/json"
Fetch jpeg image from a URL. Use this image as payload. Set content type to
iter8 launch -c load-test-http --noDownload \ --set url=http://127.0.0.1/post \ --set payloadURL=https://cdn.pixabay.com/photo/2021/09/08/17/58/poppy-6607526_1280.jpg \ --set contentType="image/jpeg"
To learn more about all the parameters of the
load-test-http chart and their default values, please refer to the chart's