Skip to content

Metrics Tasks



The metrics/collect takes enables collection of builtin metrics. It generates a stream of HTTP requests to one or more app/ML model versions, and collects latency/error metrics.


The following start action contains a metrics/collect task which is executed at the start of the experiment. The task sends a certain number of HTTP requests to each version specified in the task, and collects builtin latency/error metrics for them.

- task: metrics/collect
      # Version names must be unique. 
      # Each version name in the task must match the name of some version
      # in the versionInfo field of the experiment spec.
    - name: iter8-app
      # URL is where this version receives HTTP requests
      url: http://iter8-app.default.svc:8000
    - name: iter8-app-candidate
      url: http://iter8-app-candidate.default.svc:8000


Field name Field type Description Required
time string Duration of the metrics/collect task run. Specified in the Go duration string format. Default value is 5s. No
payloadURL string URL of JSON-encoded data. If this field is specified, the metrics collector will send HTTP POST requests to versions, and the POST requests will contain this JSON data as payload. No
versions []Version A non-empty list of versions. Yes
loadOnly bool If set to true, this task will send requests without collecting metrics. Default value is false. No


Field name Field type Description Required
name string Name of the version. Version names must be unique and must match one of the version names in the VersionInfo field of the experiment. Yes
qps float How many queries per second will be sent to this version. Default is 8.0. No
headers map[string]string HTTP headers to be used in requests sent to this version. No
url string HTTP URL of this version. Yes


This task will run for the specified duration (time), send requests to each version (versions) at the specified rate (qps), and will collect built-in metrics for each version. Builtin metric values are stored in the metrics field of the experiment status in the same manner as custom metric values.

The task may result in an error, for instance, if one or more required fields are missing or if URLs are mis-specified. In this case, the experiment to which it belongs will fail.

Start vs loop actions

If this task is embedded in start actions, it will run once at the beginning of the experiment.

If this task is embedded in loop actions, it will run in each loop of the experiment. The results from each run will be aggregated.

Load generation without metrics collection

You can use this task to send HTTP GET and POST requests to app/ML model versions without collecting metrics by setting the loadOnly input to true.