Automated A/B, A/B/n and canary tests are managed by iter8’s Kubernetes controller. The behavior of a particular experiment is controlled by an Experiment custom resource. Details of the CRD are here.
To assess the behavior of microservice versions, iter8 supports a few metrics out-of-the-box without requiring users to do any extra work. In addition, users can define their own custom metrics. Iter8’s out-of-the-box metrics as well as user-defined metrics can be referenced in the criteria of an experiment. More details about metrics are documented here.
A key goal of this project is to introduce statistically robust algorithms for decision making during cloud-native continuous experiments, including canary releases, and A/B rollouts, and A/B/n rollout experiments. Details are provided here.