Automated blue-green rollout¶

This tutorial shows how Iter8 can be used to implement a blue-green rollout of a Kubernetes application. In a blue-green rollout, a percentage of requests are directed to a candidate version of the application. The remaining requests go to the primary, or initial, version. Iter8 enables a blue-green rollout by automatically configuring routing resources to distribute requests.

After a one-time initialization step, the end user merely deploys candidate versions, evaluates them, and either promotes or deletes them. Optionally, the end user can modify the percentage of requests being sent to the candidate. Iter8 automatically handles all underlying routing configuration.

Install Iter8¶

helm install --repo https://iter8-tools.github.io/iter8 iter8 traffic

helm install --repo https://iter8-tools.github.io/iter8 iter8 traffic \
--set clusterScoped=true

kubectl apply -k 'https://github.com/iter8-tools/iter8.git/kustomize/iter8/namespaceScoped?ref=v0.15.3'

kubectl apply -k 'https://github.com/iter8-tools/iter8.git/kustomize/iter8/clusterScoped?ref=v0.15.3'

Initialize primary¶

Application¶

Deploy the primary version of the application. In this tutorial, the application is a Kubernetes Deployment. We use httpbin as our application. Initialize the resources for the primary version (v0) as follows:

kubectl create deployment httpbin-0 --image=kennethreitz/httpbin --port=80
kubectl label deployment httpbin-0 app.kubernetes.io/version=v0
kubectl label deployment httpbin-0 iter8.tools/watch=true
kubectl expose deployment httpbin-0 --port=80

You can inspect the deployed Deployment. When the AVAILABLE field becomes 1, the application is fully deployed.

kubectl get deployment httpbin-0

Routing¶

Initialize the routing resources for the application to use a blue-green rollout strategy:

cat <<EOF | helm template routing --repo https://iter8-tools.github.io/iter8 routing-actions -f - | kubectl apply -f -
appType: deployment
appName: httpbin
action: initialize
strategy: blue-green
EOF

The initialize action (with strategy blue-green) configures the (Istio) service mesh to route all requests to the primary version of the application (httpbin-0). It further defines the routing policy that will be used when changes are observed in the application resources. By default, this routing policy splits requests 50-50 between the primary and candidate versions. For detailed configuration options, see the Helm chart.

Verify routing¶

To verify the routing configuration, you can inspect the VirtualService:

kubectl get virtualservice -o yaml httpbin

To send inference requests to the model:

Note that the model version responding to each inference request is noted in the response header app-version. In the requests above, we display only the response code and this header.

Deploy candidate¶

Deploy a candidate model using a second Deployment:

kubectl create deployment httpbin-1 --image=kennethreitz/httpbin --port=80
kubectl label deployment httpbin-1 app.kubernetes.io/version=v1
kubectl label deployment httpbin-1 iter8.tools/watch=true
kubectl expose deployment httpbin-1 --port=80

Verify routing changes¶

The deployment of the candidate triggers an automatic routing reconfiguration by Iter8. Inspect the VirtualService to see that the routing has been changed. Inspect the VirtualService to see that the routing has been changed. Requests are now distributed between the primary model and the secondary model:

kubectl get virtualservice httpbin -o yaml

You can send additional inference requests as described above. They will be handled by both versions of the model.

Modify weights (optional)¶

You can modify the weight distribution of inference requests as follows:

cat <<EOF | helm template routing --repo https://iter8-tools.github.io/iter8 routing-actions -f - | kubectl apply -f -
appType: deployment
appName: httpbin
action: modify-weights
strategy: blue-green
appVersions:
  - weight: 20
  - weight: 80
EOF

Note that using the modify-weights action overrides the default traffic split for all future candidate deployments.

As above, you can verify the routing changes.

Promote candidate¶

Promoting the candidate involves redefining the primary version of the application and deleting the candidate version.

Redefine primary¶

kubectl set image deployment/httpbin-0 httpbin=kennethreitz/httpbin
kubectl label deployment httpbin-0 app.kubernetes.io/version=v1 --overwrite

Delete candidate¶

Once the primary has been redeployed, delete the candidate:

kubectl delete deployment/httpbin-1 service/httpbin-1

Verify routing changes¶

Inspect the VirtualService to see that the it has been automatically reconfigured to send requests only to the primary.

Cleanup¶

If not already deleted, delete the candidate:

kubectl delete deployment/httpbin-1 service/httpbin-1

Delete routing:

cat <<EOF | helm template routing --repo https://iter8-tools.github.io/iter8 routing-actions -f - | kubectl delete -f -
appType: deployment
appName: httpbin
action: initialize
strategy: blue-green
EOF

Delete primary:

kubectl delete deployment/httpbin-0 service/httpbin-0

Uninstall Iter8:

helm delete iter8

kubectl delete -k 'https://github.com/iter8-tools/iter8.git/kustomize/iter8/namespaceScoped?ref=v0.15.3'

kubectl delete -k 'https://github.com/iter8-tools/iter8.git/kustomize/iter8/clusterScoped?ref=v0.15.3'