Kubernetes Course Labs

Lab Solution

Here's my solution:

kubectl apply -f labs/productionizing/solution

Open two terminals and you can watch the repair and scale in action:

kubectl get pods -l app=configurable --watch

kubectl get endpoints configurable-lb --watch

Browse to the app and refresh lots. You might see failures because the app fails so frequently, but leave it a few seconds between refreshes and the app comes back online.

Or if you have watch installed (brew install watch on macOS; already on Linux; not available on Windows):

watch -n 0.5 kubectl get pods,endpoints,hpa

watch -n 0.5 curl -s http://localhost:8040

Eventually all the Pods will go into CrashLoopBackOff because Kubernetes thinks the app is unstable.

Testing the HPA

The HPA is independent of the Deployment. You can scale the Deployment manually (or delete Pods to simulate node loss) and the HPA will override it.

Scale down manually:

kubectl scale deployment/configurable --replicas 1

kubectl get hpa configurable-cpu --watch

The HPA scales back up after a few minutes - the minimum is not dependent on CPU usage and it overrides the scale setting for the Deployment.

Scale up manually:

kubectl scale deployment/configurable --replicas 8

kubectl get hpa configurable-cpu --watch

After a few more minutes the HPA scales down - it scales down to the maximum initially, but there's no CPU activity so it will repeatedly scale down until it gets to the minimum.

You can't configure the scale-up and scale-down timings for v1 HPAs. If you need that level of control you can use HorizontalPodAutoscaler (autoscaling/v2), which is a more complex HPA spec that allows for other metrics.

Back to the exercises