Using Pause Pods to Overprovision a Cluster
Pause containers are containers that run and don't do anything. They can be used for various things. One use case is for over-provisioning a cluster to decrease the time it takes for real pods to get up and running during high moments of traffic.
The pattern goes like this:
- Create a default
PriorityClass(usingglobalDefault: true) with a value of 0 - Create a
PriorityClassof -1 (this will be used for the pause pods) -- The higher the priority value, the higher scheduling priority it will get - Apply a Deployment of pause pods configured with memory requests that are close to the size of your worker node's capacity, with a PriorityClass of -1. -> This will trigger the cluster to scale out, assuming you have an autoscaler installed.
- Now that the cluster is warm, scale out your application pods -> they will have a higher priority and will be scheduled on the running nodes. the pause pods will be evicted because they have a -1 priority.
In effect, this will save you minutes of time waiting for new nodes to come up during scaling events.
I haven't actually tested this though... I wonder if the pause pods that are evicted will get scheduled on new nodes. Will the autoscaler create new nodes to satisfy these Pending pause pods? If so, you can just delete the pause pods deployment.