Scale orchestration - Kubernetes data plane
A workload can need more capacity while the cluster still has spare CPU. It can also ask for another replica and leave that Pod waiting for a Node. Both look like a scaling problem, but they require different decisions.
I separate those decisions into three questions: how much capacity each replica needs, how many replicas demand requires, and where those replicas can run.
Size, count, and room
A larger request, more replicas, and more Node capacity change different parts of the system. In the model below, one Node has room for two 500m CPU requests—or one 1 CPU request. Adding another Node creates room without changing the replicas themselves.
Size
One larger request
Count
Two smaller replicas
Room
Another place to fit
Requests are scheduling inputs, not a measurement of current demand. An unnecessarily large request can make a healthy workload hard to place. An undersized request can leave its declared needs out of step with how it actually runs. Rightsizing starts with representative observations, including startup and peak behavior, rather than a single quiet interval.
Match replicas to demand
A HorizontalPodAutoscaler changes a workload's desired replica count using observed metrics and a target. CPU utilization is measured relative to resource requests; changing a request therefore changes what the same CPU usage means to that controller.
This is why I treat request sizing and replica scaling as coupled decisions. A recommendation to reduce requests should be reviewed alongside the HPA policy, rather than applied as an isolated cost change.
2 replicas × 90% utilization ÷ 60% target → 3 desired
2 ready
Demand rises
1 pending
The third replica needs room
3 ready
After placement and startup
The arithmetic above is deliberately simplified. Missing metrics, readiness, tolerance, stabilization, and scaling policies affect the real controller. More replicas also help only when the application can share work effectively; a downstream bottleneck can remain unchanged.
Give the scheduler somewhere to put them
If a new Pod stays Pending, inspect its scheduling events before changing a scaling target. Lack of capacity is one possible cause. Placement rules, resource shapes, and other scheduling constraints can prevent placement too.
Node autoscaling can provision capacity for unschedulable Pods within its configured limits and constraints. It does not make every Pod schedulable, and provisioning a Node is only part of the delay before a workload becomes useful.
Track the entire path: demand detected → desired replicas changed → Pods scheduled → application ready. A higher desired count is not yet delivered capacity.
Right-size without fighting the other controls
Vertical Pod Autoscaler is a separately installed component that can recommend resource requests and, depending on configuration, apply changes. Applying recommendations needs an explicit rollout and disruption policy.
When vertical and horizontal scaling act on the same resource signal, changes to requests can alter the HPA's utilization calculation. Start with recommendations, understand that interaction, and choose which controller owns which decision before enabling automatic changes.
Keep a distinction between a resource request, a resource limit, and actual usage. They answer different questions about scheduling, enforcement, and observation.
A practice to keep
Begin with the symptom: latency, queue growth, resource pressure, or unschedulable Pods. Identify the constrained layer, choose a signal that represents it, and test the whole response path.
Before changing a policy, record the current requests, replica bounds, placement constraints, and startup time. Afterward, compare useful capacity and the original symptom, not just the number of Nodes or replicas. The numerical walkthrough here is illustrative; it is not a benchmark or a report of production results.
Further reading
These controller boundaries follow the Kubernetes documentation. Check the behavior supported by your cluster version and autoscaler configuration: