Capacity planning

Sizing the fleet per tier, pool economics, lead times and the signals that mean buy hardware.

Two questions, continuously: is there enough capacity to serve demand, and is pooled capacity sized so start-latency SLOs hold without the pool becoming the dominant cost.

Capacity model per placement class

required_hosts = ceil(
    (peak_concurrent_sessions × avg_resources_per_session)
    / (host_capacity × oversubscription × target_utilisation)
  )
  + pool_hosts
  + failure_domain_reserve

Term	Value
`target_utilisation`	0.80 — headroom for burst and placement choice
`failure_domain_reserve`	One zone's worth, so a zone loss does not refuse creates
`pool_hosts`	Capacity held by warm pools, computed below

Class	Oversubscription (vCPU)	Target utilisation	Notes
`sandbox` (T1)	4:1	0.85	Bursty and often idle; the densest class
`micro` (T2, T3)	2:1	0.80	Sustained CPU, longer sessions
`desktop` (T4)	1:1	0.70	Interactive and side-channel sensitive; most headroom
`gpu`	1:1	0.75	GPU memory cannot be oversubscribed usefully

Memory is barely oversubscribed anywhere, because memory pressure becomes an OOM kill, which to a customer is an unexplained failure. CPU oversubscription shows up as slower execution, which is recoverable and visible in cpu_ms.

Pool economics

pool cost   = depth × hourly cost of one idle instance
miss cost   = p(miss) × creates × (cold − warm latency) × value of latency

Pool depth is held at the point where marginal pool cost equals the marginal value of avoided cold starts. In practice that means a different posture per tier:

Tier	Idle instance cost	Posture
1	Low (dense, small)	Deep pools; a miss is cheap to avoid
2	Moderate	Deep for common templates; thin for rare ones
3	Moderate–high	Moderate; cold is 9 s so misses hurt
4	Equal to a running session	Thin platform baseline; depth beyond it is sold as a reservation

Tier 4 is the honest case: a pooled desktop costs essentially a running desktop. A platform-funded deep tier 4 pool would mean every customer paying for a latency guarantee most do not need. So the baseline is small and pool_reservation_s exists for customers who want more.

Target computation

Recomputed every 15 seconds per (tier, template@digest, size_class, zone):

target = clamp(
    ceil(burn_rate × refill_latency × safety) + baseline,
    floor, ceiling)

Input	Source
`burn_rate`	Claims/second, 5-minute EWMA plus a seasonal term from the same slot last week and yesterday
`refill_latency`	p90 seconds to bring one instance to ready, per pool
`safety`	1.5 for tiers 1–2, 2.0 for tiers 3–4
`baseline`	Minimum depth so the first request of a quiet period is warm
`floor`	Contractual reserved depth
`ceiling`	Budget guard per tier per zone

The seasonal term matters more than it looks. Agent traffic is strongly diurnal and weekly; a purely reactive EWMA always under-provisions the Monday-morning ramp.

Template proliferation

Every distinct template digest is a separate pool needing its own depth.

40 private templates × 5 size classes × 3 zones = 600 pools

Depth is allocated by demand, so rarely used templates get zero and always cold start. The remedies, in order of preference: consolidate templates, pin the few that are latency-critical, or restrict size classes for private templates. The infra console's per-template cold-rate column is what makes this visible.

Demand forecasting

Horizon	Method	Used for
Minutes	EWMA plus seasonal	Pool targets
Days	Weekly seasonality on 8 weeks	Drain scheduling, staging capacity
Weeks	Per-organization trend plus onboarding pipeline	Rack orders
Quarters	Sales pipeline plus committed-use agreements	Region expansion

The quarterly horizon is where committed-use contracts earn their keep: a signed commitment is a capacity signal months before the traffic arrives.

Lead times

Action	Lead time
Add pool depth in an existing class	Seconds
Bring a cordoned host back	Minutes
Provision a racked, un-enrolled host	Hours
Rack and enrol new hardware in an existing region	2–6 weeks
New zone in an existing region	2–4 months
New region	4–9 months

The 2–6 week line is the one that drives alerting thresholds. By the time headroom is tight, it is already too late to order; the trigger has to fire while headroom still looks comfortable.

Buy-hardware signals

Signal	Threshold	Action
`allocatable_ratio{class,zone}`	< 0.25 sustained 7 d	Start a procurement cycle
`allocatable_ratio`	< 0.15	Escalate; consider temporary pool reduction
`placement_failures{reason=no_capacity}`	> 0.1% of creates	Capacity is already being refused
`pool_hit_ratio` falling with pools at target	Any sustained decline	Demand is outgrowing the fleet
`refill_latency_p90` rising	> 2× baseline	Storage or network pressure, not a capacity shortage
Committed-use signed	Any	Add capacity before the ramp

The fifth row is the important distinction. Depth below target with high refill latency is not a demand problem — adding target depth makes it worse. Diagnose refill latency first.

Zone balance

Capacity is kept even across zones so a zone loss does not refuse creates. Two things fight this:

Volumes are zonal. A customer with volumes concentrated in one zone drags placement there. The mitigation is that volume creation without an explicit zone picks the zone with the most pool headroom.
Pool depth follows demand, which follows volumes. The ceiling term in the target computation stops one zone's pools from consuming a disproportionate share of a region's capacity.

Cost structure

Roughly, for the fleet:

Component	Share	Lever
Host hardware amortisation	55%	Oversubscription ratios, utilisation target
Power and cooling	15%	Placement density, workload shaping
Network transit	12%	Peering, egress pricing
Storage	10%	Snapshot deduplication, lifecycle rules
Pooled idle capacity	8%	Pool targets, reservation pricing

Pooled idle capacity is the smallest line and the most visible one internally, because it is the cost of a promise rather than of work done. Resist the instinct to cut it first: the pool is what the start-latency SLO is made of, and the SLO is what the product is.

Scenarios

A customer announces a 10× ramp

Identify their tier mix and template set from the last 30 days.
Check headroom in their region's relevant classes against projected peak.
Pin their templates to warm pools in their region; pre-pull layers to candidate hosts.
Raise their quotas to the new ceiling ahead of the ramp.
If headroom after the ramp falls below 25%, start procurement now — 2–6 weeks is longer than most ramps.

Tier 4 demand doubles

Tier 4 cannot absorb a surge by packing tighter — it runs 1:1. The options are: add desktop-class hosts (weeks), convert a reservation-holding customer to on-demand with notice, or refuse creates with capacity_unavailable. The third is the honest short-term answer, and it is why tier 4 quotas are low by default.

A zone fails

Sessions in the zone fail with host_failure. New placements go to the surviving zones, which absorb the load because of the failure-domain reserve. Pools rebalance within 15 seconds; cold-start rate rises until they refill. If the failure lasts, re-target pools permanently to the surviving zones rather than holding capacity for a zone that is not coming back soon.