State machines

Valid states and transitions for sessions, runs, snapshots, volumes, templates, invoices and webhook deliveries.

Every stateful object has an explicit machine. Transitions not listed here do not happen; an attempt to cause one is a 409 invalid_state_transition.

Session

Dashed outlines are terminal states. stopping also reaches failed when teardown itself errors; the reconciler releases the resources either way.

State	Billed	Meaning
`pending`	No	Admitted and queued; normally under 10 ms
`provisioning`	No	Claiming a pool instance or cold starting. Not billed
`running`	Yes	Executing. `tierN.session_s` accrues every second
`paused`	Storage only	Memory and disk retained, no CPU
`snapshotting`	Yes	Still running; the snapshot is taken live
`stopping`	Yes, until the final meter	Draining, optional final snapshot, teardown
`stopped`	No	Terminal, clean
`failed`	No	Terminal, unclean
`expired`	No	Terminal. Maximum lifetime reached

From	To	Trigger	Billing effect
—	`pending`	`POST /sessions` admitted	Concurrency slot reserved
`pending`	`provisioning`	Placed	None
`pending`	`failed`	No capacity, or admission reversed	Slot released
`provisioning`	`running`	Node health probe passed	Billing starts
`provisioning`	`failed`	Start error or provisioning deadline	None — starts are free
`running`	`paused`	`POST :pause`	Compute stops, storage continues
`paused`	`running`	`POST :resume`	Compute resumes
`paused`	`stopping`	Pause maximum of 1 h reached, or `DELETE`	—
`running`	`snapshotting`	`POST :snapshot`	Unchanged — the session is still running
`snapshotting`	`running`	Snapshot captured	—
`snapshotting`	`failed`	Capture error	Session is lost; the snapshot is not created
`running`	`stopping`	`DELETE`, idle timeout, eviction, policy kill, spend cap	—
`running`	`failed`	Host failure, OOM, guest panic	Billing stops at the last usage event
`running`	`expired`	`max_lifetime_s` reached	Billing stops at expiry
`stopping`	`stopped`	Teardown complete	Final flush, slot released
`stopping`	`failed`	Teardown error	Slot released by reconciler

Terminal: stopped, failed, expired. Terminal sessions are immutable and retained for 90 days.

state_reason is always populated on entry to a terminal state:

client_delete · idle_timeout · max_lifetime · host_failure · host_maintenance
oom · guest_panic · policy_violation · spend_cap · quota_exceeded
provision_timeout · capacity_unavailable · operator_terminated

Run

From	To	Trigger
—	`queued`	Created; waiting for the session to be `running` or for a free execution slot
`queued`	`running`	Execution started
`queued`	`cancelled`	`DELETE` before start, or the session ended
`running`	`succeeded`	Exit code 0
`running`	`failed`	Non-zero exit, or the runtime errored
`running`	`timed_out`	`timeout_ms` reached; the process is killed
`running`	`cancelled`	`DELETE /runs/{id}`

A run cannot outlive its session. When a session leaves running, every non-terminal run on it goes to cancelled with reason: session_ended.

Snapshot

Only ready snapshots restore. Billing for snapshot_gib_month starts at ready.

From	To	Trigger
—	`creating`	`POST /sessions/{id}/snapshot`; the guest is frozen
`creating`	`uploading`	Guest thawed, the session is running again
`uploading`	`ready`	Blocks durable in object storage
`uploading`	`error`	Upload failed after retries
`ready`	`deleting`	`DELETE`, or `expires_at` reached

Deleting a snapshot that is the parent of another in a dedupe chain does not free its blocks; it marks the snapshot deleted and leaves the shared blocks until the last child goes. Billing moves to the surviving child.

Volume

The dashed edge stands for every state: a provisioning or storage failure moves the volume to error from wherever it was.

From	To	Trigger
—	`creating`	`POST /volumes`
`creating`	`available`	Provisioned and formatted
`available`	`attached`	A session starts with `volume_id`
`attached`	`available`	The session reaches a terminal state
`available`	`deleting`	`DELETE`
`attached`	`deleting`	Refused — detach first

volume_gib_month bills from available onward, regardless of attachment.

Template

From	To	Trigger
—	`building`	`POST /templates`; build and scan start
`building`	`ready`	Built, scanned, signed, pre-booted snapshots produced
`building`	`failed`	Build error, or the scan found a blocking vulnerability
`ready`	`deprecated`	Superseded, or explicitly deprecated

A deprecated template still runs for sessions that pin its digest, but is excluded from warm pools and from friendly-name resolution. This is what makes deprecation safe: existing pinned workloads keep working, new ones move forward.

Invoice

A finalized invoice is immutable. Corrections are credit notes against it, never edits.

From	To	Trigger
—	`draft`	Period opens; usage accrues continuously
`draft`	`finalized`	Period closes; a consistent snapshot of rated usage is taken
`finalized`	`paid`	Payment captured, or credits covered it
`finalized`	`past_due`	Payment failed or the due date passed
`past_due`	`paid`	Payment recovered
`finalized`	`void`	Voided by billing operations, with an audit entry

Immutability is what keeps a past invoice explainable from the rated usage that produced it.

Webhook delivery

Retries use exponential backoff with jitter at roughly 10 s, 30 s, 2 m, 10 m, 1 h, 6 h — eight attempts over about 24 hours. After the final failure the delivery is failed and the endpoint's error rate counts toward its circuit breaker. An endpoint failing more than 50% of deliveries over an hour is disabled, and the organization's owners are notified.

Failed deliveries remain replayable from the events API for 30 days.

Node (internal)

A patched host returns from maintenance to ready; that edge is omitted so the main path stays readable.

State	Accepts placements	Running sessions
`provisioning`	No	None
`ready`	Yes	Normal
`cordoned`	No	Continue
`draining`	No	Preemptible evicted after notice; others run to the drain deadline
`unhealthy`	No	Sessions failed with `host_failure`
`maintenance`	No	None
`retired`	No	None

Transition guarantees

Transitions are atomic and recorded. Every change writes a row with the timestamp, reason and actor, which is what the session trace renders.

Terminal states are terminal. No object leaves a terminal state. A "restart" is a new object with a reference to the old one.

Reconcilers are idempotent. Applying the same desired state twice is a no-op, so a duplicated message or a restarted process cannot corrupt the machine.

Billing transitions are never inferred from a missing event. Billing stops on an explicit terminal transition or on the metering pipeline observing no further events past a deadline, with the gap reconciled against the node's write-ahead log. A lost message never silently extends a bill.