Data plane
The node agent, the tier runtimes, the session gateway and the host layout that executes agent actions.
The data plane runs the agent's work. It is designed so that a running session survives a complete control-plane outage: the node already has everything it needs — the compiled policy, the template, the secrets, the metering schedule.
Host layout
Hosts are bare metal, grouped into placement classes by tier. Tiers are not merely configured differently; they run on different hardware pools with different kernels, oversubscription ratios and network topologies.
| Placement class | Tiers | Oversubscription | Kernel | Notes |
|---|---|---|---|---|
tool | 0 | n/a | Host kernel | Tool executors run in the control-plane request path |
sandbox | 1 | 4:1 vCPU | Hardened host kernel, gVisor | Highest density, shortest-lived workloads |
micro | 2, 3 | 2:1 vCPU | KVM host, minimal guest kernel | Snapshot-restore fast path |
desktop | 4 | 1:1 vCPU | KVM host, full device model | No co-tenancy on the physical core |
gpu | 2, 4 with GPU | 1:1 | KVM host, vfio or vGPU | Separate pool, scheduled by GPU model |
Mixing tiers on a host is not allowed. A tier 1 workload cannot land on a desktop host, and a tier 4 VM never shares a core with a tier 2 microVM. This removes cross-tier contention and keeps each tier's latency SLO independent of the others' load.
wbnode — the node agent
One privileged daemon per host, and the only thing on the host that talks to the control plane.
Responsibilities
| Responsibility | Detail |
|---|---|
| Capacity reporting | Allocatable vCPU, memory, disk, GPU; pool depth; health score; template cache contents. Reported every 5 s. |
| Runtime lifecycle | Start, health-probe, supervise, stop and destroy every sandbox, microVM and VM on the host. |
| Warm pool keeping | Hold pre-started runtimes at the target depth pool-mgr sets; refill asynchronously after every claim. |
| Storage | Carve per-session scratch from NVMe, attach networked volumes, stream snapshots to and from object storage. |
| Networking | Create the tap device and namespace, join the session to the egress path, enforce per-session bandwidth ceilings. |
| Secrets | Fetch scoped, short-TTL credentials and inject them into the guest over vsock at start. Never written to disk. |
| Metering | Sample every runtime every 10 s, write to a local write-ahead log, stream to metering, retry for up to 24 h. |
| Enforcement | Apply the compiled policy artifact: cgroup limits, seccomp, CLI allowlist, egress ruleset. |
Why a write-ahead log for metering
A node must never lose usage, and must never bill twice. The collector appends each sample to a
local WAL before acknowledging it internally, streams from the WAL to metering, and truncates only
on acknowledgement. Events carry (session_id, meter, window_start), so a replay after a crash is
deduplicated at ingest rather than double-charged.
A node unable to reach metering keeps running sessions and keeps buffering. Past 24 hours of
buffer it raises a critical alert and begins refusing new placements, because beyond that point the
platform is doing unbilled work.
Control-stream behaviour when the control plane is unreachable
connected normal operation: pull desired state, report actual, stream usage
degraded (>30 s no control plane)
running sessions continue
idle timeouts and max lifetimes still enforced locally
warm pool held at last known target, not resized
new placements impossible (nothing is asking for them)
usage buffers to the WAL
isolated (>24 h)
refuse to accept work even if the control plane returns
until an operator clears the node, to avoid acting on stale state
Local enforcement of idle timeouts and maximum lifetimes matters: a control-plane outage must not turn into an unbounded bill.
Tier runtimes
Tier 0 — tool executor
Runs in the control-plane request path, not on a data-plane host. Built-in handlers execute in-process under a per-call timeout and a memory ceiling; remote MCP servers are reached through a pooled client with a circuit breaker. See tier 0.
Tier 1 — gVisor sandbox
A gVisor sentry per session, with a read-only template rootfs and a per-session writable overlay. The sentry runs unprivileged under a seccomp filter restricting the host syscall surface to roughly sixty calls. Pre-started sentries sit in the pool with an empty overlay; a claim binds the session's overlay and starts the interpreter. See tier 1.
Tiers 2 and 3 — microVM
A Firecracker-class VMM per session under a jailer: unprivileged user, seccomp filter, own network namespace, own chroot, cgroup limits. Devices are virtio-blk, virtio-net, virtio-vsock and a serial console, and nothing else.
The fast path is snapshot resume: the pool holds VMs already booted to a ready wbagent,
paused at a memory snapshot. A claim resumes the snapshot, swaps in the session's scratch device,
joins the egress namespace and unpauses — tens of milliseconds of VMM work.
Tier 3 adds a browser supervisor inside the guest and a recorder, and is always placed behind the egress proxy. See tier 2 and tier 3.
Tier 4 — full VM
QEMU with KVM, a full device model, a virtual framebuffer and optional vGPU. The pool holds VMs booted to a logged-in desktop and snapshotted; a claim resumes and attaches the display encoder. Instances are destroyed after use and the pool slot refilled from a clean snapshot — tier 4 instances are never reused across sessions. See tier 4.
session-gateway
The regional, client-facing multiplexer. Everything a client streams to or from a session goes through it.
| Protocol | Carries |
|---|---|
| SSE over HTTPS | exec output: stdout, stderr, exit frames |
| WebSocket | PTY sessions, CDP for tier 3, control channel for tier 4 input |
| WebRTC | Tier 4 desktop video and audio |
| HTTPS | Filesystem reads and writes, uploads, downloads |
Properties:
- Stateless and horizontally scaled. Session-to-host routing comes from the hot session index in Redis, so any gateway can serve any session in its region.
- Authenticated per connection. Connections carry either an API key or a short-lived session token. A token is scoped to one session and expires in minutes, which is what makes it safe to hand to an end user's browser.
- It holds the stream across provisioning. A client may open an exec stream against a session
still in
provisioning. The gateway buffers until the node reportsrunning, so callers never have to poll. - Backpressure is explicit. A slow consumer is paused, not dropped. Output beyond the buffer
ceiling is truncated with an explicit
output_truncatedframe rather than silently lost.
egress-proxy
Every outbound byte from every tier 1 to 4 session passes through it. There is no other route out of a guest.
Enforcement uses the compiled policy artifact the control plane produced at admission, cached by content hash. The proxy does not call the control plane per request.
Detail: networking and egress.
Storage
| Kind | Backing | Lifetime | Billing |
|---|---|---|---|
| Scratch | Local NVMe, per-session logical volume | Destroyed at session end | disk_gib_ms |
| Volume | Networked block storage, replicated | Independent of sessions | volume_gib_month |
| Snapshot | Object storage, compressed, encrypted | Until deleted | snapshot_gib_month |
| Template cache | Local NVMe, shared read-only | LRU-evicted | Not billed |
| Recording | Object storage with lifecycle policy | Retention period | recording_gib_month |
Scratch is local for latency; volumes are networked so they can outlive a host. A session with a volume attached is placed in the volume's zone, which is a hard placement constraint and a common cause of a cold start when that zone's pool is thin.
Detail: storage, volumes, snapshots.
edge-router
Serves *.run.workspaces.build, mapping p{port}-{session}.run.workspaces.build to an exposed port
on a running session. It terminates TLS with a wildcard certificate, resolves the session from the
hostname, enforces the port's visibility setting, and proxies.
A public exposed port serves whatever the agent is running, on a platform subdomain. That is
phishing-adjacent by construction, so public requires an explicit organization-level opt-in, is
recorded in the audit log, and is served under a domain with its own reputation boundary.
Node lifecycle
provisioning image installed, wbnode enrolled, attested, template cache pre-pulled ready accepting placements, holding warm pool at target cordoned no new placements, existing sessions continue draining cordoned + sessions asked to finish; preemptible ones evicted after notice maintenance empty, patched or rebooted retired removed from inventory
Drains are the routine path for kernel patching. A drain stops new placements, lets non-preemptible
sessions run to their natural end within the drain deadline, and evicts preemptible sessions with a
webhook and a prorated bill. Sessions that exceed the deadline are terminated with
host_maintenance as the reason, which is surfaced to the customer and excluded from availability
SLO calculations only if the drain was announced.