Subshell Docs

Node protocol

Understand how an enrolled node authenticates, receives work, and reports session output to its server.

Scope and transport

The node daemon opens an outbound WebSocket connection to the server's /ws/node endpoint. The server uses that connection for commands; the node uses it for results, machine reports, terminal output, and process-exit events. Browsers connect to the server, so they don't need direct access to a node's ports.

This protocol is an internal contract between the server and node releases. It is separate from the browser API and MCP tool interface. Use REST API or MCP tool reference for external integrations.

No public node listener is required for this connection. The node's separate loopback management dashboard isn't the server-to-node transport; see Node CLI and Security model.

Enrollment and identity

Enrollment redeems a single-use setup key for a persistent node ID and node credential, along with the server's public signing identity. The node stores its connection and identity state in its local configuration. Files and paths identifies that file and the associated key storage.

Three identities serve different purposes:

IdentityPurpose
Node credentialAuthenticate the WebSocket upgrade and prove the same credential inside the encrypted link. It isn't a REST API credential.
Server signing public keyVerify that commands were signed by the trusted control plane.
Link encryption public keysPin the endpoints used for application-layer encryption. These are distinct from the command-signing identity.

Changing the server address preserves the node's identity and trust state. It is suitable for another address of the same server. Joining a different server requires enrollment there; changing a URL doesn't make its signing keys interchangeable. Repointing also sends the node credential to the newly named host, so use a trusted address. See Repoint a node.

Connection setup

For a paired node, connection setup proceeds as follows:

  1. The daemon connects to /ws/node with its node bearer credential. The server checks the credential, node record, and owner's enabled state before admitting it.
  2. The node sends a kx frame with a fresh ephemeral public key. It derives connection keys using the server encryption key it already trusts.
  3. The server derives its side of the connection keys. It doesn't send a plaintext key-exchange reply.
  4. The node sends an encrypted binding containing its node ID, node credential, and protocol version. The server re-verifies that the credential belongs to the node admitted at upgrade.
  5. The server sends an encrypted ok acknowledgment.
  6. The node sends its encrypted ready report. The server applies release-version and protocol gates before marking it online.

Connection keys use libsodium crypto_kx; established traffic uses crypto_secretstream_xchacha20poly1305. Commands and events are carried in encrypted binary WebSocket frames. A broken stream or unexpected plaintext frame ends the encrypted connection rather than causing a fallback to plaintext.

The encrypted handshake has a 10-second deadline. An open socket with an incomplete handshake isn't eligible for ordinary commands.

Pairing a node without an encryption pin

A node record without a pinned encryption identity uses a registration flow. The node sends register with its public encryption key. The server records the pin, returns register-ok with its public encryption key, and closes that socket normally. The node saves the returned identity and reconnects through the encrypted handshake above.

Registration is a pairing operation, not normal execution over plaintext. The node credential and enrollment address are its trust root. Protect enrollment with trusted network access and HTTPS where appropriate; encryption after pairing doesn't independently authenticate the first server address.

Readiness and machine reports

The ready event identifies the node's release version, protocol version, operating system, architecture, hostname, data directory, and capabilities. It can also report its home directory, invocation path, runtime and service details, maintenance state, and system specs (OS release, CPU model, logical CPU count, total memory, and report time). System specs are optional so older nodes can still connect; the server retains the last report for offline discovery.

These reports let the server plan launches for the execution machine rather than guessing its paths from the server host. They can disclose machine names, local paths, and selected environment information to the trusted control plane.

The server requests binary detection using detect with manifest-derived detection rules and named environment variables. The node returns raw detection results; the server's plugins interpret them. Nodes don't download or load a separate harness plugin set. Cached detection information while offline doesn't prove a binary is still available or that the node can launch work.

Signed commands and results

The server wraps each command in a compact JWS signed with ES256. The claims include:

ClaimCheck
issExpected control-plane issuer, subshell-control.
audThe specific target node ID.
iat and expFreshness and a 30-second command lifetime.
jtiSingle-use command identifier for replay checks and result correlation.
seqConnection-local ordering. It isn't a substitute for replay detection.
cmdStructurally validated command body.

The signed envelope is then encrypted for the node connection. Signing authorizes the command and binds its target; encryption protects the bytes in transit. Neither prevents a trusted or compromised server from requesting real program execution.

The daemon decrypts the frame, verifies the signature and claims, validates the command body, and dispatches it. Verification is ordered per connection. Replay tracking survives reconnects; connection-local sequence tracking resets for a new connection.

A success event has type: "result", ref, ok: true, and optional data. A refusal has ok: false and an error string. Node events inherit the authenticated encrypted connection's identity; they aren't separately JWS-signed commands.

The server normally waits up to 10 seconds for a command result. A node update gets five minutes. harness_update and harness_install use the installer budget: a ten-minute deadline on the node, with the server waiting thirty seconds longer. archive_create and archive_extract use a five-minute deadline; tree_manifest gets a minute and each transfer_write chunk thirty seconds. The command's signature expiry governs acceptance, not the maximum duration of an already-accepted operation. A timeout doesn't prove that the node performed no work; inspect state before retrying an operation that could create or change something.

Most commands execute in arrival order on a shared queue, so a long archive operation can delay captures, resizes, and other management commands. Terminal input and harness runs (harness_update, harness_install) have their own ordered queues and don't wait behind those operations; harness runs still queue among themselves, because two package managers can race over one filesystem. Transfer relay windows are capped at 512 KiB of raw bytes per frame, and one archive at 4 GiB uncompressed.

Command families

PurposeCommand types
Session processeslaunch, terminate, kill, probe, prompt_deliver.
Terminal interaction and snapshotsinput, resize, capture, pane_size, pane_cursor.
Transcript reads and live subscriptionslog_read, tail_start, tail_stop.
Files and directoriesstat_dir, fs_ls, path_exists, write_file, remove_paths, set_allowed_dirs.
Archive transferarchive_create, file_read, transfer_write, archive_extract, tree_manifest.
Machine informationinventory, detect, ping.
Node managementservice, agent_log_read, set_server_url, set_log_level, set_maintenance, set_ssh_enabled, update.
Harness CLIsharness_update, harness_install (run one harness CLI's own update or install line; a non-zero exit is a reported result).
SSH discovery and trustssh_discover_aliases, ssh_resolve_config, ssh_register_identity, ssh_agent_identities, ssh_host_key, ssh_machine_pin_repair.
SSH relay and remote setupssh_relay_open, ssh_relay_close, ssh_exec_stdin (setup scripts on stdin), legacy ssh_exec, ssh_exec_status.

A command existing on the wire doesn't grant every user access to it. Server routes enforce ownership, grants, and administrator requirements before dispatch. The daemon also applies local checks, including directory rules, maintenance refusals, and service-manager safety checks.

Launch data

A launch command carries the session ID, tmux socket and session names, absolute working directory, harness ID, preset configuration, pane credentials, initial terminal dimensions, and server-built argv plus a binary-resolution rule. Optional fields include MCP registration data and a harness conversation identity for resumption. The answer echoes any virtual display the launch allocated. When the server asks with probeVersion (agent harnesses only), it also echoes the raw <binary> --version text the node read for the binary it spawned, so the server can record the version each subshell launched on.

The node resolves the agent binary on its own machine and launches it under its OS account. The server owns plugin interpretation; the node executes the prepared launch. Directory restrictions limit the starting location, not the agent's later filesystem access. See Directory restrictions.

Terminal output and process liveness

Browser input passes through the server's access checks and becomes node input commands. Size changes become resize commands. The server requests captures, pane dimensions, cursor position, and log data when reconstructing a terminal view.

Live output uses tail_start subscriptions. The node sends output events identifying the session, subscription ID, byte range, and base64-encoded bytes. Base64 is a byte representation, not encryption; the node link encrypts the event in transit. The server can read terminal output and relays it to authorized browser viewers.

The diagram separates input and output paths; it doesn't require a new subscription for each keystroke. tail_stop ends a subscription when it is no longer needed.

A node emits exit when a process dies and subshells_report to reconcile session liveness. It also sends heartbeat events every 15 seconds by default. A heartbeat establishes daemon activity, not proof that every agent process is alive.

Node disconnection makes remote sessions unreachable through the server. It doesn't itself prove their processes stopped: tmux processes can remain on the execution machine. Reconnection permits the server to reconcile liveness and restore output subscriptions. See Session lifecycle.

Compatibility and held connections

The server applies two distinct gates to readiness:

  1. The node release must meet the server's minimum supported release version.
  2. Its protocol version must match the server's protocol exactly.

The implemented protocol is 29. Its number is independent of the package release version. Additive wire changes also require a protocol bump and coordinated rollout; there is no per-feature compatibility window.

A refused node can remain connected in a held state for updates while staying unavailable for launches and ordinary management commands. Only update can use that held connection. Older nodes on the legacy path receive that update command in its frozen plaintext recovery format; they aren't admitted for normal plaintext operation. Invalid credentials and disabled owners are authentication failures, not update holds.

An update still requires the node to verify its signed release manifest and artifact digest. After updating and reconnecting, it must pass the same readiness gates. See Version compatibility and Node updates.

Limits and connection failures

Raw node frames are capped at 1 MiB before decoding. An oversized frame closes the connection with WebSocket code 1009. Structural parsers reject malformed command and event shapes. These checks bound individual frames; they aren't general denial-of-service protection.

Common close signals include:

CodeMeaning and response
1000Normal close, including completed encryption registration. The daemon normally reconnects unless stopping.
4401Node authentication is no longer valid, such as after key revocation. Review the node credential.
4403The node owner's account is disabled. Re-enable the account only after resolving the reason for disabling it.
4406Update-required compatibility signal. Review the server's reported release or protocol requirement.
4409Another connection superseded this daemon. The daemon exits rather than competing in a reconnect loop; stop duplicate instances.
4410Encrypted handshake or stream failure. Reconnect; this isn't permission to discard a trusted key pin.
4411The server requires re-pairing because the row has no matching encryption pin. The daemon's pre-establishment repair flow can re-register its existing identity.

The daemon uses full-jitter exponential reconnect backoff with a one-second initial ceiling and a 60-second maximum ceiling. A successful socket open resets the backoff ladder. Authentication or pairing failures require diagnosis rather than repeated launch attempts.

Key rotation replaces the node credential and clears the server-side encryption pin. Store the new node credential on the execution machine with subshell configure --key; a running daemon reloads it and reconnects. Automatic re-pairing doesn't supply the replacement credential. A lost node-side encryption identity against an already-pinned server record may require explicit credential rotation and repair. See Repoint a node and Node troubleshooting.

Preview transport

Web previews and virtual displays use the signed-command and encrypted-frame paths. These features don't open an inbound public port on the node. The server's built-in node runs the same preview primitives in process rather than sending node commands.

MessagePurpose
set_preview_enabledApply server-owned preview permissions, the default displayProvider, the allowed displayProviders set, and the change timestamp. on enables the transport; optional webPreviewsOn and displayOn distinguish web-preview access from virtual displays. When omitted, each inherits on for compatibility.
preview_display_checkCheck an optional provider or the node default for support and unmet requirements before launch; this never allocates a display.
preview_display_closeEnd a subshell's allocated virtual display without touching the pane; an unallocated pane answers closed: false.
preview_ports_getReport listeners under a subshell's process tree, its CDP port, and display metadata.
preview_openStart a loopback TCP forward or CDP screencast for a subshell, port, reference, and bounded lifetime. Forward lanes may include service: { host, protocol } for a saved HTTP(S) destination.
preview_dataCarry capped base64 bytes toward the node (in) or server (out).
preview_streamCarry sequenced JPEG browser frames and dimensions.
preview_close / preview_closedRequest or report teardown with a named reason.

Display metadata contains an opaque display identifier and a viewer connection, currently { protocol: "vnc", port }. Provider launch credentials stay on the node. An unknown provider is unsupported; it never silently selects another. Provider IDs match ^[a-z][a-z0-9-]{0,63}$; the allowed set contains at most 32 unique IDs. An empty set permits no allocations even when the feature is on.

The launch command accepts screen: { width, height, provider? }, with integer dimensions from 320 through 4,096 pixels per axis. The server resolves an omitted provider from the node default and persists it before launch. Restarts reuse that saved choice and recheck current permission. Both the server and enrolled node refuse a disabled provider before allocation. Readiness can succeed while the feature or provider is disabled; it reports prerequisites, not permission.

A forward command targets loopback unless it carries a validated saved-service destination. Frame references are bound to the authenticated node on receipt. The server's HTTP/WS proxy uses separate per-connection lanes beneath a browser capability URL, and browser watch frames are delivered only to authorized live-feed subscriptions. CDP discovery refuses HTTP redirects, caps discovery JSON at 256 KiB, and requires the returned WebSocket to stay on the requested loopback port.

Preview transport applies these limits:

ResourceLimit
Raw payload256 KiB per frame
Lane buffer, queued target writes, and queued preview traffic on the encrypted link1 MiB each
Active forward lanes / waiting lanes per subshell4 / 8
Active lanes and browser bridges per node16 total
Browser watch1–5 fps, 5 buffered frames, at most 8 subscribers
Screencast image8,192 pixels per axis and 16,777,216 pixels total
Retained forwarding capabilities per node64
Pending preview opens per node32

The browser checks encoded image headers before decoding them. Capability and pending-open limits apply independently of active lane capacity.

Forward sessions expire after 15 minutes without traffic. Individual lanes and bridges have a four-hour maximum lifetime. Close reasons include viewer-left, pane-gone, node-dropped, slow-link, over-cap, idle, revoked, not-cdp, and target-gone. Turning off either node feature setting prevents new operations for that feature without immediately closing existing connections. If an open reply times out or is malformed, the server attempts to close the allocated reference; it doesn't assume the node failed to create the resource.

See Preview architecture and Preview security for the browser isolation, access, capability-link, and local-machine boundaries.

Security boundaries and implementation reference

The server and enrolled execution account are trusted endpoints. A server compromise can authorize commands across the fleet; a compromised node account can read its processes, accessible credentials, and transcripts. Command signing and encrypted transport don't provide an agent sandbox. Browser transport, enrollment, and storage need their own protection.

Read Security model for the complete boundary and Communication security for MCP channel encryption, which is separate from this link.

For exact field definitions and validation, consult the public protocol source:

Edit on GitHub

Last updated on

On this page