Troubleshooting

Each entry below is one refusal the software really prints: the symptom as you see it, what is actually going on, and what to run or click. They are ordered by how often people hit them, not by which part of the system they belong to. Every quoted string is transcribed from the code that emits it, so if your screen shows something else, you have found something new, and we want to hear about it.

Start here: subshell-server status

Before reading symptoms, run the point-in-time report on the control-plane host:

subshell-server status

It prints what the server would boot with, without booting anything: the version line, where config.env lives, each setting with the layer its value came from (process environment, file, or default), tmux = NOT FOUND: install tmux (apt install tmux / brew install tmux) when tmux is missing, the resolved MCP entrypoint, the node artifacts = 2/3 published line, whether an admin account exists yet, whether the configured port is listening, and whether a service definition is installed at all. The service status verb goes one layer deeper: it is what prints teardown keeps panes = NO: this definition predates the fix; reinstall it before stopping or restarting. The auth secret appears masked, never in full. status --json gives the same facts for scripts. What each line means is in Logs & Debugging.

403 "Invalid origin" on sign-in

Symptom. The sign-in page (or the first-run wizard) refuses with a 403, and the error names itself: Invalid origin. Passwords are not wrong; the request never gets that far.

Cause. Subshell only accepts browser requests from addresses it trusts, and the allowlist is derived: from the instance's own addresses (the loopback spellings of its port, the configured APP_BASE_URL, a concrete bind host), the operator's TRUSTED_ORIGINS, and whatever a network plugin published. It is never read from the request's own Host header. That is the DNS-rebinding hole the list exists to close. On the default configuration the derived set is just the two loopback spellings, so opening the UI through a LAN IP or a VPN name sends an origin nothing matches.

Fix. Either browse from an address already on the list, or add the address you actually use: Settings → Service has the trusted-origins field, and from a terminal it is subshell-server configure --trusted-origins <origin,origin> (see Server CLI). Each entry is a full origin (scheme and host, no path), and wildcards are refused. The flag rewrites config.env and the CLI itself says the new value applies at the next start: restart, or use the dashboard's field, which applies live. subshell-server status prints the stored line with per-entry diagnostics for spellings a browser will not match; the list a browser is actually checked against is derived live and also includes the instance's own addresses. The long version, including why APP_BASE_URL matters separately, is Networking & Addresses.

Passkeys fail everywhere except one address

Symptom. Password sign-in works from every address you added, but passkeys only ever work on one, usually localhost or the exact host in APP_BASE_URL. Elsewhere the browser itself rejects the ceremony before the server sees it.

Cause. A passkey is bound at creation to a relying-party ID, and Subshell's is the host of the configured APP_BASE_URL, never the request's host. That is what makes the credential non-phishable, and it is also why the same key refuses to authenticate under a second spelling of the same server.

Fix. Use passkeys on the canonical address and put other devices on it too (add the name to TRUSTED_ORIGINS, then browse there). Be deliberate about changing APP_BASE_URL itself: it moves the relying-party ID, and existing passkeys stop working on the old address, including the Subshell Server desktop app's own window, which opens on loopback. Adding extra origins to the allowlist does not have that effect and is the safe lever for "also reachable at".

A launch refuses: offline, unreachable, or in maintenance

Symptom. The launch form prints one of three lines, and each sends you somewhere different:

• That node is offline. Start its subshell or pick another node.: the machine's subshell daemon is not connected right now.
• The node did not answer. Try again shortly.: the plane sent the command and got nothing back.
• That node is in maintenance. Pick another node, or end maintenance from its node page.: the machine is healthy and answering everything except launches.

Cause and fix. An offline node is not a crashed subshell: check the node, not the pane. Open the node's page: the status line says whether it is online, and Re-check asks it right now. On the machine itself, subshell status answers ONLINE from the local daemon lock or OFFLINE (no local subshell running with the commands that fix it. If a human threw maintenance on, it is the node page's switch (Maintenance); if it is simply down, Managing a Node has the start/restart/service path.

The node never comes online (usually a baked loopback URL)

Symptom. Enrollment finishes, the row exists, and it stays offline forever. Or the install printed a warning you scrolled past:

subshell: WARNING: SERVER is a loopback address; a remote node
    must dial this machine's VPN/LAN address instead (Nodes page).

Cause. The address the node dials is baked into the script at render time and used forever, and a command built from a loopback base URL sends a remote machine dialing itself, connected to nothing. The Add-node dialog's address dropdown is the guard against this: its rows come from the addresses the instance is known at, and the row you pick is the address the node gets. But a loopback-only instance (no LAN derivation, no joined network) has only that one row to offer.

Fix. Pick an address the new machine can actually reach, usually your VPN or tailnet address, in the dialog's Address the node dials dropdown before copying; the choice rides to the server as the command's server= argument and only addresses the instance already trusts are accepted. To fix it for every future install, set APP_BASE_URL itself. Either way, mint a fresh key and rerun: the old key is single-use. To diagnose an already-enrolled node, subshell status --probe on the machine dials the stored address and names it, but read its own warning first: the probe opens a live node socket and the control plane keeps the newest connection, so it kicks any daemon running elsewhere for that node. Only probe when no other copy is running. And if you started the daemon by hand with subshell run, it died with your SSH session. That verb is foreground on purpose; install the background service instead (Adding a Node walks both).

The install one-liner refuses

curl -fsSL …/install.sh?setup_key=… | bash can stop for several distinct reasons, and each says which:

When the instance is configured not to fetch releases, the Add-node dialog warns up front (This server has no node binary for: the missing triples , and it is configured not to download one.), and the honest remedies are the ones it names: publish binaries on the server (copy the subshell-node-cli-<triple> assets from a cli-node-v* GitHub Release into its node-artifacts dir), or install the node another way and run subshell setup directly. Adding a Node has the full path.

"program not found" for an agent you installed

Symptom. The harness card or launch picker says program not found even though you can run claude in your own terminal.

Cause. Detection answers per machine, and it answers for the machine you picked. The most common version of this is not a bug: the CLI is installed on the control-plane host but you are looking at a node's card, or the other way round. The plane's in-page installer only ever installs on the control-plane host; a node gets its CLIs over SSH or from your images (Installing Agent CLIs).

Fix. Press Re-check on that machine's card after installing; rows update on page load, and Re-check asks now. If it still refuses on the right machine, the binary sits where detection cannot see it: the lookup tries an explicit CLAUDE_PATH-style override, then the service's PATH, then well-known install locations, then version-manager layouts (nvm, fnm, volta, asdf, mise, n) by glob, then the login shell's PATH as a last resort. A pinned-but-wrong override is reported distinctly and is a one-edit fix; moving the binary onto a standard PATH, or setting that override, ends it.

It works in your terminal but not as a service

Symptom. tmux or an agent CLI is found in your shell, yet the background subshell daemon reports "not found" and launches fail.

Cause. Service managers start units with a stock PATH: on macOS that misses Homebrew's /opt/homebrew/bin, on Linux it misses whatever your profile adds. Subshell bakes the installing shell's PATH into the unit when it installs the service, which is why the daemon can be blind to a binary that appeared (or moved) after that install.

Fix. Install the service again from a shell that finds everything (subshell service install re-bakes the current PATH) and restart it. For the control-plane host itself the same trap exists for tmux, and subshell-server status prints exactly which tmux the server resolves, or the line NOT FOUND: install tmux. The agent-install path in the browser adds the login shell's PATH deliberately, so an install run from the page lands where detection looks (Installing Agent CLIs).

tmux is missing

Symptom. One of these, depending on where you hit it:

• enrollment or subshell setup refuses before your single-use key is spent: tmux not found (on macOS: brew install tmux); install it and retry (macOS adds the hint; the tail always names the variable SUBSHELL_CLIENT_SKIP_TMUX_CHECK=1 that skips the check, an escape hatch for tests, not a deployment plan),
• the install script warns subshell: tmux is not installed; a node needs it to run subshells. with the right command per platform,
• the setup checklist on the control-plane host shows Install tmux: Nothing can launch on this machine: every subshell runs inside a tmux pane, and the server found no tmux.

Cause and fix. Every pane is a tmux session, on every machine that runs one; there is no mode without it. Install it with your package manager (brew install tmux, sudo apt-get install tmux), or press the install button where you are an admin: the first-run wizard and the control-plane host's node row offer it, and the Subshell Server desktop app's assistant does the same with its own elevation prompt. The browser button can only run package managers it can drive without a password prompt (in practice Homebrew on macOS), and it prints the command to run yourself wherever it cannot. Version floor: tmux 3.2 or newer; 3.6 adds the crashed-pane exit code (Supported Platforms).

"Restart the server" refuses, or is greyed out

Symptom. The restart action on Settings → Service says This server is not running under a service manager; restart it where you started it., or a restart that should work instead answers The installed service definition would close every running subshell on restart; reinstall the service definition, or pass force to restart anyway.

Cause. The first refusal is availability, not permission: a restart works by exiting for your service manager to respawn, and a bun run start, a container with no init, or a hand-run process has nothing to respawn it. There is deliberately no way to exit a server into nothing. The second is pane safety: an old service definition kills the process group, which is every live tmux pane with it. subshell-server service status reads which definition you have; teardown keeps panes = NO names the stale one.

Fix. For the first: start the server from wherever it lives, or make it service-managed: subshell-server service install on the host, or let the Subshell Server desktop app supervise it. For the second: reinstall the service definition so teardown keeps panes, or pass force when you really mean it; from the CLI that is subshell-server service restart --force. Stop, start, uninstall and reset stay off the dashboard entirely by design: each leaves the server unreachable, so a page the server serves is the wrong place to ask (Service & Autostart).

The server refuses to boot in production

Symptom. With NODE_ENV=production, startup dies with:

Refusing to boot: BETTER_AUTH_SECRET is unset in production. The built-in
placeholder secret is public (it ships in the repo), so session cookies
signed with it can be forged by anyone. Set BETTER_AUTH_SECRET to a unique
value of at least 32 characters.

Cause. Session cookies are signed with BETTER_AUTH_SECRET. Outside production the built-in placeholder is convenient; under NODE_ENV=production signing with a key that ships in the repository would let anyone forge a session, so the boot refuses instead.

Fix. Set a real BETTER_AUTH_SECRET in config.env (the file subshell-server status names) or the environment, and set APP_BASE_URL while you are there: production enforces the origin check strictly, which is where the 403 above starts to bite. configure deliberately does not write this key, so edit the file directly.

A subshell says crashed after a restart

Symptom. After the server (or the machine) came back, a subshell reads as crashed, or a push said Crashed, auto-restarting and then Crashed.

Cause. For panes, tmux is the source of truth and the database row is a statement of intent; a reconcile pass re-reads it every minute. So offline is not crashed: if a node was unreachable, the row waits rather than writing a verdict. A crashed badge means the pane process genuinely died: harness exit, reboot that took the tmux server with it, an OOM kill. Auto-restart is opt-in, not default: only subshells whose preset enables "restart on unexpected exit" are revived, with exponential backoff, and Crashed (the final form) is what you see after the backoff ladder gives up.

Fix. Open the subshell's log: the transcript holds the pane's last words and the harness usually said why. Restart relaunches the same row and resumes the conversation when its transcript survived (Subshells). If the whole machine rebooted and panes are gone, they are gone on that machine. Relaunch.

The desktop app reports no server binary

Symptom. Subshell Server's assistant says this build ships no server binary.

Fix. The bundle you installed is missing the server it wraps; reinstall the app from the project's releases page. An official bundle carries the binary, so a refusal means the install itself is incomplete, not that your machine is unsupported. If a reinstall reproduces it, that is a bug worth reporting, with the OS and the app version from About.

Terminal attach glitches

Symptom. The live terminal renders garbage, stale scrollback, or a mis-sized grid on attach, sometimes.

Cause. A pane's grid is shared: every viewer's columns and rows fight for it, sized to the smallest visible viewer, and a repaint landing mid-attach is the usual suspect. Who else is attached is visible in the viewers line; Devices & Viewers explains what that line means.

Fix for a reproducible bug. On the control-plane host, set SUBSHELL_ATTACH_DEBUG=1 in the server's environment and restart it. Each attach then dumps what the pane held and exactly what the client was sent, under /tmp/subshell-attach-debug/. It is off by default because those dumps are real terminal output (potential secrets on screen), so attach them deliberately and remove the variable when done. Attach the dumps to your report and say which pane, which viewers, and which tmux version.

See also

• Networking & Addresses: the origin allowlist, the three addresses, and the DNS-rebinding hole
• Adding a Node: enrollment end to end, including its own "if it does not come online" list
• Logs & Debugging: status, the bounded server log, and debug mode
• Security Model: why several refusals above are by design
• Support: what to include when the fix above is "report it"