Skip to content

Spawn warm pool

URL: https://mkdocs.justinsforge.com/memory/general/reference_spawn_pool/

Cold-booting a worker costs 30-60s on a loaded Console, almost all of it claude + MCP-subprocess init. The /remote-control handshake itself is only ~1s. So the pool pre-warms N spares up to the ❯ prompt only, with Remote Control deferred (no bridge), and activates it at claim time. /new (claim) returns a live URL in ~1s, then refills in the background.

Why no bridge until claimed: a spare that ran /remote-control registers a live bridge and shows up in the claude.ai/code session list. Pre-warming without it keeps idle spares invisible there, so the list only shows sessions you're actually using. Idle spares burn no tokens, only a little RAM.

Components

Piece Path
Pool manager scripts/forge_spawn_pool.sh
/new skill .claude/skills/new/SKILL.md (calls forge_spawn_pool.sh claim)
Self-heal timer ~/.config/systemd/user/forge-spawn-pool.{service,timer} (OnBootSec 2min, every 15min)
State data/spawn_pool/{available,claimed}/, size (default 2), .refill.lock, .claim.lock

Spares are booted via the shared scripts/forge_spawn_session.sh in SPAWN_NO_REMOTE=1 mode (boot to ❯, skip remote-control/prompt/URL). A claimed spare is identical to a normal worker, just pre-warmed.

Commands

forge_spawn_pool.sh claim        # hand out one spare's url instantly, refill in bg (this is what /new runs)
forge_spawn_pool.sh refill [N]   # top up to N (default from size file); prune dead, recycle >90min old
forge_spawn_pool.sh status       # list available + claimed with liveness + age
forge_spawn_pool.sh drain        # kill all spares, clear state
echo 3 > data/spawn_pool/size    # change pool depth

How it works

  • claim (flock on .claim.lock): pick first alive warm spare, reserve it (mv available/ -> claimed/), run /remote-control on its pane, resolve the bridge id, append url= to the marker, print it, fork a background refill. Falls back to a synchronous cold spawn if no usable spare, so /new never breaks. Bridge resolution (rewired 2026-06-11): grep the spare's own debug log /tmp/spawn-rc-<session>.log for v1/code/sessions/(cse|session)_...; the log is per-process so a concurrent refill can't be confused for the claimed session. The old ~/.claude/sessions/*.json snapshot-diff is dead: claude 2.1.173 stopped writing those files on rc activation.
  • refill is single-instance (flock -n on .refill.lock): prune dead spares, recycle any older than POOL_MAX_AGE (6h; claude/MCP freshness, no bridge to expire), boot up to target. Separate lock from claim.
  • liveness = tmux session exists AND pane_current_command is claude/node. Do NOT grep the pane text: when an idle claude exits, its TUI (including ❯) stays in the scrollback while the pane drops to a bash prompt, so a text match false-positives a dead spare.

Gotchas (don't reintroduce)

  • fd leak into the tmux daemon. The flock fds (7,8) MUST be closed when launching the spawn ("$SPAWN" ... 7>&- 8>&-). The tmux server the spawn starts is long-lived and otherwise inherits the open lock fd and holds the lock for its entire life, so every future refill sees the lock held and silently skips. This is why the original build warmed once then never refilled. See LESSONS.md 2026-06-06.
  • Banner text drift. Claude Code changed the Remote Control banner from remote-control is active to Remote Control active to status-bar /rc active; the matcher must tolerate all three. A stale regex (missing /rc active) silently degraded every claim to a cold spawn until 2026-06-11.
  • Spares must boot with --debug-file (forge_spawn_session.sh does this for all boots including SPAWN_NO_REMOTE=1): claim-time bridge resolution reads that log. A spare booted without it always fails activation and burns itself.
  • Don't pre-activate Remote Control on spares. It registers a bridge and clutters the claude.ai/code session list with idle phantom sessions. Activation is deferred to claim (SPAWN_NO_REMOTE=1 at boot); it only costs ~1s since claude/MCP are already warm.
  • Idle claude self-exits (~2h). A bridgeless warm claude TUI sitting with no input eventually exits back to a shell. Handled by: correct liveness detection (claim skips dead spares and picks a live one; refill prunes them) plus POOL_MAX_AGE=60min proactive recycle and the 15min refill timer. Don't raise POOL_MAX_AGE near or past ~2h or claims will start hitting dead spares and cold-fall-back.
  • Liveness via pane_current_command, never pane text (see liveness above). This caused a claim to send /remote-control to a dead spare's bash prompt and fail; it fell back to a cold spawn (resilience held) but the spare was wasted.
  • Spares are full-featured (all MCP/plugins). A future optimization is a lean boot profile (strip chrome/gmail) for faster refills, not yet built; pool depth 2 already hides refill latency.

[Claude Code]

Multi-model pools (2026-06-12)

The pool is no longer opus-only. Per-model targets live in data/spawn_pool/sizes, one model=count per line (current: opus=2 fable=2 sonnet=2; the legacy size file is ignored). claim [model] (default opus, so /new is unchanged) hands out a spare of that model and cold-spawns that model if none are warm; refill tops up every model in the sizes file. A spare's model is identified by its session-name label suffix (_Opus48, _Fable5, _Sonnet46), which must keep mirroring forge_spawn_session.sh. Changing counts = edit the sizes file, done. RAM cost is ~450MB per spare; 6 spares leaves Console (~12GB) at roughly 2.4GB available, so do not grow the pool casually. Consumed by /new (opus claim) and the session-launcher API at chat.justinsforge.com (per-model claim). [session-launcher_Fable5]