Skip to content

Brain latency rewire brief, 2026-06-09 (item 5 of the 9-item batch)

Worker brief, final item of the approved batch. Goal: bot brain latency p90 from ~70s to under ~20s.

The problem

Every Telegram brain call shells out a fresh cold claude -p (audited 2026-06-09: coordinator n=162 med 10.7s p90 68.8s; capture n=49 med 8.8s p90 70.8s). The CLI cold start plus the full prompt (system + 65-tool catalog, recently pruned from 75 in commit a8be560) is most of the tail.

Hard constraints

  1. Billing: there is NO Anthropic API account; everything must bill the Claude subscription (feedback_subscription_over_api_for_automation.md). The Claude Agent SDK rides CLI auth and bills the subscription; verify this explicitly during the probe (use the claude-api skill or claude-code-guide agent, do not assume).
  2. Kill switch: env-var fallback to the current claude -p path (FORGE_BRAIN_ENGINE=subprocess|server, default the OLD path until soak passes). The nine robustness layers (reference_telegram_bot_robustness_layers.md) must survive unchanged.
  3. Concurrency model: brains run inside per-message mp.Process children with per-chat flocks (reference_coordinator_concurrent_dispatch.md). A persistent engine therefore cannot live inside the bot process; it must be a separate daemon the children call over localhost (unix socket or 127.0.0.1 HTTP). Naming: forge-brain-server.service, script scripts/forge_brain_server.py.
  4. No threads around subprocess.run (feedback_no_threads_with_subprocess_run.md).
  5. Quota tracking: keep forge_quota_tracker.record() calls; if the SDK exposes real token counts, record those instead of estimates and note the change.
  6. Console CPU lacks AVX2; only ~/.local/bin/claude works (reference_console_claude_binary_avx2_constraint.md). If the SDK bundles its own binary, verify it doesn't SIGILL.

Process

  1. /feature-plan to memory/plans/brain-latency-rewire-2026-06-09.md. Include a PROBE phase with measured numbers: cold claude -p vs warm SDK round-trip on the real brain prompt, before committing to the design.
  2. If the probe shows the SDK cannot hit <20s p90 or cannot bill subscription, STOP and write findings to this handoff instead of building; notify via scripts/forge_notify.sh info.
  3. Staged rollout: capture persona first (lowest traffic), soak, then coordinator + general-purpose. Default env stays subprocess until Justin flips it or you observe one clean day; document the flip command in the handoff.
  4. Verify with real latency numbers from the quota JSONL after a few live exchanges, plus /bots-health.

File ownership

YOURS: scripts/forge_telegram_inbox_brain.py (the claude invocation path only; do NOT touch the tool layer, it shipped today), scripts/forge_telegram_brain.py (same), new forge_brain_server.py + unit. NOT YOURS: poller scripts, forge_telegram_format.py, callbacks, forge_gmail.py, forge_email_triage.py, forge_finance_query.py, forge_monitor.py, forge_alert_fixer.py.

Today's prior commits to respect: 0b3e1b5 (Gmail direct port), f9aee5d (tg consolidation + heartbeats), 8e47bf1..dd031d1 (triage buttons, morning actions, finance), a8be560 (catalog prune + fixer loop). Pull before you start.

Update this handoff with outcomes; register the server in MEMORY.md + a topic file if it ships.

[Claude Code, forge-audit session]

Outcomes (2026-06-09, brain-latency session)

Shipped. Probe, plan, server, dispatch, and stage-1 flip all landed same day. Plan with full probe data: brain-latency-rewire-2026-06-09. Topic file: reference_brain_server.

Probe results (measured, real 39k-char capture prompt): cold claude -p 4.9-6.4s; warm SDK with max_turns=1 flat 3.5-4.6s. Billing verified empirically: zero API keys exist on Console (env + ~/.forge-secrets/ checked) and SDK calls succeed, so they ride the CLI's subscription OAuth. AVX2 trap was real: SDK 0.2.95 bundles its own CLI and prefers it over PATH; server pins cli_path=~/.local/bin/claude.

Root cause of the p90 tail (new finding): not cold start. Quota JSONL (n=247): 53 calls SUCCEEDED at median 37s producing ~734 chars, same prompt size as fast calls. The old path ran claude -p with default CLI tool access and no turn cap, so the model sometimes did agentic CLI-tool work before emitting JSON. The server's max_turns=1 removes the mechanism; cold-boot skip is the secondary win.

State: forge-brain-server.service active + enabled on Console (127.0.0.1:8771). Capture poller flipped via drop-in (FORGE_BRAIN_ENGINE=server:capture); coordinator + general remain on subprocess default. Any server failure auto-falls-back to subprocess (tested against a dead port). Server-engine quota rows now carry real token counts (engine, num_turns, tokens{...}).

Verification: server selftest PASS (1.7-3.2s round-trips); live one-shots through both brains via both engines PASS; /bots-health green after the flip (all 5 active, watchdog ok, canary pass).

Flip command after one clean capture day (also in the topic file):

sudo sed -i 's/server:capture/server/' /etc/systemd/system/forge-inbox-capture.service.d/brain-engine.conf
sudo mkdir -p /etc/systemd/system/forge-lifeos-coordinator.service.d
sudo cp /etc/systemd/system/forge-inbox-capture.service.d/brain-engine.conf /etc/systemd/system/forge-lifeos-coordinator.service.d/
sudo systemctl daemon-reload && sudo systemctl restart forge-inbox-capture forge-lifeos-coordinator

Revert/kill switch: delete the drop-in(s), daemon-reload, restart pollers; or just sudo systemctl stop forge-brain-server (bots degrade to subprocess automatically).

[Claude Code, brain-latency session]