Skip to content

Plan: Recall v2 three-layer retrieval

URL: https://mkdocs.justinsforge.com/memory/plans/recall-v2-three-layer-retrieval-2026-05-04/

Date: 2026-05-04 Approved spec: Port the 3-layer retrieval pattern into /recall. Default returns a compact index ([id] path date | summary score), --get <ids> returns full chunk bodies, --timeline <id> --window N returns same-file neighbors by chunk_idx. --full keeps v1 behavior, --json and -k preserved. Schema gains chunk_id (4-hex sha1, collision-extend) and summary (first 80 chars to word boundary) plus a meta table for schema versioning. v1→v2 upgrade triggers one full re-embed. Rollout behind FORGE_RECALL_V2=1 for a dogfood week with token-count bench logging to data/recall-bench/. Active build ~3.3h. Out of scope: - Cross-file timeline (same-file only this round) - Sonnet-generated summaries (cheap first-line only) - Cross-encoder reranking - Replacing BGE-small embeddings

Task list

Task 1: Add schema versioning + chunk_id + summary columns to indexer

  • Files: /home/justinwieb/forge/scripts/forge_search_index.py
  • What: Add a meta(key TEXT PRIMARY KEY, value TEXT) table. On startup, read schema_version; if missing or <2, drop chunks and chunk_vecs and recreate. Recreate chunks with new columns: chunk_id TEXT UNIQUE NOT NULL, summary TEXT NOT NULL. Write schema_version=2 after rebuild. No embedding logic changes yet.
  • Verification: /home/justinwieb/.forge-venvs/search/bin/python -c "import sqlite3; c=sqlite3.connect('/home/justinwieb/forge/data/search/index.db'); print([r[1] for r in c.execute('PRAGMA table_info(chunks)')]); print(c.execute(\"select value from meta where key='schema_version'\").fetchone())" returns columns including chunk_id and summary, version 2.
  • Commit: recall v2: add schema_version=2 with chunk_id + summary columns

Task 2: Generate chunk_id and summary at index time

  • Files: /home/justinwieb/forge/scripts/forge_search_index.py
  • What: When inserting a chunk: chunk_id = sha1(f"{path}:{chunk_idx}").hexdigest()[:4]; on UNIQUE collision, retry with [:5], [:6], up to 16. summary = body[:80] truncated at last whitespace, ellipsis appended if truncated. Store both. Run a full rebuild via --rebuild flag (already exists per reference doc) to populate.
  • Verification: /home/justinwieb/.forge-venvs/search/bin/python /home/justinwieb/forge/scripts/forge_search_index.py --rebuild runs to completion. Then sqlite3 /home/justinwieb/forge/data/search/index.db "select chunk_id, length(summary), substr(summary,1,40) from chunks limit 5" shows 4-char IDs and summaries <=80 chars.
  • Commit: recall v2: populate chunk_id (4-hex sha1) and summary (first 80 chars) at index time

Task 3: Add compact-index default mode to query CLI

  • Files: /home/justinwieb/forge/scripts/forge_search_query.py
  • What: When FORGE_RECALL_V2=1 and no mode flag, output [chunk_id] <path> <date> | <summary> score=0.NN, one per line. Date is from filename if it matches YYYY-MM-DD, else file mtime. Without the env var, behavior is unchanged (v1 full bodies). --json returns the same fields as a list of dicts.
  • Verification: FORGE_RECALL_V2=1 /home/justinwieb/.forge-venvs/search/bin/python /home/justinwieb/forge/scripts/forge_search_query.py "task dispatcher" prints compact lines with 4-char IDs and score= suffix; /home/justinwieb/.forge-venvs/search/bin/python /home/justinwieb/forge/scripts/forge_search_query.py "task dispatcher" (no env) still prints v1 full bodies.
  • Commit: recall v2: compact-index default mode behind FORGE_RECALL_V2

Task 4: Add --get mode for explicit chunk lookup

  • Files: /home/justinwieb/forge/scripts/forge_search_query.py
  • What: --get a3f2,b8c1 parses a comma-separated chunk_id list, looks each up in chunks, prints full body with path + heading header. Missing IDs print [no match for <id>] to stderr and the command exits 1 if any are missing.
  • Verification: Pick a known chunk_id from sqlite3 /home/justinwieb/forge/data/search/index.db "select chunk_id from chunks limit 1", then /home/justinwieb/.forge-venvs/search/bin/python /home/justinwieb/forge/scripts/forge_search_query.py --get <id> prints the full body. --get zzzz exits 1 with stderr message.
  • Commit: recall v2: --get <ids> mode for full-body lookup

Task 5: Add --timeline mode (same-file neighbors)

  • Files: /home/justinwieb/forge/scripts/forge_search_query.py
  • What: --timeline <id> --window N (default N=5) finds the chunk's path + chunk_idx, selects same-path chunks where chunk_idx BETWEEN id_idx - N AND id_idx + N, ordered by chunk_idx, prints each as compact-index line (or full body with --full).
  • Verification: Pick an ID from a multi-chunk file, run /home/justinwieb/.forge-venvs/search/bin/python /home/justinwieb/forge/scripts/forge_search_query.py --timeline <id> --window 2 and confirm 5-or-fewer entries from the same file in chunk_idx order.
  • Commit: recall v2: --timeline mode for same-file neighbors

Task 6: Add --full escape hatch

  • Files: /home/justinwieb/forge/scripts/forge_search_query.py
  • What: --full <query> forces v1 behavior (full chunk bodies) regardless of FORGE_RECALL_V2. Trivial: bypass compact-mode branch.
  • Verification: FORGE_RECALL_V2=1 /home/justinwieb/.forge-venvs/search/bin/python /home/justinwieb/forge/scripts/forge_search_query.py --full "task dispatcher" prints full bodies.
  • Commit: recall v2: --full escape hatch for v1-style output

Task 7: Rewrite the /recall skill with new flag set

  • Files: /home/justinwieb/forge/.claude/skills/recall/SKILL.md
  • What: Document the three modes, the env-var gate during dogfood, and a short decision flowchart: "default first; if a hit looks load-bearing, --get it; if you need decision context, --timeline it." Include examples for each flag.
  • Verification: grep -E "(--get|--timeline|--full|FORGE_RECALL_V2)" /home/justinwieb/forge/.claude/skills/recall/SKILL.md | wc -l returns >=4.
  • Commit: recall v2: rewrite SKILL.md with three-layer decision flow

Task 8: Bench harness for token-count comparison

  • Files: /home/justinwieb/forge/scripts/forge_recall_bench.py, /home/justinwieb/forge/data/recall-bench/.gitkeep
  • What: New script runs ~10 canned queries (e.g. "task dispatcher", "frigate config", "eight sleep credentials") through both v1 and v2 modes, measures output character count as a proxy for token cost (chars/4), writes JSONL to data/recall-bench/<YYYY-MM-DD>.jsonl with fields query, mode, chars, hits. Exit 0.
  • Verification: /home/justinwieb/.forge-venvs/search/bin/python /home/justinwieb/forge/scripts/forge_recall_bench.py runs to completion, then wc -l /home/justinwieb/forge/data/recall-bench/$(date +%F).jsonl shows ~20 lines (10 queries × 2 modes).
  • Commit: recall v2: bench harness logging v1-vs-v2 char counts to data/recall-bench/

Task 9: Refresh reference doc

  • Files: /home/justinwieb/.claude/projects/-home-justinwieb-forge/memory/reference_semantic_search.md
  • What: Fix stale scripts/search/index.py paths to actual scripts/forge_search_index.py and scripts/forge_search_query.py. Add a "## v2 three-layer mode" section with the new flags and the FORGE_RECALL_V2 gate. Update the "Files" list. Bump the body to reflect post-2026-05-04 reality.
  • Verification: grep -c "forge_search_query.py" /home/justinwieb/.claude/projects/-home-justinwieb-forge/memory/reference_semantic_search.md >=1, grep -c "FORGE_RECALL_V2" ... >=1.
  • Commit: recall v2: refresh reference_semantic_search.md with v2 modes + correct file paths

Task 10: Register and document

  • Files: /home/justinwieb/.claude/projects/-home-justinwieb-forge/memory/MEMORY.md, /home/justinwieb/forge/LESSONS.md (only if eval flags something)
  • What: Update the existing Semantic search (/recall) MEMORY.md entry to mention v2 three-layer mode (one extra clause on the existing line). Run bash /home/justinwieb/forge/scripts/forge_eval_run.sh; if any check trips, log to LESSONS.md and fix.
  • Verification: bash /home/justinwieb/forge/scripts/forge_eval_run.sh exits 0.
  • Commit: recall v2: register v2 mode in MEMORY.md index

Task 11: Flip default after dogfood

  • Files: /home/justinwieb/forge/scripts/forge_search_query.py, /home/justinwieb/forge/.claude/skills/recall/SKILL.md
  • What: After ~7 days of FORGE_RECALL_V2=1 use and the bench data confirms savings, remove the env-var gate, make compact-index the default. Keep --full for v1 behavior. Update SKILL.md to drop the env-var note.
  • Verification: /home/justinwieb/.forge-venvs/search/bin/python /home/justinwieb/forge/scripts/forge_search_query.py "task dispatcher" (no env) prints compact lines.
  • Commit: recall v2: promote compact-index to default, retire FORGE_RECALL_V2 gate