project creatortrack scripts app build 20260715T182554
name: CreatorTrack Scripts app: RAG script generation in creator's voice description: New CreatorTrack app being built (Tasks 1-19, feat/social-data-layer-and-scripts): generates video scripts in a creator's own voice via pgvector embeddings + retrieval + few-shot generation. type: project
CreatorTrack is building a "Scripts" app: given a topic, it generates a video script that reads in a specific creator's own voice, using a RAG pipeline over that creator's past video transcripts.
Stack/architecture decisions made during the 2026-07-15 build (as of Task 15, commit b8bb87c on feat/social-data-layer-and-scripts):
- Voice profile (Task 14): weighted-quantile stats over sentence length / pacing, stored per workspace in app.voice_profiles.
- Embeddings: @huggingface/transformers + BAAI/bge-small-en-v1.5, 384-dim pgvector column on app.social_videos, no ANN index needed at current scale (~500 rows/workspace).
- Retrieval: two-stage CTE, cosine-similarity pool then retention-rerank (NULLS LAST, verified live to actually reorder results).
- Generation: lib/scripts/generate.ts generateScript(), strict-JSON contract with one retry, RLS-scoped by workspace.
- Test workspace is ws 85 ("Gus the Bass", a pet-bass-in-a-tank channel) — test topics far from the creator's real content (e.g. "heavy vegetation fishing") produce stylistically-correct but content-invented scripts; retrieval correctly finds nearest exemplars but can't invent footage that doesn't exist. This motivated adding a similarity score to retrieval output so the UI (Task 16) can warn when a requested topic is off-corpus.
- Remaining tasks at last check: 16 (Scripts UI), 17 (chat tool), 18 (blind eval gate), 19 (docs), then two whole-branch reviews. A parallel "Meta lane" (feat/social-meta-ingest, phases M1-M5, worktree warm-slot-2) covering social video ingest/cross-post UI was build-complete and review-clean as of the same session.
Why: tracks an in-progress multi-day CreatorTrack feature build so future sessions don't need the task history re-explained.
How to apply: check feat/social-data-layer-and-scripts and comms/results/creatortrack-social-data-layer-and-scripts-app-task-*.md for current status before assuming this is unstarted or done; look for a rebase/ship of both lanes once Tasks 16-19 close.
[auto-memory session 60d132de-4659-485c-8044-427af0ccce7f, confidence 0.75, mode staged]