Skip to content

Forge Data Layer, the spine, 2026-06-09

The keystone build. A single forge-owned database that unifies customer + commerce + financial data across all brands, continuously synced from the source systems, queryable by AI and the coordinator bot. Everything else (email engine, segmentation, dashboards, content pipeline) becomes easy because this exists. Embodies feedback_ai_native_tool_selection: open engines + AI glue + forge as connective tissue.

Why this is the north star

SaaS silos data per brand, per tool: Klaviyo only knows its store; each Shopify only knows itself; ERPNext only knows the books. Nobody sells you a unified view across brands, because that's the moat they're selling back to you piecemeal. The data layer is the thing you own that makes every AI-native tool smart, and it's the one capability with compounding leverage: build it once, every brand and every downstream tool draws from it.

Strategic rule: build horizontal capabilities once, apply to all brands. The data layer is the most horizontal thing there is.

What it unlocks (downstream, once the spine exists)

  • Email engine: segment + personalize from real orders/engagement ("everyone who bought a kayak but not a paddle, in the last 90 days").
  • Unified customer / LTV: the same human buying across multiple stores becomes one record, real lifetime value, cross-sell.
  • Analytics / dashboards: revenue, repeat rate, cohorts, per brand and combined.
  • Coordinator bot Q&A: "how's Gus Outdoor doing this month?" answered from actual data.
  • Content + ops: feeds the content pipeline, inventory alerts, anything data-driven.

Architecture

Four layers (the forge OS model): - Engines / sources: Shopify Admin API (per store), the ESP (SES/Listmonk/Omnisend engagement), ERPNext (financials), later Notion / social analytics. - Connectors: one idempotent sync script per source, upserting into a canonical schema. - The DB (the spine): source-agnostic canonical entities. - Access: a query lib + coordinator bot tools + a dashboard. Email engine and dashboards are consumers, not owners.

Storage

Start with SQLite at data/datalayer/datalayer.db (matches the finance-app pattern, fast to ship, easy to reason about). Upgrade path to Postgres if concurrency/scale demands it (multiple writers, big order volume). One source of truth (FORGE-DOCTRINE 9.5): this DB is the canonical store for commerce data; everything else imports.

Canonical schema (source-agnostic)

  • brands — JWVR, Nova, Gus Outdoor, Gus the Bass, Sip N Serve (id, name, shopify_domain, esp, erpnext_company).
  • customers — unified person: email (key), name, first_seen, last_seen, brands[], total_spend, order_count, ltv, marketing_consent.
  • identities — links one human across brands/emails (email + fuzzy name/address) so cross-brand LTV works.
  • products — per brand: shopify_product_id, title, sku, price, image_url, url, inventory.
  • orders — shopify_order_id, brand, customer_email, total, currency, placed_at, status, fulfillment.
  • order_lines — order_id, product_id, qty, price.
  • email_events — customer_email, brand, campaign_id, type(send/open/click/unsub/bounce), at.
  • campaigns — id, brand, subject, sent_at, recipients, sender (send@ etc.).
  • financials — brand, period, revenue, expenses, source=erpnext (summary rollups).

All upserts keyed on the source id (idempotent: same sync twice = same state).

Connectors (flat, verbose, idempotent)

  • forge_datalayer_db.py — schema init + upsert helpers (like forge_finance_db).
  • forge_datalayer_shopify_sync.py — per-store Admin API pull: products, orders, customers → canonical tables. Reads token from ~/.forge-secrets/shopify-<brand>.env.
  • forge_datalayer_esp_sync.py — email engagement from the ESP (SES events / Listmonk / Omnisend API).
  • forge_datalayer_erpnext_sync.py — financial summaries from ERPNext (CT 104).
  • forge_datalayer_identity.py — resolve customers across brands (email-first, fuzzy fallback).
  • Each on a systemd timer (daily/hourly), fail-loud notify on sync failure.

Access / interface

  • forge_datalayer.py — query lib for other services (the email engine calls this for segments).
  • Coordinator bot tools: tool_brand_snapshot, tool_customer_lookup, tool_segment ("bought X not Y"), tool_email_engagement.
  • Dashboard at datalayer.justinsforge.com (or a tab on an existing one) behind CF Access: per-brand + combined revenue, customers, top products, campaign performance.

Privacy / security

Customer PII lives in a forge DB on Console. Guard it: not internet-exposed, any dashboard behind Cloudflare Access, tokens in ~/.forge-secrets/. Honor unsubscribes/deletion requests (sync the marketing_consent + suppression state from the ESP). Treat as sensitive.

Phased build

  1. Spine + Shopify (Nova first): forge_datalayer_db.py schema + forge_datalayer_shopify_sync.py pulling Nova's products/orders/customers. Prove the pattern end-to-end. (Needs the Nova Shopify Admin API token.)
  2. Multi-store + identity resolution: add Gus Outdoor + others; forge_datalayer_identity.py unifies customers across brands.
  3. ESP engagement sync: email events into the layer.
  4. ERPNext financial sync: revenue/expense rollups.
  5. Interface: coordinator bot tools + datalayer.justinsforge.com dashboard.
  6. Email engine consumes it: the Klaviyo replacement reads segments/products from here (separate build, this is its data source).

Starting point

Nova Design is the sandbox: real Shopify store, low stakes. Mint the Nova Admin API token → run Phase 1 → see products/orders/customers land in the forge DB and a sample segmented product email generate from real data. Then roll the proven pattern to Gus Outdoor (the live $70 Klaviyo replacement) and the rest.

Relation to other plans

  • Replaces the per-brand silos behind Klaviyo/Mailchimp (consolidation-roadmap Workstream B).
  • The email engine is a consumer of this layer (Klaviyo replacement).
  • finances.justinsforge.com is the personal analog; this is the business/commerce analog. They could share patterns (SQLite + sync timers + dashboard + bot tools).

[Claude Code, 2026-06-09]