@jhizzard/termdeck 0.2.4 → 0.3.0

package/README.md

@@ -127,11 +127,19 @@ Restart Claude Code. Six MCP tools appear: `memory_remember`, `memory_recall`, `
 
 ### Tier 3 — Add Rumen for async learning
 
-Rumen is a separate npm package — `@jhizzard/rumen@0.2.0` — that ships as a Supabase Edge Function designed to run on a 15-minute `pg_cron` schedule. It's the async reflection layer over Mnestra: it reads recent session memories, cross-references them with your entire historical corpus via hybrid search, synthesizes insights via Claude Haiku, and writes the results back into `rumen_insights` (a new table alongside Mnestra's `memory_items`). TermDeck's Flashback and Claude Code's `memory_recall` both automatically benefit because insights flow back into the same database.
+Rumen is a separate npm package — `@jhizzard/rumen@0.3.6` — that ships as a Supabase Edge Function designed to run on a 15-minute `pg_cron` schedule. It's the async reflection layer over Mnestra: it reads recent session memories, cross-references them with your entire historical corpus via hybrid search, synthesizes insights via Claude Haiku, and writes the results back into `rumen_insights` (a new table alongside Mnestra's `memory_items`). TermDeck's Flashback and Claude Code's `memory_recall` both automatically benefit because insights flow back into the same database.
 
-Rumen v0.2 is the current shipped version. Setup is still manual — Sprint 3 (next) ships a `termdeck init --rumen` one-liner that automates the deploy. For now, follow [github.com/jhizzard/rumen#readme](https://github.com/jhizzard/rumen) to deploy via the Supabase CLI.
+**Rumen is live.** First full-kickstart run against a production Mnestra store on 2026-04-15 19:47 UTC: **111 sessions processed, 111 insights generated** in one pass. Insights surfaced patterns like "the error detection regex in Flashback misses `No such file or directory` — same class of blind spot as X" and "Practice sessions exist as a separate model but frontend components were built and never wired into the schedule view." The cognitive loop is closed.
 
-**Why you'd want Rumen:** without it, Flashback only surfaces memories that structurally match the current context (same project, similar error). With Rumen, Flashback can surface **cross-project patterns** that Haiku synthesized while you were away — "the CORS fix you applied in Project A probably solves this error in Project B." That's the moat.
+Deploy with **one command**:
+
+```bash
+termdeck init --rumen
+```
+
+The wizard auto-resolves the latest published `@jhizzard/rumen` version from npm at deploy time, applies the self-healing SQL migration, stages the Edge Function with the correct version pin, deploys via the Supabase CLI, sets function secrets, applies the `pg_cron` schedule, and POSTs a manual test invocation. Prerequisites: Supabase CLI, Deno, and a `DATABASE_URL` pointing at a Mnestra-compatible Postgres. Full walkthrough including the five setup gotchas (the hidden IPv4 toggle in the Supabase Connect modal, the literal-string password bug, `DATABASE_URL` only / no `DIRECT_URL`, the macOS-13 + Homebrew + Deno incompatibility, and schema drift handling) is at **[rumen/install.md](https://github.com/jhizzard/rumen/blob/main/install.md)**.
+
+**Why you'd want Rumen:** without it, Flashback only surfaces memories that structurally match the current context (same project, similar error). With Rumen, Flashback surfaces **cross-project patterns** that Haiku synthesized while you were away — "the CORS fix you applied in Project A probably solves this error in Project B." That's the moat.
 
 ---
 
@@ -143,7 +151,7 @@ Honest limits, stated upfront so the skeptic has nothing to chase:
 - **Not a replacement for reading docs.** It's the shortest path to a memory you already wrote. If the memory isn't there, the feature does nothing.
 - **Not fully local by default.** Tier 2+ reaches out to Supabase for storage and OpenAI for embeddings. Tier 1 is fully local. A fully-local Tier 2 (local Postgres + local embeddings) is on the Sprint 3 roadmap.
 - **Not free forever.** Tier 2+ pays OpenAI fractions of a cent per memory for embeddings. Self-hosted embeddings via Ollama are on the roadmap.
-- **Not proven at scale.** v0.2, validated against 3,400+ memories in one developer's production store. No multi-user data yet. Bug reports and issues welcome.
+- **Not proven at scale.** v0.2.5, validated against 3,527 memories in one developer's production store. First full Rumen kickstart on 2026-04-15 processed 111 sessions into 111 insights in one pass. No multi-user data yet. Bug reports and issues welcome.
 
 ---
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@jhizzard/termdeck",
-  "version": "0.2.4",
+  "version": "0.3.0",
   "description": "Browser-based terminal multiplexer with metadata overlays, panel flashback memory recall, and AI-aware session management",
   "bin": {
     "termdeck": "./packages/cli/src/index.js"
@@ -24,6 +24,7 @@
     "dev": "node packages/server/src/index.js",
     "server": "node packages/server/src/index.js",
     "start": "NODE_ENV=production node packages/cli/src/index.js",
+    "test": "node --test packages/server/tests/**/*.test.js",
     "install:app": "bash install.sh"
   },
   "dependencies": {

package/packages/cli/src/init-rumen.js

@@ -9,6 +9,9 @@
 //   - `deno` on PATH
 //   - `~/.termdeck/secrets.env` with SUPABASE_URL + SUPABASE_SERVICE_ROLE_KEY +
 //     DATABASE_URL + ANTHROPIC_API_KEY populated (run `termdeck init --mnestra` first)
+//   - OPENAI_API_KEY in secrets.env is OPTIONAL: when present, Rumen's Relate
+//     phase uses semantic+keyword hybrid search via OpenAI text-embedding-3-large.
+//     When absent, Rumen falls back to keyword-only matching.
 //
 // Steps:
 //   1. Preflight: which supabase, which deno, read secrets.env
@@ -16,7 +19,7 @@
 //   3. supabase link --project-ref <ref>
 //   4. Apply rumen migration 001 via pg
 //   5. supabase functions deploy rumen-tick --no-verify-jwt
-//   6. supabase secrets set DATABASE_URL=... ANTHROPIC_API_KEY=...
+//   6. supabase secrets set DATABASE_URL=... ANTHROPIC_API_KEY=... [OPENAI_API_KEY=...]
 //   7. Test the function with a manual POST (fetch)
 //   8. Apply pg_cron schedule migration (002) with project ref substituted
 
@@ -34,6 +37,44 @@ const {
   pgRunner
 } = require(SETUP_DIR);
 
+// Pinned fallback used only when the npm registry is unreachable. Bump this
+// when you republish @jhizzard/rumen and can't (or won't) rely on `npm view`
+// at deploy time. The wizard prefers the live registry answer — this value
+// exists so a fully offline machine can still ship a working Edge Function.
+const FALLBACK_RUMEN_VERSION = '0.3.4';
+
+// Resolve the current published version of @jhizzard/rumen. Source of truth
+// is the npm registry (because the Edge Function runtime pulls from there,
+// not from local node_modules). On network failure we fall back to
+// FALLBACK_RUMEN_VERSION and tell the user what happened — we don't silently
+// deploy a bogus version.
+function resolveRumenVersion() {
+  const r = spawnSync('npm', ['view', '@jhizzard/rumen', 'version'], {
+    encoding: 'utf-8',
+    stdio: ['ignore', 'pipe', 'pipe'],
+    timeout: 15000
+  });
+  if (r.status === 0) {
+    const version = (r.stdout || '').trim();
+    if (/^\d+\.\d+\.\d+/.test(version)) {
+      return { version, source: 'npm registry' };
+    }
+    return {
+      version: FALLBACK_RUMEN_VERSION,
+      source: 'pinned fallback',
+      warning: `npm view returned unexpected output: ${JSON.stringify(version)}`
+    };
+  }
+  const stderr = (r.stderr || '').trim();
+  return {
+    version: FALLBACK_RUMEN_VERSION,
+    source: 'pinned fallback',
+    warning: stderr
+      ? `npm view failed: ${stderr.split('\n').pop()}`
+      : `npm view failed (exit ${r.status === null ? 'timeout' : r.status}) — offline?`
+  };
+}
+
 const HELP = [
   '',
   'TermDeck Rumen Setup',
@@ -125,6 +166,18 @@ function preflight() {
   }
   ok();
 
+  // OPENAI_API_KEY is optional: when present, Rumen's Relate phase generates
+  // real embeddings for semantic+keyword hybrid search. When absent, Rumen
+  // falls back to keyword-only matching (still works, but loses cross-project
+  // conceptual retrieval).
+  if (!secrets.OPENAI_API_KEY) {
+    process.stderr.write(
+      '\n⚠  OPENAI_API_KEY is not set in ~/.termdeck/secrets.env.\n' +
+      '   Rumen will run in keyword-only mode — for full cross-project conceptual\n' +
+      '   retrieval, add OPENAI_API_KEY to secrets.env and re-run `termdeck init --rumen`.\n\n'
+    );
+  }
+
   return { supabaseBin: sb, denoBin: deno, secrets };
 }
 
@@ -196,7 +249,7 @@ async function applyRumenTables(secrets, dryRun) {
   }
 }
 
-function deployFunction(dryRun) {
+function deployFunction(rumenVersion, dryRun) {
   step('Running: supabase functions deploy rumen-tick --no-verify-jwt...');
   if (dryRun) { ok('(dry-run)'); return true; }
 
@@ -204,7 +257,7 @@ function deployFunction(dryRun) {
   // `supabase/functions/rumen-tick/`. The TermDeck install does NOT include
   // a `supabase/` directory at the project root, so we stage a tiny working
   // directory under `os.tmpdir()` that mirrors what the CLI expects.
-  const stage = stageRumenFunction();
+  const stage = stageRumenFunction(rumenVersion);
   if (!stage) {
     fail('could not stage rumen-tick function source');
     return false;
@@ -222,7 +275,10 @@ function deployFunction(dryRun) {
 //   <stage>/supabase/functions/rumen-tick/{index.ts, tsconfig.json}
 // Also write a minimal `supabase/config.toml` so `supabase functions deploy`
 // doesn't complain about a missing project root.
-function stageRumenFunction() {
+function stageRumenFunction(rumenVersion) {
+  if (!rumenVersion || !/^\d+\.\d+\.\d+/.test(rumenVersion)) {
+    throw new Error(`stageRumenFunction: invalid rumenVersion ${JSON.stringify(rumenVersion)}`);
+  }
   const stage = fs.mkdtempSync(path.join(os.tmpdir(), 'termdeck-rumen-stage-'));
   const functionSrc = migrations.rumenFunctionDir();
   if (!fs.existsSync(functionSrc)) return null;
@@ -230,7 +286,22 @@ function stageRumenFunction() {
   const dest = path.join(stage, 'supabase', 'functions', 'rumen-tick');
   fs.mkdirSync(dest, { recursive: true });
   for (const f of fs.readdirSync(functionSrc)) {
-    fs.copyFileSync(path.join(functionSrc, f), path.join(dest, f));
+    const srcPath = path.join(functionSrc, f);
+    const destPath = path.join(dest, f);
+    // Substitute the version placeholder in the Deno entry point. Other files
+    // in the directory (tsconfig.json, etc.) are copied verbatim.
+    if (f === 'index.ts') {
+      const raw = fs.readFileSync(srcPath, 'utf-8');
+      if (!raw.includes('__RUMEN_VERSION__')) {
+        throw new Error(
+          `rumen-tick/index.ts is missing the __RUMEN_VERSION__ placeholder — ` +
+          `has someone reintroduced a hardcoded version?`
+        );
+      }
+      fs.writeFileSync(destPath, raw.replace(/__RUMEN_VERSION__/g, rumenVersion));
+    } else {
+      fs.copyFileSync(srcPath, destPath);
+    }
   }
 
   const configToml = `# staged by termdeck init --rumen
@@ -245,19 +316,27 @@ verify_jwt = false
 }
 
 function setFunctionSecrets(secrets, dryRun) {
-  step('Setting function secrets (DATABASE_URL, ANTHROPIC_API_KEY)...');
+  const haveOpenAI = Boolean(secrets.OPENAI_API_KEY);
+  const label = haveOpenAI
+    ? 'DATABASE_URL, ANTHROPIC_API_KEY, OPENAI_API_KEY'
+    : 'DATABASE_URL, ANTHROPIC_API_KEY';
+  step(`Setting function secrets (${label})...`);
   if (dryRun) { ok('(dry-run)'); return true; }
-  const r = runShellCaptured('supabase', [
+  const args = [
     'secrets', 'set',
     `DATABASE_URL=${secrets.DATABASE_URL}`,
     `ANTHROPIC_API_KEY=${secrets.ANTHROPIC_API_KEY}`
-  ]);
+  ];
+  if (haveOpenAI) {
+    args.push(`OPENAI_API_KEY=${secrets.OPENAI_API_KEY}`);
+  }
+  const r = runShellCaptured('supabase', args);
   if (!r.ok) {
     fail(`secrets set failed (exit ${r.code})`);
     if (r.stderr) process.stderr.write(r.stderr + '\n');
     return false;
   }
-  ok();
+  ok(haveOpenAI ? '(hybrid mode)' : '(keyword-only mode — OPENAI_API_KEY not set)');
   return true;
 }
 
@@ -400,7 +479,17 @@ async function main(argv) {
 
   if (!(await link(projectRef, flags.dryRun))) return 4;
   if (!(await applyRumenTables(secrets, flags.dryRun))) return 5;
-  if (!deployFunction(flags.dryRun)) return 6;
+
+  step('Resolving @jhizzard/rumen version from npm registry...');
+  const resolved = resolveRumenVersion();
+  ok();
+  process.stdout.write(`→ Using rumen version: ${resolved.version} (from ${resolved.source})\n`);
+  if (resolved.warning) {
+    process.stderr.write(`  ! ${resolved.warning}\n`);
+    process.stderr.write(`  ! falling back to pinned FALLBACK_RUMEN_VERSION=${FALLBACK_RUMEN_VERSION}\n`);
+  }
+
+  if (!deployFunction(resolved.version, flags.dryRun)) return 6;
   if (!setFunctionSecrets(secrets, flags.dryRun)) return 7;
   if (!(await testFunction(projectRef, secrets, flags.dryRun))) return 8;
   if (!flags.skipSchedule) {