spectoflow 0.32.0 → 0.33.0

package/README.md

@@ -307,8 +307,9 @@ off by default):
 - **Info** — a project-at-a-glance summary.
 - **Documentation** — the live supported-agents table (your own install status + links) plus the CLI
   command reference.
-- **Second brain** — what spectoflow has learned about you, shared by all your projects: read, add, fix,
-  confirm (see [Second brain](#second-brain)). Local only.
+- **Second brain** — two sections: **You** (what spectoflow has learned about you, shared by all your projects,
+  local only) and **This project** (the project memory, committed). Read, add, fix, confirm, move a fact to the
+  other one (see [Second brain](#second-brain)).
 - **Personalize** — autonomy mode, language, design, the active agent, **navigation tabs**, **slash
   commands**, and **Extend spectoflow** (see *Customize* below).
 
@@ -412,11 +413,37 @@ agents then read and grow your second brain through that server: nothing is copi
   `::spectoflow learn category=… msg=…` line instead. Those always wait in *To confirm*, whatever the setting:
   that output also carries command output and file contents, so a line hidden in a repository can't slip
   in unseen.
-- **Private:** the Second brain tab and its API answer this machine only — not the online dashboard
+- **Private:** the *You* section and its API answer this machine only — not the online dashboard
   (`server/` refuses them for everyone, the project owner included), and not other machines or websites.
   A run started from either of those can't write into it, and a learned fact never goes into a project's chat
   log.
 
+### Project memory
+
+Some facts aren't about you but about **one project**: its conventions, what breaks, its vocabulary, its
+constraints. Those go in `.spectoflow/memory.md`, **committed with the code**, so your team and their agents
+share them — and your other projects don't inherit them.
+
+```markdown
+# Project memory
+
+## Conventions
+- Tests run with `npm test -- --runInBand` (shared DB fixtures)
+
+## Glossary
+- "Ticket" means a customer support case, never a Jira issue
+```
+
+- **Four categories:** Conventions, Pitfalls, Glossary, Constraints.
+- **The agent reads and writes the file directly** — no setup. About you → second brain; about the project →
+  this file. Never anything personal in it.
+- **Facts the agent adds land directly by default.** To review them first, untick *Add what the agent learns
+  directly* in the *This project* section: that sets `memoryAutoAdd: false` in `.spectoflow/config.json`, for
+  the whole team.
+- **Wrong memory?** *Move to…* on any fact sends it to the other one, in the category you pick (local dashboard
+  only).
+- Created on the first fact; `spectoflow update` never touches it.
+
 ## A workflow that fits the project
 
 The workflow (`.spectoflow/workflow.md`) isn't one-size-fits-all anymore.

package/lib/adapters.js

@@ -30,6 +30,11 @@ learn one (a stated preference, a correction of how you work, their role), recor
 \`::spectoflow learn category=<profile|preferences|workflow|avoid> msg=<fact>\` if that tool is unavailable. Never
 secrets or sensitive data.
 
+**Project memory.** Facts about *this project* (conventions, pitfalls, domain vocabulary, constraints) live in
+\`.spectoflow/memory.md\`, committed — read it at session start and apply it. When you learn one, add a line under
+its section (or under \`## To confirm\` as \`- [category] fact\` when \`.spectoflow/config.json\` → \`memoryAutoAdd\` is
+\`false\`). About the user → second brain; about the project → this file. Never anything personal there.
+
 **Workflow.** If a request needs a step that is disabled in \`.spectoflow/workflow.md\` (e.g. writing code while
 *Develop* is off): enable it yourself and say so when \`.spectoflow/config.json\` → \`workflowAutoEnable\` is \`true\`;
 otherwise ask first. Never disable a step without the user.
@@ -53,6 +58,11 @@ learn one (a stated preference, a correction of how you work, their role), recor
 \`::spectoflow learn category=<profile|preferences|workflow|avoid> msg=<fact>\` if that tool is unavailable. Never
 secrets or sensitive data.
 
+**Project memory.** Facts about *this project* (conventions, pitfalls, domain vocabulary, constraints) live in
+\`.spectoflow/memory.md\`, committed — read it at session start and apply it. When you learn one, add a line under
+its section (or under \`## To confirm\` as \`- [category] fact\` when \`.spectoflow/config.json\` → \`memoryAutoAdd\` is
+\`false\`). About the user → second brain; about the project → this file. Never anything personal there.
+
 **Workflow.** If a request needs a step that is disabled in \`.spectoflow/workflow.md\` (e.g. writing code while
 *Develop* is off): enable it yourself and say so when \`.spectoflow/config.json\` → \`workflowAutoEnable\` is \`true\`;
 otherwise ask first. Never disable a step without the user.
@@ -73,6 +83,11 @@ learn one (a stated preference, a correction of how you work, their role), recor
 \`::spectoflow learn category=<profile|preferences|workflow|avoid> msg=<fact>\` if that tool is unavailable. Never
 secrets or sensitive data.
 
+**Project memory.** Facts about *this project* (conventions, pitfalls, domain vocabulary, constraints) live in
+\`.spectoflow/memory.md\`, committed — read it at session start and apply it. When you learn one, add a line under
+its section (or under \`## To confirm\` as \`- [category] fact\` when \`.spectoflow/config.json\` → \`memoryAutoAdd\` is
+\`false\`). About the user → second brain; about the project → this file. Never anything personal there.
+
 **Workflow.** If a request needs a step that is disabled in \`.spectoflow/workflow.md\` (e.g. writing code while
 *Develop* is off): enable it yourself and say so when \`.spectoflow/config.json\` → \`workflowAutoEnable\` is \`true\`;
 otherwise ask first. Never disable a step without the user.

package/lib/brain.js

@@ -2,256 +2,25 @@
 /*
  * The second brain — what spectoflow has learned about the user, shared by all their projects.
  * One markdown file, ~/.spectoflow/brain.md (never inside a project, never copied), read and written
- * by the dashboard page, the `spectoflow mcp` server and the `::spectoflow learn` run line. Zero
- * dependency.
- *
- *   # Second brain
- *   ## Profile
- *   - Scrum master and full-stack developer <!-- id:b7k2 by:agent at:2026-09-16 -->
- *   ## To confirm
- *   - [preferences] Prefers pnpm over npm <!-- id:b7k6 by:agent at:2026-09-16 -->
- *
- * The file is the user's too: every line spectoflow doesn't change is written back byte for byte —
- * titles, blank lines, comments, code blocks, unknown sections and their order, hand-written entries.
- * Writers in different processes (hub, MCP servers, runs) take a lock file, then write-then-rename.
+ * by the dashboard page, the `spectoflow mcp` server and the `::spectoflow learn` run line. Facts about
+ * one project go to the project memory instead (lib/project-memory.js). The file format, its fidelity
+ * and its lock live in lib/memory-store.js.
  */
-const fs = require('fs');
 const path = require('path');
-const crypto = require('crypto');
 const globalConfig = require('./global-config');
-
-const CATEGORIES = ['profile', 'preferences', 'workflow', 'avoid'];
-const HEADINGS = { profile: 'Profile', preferences: 'Preferences', workflow: 'Working style', avoid: 'Avoid' };
-const PENDING = 'To confirm';
-const MAX_TEXT = 500;
-const FALLBACK_CATEGORY = 'preferences';
+const { createMemoryStore } = require('./memory-store');
 
 function brainPath() { return path.join(globalConfig.homeDir(), 'brain.md'); }
 
-const normCategory = (c) => (CATEGORIES.includes(String(c || '').trim().toLowerCase()) ? String(c).trim().toLowerCase() : FALLBACK_CATEGORY);
-// One line, no HTML-comment delimiters (they would break the metadata), capped.
-function cleanText(t) {
-  return String(t == null ? '' : t).replace(/<!--|-->/g, '').replace(/\s+/g, ' ').trim().slice(0, MAX_TEXT);
-}
-const newId = () => 'b' + Date.now().toString(36) + crypto.randomBytes(3).toString('hex');
-// A hand-written line has no id: derive a stable one. `n` tells identical lines of a category apart.
-const derivedId = (category, text, n) => 'h' + crypto.createHash('sha1').update(`${category}\n${text}\n${n}`).digest('hex').slice(0, 10);
-const today = () => new Date().toISOString().slice(0, 10);
-const isBlank = (it) => it.raw !== undefined && !it.raw.trim();
-
-function headingKind(title) {
-  const t = title.trim().toLowerCase();
-  if (t === PENDING.toLowerCase()) return { kind: 'pending' };
-  const cat = CATEGORIES.find((c) => c === t || HEADINGS[c].toLowerCase() === t);
-  return cat ? { kind: 'category', id: cat } : { kind: 'unknown' };
-}
-
-const ENTRY_RE = /^-\s+(?:\[([\w-]+)\]\s+)?(.*?)\s*(?:<!--\s*(.*?)\s*-->)?\s*$/;
-function parseMeta(s) {
-  const out = {};
-  for (const m of String(s || '').matchAll(/(\w+):(\S+)/g)) out[m[1]] = m[2];
-  return out;
-}
-
-// → { title, blocks: [{ kind: preamble|category|pending|unknown, id?, heading?, items: [...] }] }
-// item = { raw } (a line kept verbatim) or an entry { id, category, text, by, at, status, line }.
-function parse(text) {
-  const lines = String(text || '').split(/\r?\n/);
-  while (lines.length && !lines[lines.length - 1].trim()) lines.pop();
-  const model = { title: null, eol: /\r\n/.test(String(text || '')) ? '\r\n' : '\n', blocks: [{ kind: 'preamble', items: [] }] };
-  let block = model.blocks[0], inFence = false;
-  const seen = new Map();
-  for (const line of lines) {
-    if (/^\s*(```|~~~)/.test(line)) { inFence = !inFence; block.items.push({ raw: line }); continue; }
-    if (inFence) { block.items.push({ raw: line }); continue; }
-    if (model.title === null && model.blocks.length === 1 && block.items.every(isBlank) && /^#\s+/.test(line)) { model.title = line; continue; }
-    const h = line.match(/^##\s+(.*)$/);
-    if (h) {
-      block = { ...headingKind(h[1]), heading: line, items: [] };
-      model.blocks.push(block);
-      continue;
-    }
-    const m = (block.kind === 'category' || block.kind === 'pending') && line.match(ENTRY_RE);
-    const meta = m ? parseMeta(m[3]) : null;
-    // A trailing comment is never part of the fact (a user's own comment stays hidden: the line itself is
-    // written back verbatim while the entry is untouched; editing the entry replaces the line).
-    const body = m ? m[2] : '';
-    if (m && cleanText(body)) {
-      const pending = block.kind === 'pending';
-      const category = pending ? normCategory(m[1]) : block.id;
-      const entryText = cleanText(pending || !m[1] ? body : `[${m[1]}] ${body}`);
-      let id = meta.id;
-      if (!id) { const k = `${category}\n${entryText}`; const n = seen.get(k) || 0; seen.set(k, n + 1); id = derivedId(category, entryText, n); }
-      block.items.push({ id, category, text: entryText, by: meta.by || 'user', at: meta.at || null, status: pending ? 'pending' : 'confirmed', line });
-    } else {
-      block.items.push({ raw: line });
-    }
-  }
-  return model;
-}
-
-function entryLine(e) {
-  const meta = `id:${e.id} by:${e.by}${e.at ? ' at:' + e.at : ''}`;
-  return `- ${e.status === 'pending' ? `[${e.category}] ` : ''}${e.text} <!-- ${meta} -->`;
-}
-function serialize(model) {
-  const out = [model.title || '# Second brain'];
-  for (const b of model.blocks) {
-    if (b.kind === 'pending' && !b.items.some((it) => it.raw === undefined) && b.items.every(isBlank)) continue;
-    if (b.kind !== 'preamble') {
-      if (out[out.length - 1].trim()) out.push(''); // a section always follows a blank line
-      out.push(b.heading || `## ${b.kind === 'pending' ? PENDING : HEADINGS[b.id]}`);
-    }
-    for (const it of b.items) out.push(it.raw !== undefined ? it.raw : (it.line && !it.dirty ? it.line : entryLine(it)));
-  }
-  while (out.length > 1 && !out[out.length - 1].trim()) out.pop();
-  const eol = model.eol || '\n';
-  return out.join(eol) + eol;
-}
-
-function emptyModel() {
-  return { title: null, blocks: [{ kind: 'preamble', items: [] }, ...CATEGORIES.map((id) => ({ kind: 'category', id, heading: null, items: [] }))] };
-}
-function load(file = brainPath()) {
-  let text = '';
-  try { text = fs.readFileSync(file, 'utf8'); } catch (e) { if (e.code !== 'ENOENT') throw e; }
-  return text.trim() ? parse(text) : emptyModel();
-}
-function save(model, file = brainPath()) {
-  fs.mkdirSync(path.dirname(file), { recursive: true });
-  const tmp = `${file}.${process.pid}.${crypto.randomBytes(4).toString('hex')}.tmp`;
-  fs.writeFileSync(tmp, serialize(model));
-  fs.renameSync(tmp, file);
-}
-
-// Cross-process lock: the hub, any number of `spectoflow mcp` processes and runs may write in the
-// same instant. A lock older than 10s is a crashed writer's and is taken over.
-const sleepSync = (ms) => Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, ms);
-function withLock(file, fn) {
-  fs.mkdirSync(path.dirname(file), { recursive: true });
-  const lock = `${file}.lock`, deadline = Date.now() + 3000;
-  for (;;) {
-    try { fs.writeFileSync(lock, String(process.pid), { flag: 'wx' }); break; }
-    catch (e) {
-      if (e.code !== 'EEXIST') throw e;
-      try { if (Date.now() - fs.statSync(lock).mtimeMs > 10000) { fs.unlinkSync(lock); continue; } } catch (_) {}
-      if (Date.now() > deadline) throw Object.assign(new Error('The second brain is busy, try again.'), { status: 503 });
-      sleepSync(15);
-    }
-  }
-  try { return fn(); } finally { try { fs.unlinkSync(lock); } catch (_) {} }
-}
-// Read, change, save — under the lock. `fn` returns { result, changed }.
-function mutate(file, fn) {
-  return withLock(file, () => {
-    const model = load(file);
-    const { result, changed } = fn(model);
-    if (changed) save(model, file);
-    return result;
-  });
-}
-
-const entriesOf = (model) => model.blocks.flatMap((b) => b.items.filter((it) => it.raw === undefined));
-function find(model, id) {
-  for (const b of model.blocks) {
-    const i = b.items.findIndex((it) => it.raw === undefined && it.id === id);
-    if (i >= 0) return { block: b, i, entry: b.items[i] };
-  }
-  return null;
-}
-// Append before the block's trailing blank lines, so the spacing before the next heading stays.
-function append(block, entry) {
-  let i = block.items.length;
-  while (i > 0 && isBlank(block.items[i - 1])) i--;
-  block.items.splice(i, 0, entry);
-}
-function blockFor(model, status, category) {
-  const want = status === 'pending' ? (b) => b.kind === 'pending' : (b) => b.kind === 'category' && b.id === category;
-  let b = model.blocks.find(want);
-  if (!b) {
-    b = status === 'pending' ? { kind: 'pending', heading: null, items: [] } : { kind: 'category', id: category, heading: null, items: [] };
-    const at = status === 'pending' ? -1 : model.blocks.findIndex((x) => x.kind === 'pending');
-    if (at < 0) model.blocks.push(b); else model.blocks.splice(at, 0, b);
-  }
-  return b;
-}
-const notFound = () => Object.assign(new Error('Entry not found.'), { status: 404 });
-const emptyText = () => Object.assign(new Error('Text is required.'), { status: 400 });
-
-// { entries (confirmed, in category order), pending, autoAdd }
-function read(file = brainPath()) {
-  const all = entriesOf(load(file));
-  return {
-    entries: CATEGORIES.flatMap((c) => all.filter((e) => e.status === 'confirmed' && e.category === c)),
-    pending: all.filter((e) => e.status === 'pending'),
-    autoAdd: autoAdd(),
-  };
-}
-
-// Adds one fact. status 'confirmed' or 'pending'. A case-insensitive duplicate of any entry is not
-// added again: → { duplicate: true, entry: <the existing one> }.
-function add({ category, text, by = 'user', status = 'confirmed' }, file = brainPath()) {
-  const t = cleanText(text);
-  if (!t) throw emptyText();
-  return mutate(file, (model) => {
-    const existing = entriesOf(model).find((e) => e.text.toLowerCase() === t.toLowerCase());
-    if (existing) return { result: { duplicate: true, entry: existing }, changed: false };
-    const entry = { id: newId(), category: normCategory(category), text: t, by, at: today(), status: status === 'pending' ? 'pending' : 'confirmed' };
-    append(blockFor(model, entry.status, entry.category), entry);
-    return { result: { duplicate: false, entry }, changed: true };
-  });
-}
-
-// What the agent learned through MCP: confirmed right away, or "to confirm", per brain.autoAdd.
-function learn({ category, text }, file = brainPath()) {
-  return add({ category, text, by: 'agent', status: autoAdd() ? 'confirmed' : 'pending' }, file);
-}
-
-function update(id, { text, category } = {}, file = brainPath()) {
-  const t = text === undefined ? undefined : cleanText(text);
-  if (text !== undefined && !t) throw emptyText();
-  return mutate(file, (model) => {
-    const hit = find(model, id); if (!hit) throw notFound();
-    const e = hit.entry;
-    if (t !== undefined) e.text = t;
-    if (category !== undefined) {
-      const c = normCategory(category);
-      if (c !== e.category && e.status === 'confirmed') { hit.block.items.splice(hit.i, 1); append(blockFor(model, 'confirmed', c), e); }
-      e.category = c;
-    }
-    e.dirty = true;
-    return { result: e, changed: true };
-  });
-}
-
-function remove(id, file = brainPath()) {
-  return mutate(file, (model) => {
-    const hit = find(model, id); if (!hit) throw notFound();
-    hit.block.items.splice(hit.i, 1);
-    return { result: { ok: true }, changed: true };
-  });
-}
-
-function confirm(id, file = brainPath()) {
-  return mutate(file, (model) => {
-    const hit = find(model, id); if (!hit) throw notFound();
-    if (hit.entry.status !== 'pending') return { result: hit.entry, changed: false };
-    hit.block.items.splice(hit.i, 1);
-    hit.entry.status = 'confirmed'; hit.entry.dirty = true;
-    append(blockFor(model, 'confirmed', hit.entry.category), hit.entry);
-    return { result: hit.entry, changed: true };
-  });
-}
-
-// Confirmed entries as markdown, grouped by category — what an agent is given.
-function renderForAgent(file = brainPath()) {
-  const { entries } = read(file);
-  return CATEGORIES.map((c) => {
-    const items = entries.filter((e) => e.category === c);
-    return items.length ? `## ${HEADINGS[c]}\n${items.map((e) => `- ${e.text}`).join('\n')}` : '';
-  }).filter(Boolean).join('\n\n');
-}
-
-function autoAdd() { return globalConfig.get('brain.autoAdd').value !== false; }
-
-module.exports = { CATEGORIES, HEADINGS, MAX_TEXT, brainPath, parse, serialize, read, add, learn, update, remove, confirm, renderForAgent, autoAdd };
+const store = createMemoryStore({
+  name: 'second brain',
+  title: 'Second brain',
+  categories: ['profile', 'preferences', 'workflow', 'avoid'],
+  headings: { profile: 'Profile', preferences: 'Preferences', workflow: 'Working style', avoid: 'Avoid' },
+  fallback: 'preferences',
+  idPrefix: 'b',
+  defaultFile: brainPath,
+  autoAdd: () => globalConfig.get('brain.autoAdd').value !== false,
+});
+
+module.exports = { ...store, brainPath };

package/lib/dashboard/ops.js

@@ -18,6 +18,7 @@ const orchestrator = require('./orchestrator');
 const adapters = require('../adapters');
 const detect = require('../detect');
 const brain = require('../brain');
+const projectMemory = require('../project-memory');
 const brainSetup = require('../brain-setup');
 const globalConfig = require('../global-config');
 const workflowDetect = require('../workflow-detect');
@@ -74,6 +75,7 @@ function writeConfig(root, patch, detectOpts) {
   if (typeof patch.activeTab === 'string' && patch.activeTab.trim()) cfg.activeTab = patch.activeTab.trim();
   if (typeof patch.chatOpen === 'boolean') cfg.chatOpen = patch.chatOpen;
   if (typeof patch.workflowAutoEnable === 'boolean') cfg.workflowAutoEnable = patch.workflowAutoEnable;
+  if (typeof patch.memoryAutoAdd === 'boolean') cfg.memoryAutoAdd = patch.memoryAutoAdd;
   // kanbanColumns: reject the whole patch (leave the current value untouched) rather than silently
   // filtering out bad entries — an invalid/unknown status id here means the client sent something it
   // shouldn't have, and a real product-safety rule (never persist zero visible columns) applies too.
@@ -150,8 +152,12 @@ const changed = (ctx, result) => { ctx.emit({ type: 'change' }); return result;
 // watches ~/.spectoflow/brain.md and tells local tabs only, whoever wrote it (page, MCP, run line).
 function brainOp(ctx, fn) {
   if (ctx && ctx.remote) throw new OpError(404, 'Unknown operation.');
-  try { return fn(); } catch (e) { if (e.status && !(e instanceof OpError)) throw new OpError(e.status, e.message); throw e; }
+  return storeOp(fn);
 }
+const storeOp = (fn) => { try { return fn(); } catch (e) { if (e.status && !(e instanceof OpError)) throw new OpError(e.status, e.message); throw e; } };
+// The project memory is the project's (.spectoflow/memory.md, committed): gated like any project read/write,
+// online included. Only `memory.move` touches the personal brain, so only it is local.
+const memOp = (root, ctx, fn) => { const r = storeOp(() => fn(projectMemory.fileFor(root))); ctx.emit({ type: 'change' }); return r; };
 
 const ops = {
   'project.read': async (root) => {
@@ -307,6 +313,26 @@ const ops = {
     return { autoAdd: globalConfig.set('brain.autoAdd', autoAdd) };
   }),
 
+  'memory.read': async (root) => storeOp(() => ({ ...projectMemory.read(projectMemory.fileFor(root)), path: projectMemory.REL })),
+  'memory.add': async (root, { category, text: body }, ctx) => memOp(root, ctx, (f) => projectMemory.add({ category, text: body, by: 'user' }, f)),
+  'memory.update': async (root, { id, patch }, ctx) => memOp(root, ctx, (f) => ({ entry: projectMemory.update(id, patch || {}, f) })),
+  'memory.remove': async (root, { id }, ctx) => memOp(root, ctx, (f) => projectMemory.remove(id, f)),
+  'memory.confirm': async (root, { id }, ctx) => memOp(root, ctx, (f) => ({ entry: projectMemory.confirm(id, f) })),
+  // The agent picked the wrong memory: move one entry to the other, under the category the user chose. Added
+  // to the target first, removed from the source after — a failure never loses the fact. Local only: one of
+  // the two files is always the user's personal brain.
+  'memory.move': async (root, { from, id, category }, ctx) => brainOp(ctx, () => {
+    if (from !== 'user' && from !== 'project') bad('from must be "user" or "project".');
+    const pf = projectMemory.fileFor(root);
+    const [src, srcFile, dst, dstFile] = from === 'user' ? [brain, undefined, projectMemory, pf] : [projectMemory, pf, brain, undefined];
+    const entry = src.get(id, srcFile);
+    if (!dst.CATEGORIES.includes(category)) bad(`Unknown category: ${category}.`);
+    const added = dst.add({ category, text: entry.text, by: entry.by }, dstFile);
+    src.remove(id, srcFile);
+    ctx.emit({ type: 'change' });
+    return added;
+  }),
+
   'attention.remove': async (root, { id }, ctx) => {
     const rt = store.readRuntime(root); rt.attention = (rt.attention || []).filter((x) => x.id !== id); store.writeRuntime(root, rt);
     return changed(ctx, { ok: true });