@tpsdev-ai/flair 0.31.1 → 0.32.0

package/README.md

@@ -40,7 +40,19 @@ Self-hosted on [Harper](https://harper.fast) as a single process. No sidecars, n
 
 ## Quick start
 
-Needs **Node.js 22+** and a **user-writable npm global prefix**.
+Two front doors. Pick by where your code already runs.
+
+### Already on Harper? Load Flair as a component
+
+Flair *is* a Harper component. Deploy it into the instance your application already runs in and call its resources directly — `await h.post({ agentId, content })` is a **method call**, not a network call. No second service to operate, no HTTP round trip, and no key to distribute: a caller in the same process is already inside the trust boundary and names the agent it is acting as, per call.
+
+Adding it takes nothing away. The HTTP surface keeps serving MCP clients and remote agents exactly as before.
+
+**→ [Embedding Flair in a Harper app](docs/embedding-in-a-harper-app.md)** — the whole in-process contract, measured against a real instance: resolving the resource, the table-vs-resource distinction that decides whether your memories are scoped at all, N agents in one process, and registering agents with no shell on the node.
+
+### Everywhere else — install the CLI
+
+A laptop, a VPS, an MCP client, or any language over HTTP. Needs **Node.js 22+** and a **user-writable npm global prefix**.
 
 > ⚠️ **Never `sudo npm install -g @tpsdev-ai/flair`.** A root-owned install can't write the embedding model into its own package directory, so semantic search silently degrades to keyword-only. `flair init` and `flair doctor` will warn you loudly. Use `nvm`, or point npm at your home directory: `npm config set prefix ~/.npm-global` and add `~/.npm-global/bin` to `PATH`.
 
@@ -73,6 +85,14 @@ That trailing figure is a rank score, normalized so the top hit is always near 1
 
 Full walkthrough with expected output at every step: **[docs/quickstart.md](docs/quickstart.md)**.
 
+### Where the agent's key lives
+
+`flair init --agent mybot` writes the private key to `~/.flair/keys/mybot.key` (mode `0600`) and the public half beside it as `mybot.pub`. Only the **public** key is registered on the instance; the private key never leaves the machine. `--keys-dir` writes both somewhere else.
+
+**Back that file up — it is the agent's identity, and there is one copy.** Memories are not encrypted with it, so losing it costs the identity, not the data: the agent can no longer sign, and every HTTP call it makes fails. Recovery is `flair agent rotate-key mybot`, which mints a new pair and re-registers the public half — it needs the admin password `flair init` wrote to `~/.flair/admin-pass`, so back that up too. Otherwise treat the key like an SSH key: one per agent per host, never copied between machines ([docs/secrets-and-keys.md](docs/secrets-and-keys.md)).
+
+**The in-process path needs no key at all.** Keys are how an agent *outside* the process proves who it is. Code running inside the same Harper instance asserts identity through the call context instead — `agentContext("mybot")` — which Flair reads and acts on with no signature, no `Agent`-table lookup and no registration. That is deliberate: same-process code could write the storage tables directly, so demanding a signature from it would be theatre. It is also why that id must come from your own server-side state and never from request data.
+
 ### Useful flags
 
 ```bash
@@ -278,24 +298,30 @@ Sign `agentId:timestamp:nonce:METHOD:/path` with the agent's private key. Protoc
 
 ### Embedded in a Harper app (in-process)
 
-Flair *is* a Harper component. If your application already runs on Harper, load Flair into the same instance and call its resources directly — a method call instead of an HTTP round trip.
+Flair *is* a Harper component. If your application already runs on Harper, load Flair into the same instance and call its resources directly — a method call instead of an HTTP round trip, and no key anywhere.
 
 ```javascript
 import { server } from "harper";
+import { agentContext, collectionResource } from "@tpsdev-ai/flair/dist/resources/in-process.js";
+
+// The RESOURCE — auth, read-scoping, visibility, embedding.
+// Registry keys carry no leading slash: get("Memory"), never get("/Memory").
+const Memory = server.resources.get("Memory").Resource;
 
-const Memory = server.resources.get("Memory").Resource;   // the resource, not the table
-const h = new Memory(undefined, { request: { tpsAgent: "mybot" } });
+const h = await collectionResource(Memory, agentContext("mybot"));
 await h.post({ agentId: "mybot", content: "...", durability: "standard" });
 ```
 
-`databases.flair.Memory` is the **table** (raw storage); the exported `Memory` class is the **resource**, where auth, read-scoping, visibility and embedding live. A context-less call runs unfiltered. Full guide: **[docs/embedding-in-a-harper-app.md](docs/embedding-in-a-harper-app.md)**.
+`databases.flair.Memory` is the **table** (raw storage); the exported `Memory` class is the **resource**, where auth, read-scoping, visibility and embedding live. `new Memory(...)` is not a substitute for `collectionResource` — a create needs a collection-bound instance only Harper can produce. Both helpers refuse a missing agent id rather than defaulting it, because a resource invoked with no context resolves to Flair's trusted `internal` verdict and runs unfiltered across every agent. Full guide: **[docs/embedding-in-a-harper-app.md](docs/embedding-in-a-harper-app.md)**.
 
 ### Auth across surfaces
 
-The default everywhere is **Ed25519 per-agent**: each agent holds its own key at `~/.flair/keys/<agent>.key` and signs every request. That gives write isolation — no agent can write as another — and identity-verified reads. It does *not* refuse cross-agent reads: within one instance, any verified agent can read any other agent's non-private memory by design. The hard boundary is the federation edge, not intra-instance reads. See [SECURITY.md](SECURITY.md).
+For every caller that reaches Flair over the network the default is **Ed25519 per-agent**: each agent holds its own key at `~/.flair/keys/<agent>.key` and signs every request. That gives write isolation — no agent can write as another — and identity-verified reads. It does *not* refuse cross-agent reads: within one instance, any verified agent can read any other agent's non-private memory by design. The hard boundary is the federation edge, not intra-instance reads. See [SECURITY.md](SECURITY.md).
 
 One exception: the **`n8n-nodes-flair`** node authenticates with the Harper **admin password** (Basic auth), which bypasses agent scoping entirely — it can read other agents' `visibility: private` memories and write as anyone. That is acceptable only on a single-tenant, operator-controlled n8n with trusted workflow inputs. Otherwise prefer the Ed25519 path. Full breakdown in **[docs/auth.md](docs/auth.md#auth-across-surfaces-read-this-first)**.
 
+In-process callers are a different model, not an exception to this one: they never sign, because identity is asserted through the call context rather than proven. Co-location *is* the grant — which is why Flair beside untrusted co-tenants on a shared instance is a different proposition to Flair inside your own app.
+
 ## Deployment
 
 ### Local (default)

package/dist/cli.js

@@ -269,7 +269,19 @@ function migrateLegacyLaunchdLabel(dataDir, runLaunchctl, launchAgentsDir = defa
     }
     catch { /* best effort */ }
     const legacyContent = readFileSync(resolved.plistPath, "utf-8");
-    const newContent = legacyContent.replace(`<key>Label</key><string>${LEGACY_LAUNCHD_LABEL}</string>`, `<key>Label</key><string>${newLabel}</string>`);
+    // Use a function replacer to avoid $-sensitivity in the replacement
+    // string (flair#919). String.prototype.replace interprets $&, $', $`
+    // etc. in the replacement even when the search value is a plain string.
+    const labelSearch = `<key>Label</key><string>${LEGACY_LAUNCHD_LABEL}</string>`;
+    const labelReplacement = `<key>Label</key><string>${newLabel}</string>`;
+    const newContent = legacyContent.replace(labelSearch, () => labelReplacement);
+    // Refuse to propagate a malformed plist: if the Label wasn't found,
+    // the plist is not what we expect and migration must not write it.
+    if (newContent === legacyContent) {
+        throw new Error(`Legacy plist at ${resolved.plistPath} does not contain the expected ` +
+            `Label key — it may be malformed or from an unknown Flair version. ` +
+            `Remove it manually and re-run 'flair init'.`);
+    }
     writeFileSync(newPlistPath, newContent);
     try {
         unlinkSync(resolved.plistPath);
@@ -277,6 +289,58 @@ function migrateLegacyLaunchdLabel(dataDir, runLaunchctl, launchAgentsDir = defa
     catch { /* best effort */ }
     return { migrated: true, label: newLabel, plistPath: newPlistPath };
 }
+/**
+ * Clean up a pre-flair#693 legacy launchd plist (ai.tpsdev.flair) during
+ * init, but ONLY when it belongs to the data dir being initialised.
+ *
+ * flair#966: the legacy plist is a single global label — init must not
+ * unload/delete it unless ROOTPATH proves it serves this data dir.
+ *
+ * `runLaunchctl` is injected so tests can record/mock without touching
+ * real launchd. The caller (init) passes a real execSync wrapper; tests
+ * pass a recording stub.
+ */
+function cleanupLegacyLaunchdPlist(dataDir, plistDir, runLaunchctl) {
+    const legacyPlistPath = launchdPlistPath(LEGACY_LAUNCHD_LABEL, plistDir);
+    if (!existsSync(legacyPlistPath))
+        return { action: "none" };
+    const legacyRootPath = readPlistRootPath(legacyPlistPath);
+    const legacyOwnedByUs = legacyRootPath !== null && resolve(legacyRootPath) === resolve(dataDir);
+    if (legacyOwnedByUs) {
+        let unloadFailed;
+        let deleteFailed;
+        try {
+            runLaunchctl(`launchctl unload "${legacyPlistPath}"`);
+        }
+        catch (err) {
+            unloadFailed = err?.message ?? String(err);
+            console.error(`Failed to unload legacy launchd service (${LEGACY_LAUNCHD_LABEL}): ` +
+                `${unloadFailed}. ` +
+                `The plist at ${legacyPlistPath} may still be loaded — ` +
+                `unload it manually with: launchctl unload "${legacyPlistPath}"`);
+        }
+        try {
+            unlinkSync(legacyPlistPath);
+            console.log(`Migrated off legacy launchd label (${LEGACY_LAUNCHD_LABEL}) ✓`);
+        }
+        catch (err) {
+            deleteFailed = err?.message ?? String(err);
+            console.error(`Failed to remove legacy launchd plist at ${legacyPlistPath}: ` +
+                `${deleteFailed}. Remove it manually.`);
+        }
+        return { action: "unloaded", unloadFailed, deleteFailed };
+    }
+    if (legacyRootPath !== null) {
+        console.log(`Skipped legacy launchd cleanup: the plist at ${legacyPlistPath} ` +
+            `belongs to data dir ${resolve(legacyRootPath)}, not ${resolve(dataDir)} — ` +
+            `that is a different Flair instance.`);
+        return { action: "skipped-foreign", foreignDataDir: resolve(legacyRootPath) };
+    }
+    console.log(`Skipped legacy launchd cleanup: could not determine which data dir ` +
+        `the plist at ${legacyPlistPath} serves. ` +
+        `If it is yours, remove it manually with: rm "${legacyPlistPath}"`);
+    return { action: "skipped-unknown" };
+}
 /**
  * Load + start `dataDir`'s launchd service, migrating off a pre-flair#693
  * legacy registration FIRST if one is found (migrateLegacyLaunchdLabel
@@ -290,6 +354,13 @@ function migrateLegacyLaunchdLabel(dataDir, runLaunchctl, launchAgentsDir = defa
  */
 function ensureLaunchdServiceLoaded(dataDir, runLaunchctl, launchAgentsDir = defaultLaunchAgentsDir()) {
     const migration = migrateLegacyLaunchdLabel(dataDir, runLaunchctl, launchAgentsDir);
+    // Unload first so a rewritten plist is re-read (flair#872).
+    // launchd caches the environment of an already-loaded job; load
+    // alone does not pick up changes to the plist on disk.
+    try {
+        runLaunchctl(`launchctl unload "${migration.plistPath}"`);
+    }
+    catch { /* not loaded, etc. — best effort */ }
     try {
         runLaunchctl(`launchctl load "${migration.plistPath}"`);
     }
@@ -2933,25 +3004,20 @@ program
                 const plistDir = defaultLaunchAgentsDir();
                 mkdirSync(plistDir, { recursive: true });
                 const plistPath = launchdPlistPath(label, plistDir);
-                // flair#693 migration: a pre-flair#693 install registered under
+                // flair#693 + flair#966: a pre-flair#693 install registered under
                 // the bare LEGACY_LAUNCHD_LABEL. init always writes fresh plist
                 // content below (it has the current ports/creds in hand), so
                 // migration here is just "clean up the old registration" —
                 // unload + remove it BEFORE writing the new one, so re-running
                 // init never leaves two services behind for this data dir.
-                const legacyPlistPath = launchdPlistPath(LEGACY_LAUNCHD_LABEL, plistDir);
-                if (existsSync(legacyPlistPath)) {
-                    try {
-                        const { execSync } = await import("node:child_process");
-                        execSync(`launchctl unload "${legacyPlistPath}"`, { stdio: "pipe" });
-                    }
-                    catch { /* best effort */ }
-                    try {
-                        unlinkSync(legacyPlistPath);
-                    }
-                    catch { /* best effort */ }
-                    console.log(`Migrated off legacy launchd label (${LEGACY_LAUNCHD_LABEL}) ✓`);
-                }
+                //
+                // flair#966: the legacy plist is NOT scoped to this data dir —
+                // it is a single global label. cleanupLegacyLaunchdPlist reads
+                // ROOTPATH to establish ownership before touching it.
+                cleanupLegacyLaunchdPlist(dataDir, plistDir, (cmd) => {
+                    const { execSync } = require("node:child_process");
+                    execSync(cmd, { stdio: "pipe" });
+                });
                 const opsSocket = join(dataDir, "operations-server");
                 // authorizeLocal: false (flair#654) — same posture as the initial spawn
                 // above; the launchd-managed process must not diverge from it.
@@ -9741,20 +9807,28 @@ program
  * Never logs plist contents — the plist embeds HDB_ADMIN_PASSWORD. Only the
  * extracted ROOTPATH path ever reaches a message.
  */
-export function assertLaunchdServiceOwnedBy(dataDir, label, plistPath, action) {
-    let declared = null;
+/**
+ * Read the ROOTPATH declared in a launchd plist, or null if it cannot be
+ * determined (file missing, unreadable, or no ROOTPATH key).
+ *
+ * The plist stores this XML-escaped (buildLaunchdPlist), so the returned
+ * value is decoded through unescapeXml before being returned — a data dir
+ * containing `&` is on disk as `&amp;` and this returns the literal `&`.
+ *
+ * Never logs plist contents — the plist embeds HDB_ADMIN_PASSWORD.
+ */
+export function readPlistRootPath(plistPath) {
     try {
         const raw = readFileSync(plistPath, "utf-8");
         const m = raw.match(/<key>ROOTPATH<\/key>\s*<string>([^<]*)<\/string>/);
-        // The plist stores this XML-escaped (buildLaunchdPlist), so a data dir
-        // containing `&` is on disk as `&amp;`. Decode before comparing, or the
-        // path would never equal itself and this guard would refuse a legitimate
-        // stop/start on any instance whose path contains an escaped character.
-        declared = m ? unescapeXml(m[1]) : null;
+        return m ? unescapeXml(m[1]) : null;
     }
     catch {
-        return; // unreadable — no evidence, don't block
+        return null;
     }
+}
+export function assertLaunchdServiceOwnedBy(dataDir, label, plistPath, action) {
+    const declared = readPlistRootPath(plistPath);
     if (declared === null)
         return;
     if (resolve(declared) === resolve(dataDir))
@@ -9835,18 +9909,15 @@ async function stopFlairProcess(port, dataDir) {
             assertLaunchdServiceOwnedBy(dataDir, label, plistPath, "stop");
             try {
                 const { execSync } = await import("node:child_process");
-                // Ensure the service is loaded (init writes the plist but doesn't load it)
-                try {
-                    execSync(`launchctl load "${plistPath}"`, { stdio: "pipe" });
-                }
-                catch { }
-                // Capture the current PID *before* stopping so callers that
+                // Capture the current PID *before* unloading so callers that
                 // immediately restart can verify exit. Without this, waitForHealth
                 // can race against the still-shutting-down old process and return
-                // success before KeepAlive brings the new one up.
+                // success before the new one comes up.
                 const oldPid = readHarperPid(dataDir);
+                // unload stops the job AND prevents KeepAlive from respawning it.
+                // launchctl stop alone is insufficient for a KeepAlive job (flair#874).
                 try {
-                    execSync(`launchctl stop ${label}`, { stdio: "pipe" });
+                    execSync(`launchctl unload "${plistPath}"`, { stdio: "pipe" });
                 }
                 catch { }
                 if (oldPid)
@@ -14068,6 +14139,19 @@ program
         console.error("Error: --admin-pass, --admin-pass-file, or FLAIR_ADMIN_PASS required for backup");
         process.exit(1);
     }
+    // flair#968: `flair backup > file.json` captures the progress report, not
+    // the archive (which goes to --output, defaulting to ~/.flair/backups/...).
+    // The result was exit 0 and a plausible-looking file of a few hundred bytes —
+    // a false success immediately before a destructive upgrade.
+    //
+    // When stdout is not a TTY, route progress output to stderr. The archive
+    // still goes to --output / the default path. This makes `flair backup >
+    // file.json` produce an EMPTY file — unmistakably not a valid archive —
+    // while leaving default-path callers (schedulers, cron) completely
+    // unaffected.
+    const log = process.stdout.isTTY
+        ? console.log.bind(console)
+        : console.error.bind(console);
     const auth = `Basic ${Buffer.from(`${adminUser}:${adminPass}`).toString("base64")}`;
     async function adminGet(path) {
         const res = await fetch(`${baseUrl}${path}`, {
@@ -14080,11 +14164,11 @@ program
         }
         return res.json();
     }
-    console.log("Fetching agents...");
+    log("Fetching agents...");
     const allAgents = await adminGet("/Agent/");
     const filterIds = opts.agents ? opts.agents.split(",").map((s) => s.trim()) : null;
     const agents = filterIds ? allAgents.filter((a) => filterIds.includes(a.id)) : allAgents;
-    console.log(`Fetching memories for ${agents.length} agent(s)...`);
+    log(`Fetching memories for ${agents.length} agent(s)...`);
     const memories = [];
     for (const agent of agents) {
         try {
@@ -14096,7 +14180,7 @@ program
             console.warn(`  Warning: could not fetch memories for ${agent.id}: ${err.message}`);
         }
     }
-    console.log("Fetching souls...");
+    log("Fetching souls...");
     const souls = [];
     for (const agent of agents) {
         try {
@@ -14124,11 +14208,11 @@ program
     const tmp = outputPath + ".tmp";
     writeFileSync(tmp, JSON.stringify(backup, null, 2) + "\n", "utf-8");
     renameSync(tmp, outputPath);
-    console.log(`\n${render.icons.ok} ${render.wrap(render.c.green, "Backup complete")}`);
-    console.log(render.kv("Agents", render.wrap(render.c.bold, String(agents.length))));
-    console.log(render.kv("Memories", render.wrap(render.c.bold, String(memories.length))));
-    console.log(render.kv("Souls", render.wrap(render.c.bold, String(souls.length))));
-    console.log(render.kv("Output", render.wrap(render.c.dim, outputPath)));
+    log(`\n${render.icons.ok} ${render.wrap(render.c.green, "Backup complete")}`);
+    log(render.kv("Agents", render.wrap(render.c.bold, String(agents.length))));
+    log(render.kv("Memories", render.wrap(render.c.bold, String(memories.length))));
+    log(render.kv("Souls", render.wrap(render.c.bold, String(souls.length))));
+    log(render.kv("Output", render.wrap(render.c.dim, outputPath)));
 });
 // ─── flair restore ────────────────────────────────────────────────────────────
 program
@@ -15137,4 +15221,4 @@ export { runCli, resolveKeyPath, buildEd25519Auth, readPortFromConfig, readOpsBi
 // Harper's own config — the per-instance port record (flair#914)
 harperConfigPath, readHarperConfig, readPortFromHarperConfig, persistDefaultInstallCoordinates, resolveTarget, resolveOpsTarget, resolveEffectiveOpsUrl, resolveOpsUrlFromTarget, signRequestBody, b64, b64url, program, api, VALID_PRESENCE_ACTIVITIES, MAX_TASK_LENGTH, MAX_WORKSPACE_FIELD_LENGTH, MAX_ORGEVENT_SUMMARY_LENGTH, MAX_ORGEVENT_DETAIL_LENGTH, isLocalBase, isLikelyRealSecret, shouldShowInlineSecretWarning, parseTokenFromFile, resolveLocalAdminPass, readAdminPassFileSecure, 
 // launchd label (flair#693)
-LEGACY_LAUNCHD_LABEL, launchdLabel, launchdPlistPath, resolveLaunchdLabel, migrateLegacyLaunchdLabel, ensureLaunchdServiceLoaded, };
+LEGACY_LAUNCHD_LABEL, launchdLabel, launchdPlistPath, cleanupLegacyLaunchdPlist, resolveLaunchdLabel, migrateLegacyLaunchdLabel, ensureLaunchdServiceLoaded, };