@tpsdev-ai/flair 0.20.1 → 0.22.0

package/README.md

@@ -67,13 +67,13 @@ If you need any of those specifically, use them. If you need crypto-pinned ident
 
 Anthropic shipped [Claude Dreams](https://platform.claude.com/docs/en/managed-agents/dreams) (research preview, April 2026) — async pipeline that reads a memory store + session transcripts and produces a curated output store: duplicates merged, stale entries replaced, insights surfaced. Validates the category: agent memory accumulates drift and needs cleanup.
 
-Flair ships both the on-demand curation surface (`flair rem rapid`) AND the scheduled nightly cycle in v0.9.0, per the [FLAIR-NIGHTLY-REM spec](specs/FLAIR-NIGHTLY-REM.md).
+Flair ships both the on-demand curation surface (`flair rem rapid`) AND the scheduled nightly cycle, per the [FLAIR-NIGHTLY-REM spec](specs/FLAIR-NIGHTLY-REM.md) and its [in-process distillation slice](specs/FLAIR-NIGHTLY-REM-SLICE-2-DISTILLATION.md). Config recipe (Ollama zero-key default, hosted-provider egress warning, clustered-deploy rules): [`docs/rem.md`](docs/rem.md).
 
-- **`flair rem rapid`** — on-demand reflection. `--focus {lessons_learned, patterns, decisions, errors}` mirrors Dreams' `instructions` parameter. Outputs *candidates*, not a wholesale store swap.
+- **`flair rem rapid`** — reflects and distills server-side by default, staging candidates in one bounded call. `--focus {lessons_learned, patterns, decisions, errors}` mirrors Dreams' `instructions` parameter. `--prompt-only` falls back to the bring-your-own-model handoff. Outputs *candidates*, not a wholesale store swap.
 - **`flair rem candidates` / `flair rem promote <id> --rationale "<why>"` / `flair rem reject <id>`** — review and promote distilled candidates with required rationale.
-- **`flair rem nightly enable [--at HH:MM]`** — scheduled automation via platform-native scheduler (launchd / systemd). Pre-cycle snapshot to `~/.flair/snapshots/<agent>/<iso>.tar.gz`. Maintenance step (soft-delete expired + soft-archive stale). Audit log to `~/.flair/logs/rem-nightly.jsonl`. `flair rem pause` / `resume` for emergency stop.
+- **`flair rem nightly enable [--at HH:MM]`** — scheduled automation via platform-native scheduler (launchd / systemd). Pre-cycle snapshot to `~/.flair/snapshots/<agent>/<iso>.tar.gz`. Maintenance (soft-delete expired + soft-archive stale) and distillation (staged candidates) both run each cycle. Audit log to `~/.flair/logs/rem-nightly.jsonl`. `flair rem pause` / `resume` for emergency stop.
 - **`flair rem restore <date> --apply`** — rewinds Harper state to a snapshot. Takes a pre-restore snapshot of current state first, so the rewind itself is reversible.
-- *Slice-3 (1.1)* — automated server-side distillation (requires pluggable LLM provider), trust-tier filter on REM input, cross-agent restore. Until then, distillation is operator-triggered via `flair rem rapid` followed by manual promote/reject.
+- *Next* — trust-tier filter on REM input, cross-agent restore. Until the trust-tier arc lands, the input filter stays scope-based (own agent, non-archived, non-permanent, scope window); the structural safety net is unchanged either way — candidates are staged, never auto-promoted.
 
 The substantive difference is the **promotion contract**:
 
@@ -94,12 +94,14 @@ An agent that can't remember what it learned yesterday, can't prove who it is to
 
 Flair fixes that with three primitives:
 
-- **Identity** — Ed25519 key pairs. Agents sign every request. No passwords, no API keys, no shared secrets. Cross-agent reads are refused at the server, not by client convention.
+- **Identity** — Ed25519 key pairs. Agents sign every request. No passwords, no API keys, no shared secrets. Write isolation (no agent can write as another) is enforced at the server, not by client convention; reads are open within the org for non-private memories by design (see [SECURITY.md](SECURITY.md)).
 - **Memory** — Persistent knowledge with semantic search (in-process [nomic-embed-text](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF), 768-dim, no API calls). Tiered durability (`permanent` / `persistent` / `standard` / `ephemeral`). Temporal validity. Decay-and-retrieval-aware composite scoring.
 - **Soul** — Personality, values, procedures. The stuff that makes an agent *that agent*. Re-injected every turn via the context-engine plugin so it doesn't drift across long sessions.
 
 Built on [Harper](https://harper.fast). Single process. No sidecars. Zero external API calls for embeddings. **[Supply-chain policy](docs/supply-chain-policy.md)** documents the bake-time + dep-pinning we run to keep this honest.
 
+See **[DESIGN.md](DESIGN.md)** for the design invariants behind these three primitives — why they're shaped the way they are.
+
 ## How It Works
 
 Flair is a native [Harper v5](https://harper.fast) application. Harper handles HTTP, persistence (RocksDB), and application logic in a single process.
@@ -152,7 +154,7 @@ Entity-to-entity triples with temporal bounds. Model structured knowledge — "A
 Subscribe to memory or soul changes via WebSocket/SSE. Useful for dashboards, cross-agent sync, or audit trails.
 
 ### Multi-Agent
-One Flair instance serves any number of agents. Each agent has its own keys, memories, and soul. Agents can't read each other's data without explicit access grants.
+One Flair instance serves any number of agents. Each agent has its own keys, memories, and soul. Reads are open within the org: any agent can read any other agent's non-private memory, no grant required — `private` is the one owner-only exception. See [DESIGN.md](DESIGN.md#access-model-open-within-the-org-closed-at-the-federation-edge) for the full model.
 
 ### OAuth 2.1 Authorization Server
 Built-in OAuth 2.1 server with PKCE, dynamic client registration, and a standards-compliant token endpoint. Agents and services can delegate auth to Flair without a separate IdP.
@@ -175,6 +177,9 @@ Passive extraction of entities from memory content on write. Entities are indexe
 ### Memory Bridges
 Pluggable import/export to foreign memory systems. Every agent-memory format (agentic-stack, Mem0, Letta, Anthropic memory, the next viral one) shouldn't need its own Flair PR. Bridges give you one contract with two shapes — a YAML descriptor for file-format targets or a code plugin for API targets — and a scaffold/test loop that lets an agent ship a working adapter in one pass. See [docs/bridges.md](docs/bridges.md).
 
+### Embedding benchmarks (`flair-bench`)
+A standalone benchmark you can run anywhere — no Flair install, no server, just Node and a GGUF file. It ships the same 126-query recall corpus and scoring Flair's own CI gates on, so its numbers are directly comparable: benchmark candidate embedding models (`--model-file`, batchable), get a model recommendation for the host you're on backed by the numbers it just measured (`recommend`), and optionally save a redacted, shareable result (`--share`) — including a `--label` for tagging host classes when characterizing fleet hardware. See [packages/flair-bench](packages/flair-bench/README.md).
+
 ## Quick Start
 
 ### Prerequisites
@@ -238,6 +243,29 @@ flair uninstall     # Remove the service (keeps data)
 flair uninstall --purge  # Remove everything including data and keys
 ```
 
+### Upgrading
+
+`flair status` and `flair doctor` both check your installed version against the latest published release (cached, offline-tolerant — this never blocks or fails either command) and print a nudge when you're behind:
+
+```
+⚠ flair 0.16.1 is behind — latest is 0.20.1 (4 releases behind). Upgrade: npm i -g @tpsdev-ai/flair@latest
+```
+
+To upgrade:
+
+```bash
+npm install -g @tpsdev-ai/flair@latest
+
+# Or check what's outdated across the whole toolchain (flair, flair-mcp, the
+# openclaw-flair plugin) without installing anything:
+flair upgrade --check
+
+# Then apply:
+flair upgrade
+```
+
+`flair upgrade` also restarts the running instance for you with `--restart`. If you deployed Flair to a Harper Fabric cluster (rather than running it locally), use `flair upgrade --target <fabric-url>` instead — see `flair upgrade --help`.
+
 ### Advanced / manual setup
 
 Prefer to drive each step yourself, or scripting an unattended setup? Run `flair init --no-mcp` to bootstrap the instance + agent without wiring any MCP clients, then drive each step on its own:
@@ -384,9 +412,9 @@ Auth is Ed25519 — sign `agentId:timestamp:nonce:METHOD:/path` with your privat
 
 ### Auth across surfaces
 
-The default, secure path everywhere is **Ed25519 per-agent**: each agent holds its own key (`~/.flair/keys/<agent>.key`), signs every request, and the server refuses cross-agent reads. The CLI, the `flair-mcp` server, and the OpenClaw / pi / Hermes plugins all use it.
+The default, secure path everywhere is **Ed25519 per-agent**: each agent holds its own key (`~/.flair/keys/<agent>.key`) and signs every request. That guarantees write isolation (no agent can write as another) and identity-verified reads — it does **not** mean cross-agent reads are refused: within one Flair instance, any verified agent can read any other agent's non-private memory by design (open-within-org read; the hard boundary is the federation edge, not intra-instance reads — see [SECURITY.md](SECURITY.md)). The CLI, the `flair-mcp` server, and the OpenClaw / pi / Hermes plugins all use this model.
 
-One exception: the **`n8n-nodes-flair`** community node authenticates with the Harper **admin password** (Basic auth), which grants **whole-instance read/write** access — not per-agent isolation. That's acceptable only on a single-tenant, operator-controlled n8n with trusted workflow inputs; otherwise prefer the CLI/SDK Ed25519 path. Full breakdown in **[docs/auth.md](docs/auth.md#auth-across-surfaces-read-this-first)**.
+One exception: the **`n8n-nodes-flair`** community node authenticates with the Harper **admin password** (Basic auth), which bypasses agent scoping entirely — including read of other agents' `visibility: private` memories and write-as-anyone, not just the org-wide non-private reads an Ed25519 identity already gets. That's acceptable only on a single-tenant, operator-controlled n8n with trusted workflow inputs; otherwise prefer the CLI/SDK Ed25519 path. Full breakdown in **[docs/auth.md](docs/auth.md#auth-across-surfaces-read-this-first)**.
 
 ## Architecture
 
@@ -457,10 +485,10 @@ See [SECURITY.md](SECURITY.md) for the full security model, threat analysis, and
 
 **Key points:**
 - Ed25519 cryptographic identity — agents sign every request
-- Collection-level data isolation — agents can't read each other's memories
+- Reads are open within the org — any agent can read any other agent's non-private memory, no grant required; `private` is the one owner-only exception (see [DESIGN.md](DESIGN.md))
 - Admin password auto-generated on init, stored at `~/.flair/admin-pass` (mode 0600)
 - Key rotation via `flair agent rotate-key`
-- Cross-agent access requires explicit grants
+- Writes are always agent-scoped — an agent can only write its own records
 
 ## Development
 
@@ -479,7 +507,7 @@ Integration tests spin up a real Harper instance on a random port, run the test
 | Category | Tests | What's covered |
 |----------|-------|----------------|
 | **Auth & Identity** | auth-middleware, auth-scoping, key-paths-and-rotation | Ed25519 signature verification, agent isolation, key rotation |
-| **Memory** | data-scoping, backup-restore, agent-remove-and-grants | Cross-agent access denied, data durability, grant lifecycle |
+| **Memory** | data-scoping, backup-restore, agent-remove-and-grants | Private-memory exclusion, open within-org read scoping, data durability, grant lifecycle |
 | **Content Safety** | content-safety | Prompt injection detection, identity hijacking, format injection, exfiltration patterns |
 | **Search** | temporal-scoring, embeddings | Temporal decay, relevance scoring, embedding generation |
 | **Rate Limiting** | rate-limiter | Per-agent rate limiting, bucket isolation |

package/SECURITY.md

@@ -2,8 +2,10 @@
 
 ## Overview
 
-Flair uses cryptographic identity (Ed25519) for agent authentication and
-collection-level scoping to enforce data isolation between agents.
+Flair uses cryptographic identity (Ed25519) for agent authentication. Within
+an org, memory reads are open — agents see each other's non-private memories —
+while writes are isolated per agent and private memories stay strictly
+owner-only. The hard access boundary is the federation edge (cross-instance).
 
 ## Authentication
 
@@ -32,30 +34,49 @@ built-in HTTP Basic auth:
 Authorization: Basic <base64(admin:password)>
 ```
 
-The admin password is set via `HDB_ADMIN_PASSWORD` environment variable
-at Harper startup. **It is never stored on the filesystem by Flair.**
+The admin password is generated on `flair init` and stored at
+`~/.flair/admin-pass` (mode `0600`, owner-only) — it can also be set
+explicitly via `--admin-pass-file`, `FLAIR_ADMIN_PASS`, or
+`HDB_ADMIN_PASSWORD`. It is never printed to the console. See
+[docs/secrets-and-keys.md](docs/secrets-and-keys.md) for the full
+precedence and rotation model.
+
+The Harper **operations API** is the admin/management surface. Network
+requests to it require admin Basic auth: Flair ships with
+`authorizeLocal: false`, so an unauthenticated loopback request is
+rejected (`401`) instead of being auto-elevated to super_user. The same
+API is also reachable over a Unix **domain socket**
+(`~/.flair/data/operations-server`, owner-write-only), which Harper
+treats as an inherent local-admin channel — a request over the socket is
+authorized as super_user **without** credentials. This is by design (the
+admin-password rotation flow relies on it) and is constrained to the box
+owner; see Threats NOT Mitigated below.
 
 ## Data Scoping
 
-### Memory Isolation
+### Memory Read/Write Model
 
-Each agent can only read and write its own memories. This is enforced at
-the database layer (Harper resource `search()` override), not application
-logic:
+Reads are open **within an org**; writes are isolated per agent. Enforced at
+the database layer (Harper resource `search()` override via a single
+centralized read-scope rule), not application logic:
 
-- `GET /Memory/` — returns only the authenticated agent's memories
-- `POST /Memory` — writes with the authenticated agent's ID
-- `POST /MemorySearch` — searches only the authenticated agent's memories
+- `GET /Memory/` — returns the authenticated agent's own memories (any
+  visibility) plus every **non-private** memory in the org
+- `POST /MemorySearch` — same read scope: own memories plus all non-private
+- `POST /Memory` — writes are attributed to the authenticated agent; an
+  agent cannot write as another (write isolation)
 
-### Cross-Agent Access
+**Private memories are strictly owner-only** — a memory written with
+`visibility: private` is returned only to its author, never to another agent.
 
-Explicit grants allow one agent to read another's memories:
+### Cross-Agent Access (within an org)
 
-```
-MemoryGrant { fromAgentId, toAgentId, scope: "read" }
-```
-
-Grants are created by admin or by the granting agent.
+Within an org, an agent reads every non-private memory directly — no grant
+required. This is deliberate: the goal is relevance (surface what's useful to
+the org) over secrecy. A writer keeps something private by marking it
+`visibility: private`, which keeps it strictly owner-only. The access boundary
+that stays hard is the **federation edge** (cross-instance / cross-org), not
+within-org reads.
 
 ### Admin Bypass
 
@@ -88,8 +109,12 @@ public key in Flair, and backs up the old key as `<agentId>.key.bak`.
 
 - **Agent impersonation:** Ed25519 signatures prevent one agent from
   acting as another without possessing the private key.
-- **Cross-agent data leakage:** Collection-level scoping ensures agents
-  can only query their own data.
+- **Private-memory leakage:** memories marked `visibility: private` are
+  strictly owner-only and never returned to another agent. Non-private
+  memories are intentionally readable within the org (by design, not
+  leakage); the cross-instance boundary is the federation edge.
+- **Write forgery:** writes are attributed to the authenticated agent —
+  an agent cannot write memories as another.
 - **Replay attacks:** 30-second window + nonce deduplication.
 - **Privilege escalation via SQL:** SQL and GraphQL endpoints blocked
   for non-admin agents.
@@ -105,6 +130,13 @@ public key in Flair, and backs up the old key as `<agentId>.key.bak`.
   process could extract them.
 - **Network sniffing:** Flair uses HTTP by default. Use HTTPS in
   production or restrict to localhost.
+- **Same-OS-user ops-API socket access:** the operations-API domain
+  socket (`operations-server`, owner-write-only) is an unauthenticated
+  super_user channel by Harper design — any process running as the box
+  owner can perform admin operations through it (the admin-password
+  rotation flow uses this path). Owner-write permissions keep it
+  unreachable by other OS users, co-tenants, and the network; isolate
+  untrusted workloads to separate OS users.
 
 ## Recommendations
 

package/config.yaml

@@ -12,11 +12,38 @@ graphqlSchema:
 jsResource:
   files: dist/resources/*.js
 
-'harper-fabric-embeddings':
-  package: 'harper-fabric-embeddings'
-  modelName: nomic-embed-text
+# Phase 1 (flair#504): embeddings now run through Harper's native
+# models.embed() facade, backed by harper-fabric-embeddings registered as the
+# `embedding` backend. That registration is NOT configured here, and (as of
+# flair#694) is no longer config-driven anywhere: a `models:` block in THIS
+# file would silently never be read (this application always loads with
+# `isRoot: false` — components/componentLoader.ts gates `bootstrapModels()`
+# on `isRoot`), and the earlier fix for that — reasserting the block into the
+# Harper INSTANCE-ROOT config via the HARPER_CONFIG env var on every spawn —
+# turned out to PERSIST that block into harper-config.yaml, which an
+# older/downgraded build's boot (never having set the env var) would tear
+# down to an invalid empty shell and refuse to boot against (flair#694; see
+# flair#695 for the invariant this violated). The registration now happens
+# in-process instead: `dist/resources/embeddings-boot.js` (built from
+# resources/embeddings-boot.ts, loaded by the `jsResource` glob below like
+# every other file under resources/) calls harper-fabric-embeddings'
+# `register()` factory directly on every boot — nothing is ever written to
+# the config file, so there is nothing for a downgrade to trip over. This
+# also means a `package:`-style sub-component entry
+# (`'harper-fabric-embeddings': { package: ... }`, which used to sit here and
+# drove `handleApplication`) still must not be reintroduced alongside it —
+# that hook populates a SEPARATE raw-API engine, not `models.embed()`, and
+# running both would double-init (two separate EmbeddingEngine instances:
+# one unused, one backing models.embed).
 
 authentication:
-  # Default secure. Set to false for local development only.
-  authorizeLocal: true
+  # Default secure (flair#654): a credential-less loopback request to the
+  # Harper ops API (:9925) is no longer auto-authorized as super_user. Local
+  # admin operations now require a real credential — ~/.flair/admin-pass
+  # (written by `flair init`), --admin-pass, or FLAIR_ADMIN_PASS. flair's own
+  # application-layer resources were already immune to this forgery (#655's
+  # credential-evidence gate); this closes the remaining gap below it, in the
+  # raw Harper ops API itself. Set true only for local development, at your
+  # own risk.
+  authorizeLocal: false
   enableSessions: true