total-recall-brain 3.37.0 → 3.39.0

package/README.md

@@ -1,158 +1,71 @@
 # Total Recall
 
-[![Version](https://img.shields.io/badge/version-3.32.4-indigo.svg)](package.json)
-[![License](https://img.shields.io/badge/license-MIT-emerald.svg)](LICENSE)
+**Portable personal memory for any IDE** — open source, with canonical SSSS Markdown and disposable search indexes.
 
-**Portable personal memory for any IDE** — filesystem-native, database-free, open source.
+Total Recall stores rules, preferences, facts and project knowledge. The portable runtime provides validated memory operations, local retrieval, task context and instruction surfaces. Optional capabilities run through explicitly installed plugins. Initialization creates an empty brain.
 
-Plugin status (September 30, 2026): the tree contains five bundled plugins; standalone plugin repos retain their own documents. Current review found incomplete operations, fabricated preview data and unsupported readiness claims. The [audit](docs/projects/in-progress/PLUGIN_IMPLEMENTATION_CORRECTIONS/PLUGIN_IMPLEMENTATION_CORRECTIONS_AUDIT.md) and [correction tracker](docs/projects/in-progress/PLUGIN_IMPLEMENTATION_CORRECTIONS/PLUGIN_IMPLEMENTATION_CORRECTIONS_PROJECT_TRACKER.md) record evidence, owners and acceptance. Findings remain open. The version badge reflects the local manifest, not publication verification.
+## Memory CLI
 
-Total Recall stores your rules, preferences, facts, and project knowledge as plain Markdown (SSSS). It compiles them into IDE instruction surfaces, runs a dream consolidation cycle, and lets agents enqueue background tasks. Host apps and product repos are equal implementations: **nothing is hard-coded to a specific codebase.**
-
-```text
-write   →  remember / session ingest
-sleep   →  dream  (consolidate, conflict, compile, prune)
-read    →  recall + compiled surfaces
-async   →  daemon task queue (agents may enqueue anything under policy)
-```
-
-```text
-your brain  =  ~/.agent/skills/total-recall/
-your project  =  <repo>/.agent/skills/total-recall/
-```
-
----
-
-## Quick start (IDE memory, ~2 minutes)
-
-No always-on LLM and no research daemon required.
-
-```bash
-cd /path/to/any-project
-npx total-recall init --project    # project brain + openwiki scaffold
-npx total-recall connect claude-code   # or: cursor | codex | gemini | aider | obsidian | http-api
-npx total-recall remember preference "Prefer clear, short answers."
-npx total-recall recall "short answers"
-npx total-recall compile           # rebuild INSTRUCTIONS.md inject blocks
+```sh
+npm install total-recall-brain
+npx total-recall-memory init --project
+npx total-recall-memory remember preference "Prefer clear, short answers." --project
+npx total-recall-memory recall "short answers" --local --project
+npx total-recall-memory context "Implement a memory change" --action edit,test --project
+npx total-recall-memory compile --project
 ```
 
-Injected rules sit only between:
+The package preserves the `total-recall` executable, dashboard and plugin scaffold during migration. The isolated memory CLI is available as `total-recall-memory`. Use `npx --package total-recall-brain total-recall-memory <command>` without a local installation.
 
-```text
-<!-- BEGIN INJECTED MEMORY: do not edit by hand; rebuilt by total-recall surface -->
-…
-<!-- END INJECTED MEMORY -->
-```
-
-Existing local rules outside those markers are never clobbered.
-
----
-
-## Core workflows
+`context` includes required rules before ranked knowledge. Check `ready: true`; exit 2 means the required rules exceed the selected budget. Increase the explicit budget or curate applicability before continuing.
 
-| Workflow | Commands |
-|----------|----------|
-| **Remember / recall** | `remember`, `forget`, `recall` |
-| **Dream (sleep)** | `dream` — or daemon system dream on empty queue |
-| **Defer work** | `task add "<intent>"` then `daemon start` |
-| **Skills across repos** | `skill track <path>`, `skill deploy`, `skill sync` |
-| **Secrets (not vault)** | `secret set\|list\|rotate\|usage\|check-surfaces` |
-| **Connect an IDE** | `connect <client>` |
+Project memory lives at `<repo>/.agent/skills/total-recall/`; global memory lives at `<home>/.agent/skills/total-recall/`. `--project` and `--global` select the layer. `AGENT_DIR` explicitly pins all layer operations to one agent root. Canonical memory remains separate between roots.
 
-### Multi-repo skills (any path you choose)
+Local retrieval needs no server or embedding provider. Set `TR_EMBEDDINGS_DISABLED=1` for provider-free operation. Semantic search requires a configured provider. Compilation preserves existing embeddings; mutation commands await local indexing.
 
-```bash
-npx total-recall skill track /path/to/any-app
-npx total-recall skill sync --repo /path/a --repo /path/b
-# or: export TR_SYNC_REPOS="/path/a:/path/b"
-```
-
-Roots come only from: project registry, install map, `TR_SYNC_REPOS`, CLI `--repo`, and cwd when it looks like a project.
+Run `total-recall --help` for the current commands. `import` preserves authored rule files and imports validated memory. `lint` validates memory and can validate installed manifests with `--plugins`; feature records require their owning validator.
 
-### Secrets
+## Optional capabilities
 
-```bash
-npx total-recall secret set some_api_key "$KEY" --provider example
-npx total-recall secret list                 # metadata only
-npx total-recall secret check-surfaces       # fail if values leaked into surfaces
+```sh
+total-recall plugin install /path/to/capability --global
+total-recall plugin list
+total-recall plugin info capability-id
+total-recall capability-command --help
+total-recall plugin remove capability-id --global
 ```
 
-Optional AES: `TR_SECRETS_PASSWORD`. Values never belong in vault markdown or openwiki.
+Local directories and Git sources use the same manifest validation, canonical installation records and artifact digest. Installed `cli` and `commands` declarations use an isolated child process with bounded output and timeout. Removal disables dispatch and preserves memory and recorded installation history; reinstalling the same artifact restores its digest.
 
-### Optional background daemon
+Plugins are trusted code with the operator's OS permissions. A child process protects the host from handler exits and output interference; it provides no OS sandbox. Background jobs, services, interfaces, distribution and feature-specific validation belong to the capability owner.
 
-```bash
-npx total-recall daemon start    # idle invent OFF unless TR_IDLE_TASKS=1
-npx total-recall task add "Extract decisions from last session" --cap vault:write
-```
+Dashboard plugin interfaces run in scripts-only frames with an opaque origin. They cannot access dashboard DOM/storage or fetch services directly. Self-contained owner modules receive themed variables and a `host.run(subcommand, args)` bridge restricted to manifest-declared commands and server authorization. Native command handlers retain the trust boundary described above. Never embed secret values in generated interface code.
 
----
+## Memory HTTP service
 
-## Dual-layer brain
+After installing the package locally:
 
-```text
-GLOBAL   ~/.agent/skills/total-recall/     identity, shared preferences
-PROJECT  <repo>/.agent/skills/total-recall/  repo facts, decisions, openwiki
+```sh
+AGENT_DIR=/path/to/agent TR_EMBEDDINGS_DISABLED=1 total-recall-memory init
+AGENT_DIR=/path/to/agent HOST=127.0.0.1 PORT=3900 node node_modules/total-recall-brain/src/server/memory-server.mjs
 ```
 
-CLI flags: `--global` / `--project`. Project wins on slug conflicts when both apply.
-
----
+In the source checkout, `npm run start:memory` selects the memory server. `npm start` preserves the existing dashboard service.
 
-## CLI inventory
+The service exposes memory, context, instructions, rules, scoped keys and authentication. `/health` reports initialization, node count, runtime and package version. Health requires authentication except direct local access. `/.well-known/total-recall.json` lists available memory endpoints. Configure a scoped credential with `total-recall key --help` or `generate-pat --help` before remote use.
 
-Commands are classified as **core** (default product path), **optional** (power features), or **legacy** (still shipped, not the focus).
+The `total-recall-brain/memory` export provides `createMemoryApp()`. Existing package and secret-store exports remain available for installed consumers.
 
-Full table: [`docs/reference/CLI_INVENTORY.md`](docs/reference/CLI_INVENTORY.md)
+## Source and migration
 
-| Tier | Examples |
-|------|----------|
-| **Core** | `init`, `connect`, `remember`, `forget`, `recall`, `compile`, `dream`, `task`, `daemon`, `skill`, `secret`, `brain`, `status`, `doctor` |
-| **Optional** | `research`, `relay`, `deploy`, `setup`, `backup`/`restore`, `chat`, `map`, `export`/`import`, `ingest` |
-| **Legacy / niche** | `collab`, `friction`, `upgrade`, `migrate`, `snapshot`, `command` |
-
-Default install story: **init → connect → remember/recall → dream**. No LLM required.
-
----
-
-## Optional full server
-
-For a local REST API + dashboard (not required for IDE memory):
-
-```bash
-npx total-recall start
-# or: npm start
-```
-
-Deploy/tunnels are optional (`deploy`, `setup`). Configure only what you need.
-
----
-
-## Architecture sketch
-
-- **SSSS vault** — Markdown + YAML frontmatter (filesystem SSOT)
-- **Openwiki** — human/agent long-form docs (ships with init)
-- **Surfaces** — compiled inject blocks for IDEs
-- **Dream** — consolidation cycle (deterministic first)
-- **Daemon tasks** — open envelope + capability policy
-- **Secrets** — separate store from vault
-
-Host apps consume TR the same way: PAT + brain URL (`connect http-api`) or local files.
-
----
-
-## Uninstall
-
-```bash
-npx total-recall uninstall
-```
+The source checkout and this compatible npm release retain existing product implementations during migration, including the dashboard, update command and plugin creation scaffold. The package gate verifies the complete memory file inventory alongside these compatibility surfaces and excludes private brain state. The isolated memory entrypoint does not load the product worker or dashboard. A default memory-only package requires a breaking release and completed capability migration.
 
-Project brains inside git repos are preserved when possible so you do not lose custom rules and memories.
+Versions ahead of the public npm release display **Preview features**. Host updates and older registered projects are reported separately; updating projects never downgrades a preview installation.
 
----
+The [prior product README](docs/reference/legacy-product-readme.md) preserves the original documentation. The [active core/plugin project](docs/projects/in-progress/TR_CORE_PLUGIN_SPLIT/TR_CORE_PLUGIN_SPLIT_PROJECT_TRACKER.md) records implementation and verification; its unfinished acceptance items are not release claims.
 
-## License
+## Development
 
-MIT — see [LICENSE](LICENSE).
+The source uses Node ESM and Vitest. Run the repository's configured background gates on its sanctioned test host. `npm run check:package` inspects npm's actual file inventory; `npm run check:ssss-registry` verifies the composed registry lock. Clean installed-artifact and native service checks complement the source suite.
 
-*Portable memory. Your files. Any IDE.*
+MIT license. See [LICENSE](LICENSE).

package/bin/total-recall-memory.mjs

@@ -0,0 +1,4 @@
+#!/usr/bin/env node
+import { main } from '../src/cli/memory-entry.mjs';
+try { await main(process.argv.slice(2)); }
+catch (err) { console.error(`Error: ${err.message}`); process.exitCode = 1; }

package/bin/total-recall.mjs

@@ -62,6 +62,7 @@ const COMMANDS = {
   edit:     'edit.mjs',
   recall:   'recall.mjs',
   context:  'context.mjs',
+  rules:    'rules.mjs',
   search:   'recall.mjs',
   research: 'research.mjs',
   proposals: 'proposals.mjs',
@@ -131,7 +132,7 @@ async function printHelp() {
 
   Core (default product path):
     init [--project]    Bootstrap global or project brain + openwiki
-    connect <client>    Wire IDE / Obsidian / http-api host
+    connect <client>    Wire IDE surface (Cursor, Claude, Codex, Grok, Replit, Lovable)
     remember / forget / edit   Write-path memory (SSSS vault)
     recall              Read-path hybrid search
     compile             Rebuild instruction surfaces
@@ -143,10 +144,11 @@ async function printHelp() {
     plugin <cmd>        Plugin manager: list, install, search, remove, info
     secret <cmd>        Secrets store + usage (not the vault)
     mesh <cmd>          Control-server mesh: nodes, ACL policy, Tailscale SSH
-    harness <cmd>       Meta-harness orchestration & council across IDEs & local LLMs
+    harness <cmd>       Meta-harness manager: list, usage (5h/weekly quotas), dispatch, council
     agent <cmd>         Process controller: spawn, list, logs, kill across mesh
     brain <cmd>         Register / ensure any project brain
     status / doctor     Health and diagnostics
+    brief               Session-start brief: state, runtime, rules, harnesses, trackers
     startup check|ensure Readiness checks / missing managed local runtime start
 ${pluginsSection}
   Optional:
@@ -279,26 +281,16 @@ async function main() {
       }
 
       // Check installed plugins for custom CLI commands declared in manifest
-      const { discoverPlugins, PACKAGE_ROOT } = await import('../src/core/plugin-loader.mjs');
+      const { discoverPlugins } = await import('../src/core/plugin-loader.mjs');
       const plugins = discoverPlugins(process.cwd());
       for (const p of plugins) {
         if (p.valid && p.manifest.cli?.command === command) {
           const handlerRel = p.manifest.cli.handler;
           if (handlerRel) {
-            const absHandler = path.resolve(p.dir, handlerRel);
-            if (fs.existsSync(absHandler)) {
-              // Match the host context supplied to dashboard/daemon handlers.
-              process.env.TR_PACKAGE_ROOT = PACKAGE_ROOT;
-              process.env.TR_PLUGIN_ID = p.id;
-              process.env.TR_PLUGIN_DIR = p.dir;
-              const handler = await import(absHandler);
-              if (handler.run) {
-                await handler.run(process.argv);
-              } else if (handler.default) {
-                await handler.default(process.argv.slice(3));
-              }
-              process.exit(process.exitCode ?? 0);
-            }
+            const { runPluginCommand } = await import('../src/core/plugin-runner.mjs');
+            const result = await runPluginCommand(p, { args: args.slice(1), stdin: 'inherit' });
+            if (result.output) process.stdout.write(result.output);
+            process.exit(result.ok ? 0 : (result.exitCode || 1));
           }
         }
       }

package/docs/ARCHITECTURE.md

@@ -1,6 +1,8 @@
 # Total Recall — System Architecture
 
-> September 30, 2026: the older narrative below is historical background. The [current plugin correction architecture](projects/in-progress/PLUGIN_IMPLEMENTATION_CORRECTIONS/PLUGIN_IMPLEMENTATION_CORRECTIONS_ARCHITECTURE.md) records reviewed boundaries/gaps. Core-only portable memory remains an extraction target; five plugin directories are still bundled. Reviews fail the actual SSSS contract; DSH/search generators do not satisfy host consumption/UI generation. Declared previews do not establish operational UI. Earlier provider/model examples and readiness claims have not been recertified by this review. Resolve endpoints/models from configuration and verify official contracts before integration changes.
+> October 7, 2026: [TR_CORE_PLUGIN_SPLIT](projects/in-progress/TR_CORE_PLUGIN_SPLIT/TR_CORE_PLUGIN_SPLIT_ARCHITECTURE.md) is the current architecture and single active implementation project. The target is portable memory with minimal safe plugin interfaces; feature implementations belong to their plugin owners. Current source still embeds feature routes, tools and daemon behavior. The npm files list already excludes the old bundled plugin directory. Older topology, counts, ownership and readiness statements below are historical and require current source evidence.
+
+> September 30, 2026: the older narrative below is historical background. The [current plugin correction architecture](projects/archived/consolidated-into-tr-core-plugin-split/PLUGIN_IMPLEMENTATION_CORRECTIONS/PLUGIN_IMPLEMENTATION_CORRECTIONS_ARCHITECTURE.md) records reviewed boundaries/gaps. Core-only portable memory remains an extraction target; five plugin directories are still bundled. Reviews fail the actual SSSS contract; DSH/search generators do not satisfy host consumption/UI generation. Declared previews do not establish operational UI. Earlier provider/model examples and readiness claims have not been recertified by this review. Resolve endpoints/models from configuration and verify official contracts before integration changes.
 
 > Verified against codebase: `relay.mjs`, `session-watcher.mjs`, `daemon-loop.mjs`, `surface.mjs`, `scheduler.mjs`, `embeddings.mjs`, `dispatch.mjs`, `vault-cache.mjs`, `config.mjs`, `sandbox.mjs`.
 > Last Updated: May 25, 2026

package/docs/reference/cli-reference.md

@@ -56,14 +56,14 @@ Wire an IDE editor or client application to your remote brain.
   ```bash
   npx total-recall connect <client> [options]
   ```
-- **Clients**: `claude-code`, `cursor`, `codex`, `antigravity`, `gemini`, `aider`, `pi`, `hermes`, `dsh`, `openclaw`, `http-api`, `obsidian`, `generic`
+- **Clients**: `claude-code`, `cursor`, `codex`, `antigravity`, `gemini`, `grok`, `replit`, `lovable`, `windsurf`, `devin`, `trae`, `goose`, `aider`, `pi`, `hermes`, `dsh`, `openclaw`, `http-api`, `obsidian`, `generic`
 - **Options**:
   - `--brain <url>`: remote brain API base URL.
   - `--token <pat>`: Personal Access Token to embed in generated config targets.
   - `--vault <path>`: Obsidian vault target directory path.
   - `--force`: Overwrite existing projection and rules files.
 
-*Symlink clients (`claude-code`, `codex`, `antigravity`, `gemini`, `pi`, `dsh`, `openclaw`) create a platform symlink linking editor shims (like `CLAUDE.md`, `AGENTS.md`, or `GEMINI.md`) to the compiled rules. File-based clients (`cursor`, `cline`, `hermes`, `aider`) write standard rule and memory files directly.*
+*Symlink clients (`claude-code`, `codex`, `antigravity`, `gemini`, `grok`, `replit`, `lovable`, `windsurf`, `devin`, `pi`, `dsh`, `openclaw`) create a platform symlink linking editor shims (such as `CLAUDE.md`, `AGENTS.md`, `GEMINI.md`, or `replit.md`) to the compiled rules. File-based clients (`cursor`, `cline`, `hermes`, `aider`) write standard rule and memory files directly.*
 
 ---
 
@@ -123,6 +123,15 @@ Malformed, conflicting, expired or source-mismatched curation falls back to the
 canonical rule. Never generate applicability from semantic ranking. Curation
 is private runtime state and must not be included in scaffolds or releases.
 
+Shorten rules per repo with `total-recall rules`: `audit` shows capsule tokens per
+action and the largest rules; `draft --out policy.json` scaffolds the policy with
+each rule's original text; shorten the directives by hand (keep every operative
+constraint), narrow `actions`, or exclude locally with `enabled:false` plus a
+`reason`; `apply policy.json` validates hashes and merges into the single
+`context:policy` decision; `verify` exits 1 on stale or orphan curation or an
+action capsule over budget; `prune [--apply] [--archive]` drops stale entries and
+lists expired, superseded or duplicate rules. Originals are never modified.
+
 ### `recall`
 Search local metadata and indexed full text across enabled memory layers.
 - **What it does**: Searches a disposable local index and reads only selected Markdown documents. Semantic enrichment is opt-in and exact hits bypass providers.
@@ -325,8 +334,9 @@ Meta-Harness orchestration across connected external IDEs, CLI tools, and local
   npx total-recall harness <command> [options]
   ```
 - **Commands**:
-  - `list`: Inspect all supported developer harnesses (`agy`, `claude`, `codex`, `gemini`, `ollama`) with availability and binary locations.
-  - `dispatch <id> [--node <node>] "<task>"`: Headlessly invoke a specific harness with task instructions. Supports cross-mesh execution via `--node` and non-interactive `pipe_stdin` for Ollama models (e.g. `gemma4:latest`).
+  - `list`: Inspect all supported developer harnesses (`agy`, `claude`, `codex`, `grok`, `gemini`, `ollama`) with availability, versions, currency, and authentication status.
+  - `usage [--json]`: Query harness account APIs for remaining 5-hour rolling session window and weekly quota percentages, version currency against upstream registries, and authentication freshness.
+  - `dispatch <id> [--node <node>] "<task>"`: Headlessly invoke a specific harness with task instructions. Supports cross-mesh execution via `--node` and non-interactive `pipe_stdin` for Ollama models (such as `gemma4:latest`).
   - `council "<task>"`: Execute multi-harness consensus deliberations concurrently across all available engines and compare responses.
 
 ---