total-recall-brain 3.32.3 → 3.33.0

package/README.md

@@ -1,10 +1,12 @@
 # Total Recall
 
-[![Version](https://img.shields.io/badge/version-3.13.0-indigo.svg)](package.json)
+[![Version](https://img.shields.io/badge/version-3.32.4-indigo.svg)](package.json)
 [![License](https://img.shields.io/badge/license-MIT-emerald.svg)](LICENSE)
 
 **Portable personal memory for any IDE** — filesystem-native, database-free, open source.
 
+Plugin status (September 30, 2026): the tree contains five bundled plugins; standalone plugin repos retain their own documents. Current review found incomplete operations, fabricated preview data and unsupported readiness claims. The [audit](docs/projects/in-progress/PLUGIN_IMPLEMENTATION_CORRECTIONS/PLUGIN_IMPLEMENTATION_CORRECTIONS_AUDIT.md) and [correction tracker](docs/projects/in-progress/PLUGIN_IMPLEMENTATION_CORRECTIONS/PLUGIN_IMPLEMENTATION_CORRECTIONS_PROJECT_TRACKER.md) record evidence, owners and acceptance. Findings remain open. The version badge reflects the local manifest, not publication verification.
+
 Total Recall stores your rules, preferences, facts, and project knowledge as plain Markdown (SSSS). It compiles them into IDE instruction surfaces, runs a dream consolidation cycle, and lets agents enqueue background tasks. Host apps and product repos are equal implementations: **nothing is hard-coded to a specific codebase.**
 
 ```text

package/bin/total-recall.mjs

@@ -44,6 +44,7 @@ const COMMANDS = {
   init:     'init.mjs',
   setup:    'setup.mjs',
   start:    'start.mjs',
+  startup:  'startup.mjs',
   deploy:   'deploy.mjs',
   doctor:   'doctor.mjs',
   backup:   'backup.mjs',
@@ -60,6 +61,7 @@ const COMMANDS = {
   forget:   'forget.mjs',
   edit:     'edit.mjs',
   recall:   'recall.mjs',
+  context:  'context.mjs',
   search:   'recall.mjs',
   research: 'research.mjs',
   proposals: 'proposals.mjs',
@@ -132,6 +134,7 @@ function printHelp() {
     agent <cmd>         Process controller: spawn, list, logs, kill across mesh
     brain <cmd>         Register / ensure any project brain
     status / doctor     Health and diagnostics
+    startup check|ensure Readiness checks / missing managed local runtime start
 
   Optional:
     update [--apply]    Auto-download total-recall-brain for registered repos

package/docs/ARCHITECTURE.md

@@ -1,5 +1,7 @@
 # Total Recall — System Architecture
 
+> September 30, 2026: the older narrative below is historical background. The [current plugin correction architecture](projects/in-progress/PLUGIN_IMPLEMENTATION_CORRECTIONS/PLUGIN_IMPLEMENTATION_CORRECTIONS_ARCHITECTURE.md) records reviewed boundaries/gaps. Core-only portable memory remains an extraction target; five plugin directories are still bundled. Reviews fail the actual SSSS contract; DSH/search generators do not satisfy host consumption/UI generation. Declared previews do not establish operational UI. Earlier provider/model examples and readiness claims have not been recertified by this review. Resolve endpoints/models from configuration and verify official contracts before integration changes.
+
 > Verified against codebase: `relay.mjs`, `session-watcher.mjs`, `daemon-loop.mjs`, `surface.mjs`, `scheduler.mjs`, `embeddings.mjs`, `dispatch.mjs`, `vault-cache.mjs`, `config.mjs`, `sandbox.mjs`.
 > Last Updated: May 25, 2026
 
@@ -132,7 +134,7 @@ Total Recall partitions your brain memory into two virtual directories:
 
 ## 🌀 The Daemon Loop & Priority Task Scheduler
 
-The background daemon (`dream.mjs`) manages the **Continuous Intelligent Scheduler** ([src/core/scheduler.mjs](src/core/scheduler.mjs)) and task execution loops:
+The background daemon (`dream.mjs`) manages the **Continuous Intelligent Scheduler** ([src/core/scheduler.mjs](../src/core/scheduler.mjs)) and task execution loops:
 
 ### 1. Priority-Driven Queue Mechanics
 Tasks are enqueued dynamically as standard Markdown files (`type: task`) located in your vault's `scheduler/queue/` directory. The scheduler resolves an **Effective Priority** by multiplying the task's base `priority` (10 to 100) by a strictly-regulated **layer weight** (`LAYER_WEIGHTS`), prioritizing critical correctness and active developer guidance over passive exploration:
@@ -161,7 +163,7 @@ When the active developer queues are empty, the scheduler **automatically auto-g
 
 ## 🏃 hard Task Execution
 
-Pending priority tasks are executed by the **Task Runner** ([src/core/task_runner.mjs](src/core/task_runner.mjs)):
+Historical task-runner description: `src/core/task_runner.mjs` is absent from the current tree. Current executor source is [src/core/task-executors.mjs](../src/core/task-executors.mjs); the older routing description below has not been recertified:
 - **State Machine Updates**: Tasks transition transparently through states: `pending` → `in_progress` → `done` or `failed` (logging full exceptions to disk).
 - **Subagent Routing**: Routes tasks to specialized subagents depending on category:
   - `proactive-research`: Spawns autonomous web crawls and gathers cited research reports (`research.mjs`).
@@ -226,7 +228,7 @@ Total Recall consolidates all data folders under the meta-skill `skills/total-re
 
 ## 💬 Interactive CLI Agent Chat REPL
 
-Total Recall features a native terminal-based chat REPL (`npx total-recall chat`) running [src/cli/chat.mjs](src/cli/chat.mjs) that enables direct, conversational access to your active brain kernel from the terminal:
+Total Recall features a native terminal-based chat REPL (`npx total-recall chat`) running [src/cli/chat.mjs](../src/cli/chat.mjs) that enables direct, conversational access to your active brain kernel from the terminal:
 
 - **CLI-Agent Execution**: Conversation turns are dispatched dynamically to your prioritized **Unified Headless CLI Agents Registry** (`antigravity`, `gemini`, `claude`, `codex`) via `spawnSync`.
 - **Pre-flight Health Routing**: Upon starting, `chat` checks agent binary availability, selecting the highest-priority active subagent found in your `$PATH`.
@@ -318,9 +320,24 @@ For web dashboard access, Total Recall strictly avoids external third-party OAut
 - **TLS Protection**: The daemon blocks non-secure remote connections, requiring active HTTPS TLS layers (automated via Caddy proxying) for all production traffic.
 
 ### 2. Personal Access Token (PAT) Key Lifecycle Manager
-Headless tools, IDE editors, and session Relays authenticate via standard HTTP Bearer headers (`Authorization: Bearer tr_<token>`), managed by the **Keys Lifecycle Manager** ([src/server/keys.mjs](src/server/keys.mjs)):
+Headless tools, IDE editors, and session Relays authenticate via standard HTTP Bearer headers (`Authorization: Bearer tr_<token>`), managed by the **Keys Lifecycle Manager** ([src/server/keys.mjs](../src/server/keys.mjs)):
 - **SHA-256 Hash Preservation**: To eliminate credential exposure risk, the server **never writes tokens to disk in plaintext**. It persists only a timing-safe SHA-256 hash (`token_hash`) and a short identifying prefix (`token_prefix`) inside the owner-exclusive `config/keys.jsonl` file (written with `0o600` access modes). Diffs are cleanly git-versioned.
 - **Timing-Attack Protection**: Inbound Bearer PAT tokens are verified using cryptographically secure timing-safe comparisons (`crypto.timingSafeEqual`) on SHA-256 hashes, eliminating timing vector leaks.
 - **Role-Based Granular Scopes**: Validated PAT keys carry granular permission scopes (e.g., `chat:write`, `memory:read`, `sandbox:run`), permitting fine-grained access control boundaries for different editors.
 
 ```
+
+## Task context and local retrieval
+
+Connected instruction files contain a small routing bootstrap. Before action,
+`total-recall context` assembles required rules using `src/core/context-policy.mjs`.
+Activation uses existing validated memory tags, not semantic inference or storage
+scope. Unknown applicability stays required; oversized required sets report
+not-ready status. Contributions are admitted whole under complete-render token
+estimates. User-authored text outside generated blocks remains intact.
+
+`recall` defaults to local retrieval. Compilation builds `local-search.json` as a
+disposable full-text posting/locator projection; matching documents are hydrated
+individually after path and source checks. `--semantic` opts into the existing
+provider path. Static files do not erase context already loaded in a client.
+Core and generated expert skill entrypoints route into package-relative references.

package/docs/SSSS.md

@@ -0,0 +1,230 @@
+# SSSS — Structured Semantic Syntax System
+
+The official database-free schema specification utilized by the **Total Recall Local AI OS** for SSSS memory vaults, conflict quarantine files, session traces, and background research queue manifests.
+
+---
+
+## ⚡ 1. Core Principles
+
+The SSSS specification operates strictly on the following local architecture:
+
+| Principle | Specification |
+| :--- | :--- |
+| **No Relation Databases** | System configuration and memories must never live in Postgres, SQLite, or third-party database engines. |
+| **Markdown is Law** | Every primary memory component (rules, patterns, concepts, decisions) exists as a plain `.md` file. |
+| **Semantic YAML** | Every Markdown file MUST contain valid YAML frontmatter specifying its SSSS ontology and parameters. |
+| **Disposable Indexes** | Derived cached indexes (`graph-index.jsonl`, embeddings vectors) are disposable. They are fully rebuildable from the canonical Markdown vaults. |
+| **Git Provenance** | The vault directories are version-controlled, providing a transparent audit trail of intelligence shifts. |
+
+---
+
+## 📂 2. Virtual File System VFS Topography
+
+All user data directories are consolidated directly under the meta-skill `skills/total-recall/` path (in the global home directory layer `~/.agent/` or project repository layers `<repo>/.agent/`):
+
+```text
+.agent/
+└── skills/
+    └── total-recall/                  # THE BRAIN Root Folder
+        ├── memory-vault/              # Canonical SSSS Markdown Nodes
+        │   ├── invariants/            # absolute invariants (compiled to T1)
+        │   ├── patterns/              # best practices ("Always do X")
+        │   ├── anti-patterns/         # negative constraints ("Never do X")
+        │   ├── preferences/           # style and editor rules
+        │   ├── decisions/             # architectural histories
+        │   ├── concepts/              # domain models and definitions
+        │   └── facts/                 # verified evidence and research outputs
+        ├── memory-derived/            # Ephemeral cached indexes (JSONL/JSON)
+        ├── memory-inbox/              # Staging area for new nodes
+        │   ├── pending/               # observational drafts awaiting check
+        │   └── conflicts/             # quarantined rule collisions
+        └── sessions/                  # Ingested conversation history files (JSONL)
+```
+
+---
+
+## 📝 3. File Type Specifications
+
+### 3.1 Memory Node Specification (`type: memory`)
+
+Lives under `.agent/skills/total-recall/memory-vault/<category>/<slug>.md`.
+
+```markdown
+---
+type: memory
+slug: prefer-atomic-writes
+category: patterns
+title: "Always write files atomically (write-then-rename)"
+schema_version: 2
+status: active
+confidence: 0.95
+importance: 4
+modality: must
+priority: normal
+created: '2026-05-25T14:00:00Z'
+updated: '2026-05-25T14:02:15Z'
+last_accessed: '2026-05-25T14:05:00Z'
+source:
+  type: manual
+  session_id: active-session-id
+  evidence_count: 3
+supersedes: []
+superseded_by: null
+contradicts: []
+tags: [filesystem, reliability]
+routes_to_skills: [deploy]
+subject: agent
+predicate: use_atomic_write
+object: file_system
+sentiment_polarity: directive_must
+sentiment_target: file writes
+decay:
+  half_life_days: 180
+  access_count: 5
+---
+
+Always write to a temporary file first, then synchronously execute `fs.renameSync()` to overwrite the target path. `rename` is atomic on POSIX-compliant filesystems, preventing partial file corruption.
+```
+
+#### Category Taxonomy
+- `invariants/`: Absolute rules that must never be bypassed. Injected directly into Tier 1 shims.
+- `patterns/`: Best practices and standard operations ("Always do X").
+- `anti-patterns/`: Anti-patterns and constraints to avoid ("Never do X").
+- `preferences/`: Custom style preferences, styling tokens, or coding conventions.
+- `decisions/`: One-time design choices or architectural histories.
+- `concepts/`: Deep technical domain models, explanations, or definitions.
+- `facts/`: Validated factual assertions, library signatures, or platform constraints.
+
+#### Required Fields (Zod Schema Spec v2.0)
+- `type`: Must be strictly `"memory"`.
+- `slug`: kebab-case identifier that matches the filename on disk.
+- `category`: Must match the parent directory name.
+- `title`: A single-line human-readable summary of the rule.
+- `schema_version`: Integer specifying the schema standard (must be `2`).
+- `status`: Lifecycle enum: `active | draft | superseded | deprecated`.
+- `confidence`: Floating-point value between `0.0` and `1.0`.
+- `importance`: Integer between `1` and `5`.
+- `modality`: Enforcement strength enum: `must | must_not | should | should_not`.
+- `subject`/`predicate`/`object`: Structured triple checks used for O(1) semantic clash detection.
+- `sentiment_polarity`: Sentiment weight mapping: `directive_must | directive_must_not | descriptive | preference`.
+- `decay.half_life_days`: Interval in days before confidence halves from disuse.
+
+---
+
+### 3.2 Conflict Record Specification (`type: conflict`)
+
+Quarantined under `.agent/skills/total-recall/memory-inbox/conflicts/<conflict-id>.md` when two rules collide.
+
+```markdown
+---
+type: conflict
+conflict_id: conflict-2026-05-25-001
+status: auto-resolved
+new_slug: use-pm2-reload
+existing_slug: use-pm2-restart
+detected_at: '2026-05-25T14:10:00Z'
+reason: "Semantic clash on subject 'pm2' with cosine similarity 0.89 ≥ 0.75"
+resolution: "supersede: use-pm2-reload"
+resolved_at: '2026-05-25T14:10:01Z'
+---
+
+### Collision Targets:
+- New: `use-pm2-reload` (User-written preferences)
+- Existing: `use-pm2-restart` (Research-derived facts)
+```
+
+#### Collision Tier Resolution Matrix
+When conflicts are checked, the steering engine attempts auto-resolution in order:
+1. **Double Absolute**: Both are `priority: absolute` → **Quarantine** (Requires manual developer intervention via `total-recall resolve`).
+2. **Invariant Dominance**: One is a protected invariant → **Invariant wins** automatically.
+3. **User Authority**: One is user-created and the other is machine-derived → **User intent wins**.
+4. **Temporal Cascade**: Same source authority → **Most recent timestamp wins**.
+
+---
+
+### 3.3 Session Log Specification (`type: session`, JSONL)
+
+Stored under `.agent/skills/total-recall/sessions/<session-id>.jsonl`. One flat JSON object per conversation trace.
+
+```jsonl
+{"id":"a1","parentId":null,"type":"task","ts":"2026-05-25T14:00:00Z","content":"Provision REST router."}
+{"id":"a2","parentId":"a1","type":"tool_call","ts":"2026-05-25T14:00:15Z","content":"write_file src/server/routes/keys.mjs"}
+{"id":"a3","parentId":"a2","type":"observation","ts":"2026-05-25T14:01:00Z","content":"Keys router successfully mounted."}
+```
+
+---
+
+### 3.4 Skill Manifest Specification (`type: skill`)
+
+Stored under `.agent/skills/<name>/SKILL.md` (or consolidated namespaces).
+
+```markdown
+---
+type: skill
+name: code-quality
+description: "Verify TypeScript compiles and lint checks pass cleanly."
+needs: []
+token_budget: 4000
+last_compiled: '2026-05-25T14:02:00Z'
+schema_version: 1
+---
+
+# Code Quality
+
+## Authoritative Rules (compiled from memory-vault)
+
+<!-- BEGIN INJECTED MEMORY: do not edit by hand; rebuilt by total-recall surface -->
+<!-- @route: hybrid-bm25, generated_at: 2026-05-25T14:02:00Z -->
+
+- **never-run-tsc-directly** (confidence 1.0, importance 5):
+  NEVER run tsc or eslint directly. Always use standard skills scripts.
+
+<!-- END INJECTED MEMORY -->
+
+## Mechanics
+...
+```
+
+---
+
+### 3.5 Research Queue Task Specification (`type: task`)
+
+Lives under `.agent/skills/total-recall/scheduler/queue/<slug>.md`. Outlines background research agendas.
+
+```markdown
+---
+type: task
+priority: 85
+category: proactive-research
+target: concepts/cloudflare-worker-bindings.md
+estimated_calls: 30
+deadline: '2026-06-01'
+created_by: research-daemon
+status: pending
+progress: 0
+---
+
+## Objective
+Research Cloudflare Workers bind capabilities for environment mappings.
+
+## Steps
+1. Crawl official Cloudflare Worker environment binding documentation.
+2. Draft a conceptual node with code examples.
+3. Validate node against SSSS Zod v2 specifications.
+
+## Success Criteria
+- [ ] MD report has valid schema-v2 frontmatter
+- [ ] Zero lint issues detected
+```
+
+---
+
+## ⚡ 4. Three-Tier Memory Surfacing Hierarchy
+
+To ensure local IDE sessions remain lightning-fast and under token thresholds, memory nodes are progressively disclosed based on context relevance:
+
+| Surfacing Tier | File Location | Intended Scope / Purpose | Token Overhead | Latency |
+| :--- | :--- | :--- | :--- | :--- |
+| **Tier 1: Hot** | `INSTRUCTIONS.md` / `.cursorrules` | Absolute invariants that must govern every single prompt. | < 1,000 tokens | 0ms (pre-loaded) |
+| **Tier 2: Skills** | `SKILL.md` files | Behavioral rule capsules loaded dynamically on demand. | ≤ 7 rules / skill | ~100ms (file read) |
+| **Tier 3: Vault** | `memory-vault/` directory | Full vector-indexed catalog of experiences and facts. | Unlimited | ~500ms (semantic scan) |

package/docs/reference/CLI_INVENTORY.md

@@ -0,0 +1,77 @@
+---
+title: Total Recall CLI inventory
+tags: [cli, TR_CORE_FOCUS]
+---
+
+# CLI inventory (core | optional | legacy)
+
+Classification for the portable-memory product focus. Commands remain available unless noted; **core** is what default docs and onboarding emphasize.
+
+## Core
+
+| Command | Role |
+|---------|------|
+| `init` | Bootstrap global or project brain + openwiki |
+| `connect <client>` | Wire IDE / Obsidian / generic HTTP host |
+| `remember` / `forget` | Write-path memory |
+| `recall` / `search` | Read-path hybrid search |
+| `compile` / `rebuild` | Surfaces + indexes |
+| `dream` | Sleep-path consolidation |
+| `task` | Open daemon task envelope |
+| `daemon` | Background worker (ingest, tasks, system dream) |
+| `skill` | Registry, deploy, track, multi-repo sync |
+| `secret` / `secrets` | Secrets store + usage (not vault) |
+| `mesh` | Control server: nodes, ACL policy, Tailscale SSH |
+| `harness` | Multi-harness orchestration, task dispatch, council consensus |
+| `agent` | Process controller (spawn, list, logs, kill) across mesh |
+| `brain` | List / register / ensure / unregister project brains |
+| `status` | Health / brain connection summary |
+| `doctor` | Environment diagnostics |
+| `help` | Offline help topics |
+
+## Optional
+
+| Command | Role |
+|---------|------|
+| `research` | Long-horizon research queue (user-enqueued) |
+| `share` | Quick capture to remember/research |
+| `ingest` | IDE log / takeout ingest (also runs in dream/daemon) |
+| `import` | Import existing rule files into vault |
+| `export` | OKF bundle export |
+| `relay` | Ship local sessions to remote brain |
+| `setup` | Interactive provision wizard |
+| `deploy` | Host provision (Caddy, tunnels, services) |
+| `start` | Foreground brain server |
+| `backup` / `restore` | Encrypted VFS backup |
+| `config` | security.yml / budget.yml style settings |
+| `map` | Vault category visualization |
+| `key` | Dashboard/API PATs |
+| `hash-password` / `reset-password` | Dashboard auth helpers |
+| `lint` | SSSS schema validation |
+| `sync` | Pull remote brain instructions / vault git helpers |
+
+## Legacy / niche
+
+| Command | Role |
+|---------|------|
+| `chat` | Terminal chat against local server |
+| `collab` | Collab sandbox dev server |
+| `friction` | Watchdog latency analysis |
+| `upgrade` | Kernel model swap helpers |
+| `migrate` | Schema migrations |
+| `snapshot` | VFS snapshots |
+| `command` | Project-local custom CLI commands |
+
+## Not product surface (repo hygiene)
+
+Root one-off scripts (`fix-*.mjs`, `patch-*.mjs`) are **not** CLI entrypoints. Prefer deleting or moving them out of the package root in a follow-up hygiene PR; they are not part of the published command surface.
+
+## Env flags (power)
+
+| Env | Effect |
+|-----|--------|
+| `TR_IDLE_TASKS=1` | Allow daemon idle task invent |
+| `TR_POWER_EXECUTORS=1` | Enable system2 / cutoff / self-diagnosis style jobs |
+| `TR_REMOTE_VAULT_SYNC=1` | Optional remote vault content sync |
+| `TR_SECRETS_PASSWORD` | AES-encrypt secrets.enc |
+| `TR_SYNC_REPOS` | Extra multi-repo roots (`:` / `,` separated) |