@psiclawops/hypermem 0.7.0 → 0.8.1

package/ARCHITECTURE.md

@@ -1,13 +1,13 @@
 # hypermem Architecture
 
-_Agent-centric memory that outlives sessions._
+_Agent-centric memory that outlives sessions, backed by SQLite memory databases._
 
 ---
 
 ## Memory Layers
 
 ```
-L1  Redis (Hot)              Active session working memory
+L1  SQLite Cache (Hot)       Active session working memory
      │                       Slots: system, identity, messages, facts, context
      │                       Sub-millisecond reads, evicts on session end
      │                       Fleet cache: agent profiles, fleet summary
@@ -28,9 +28,13 @@ L4  Library DB               Fleet-wide structured knowledge
                               Knowledge graph (DAG links between entities)
 ```
 
+> Note: some internal method names and telemetry reasons still contain `redis`
+> for backward compatibility. The runtime hot layer is SQLite `:memory:` cache,
+> not an external Redis service.
+
 ## Database Schema
 
-### messages.db (per agent, schema v3)
+### messages.db (per agent, schema v10)
 - `agent_meta` — agent metadata
 - `conversations` — session tracking
 - `messages` — raw message log (text, tool calls, tool results)
@@ -46,7 +50,7 @@ L4  Library DB               Fleet-wide structured knowledge
 - `vec_index_map` — tracks what's been indexed (source_table, source_id, source_db)
 - `embedding_cache` — avoids redundant Ollama API calls
 
-### library.db (shared, schema v5)
+### library.db (shared, schema v19)
 - `facts` — verifiable claims with confidence, domain, expiry, supersedes chains
 - `knowledge` — domain/key/value structured data
 - `knowledge_links` — DAG edges between entities (fact↔fact, fact↔knowledge, etc.)
@@ -70,7 +74,7 @@ Assembles LLM prompts from all four layers with token budgeting:
 ```
 User message arrives
   │
-  ├── L1 Redis: system prompt, identity, cached slots
+  ├── L1 Hot cache: system prompt, identity, cached slots
   ├── L2 Messages: recent conversation history (budget-truncated)
   ├── L3 Vectors: KNN semantic recall on user's latest message
   │     └── Related facts/knowledge/episodes with relevance scores
@@ -104,14 +108,14 @@ Compositor behavior is tuned via parameters tracked in `tune/TUNING_REGISTRY.md`
 - **Compaction fence:** Per-conversation boundary protecting the LLM's recent tail from compaction. Only moves forward (monotone progress). No fence = no compaction (explicit opt-in).
 - **Preservation gate:** Nomic-space geometric verification that summaries stay faithful to source content. Centroid alignment + source coverage → combined score (threshold: 0.65).
 
-## Fleet Cache (Redis Hot Layer)
+## Fleet Cache (Hot Cache Layer)
 
 ```
 fleet:agent:{id}   — Composite profile: registry + capabilities + desired state
 fleet:summary      — Fleet-wide stats: agent count, drift count, tier breakdown
 ```
 
-- **Cache-aside** on reads: Redis first, SQLite fallback, warm on miss
+- **Cache-aside** on reads: hot cache first, SQLite fallback, warm on miss
 - **Write-through invalidation** on fleet mutations
 - **Hydration** on gateway startup: bulk-populate from library.db
 - TTL: agent profiles 10min, summary 2min
@@ -148,7 +152,7 @@ Visibility-tiered access model for cross-agent knowledge queries:
 
 `visibilityFilter()` resolves access levels using an `OrgRegistry` — a mapping of agents to tiers, orgs, and capabilities. Currently loaded from a hardcoded `defaultOrgRegistry()` in `cross-agent.ts`.
 
-**Known limitation:** This duplicates fleet structure that lives authoritatively in `fleet_agents` + `fleet_orgs` in library.db. Near-term roadmap item: replace with live-loaded registry from library.db, with the hardcoded version as cold-start fallback only.
+**Known limitation:** `defaultOrgRegistry()` duplicates fleet structure that lives authoritatively in `fleet_agents` + `fleet_orgs` in library.db. See [docs/ROADMAP.md](docs/ROADMAP.md) for the planned live-load migration.
 
 ### Unknown Agent Fallback (Restrictive Default)
 
@@ -164,10 +168,10 @@ This means:
 `plugin/src/index.ts` — OpenClaw context engine plugin (`hypercompositor`, fills `contextEngine` slot):
 
 ```
-gateway:startup     → Init hypermem, auto-rotate DBs, hydrate fleet cache
-agent:bootstrap     → Warm session (history, facts, profile → Redis)
+gateway:startup     → Init hypermem, auto-rotate DBs, seed fleet registry from workspace identities, hydrate fleet cache
+agent:bootstrap     → Warm session (history, facts, profile → hot cache)
 context:assemble    → Full four-layer prompt assembly within token budget
-agent:afterTurn     → Ingest new messages to SQLite + Redis, trigger background indexer
+agent:afterTurn     → Ingest new messages to SQLite + hot cache, trigger background indexer
 ```
 
 Registers with `ownsCompaction: true` — runtime skips legacy compaction entirely.
@@ -185,7 +189,7 @@ Registers with `ownsCompaction: true` — runtime skips legacy compaction entire
 
 ```
                     ┌──────────────────────────────────────────────────┐
-                    │             REDIS (L1 Hot Layer)                  │
+                    │      HOT CACHE (SQLite :memory: CacheLayer)       │
                     │                                                  │
                     │  hm:{a}:{s}:history  ── Session archive (250 cap │
                     │    (append-only)        at bootstrap, 1000 soft  │
@@ -210,34 +214,22 @@ Data Flow (current — P0 stabilized, window/cursor active):
   ▸ sessionExists() → skip if hot  compose()                    slice(prePromptCount)
   ▸ SQLite ─→ warmSession()        ─→ getHistory(limit) ✅      ─→ record*Message()
            ─→ pushHistory(250)     ─→ dedup by id               ─→ pushHistory(1, dedup)
-           ─→ Redis :history       ─→ budget assembly            ─→ Redis :history
-                                   ─→ write :window (120s)      ─→ invalidateWindow()
-                                   ─→ write :cursor (24h)       ─→ background indexer
+           ─→ cache history        ─→ budget assembly            ─→ cache history
+                                   ─→ write window bundle       ─→ invalidateWindow()
+                                   ─→ write cursor metadata     ─→ background indexer
                                    ─→ → runtime → provider
 
 ### Key Invariants
 
-1. Redis `history` is the warm archive. Append-only. Nothing reads it for direct submission.
-2. Redis `window` is the compositor's output cache. Written ONLY by `compose()`. Read ONLY by `assemble()`. Invalidated by `afterTurn`.
-3. Redis `cursor` tracks the newest message in the last window. Used by background indexer for high-signal mining.
+1. Hot-cache `history` is the warm archive. Append-only. Nothing reads it for direct submission.
+2. Hot-cache `window` is the compositor's output cache. Written ONLY by `compose()`. Read ONLY by `assemble()`. Invalidated by `afterTurn`.
+3. Hot-cache `cursor` tracks the newest message in the last window. Used by background indexer for high-signal mining.
 4. `warmSession()` seeds `history` only (capped at 250). Never writes `window`.
-5. `pushHistory()` tail-checks before append (no duplicate IDs in Redis list).
+5. `pushHistory()` tail-checks before append (no duplicate IDs in the hot-cache history list).
 6. `compose()` deduplicates history by `id` before budget assembly.
-7. `getHistory()` honors its `limit` parameter on BOTH Redis and SQLite paths.
-
-Design spec: `specs/HYPERMEM_QUEUE_SPLIT.md`
-Incident history: `specs/HYPERMEM_INCIDENT_HISTORY.md`
+7. `getHistory()` honors its `limit` parameter on BOTH hot-cache and SQLite paths.
 
-### Open Items (Tracked)
-
-| Item | WQ | Status | Notes |
-|---|---|---|---|
-| Cross-session context boundary markers | WQ-20260402-001 | 🟡 OPEN | `buildCrossSessionContext()` renders flat previews, no per-message boundaries or sender identity. Incident 6. |
-| Cursor durability (SQLite dual-write) | — | 🟡 DEFERRED | Cursor TTL = 24h. Dual-write to SQLite required before background indexer reads cursor. Gate 2. |
-| Plugin type unification | — | 🟡 DEFERRED | Plugin uses dynamic imports; can't use TS types from core. Shims are intentional. Structural change needed. |
-| Strict topic mode: legacy NULL backfill | — | 🟡 DEFERRED | After ≥2 weeks of topic detection in production, run backfill to assign `topic_id` to legacy NULL messages, then narrow `getRecentMessagesByTopic()` to exclude NULL. Gate: topic detection must be stable and coverage >80% of new messages before narrowing. Tracked in `specs/DEFERRED.md`. |
-| ACA Step 4 — retrieval stubs replace static files | — | 🔲 PENDING | `systemPromptAddition` carries governance doc chunks instead of embedding full workspace files. Blocked on Step 3 ✅ |
-| ACA Step 5 — governance context assembly | — | 🔲 PENDING | Full on-demand assembly replaces static prompt injection. Requires Step 4. |
+For open and deferred items, see [docs/ROADMAP.md](docs/ROADMAP.md).
 
 ### Runtime Contract
 
@@ -251,14 +243,14 @@ Incident history: `specs/HYPERMEM_INCIDENT_HISTORY.md`
 |---|---|---|---|
 | `index.ts` | ~1,340 | All | Facade — all public API |
 | `compositor.ts` | ~1,140 | L1-L4 | Prompt assembly + token budgeting + safety valve + window/cursor write |
-| `library-schema.ts` | ~780 | L4 | Library schema v5 + migrations |
+| `library-schema.ts` | ~780 | L4 | Library schema v19 + migrations |
 | `background-indexer.ts` | ~680 | L2-L4 | LLM-powered extraction framework |
 | `vector-store.ts` | ~600 | L3 | Semantic search + embedding |
 | `hybrid-retrieval.ts` | ~450 | L3-L4 | FTS5 + KNN with Reciprocal Rank Fusion |
 | `fleet-store.ts` | ~440 | L4 | Fleet registry + capabilities |
 | `db.ts` | ~440 | - | Database manager + rotation |
 | `knowledge-graph.ts` | ~420 | L4 | DAG traversal + shortest path |
-| `redis.ts` | ~530 | L1 | Redis operations, window cache, cursor, fleet cache |
+| `cache.ts` | ~700 | L1 | SQLite `:memory:` hot-cache operations, window cache, cursor, fleet cache |
 | `doc-chunker.ts` | ~400 | - | Section-aware markdown/file parser |
 | `work-store.ts` | ~400 | L4 | Work queue + FTS5 |
 | `provider-translator.ts` | ~390 | - | Neutral ↔ provider format conversion |
@@ -287,12 +279,12 @@ _Test count reflects assertions, not individual test blocks. Suites contain inli
 | Suite | Key coverage |
 |---|---|
 | smoke | End-to-end create/write/read/close, provider translation |
-| redis-integration | Redis ops, slots, history limits, window cache, cursor, warming, dedup |
+| redis-integration | Legacy suite name, covers hot-cache ops, slots, history limits, window cache, cursor, warming, dedup |
 | cross-agent | Cross-agent queries, fleet search, visibility tiers |
 | vector-search | Embedding, KNN, batch indexing |
 | library | All L4 collections (facts → desired state) |
 | compositor | Four-layer composition, budgets, providers, safety valve, Gate 1 |
-| fleet-cache | Redis fleet cache, hydration, cache-aside |
+| fleet-cache | Fleet hot-cache hydration and cache-aside behavior |
 | rotation | DB rotation, auto-rotate, collision handling |
 | knowledge-graph | DAG traversal, shortest path, analytics |
 | rate-limiter | Token bucket, priority, timeout, embedder |
@@ -301,6 +293,6 @@ _Test count reflects assertions, not individual test blocks. Suites contain inli
 ## Dependencies
 
 - `node:sqlite` (Node 22+ built-in) — zero-dependency SQLite
-- `ioredis` — Redis client
+- No external cache service dependency — hot cache is SQLite `:memory:`
 - `sqlite-vec` — optional, vector search extension
 - Ollama (localhost:11434) — optional, embedding generation

package/README.md

@@ -6,7 +6,7 @@
 
 ---
 
-hypermem is a runtime context engine for OpenClaw agents.
+hypermem is a SQLite-backed runtime context engine for OpenClaw agents.
 
 ```bash
 curl -fsSL https://raw.githubusercontent.com/PsiClawOps/hypermem/main/install.sh | bash
@@ -32,6 +32,8 @@ The difference isn't intelligence. It's what was in the prompt. Two failure mode
 
 **Compaction crunch.** Long sessions fill the context window. The runtime summarizes to make room. Specifics (tool output, exact decisions, file paths) are lost in the summary. The agent keeps running, but degraded.
 
+**Bloated context.** 128k tokens doesn't mean 128k of useful prompt. Without active curation, agents fill the window with stale history, redundant instructions, and memory that isn't relevant to this turn. A bigger context window just means more room to waste. The information is in the prompt somewhere, buried under content irrelevant to this turn.
+
 ---
 
 ## What OpenClaw provides today
@@ -53,18 +55,18 @@ OpenClaw also ships compaction safeguards and hybrid file search. That's a solid
 
 ## hypermem
 
-Four storage layers, sub-millisecond retrieval, no external database services required. Runs in-process with local SQLite storage and local Nomic embeddings by default, with optional hosted embeddings for L3.
+Four SQLite-backed memory databases, sub-millisecond retrieval, no external database services required. Runs in-process with local SQLite storage and local Nomic embeddings by default, with optional hosted embeddings for L3.
 
 | Layer | What it holds | Speed |
 |---|---|---|
-| **L1 In-memory** | What the agent needs right now. Identity, recent history, active state. | 0.08ms |
+| **L1 SQLite `:memory:`** | What the agent needs right now. Identity, recent history, active state. | 0.08ms |
 | **L2 History** | Every conversation, queryable and concurrent-safe. Per-agent. | 0.13ms |
 | **L3 Semantic** | Finds related content even when the words don't match. | 0.29ms |
 | **L4 Knowledge** | Facts, wiki pages, episodes, preferences. Shared across agents. | 0.09ms |
 
 Everything is retained. Storage survives session boundaries. The retry logic decision from last week, the deployment preferences from last month, the architecture choices from day one: all queryable, all available for composition.
 
-**Session warming.** Before the first turn fires, hypermem pre-loads the agent's full working state from the in-memory SQLite cache: recent history, facts ranked by confidence and recency, active topic context, cached embeddings for fast semantic recall. The agent's first reply draws from everything that was in scope at the end of the last session. The agent picks up where it left off.
+**Session warming.** Before the first turn fires, hypermem pre-loads the agent's full working state from its SQLite-backed memory stores and hot `:memory:` cache: recent history, facts ranked by confidence and recency, active topic context, cached embeddings for fast semantic recall. The agent's first reply draws from everything that was in scope at the end of the last session. The agent picks up where it left off.
 
 ---
 
@@ -77,7 +79,7 @@ Your agent has four layers of stored context, but what shows up in the prompt? H
 The hypercompositor queries all four layers in parallel on every turn and composes context within a fixed token budget. No transcript accumulates. No lossy transcript summarization. Amnesia isn't a storage problem; the memories exist, but nobody composed them into a coherent prompt. Compaction isn't inevitable; content that doesn't fit this turn stays in storage instead of being destroyed.
 
 **Bigger context windows don't help if you fill them with stale history.**
-128k tokens of stale history and irrelevant memory is worse than 32k of precisely selected content. 10 budget categories, priority-ordered, greedy-fill. Every token in the prompt earned its spot.
+128k tokens of stale history and irrelevant memory is worse than 32k of precisely selected content. 9 budget categories, priority-ordered, greedy-fill. Every token in the prompt earned its spot.
 
 ### What the model actually sees
 
@@ -104,14 +106,14 @@ What's in storage, not in this prompt:
   Change the topic, and the next turn pulls different content from the same storage.
 ```
 
-### Standard context engine vs. hypercompositor
+### OpenClaw default vs. hypercompositor
 
 ```
-Standard                                hypercompositor
+OpenClaw default                        hypercompositor
 ────────────────────────────────        ────────────────────────────────
 message → append to transcript          message → detect active topic
 transcript full → trim oldest           query 4 storage layers in parallel
-trimmed content → summarize (lossy)     budget allocator: 10 slots, fixed cap
+trimmed content → summarize (lossy)     budget allocator: 9 slots, fixed cap
 send transcript to model                tool compression by turn age
 model responds → append again           keystone guard + hyperform profile
                                         composed prompt → model
@@ -125,7 +127,7 @@ When it fills:                          When budget is exceeded:
   no recovery path                        change topic back → retrieved again
 ```
 
-| | Standard | hypercompositor |
+| | OpenClaw default | hypercompositor |
 |---|---|---|
 | Context source | Growing transcript only | Transcript + 3 additional storage layers |
 | When context fills | Trim + summarize (lossy) | Budget allocation (lossless storage) |
@@ -164,9 +166,9 @@ Different models have different default behaviors. GPT-5.4 tends toward 2x verbo
 
 Adaptation entries are stored in the `model_output_directives` table and matched by model ID using exact match, then glob pattern (longest wins), then wildcard fallback. Each entry contains:
 
-- **Calibration** — known model tendencies and specific adjustments (e.g., "2x verbosity: cut first drafts in half")
-- **Corrections** — hard/medium/soft severity rules applied in order (e.g., "No preamble before the answer")
-- **Task overrides** — per-task-type adjustments
+- **Calibration:** known model tendencies and specific adjustments (e.g., "2x verbosity: cut first drafts in half")
+- **Corrections:** hard/medium/soft severity rules applied in order (e.g., "No preamble before the answer")
+- **Task overrides:** per-task-type adjustments
 
 Model adaptation is only active at the `full` tier. At `light` and `standard`, model-specific corrections are suppressed.
 
@@ -196,7 +198,7 @@ Would you like me to go deeper on any of these?
 WITH outputProfile: "light":
 For a 128k window: reserve 14k for identity/system, target 46k for history, 10k for recent
 tool context, and leave ~30k as allocator reserve. hypermem handles slot competition
-automatically — set `reserveFraction` to your preferred floor and let the compositor fill.
+automatically. Set `reserveFraction` to your preferred floor and let the compositor fill.
 ```
 
 **Confabulation resistance** checks output against stored facts before claims are recorded. No LLM call. Pattern matching against the fact corpus, with confidence scoring and contradiction detection. Unsupported claims are flagged, contradictions surface in diagnostics, and a confabulation risk score is attached to the stored episode.
@@ -217,7 +219,7 @@ Most memory systems store what was said. hypermem synthesizes what was learned.
 
 When a topic goes quiet, hypermem compiles the thread into a structured wiki page: decisions, open questions, artifacts, participants. When the topic resurfaces, the agent gets a compact structured summary rather than a raw history replay.
 
-OpenClaw 2026.4.7 ships memory wiki for structured storage. hypermem goes further: wiki pages are synthesized automatically and injected by the compositor within token budget.
+OpenClaw 2026.4.7 ships memory wiki for structured storage. hypermem goes further: wiki pages are synthesized automatically and injected by the compositor within token budget, backed by SQLite memory databases instead of an external cache service.
 
 ### Subagents that hit the ground running
 
@@ -241,16 +243,7 @@ SQL queries that interpolate datetime values are fully parameterized. FTS5 trigg
 
 ## Pressure management
 
-hypermem composes context fresh on every turn, but a long-running session still accumulates history in its JSONL transcript. When that grows large enough, incoming tool results have nowhere to land and get silently stripped. Four automatic paths handle this:
-
-| Path | Trigger | Action |
-|---|---|---|
-| **Pressure-tiered tool-loop trim** | Any tool-loop turn | Measures projected occupancy before results land; trims large results at 80%+ and truncates the messages[] array for the current turn |
-| **AfterTurn trim** | Every turn at >80% | Pre-emptive headroom cut after the assistant replies, before the next turn arrives |
-| **Deep compaction** | compact() at >85% | Cuts in-memory cache to 25% budget and truncates JSONL to ~20% depth. Bypasses the normal reshape guard |
-| **Reshape guard** | Structured tool history on downshift | `canPersistReshapedHistory()` blocks a lower-context snapshot from overwriting the full JSONL history |
-
-**The one thing these paths cannot fix:** a session whose JSONL transcript on disk is already at 98% when the gateway restarts. The JSONL loads into runtime context before any compaction runs. Check `session_status` on startup. If you're above 85%, start a fresh session.
+hypermem manages context pressure automatically through four escalating paths. Most sessions never need manual intervention. For trigger thresholds and path details, see [Pressure management](#pressure-management-1) below.
 
 ---
 
@@ -383,7 +376,7 @@ Slot-level budget allocation is shown in the [hypercompositor diagram](#what-the
 
 ## Requirements
 
-**Current release: hypermem 0.5.6.** Changelog: [CHANGELOG.md](./CHANGELOG.md)
+**Current release: hypermem 0.8.1.** Changelog: [CHANGELOG.md](./CHANGELOG.md)
 
 | Requirement | Version | Notes |
 |---|---|---|
@@ -396,11 +389,11 @@ SQLite is a library, not a service. All four layers run in-process with no exter
 **Runtime version constants** (importable from the package):
 ```typescript
 import {
-  ENGINE_VERSION,        // '0.5.6'
+  ENGINE_VERSION,        // '0.8.1'
   MIN_NODE_VERSION,      // '22.0.0'
   SQLITE_VEC_VERSION,    // '0.1.9'
-  MAIN_SCHEMA_VERSION,   // 6  (hypermem.db)
-  LIBRARY_SCHEMA_VERSION_EXPORT, // 12 (library.db)
+  MAIN_SCHEMA_VERSION,   // 10 (messages.db)
+  LIBRARY_SCHEMA_VERSION_EXPORT, // 19 (library.db)
 } from '@psiclawops/hypermem';
 ```
 
@@ -410,29 +403,63 @@ Schema versions are stamped into each database on startup and checked on open. A
 
 ## Installation
 
+**Requirements:** Node.js 22+, OpenClaw with context engine plugin support. No standalone SQLite install needed (uses Node 22 built-in `node:sqlite`). Embedding provider is optional for first install.
+
+### From source
+
 ```bash
-git clone https://github.com/PsiClawOps/hypermem.git ~/.openclaw/workspace/repo/hypermem
-cd ~/.openclaw/workspace/repo/hypermem
+git clone https://github.com/PsiClawOps/hypermem.git
+cd hypermem
 npm install && npm run build
 npm --prefix plugin install && npm --prefix plugin run build
 npm --prefix memory-plugin install && npm --prefix memory-plugin run build
+npm run install:runtime
+```
+
+`install:runtime` stages the runtime payload into `~/.openclaw/plugins/hypermem` and prints the exact config commands to wire the plugins. Before running them, create the data directory and config:
+
+```bash
+mkdir -p ~/.openclaw/hypermem
+cat > ~/.openclaw/hypermem/config.json <<'JSON'
+{
+  "embedding": {
+    "provider": "none"
+  }
+}
+JSON
+```
+
+This sets lightweight mode (FTS5 keyword search, no embedding provider needed). Add an embedding provider later for semantic search without losing stored data. See [INSTALL.md](./INSTALL.md#embedding-providers) for options.
+
+Wire the plugins into OpenClaw:
 
+```bash
+openclaw config set plugins.load.paths "[\"$HOME/.openclaw/plugins/hypermem/plugin\",\"$HOME/.openclaw/plugins/hypermem/memory-plugin\"]" --strict-json
 openclaw config set plugins.slots.contextEngine hypercompositor
 openclaw config set plugins.slots.memory hypermem
-openclaw config set plugins.load.paths '["~/.openclaw/workspace/repo/hypermem/plugin","~/.openclaw/workspace/repo/hypermem/memory-plugin"]' --strict-json
 openclaw config set plugins.allow '["hypercompositor","hypermem"]' --strict-json
 openclaw gateway restart
 ```
 
-Or use the one-line installer:
+Verify (run from the repo clone directory):
+
+```bash
+openclaw plugins list                    # hypercompositor and hypermem should show as loaded
+node bin/hypermem-status.mjs --health    # confirms database initialization
+openclaw logs --limit 50 | grep hypermem # should show "hypermem initialized"
+```
+
+If you see `falling back to default engine "legacy"` in the logs, the install is not active. Check [INSTALL.md troubleshooting](./INSTALL.md#troubleshooting-clean-installs).
+
+### One-line installer
 
 ```bash
 curl -fsSL https://raw.githubusercontent.com/PsiClawOps/hypermem/main/install.sh | bash
 ```
 
-**Requirements:** Node.js 22+, OpenClaw with context engine plugin support, and either Ollama (local) or an OpenRouter API key (hosted) for embeddings. No standalone SQLite install is required for the documented repo install: hypermem uses the SQLite bundled with Node 22 via `node:sqlite`, and `sqlite-vec` provides the platform-specific extension through npm dependencies.
+Interactive: detects hardware, selects embedding tier, writes config, registers plugins.
 
-Full guide with deployment-specific options: **[INSTALL.md](./INSTALL.md)**
+Full guide with embedding tiers, reranker setup, fleet config, and tuning: **[INSTALL.md](./INSTALL.md)**
 
 ### Agent-assisted install
 
@@ -440,9 +467,15 @@ If you prefer, hand the install to your OpenClaw agent:
 
 > "Install hypermem following INSTALL.md. I'm running a [solo / multi-agent] setup."
 
+### Operator guides
+
+- **[docs/MEMORY_MD_AUTHORING.md](./docs/MEMORY_MD_AUTHORING.md)**, how to keep `MEMORY.md` compact, durable, and reviewable
+- **[docs/TUNING.md](./docs/TUNING.md)**, context assembly and output shaping profiles
+- **[docs/MIGRATION_GUIDE.md](./docs/MIGRATION_GUIDE.md)**, moving data in from existing memory systems
+
 ### Tuning
 
-Two independent surfaces: **context assembly** (what fills the context window) and **output shaping** (how the model writes). Pick a profile first — most deployments adjust one or two settings on top.
+Two independent surfaces: **context assembly** (what fills the context window) and **output shaping** (how the model writes). Pick a profile first. Most deployments adjust one or two settings on top.
 
 | Profile | Target window | Best for |
 |---|---|---|
@@ -550,6 +583,21 @@ node bin/hypermem-status.mjs --health        # health checks only (exit 1 on fai
 
 ---
 
+## Pressure management
+
+hypermem composes context fresh on every turn, but a long-running session still accumulates history in its JSONL transcript. When that grows large enough, incoming tool results have nowhere to land and get silently stripped. Four automatic paths handle this:
+
+| Path | Trigger | Action |
+|---|---|---|
+| **Pressure-tiered tool-loop trim** | Any tool-loop turn | Measures projected occupancy before results land; trims large results at 80%+ and truncates the messages[] array for the current turn |
+| **AfterTurn trim** | Every turn at >80% | Pre-emptive headroom cut after the assistant replies, before the next turn arrives |
+| **Deep compaction** | compact() at >85% | Cuts in-memory cache to 25% budget and truncates JSONL to ~20% depth. Bypasses the normal reshape guard |
+| **Reshape guard** | Structured tool history on downshift | `canPersistReshapedHistory()` blocks a lower-context snapshot from overwriting the full JSONL history |
+
+**The one thing these paths cannot fix:** a session whose JSONL transcript on disk is already at 98% when the gateway restarts. The JSONL loads into runtime context before any compaction runs. Check `session_status` on startup. If you're above 85%, start a fresh session.
+
+---
+
 ## Data directory
 
 ```text

package/dist/background-indexer.d.ts

@@ -17,8 +17,9 @@
  *   - Observable: logs extraction stats for monitoring
  */
 import type { DatabaseSync } from 'node:sqlite';
-import type { IndexerConfig, SessionCursor } from './types.js';
+import type { IndexerConfig, SessionCursor, MaintenanceTickDiagnostics } from './types.js';
 import { type DreamerConfig } from './dreaming-promoter.js';
+import { type ContradictionResolutionPolicy } from './contradiction-resolution-policy.js';
 import type { VectorStore } from './vector-store.js';
 export interface IndexerStats {
     agentId: string;
@@ -29,6 +30,12 @@ export interface IndexerStats {
     knowledgeUpserted: number;
     /** Number of superseded fact vectors tombstoned from the vector index this tick. */
     tombstoned: number;
+    /** Number of contradiction audits recorded for review this tick. */
+    contradictionAuditsLogged: number;
+    /** Number of old facts auto-superseded via contradiction policy this tick. */
+    contradictionsAutoSuperseded: number;
+    /** Number of old facts auto-invalidated via contradiction policy this tick. */
+    contradictionsAutoInvalidated: number;
     elapsedMs: number;
     /** Number of messages that were post-cursor (unseen by model, high-signal priority). */
     postCursorMessages: number;
@@ -51,6 +58,8 @@ export declare class BackgroundIndexer {
     private getCursor?;
     private readonly config;
     private readonly dreamerConfig;
+    private readonly globalWritePolicy;
+    private readonly contradictionPolicy;
     private intervalHandle;
     private running;
     private vectorStore;
@@ -60,7 +69,9 @@ export declare class BackgroundIndexer {
     private consecutiveFailures;
     /** True when the indexer is running in backoff mode due to repeated failures. */
     private inBackoff;
-    constructor(config?: Partial<IndexerConfig>, getMessageDb?: ((agentId: string) => DatabaseSync) | undefined, getLibraryDb?: (() => DatabaseSync) | undefined, listAgents?: (() => string[]) | undefined, getCursor?: CursorFetcher | undefined, dreamerConfig?: Partial<DreamerConfig>);
+    private readonly _conversationLastProcessed;
+    lastMaintenanceDiagnostics: MaintenanceTickDiagnostics | null;
+    constructor(config?: Partial<IndexerConfig>, getMessageDb?: ((agentId: string) => DatabaseSync) | undefined, getLibraryDb?: (() => DatabaseSync) | undefined, listAgents?: (() => string[]) | undefined, getCursor?: CursorFetcher | undefined, dreamerConfig?: Partial<DreamerConfig>, globalWritePolicy?: import('./types.js').GlobalWritePolicy, contradictionPolicy?: ContradictionResolutionPolicy);
     /**
      * Set the vector store for embedding new facts/episodes at index time.
      * Optional — if not set, indexer runs without embedding (FTS5-only mode).
@@ -146,5 +157,5 @@ export declare class BackgroundIndexer {
  * Create and start a background indexer connected to hypermem databases.
  * Used by the hook or a standalone daemon.
  */
-export declare function createIndexer(getMessageDb: (agentId: string) => DatabaseSync, getLibraryDb: () => DatabaseSync, listAgents: () => string[], config?: Partial<IndexerConfig>, getCursor?: CursorFetcher, vectorStore?: VectorStore, dreamerConfig?: Partial<DreamerConfig>): BackgroundIndexer;
+export declare function createIndexer(getMessageDb: (agentId: string) => DatabaseSync, getLibraryDb: () => DatabaseSync, listAgents: () => string[], config?: Partial<IndexerConfig>, getCursor?: CursorFetcher, vectorStore?: VectorStore, dreamerConfig?: Partial<DreamerConfig>, globalWritePolicy?: import('./types.js').GlobalWritePolicy): BackgroundIndexer;
 //# sourceMappingURL=background-indexer.d.ts.map

package/dist/background-indexer.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"background-indexer.d.ts","sourceRoot":"","sources":["../src/background-indexer.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;GAiBG;AAEH,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAChD,OAAO,KAAK,EAAiB,aAAa,EAAe,aAAa,EAAE,MAAM,YAAY,CAAC;AAK3F,OAAO,EAA2B,KAAK,aAAa,EAAE,MAAM,wBAAwB,CAAC;AAOrF,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,mBAAmB,CAAC;AA+CrD,MAAM,WAAW,YAAY;IAC3B,OAAO,EAAE,MAAM,CAAC;IAChB,iBAAiB,EAAE,MAAM,CAAC;IAC1B,cAAc,EAAE,MAAM,CAAC;IACvB,gBAAgB,EAAE,MAAM,CAAC;IACzB,aAAa,EAAE,MAAM,CAAC;IACtB,iBAAiB,EAAE,MAAM,CAAC;IAC1B,oFAAoF;IACpF,UAAU,EAAE,MAAM,CAAC;IACnB,SAAS,EAAE,MAAM,CAAC;IAClB,wFAAwF;IACxF,kBAAkB,EAAE,MAAM,CAAC;CAC5B;AAED;;;;GAIG;AACH,MAAM,MAAM,aAAa,GAAG,CAAC,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,KAAK,OAAO,CAAC,aAAa,GAAG,IAAI,CAAC,CAAC;AAEnG,MAAM,WAAW,cAAc;IAC7B,OAAO,EAAE,MAAM,CAAC;IAChB,aAAa,EAAE,MAAM,CAAC;IACtB,SAAS,EAAE,MAAM,CAAC;CACnB;AA+XD,qBAAa,iBAAiB;IAe1B,OAAO,CAAC,YAAY,CAAC;IACrB,OAAO,CAAC,YAAY,CAAC;IACrB,OAAO,CAAC,UAAU,CAAC;IACnB,OAAO,CAAC,SAAS,CAAC;IAjBpB,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAgB;IACvC,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAyB;IACvD,OAAO,CAAC,cAAc,CAA+C;IACrE,OAAO,CAAC,OAAO,CAAS;IACxB,OAAO,CAAC,WAAW,CAA4B;IAC/C,OAAO,CAAC,WAAW,CAAiC;IACpD,OAAO,CAAC,SAAS,CAAa;IAC9B,0EAA0E;IAC1E,OAAO,CAAC,mBAAmB,CAAa;IACxC,iFAAiF;IACjF,OAAO,CAAC,SAAS,CAAkB;gBAGjC,MAAM,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC,EACvB,YAAY,CAAC,GAAE,CAAC,OAAO,EAAE,MAAM,KAAK,YAAY,aAAA,EAChD,YAAY,CAAC,GAAE,MAAM,YAAY,aAAA,EACjC,UAAU,CAAC,GAAE,MAAM,MAAM,EAAE,aAAA,EAC3B,SAAS,CAAC,EAAE,aAAa,YAAA,EACjC,aAAa,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC;IA8BxC;;;OAGG;IACH,cAAc,CAAC,EAAE,EAAE,WAAW,GAAG,IAAI;IAIrC;;OAEG;IACH,KAAK,IAAI,IAAI;IAkDb;;;;;;;OAOG;IACH,OAAO,CAAC,gBAAgB;IAkDxB;;;OAGG;IACH,OAAO,CAAC,oBAAoB;IAiB5B;;OAEG;IACH,IAAI,IAAI,IAAI;IAOZ;;OAEG;IACG,IAAI,IAAI,OAAO,CAAC,YAAY,EAAE,CAAC;IA2IrC;;;;;;;;;OASG;YACW,YAAY;IA4M1B;;OAEG;IACH,OAAO,CAAC,oBAAoB;IA+B5B;;OAEG;IACH,OAAO,CAAC,uBAAuB;IAK/B;;OAEG;IACH,OAAO,CAAC,YAAY;IAsBpB;;OAEG;IACH,OAAO,CAAC,YAAY;IAWpB;;;OAGG;IACH,OAAO,CAAC,UAAU;IA8ClB;;OAEG;IACH,OAAO,CAAC,aAAa;IAarB;;;;;;;OAOG;IACG,sBAAsB,IAAI,OAAO,CAAC,IAAI,CAAC;IAgF7C;;OAEG;IACH,aAAa,CAAC,SAAS,EAAE,YAAY,GAAG,cAAc,EAAE;CAezD;AAID;;;GAGG;AACH,wBAAgB,aAAa,CAC3B,YAAY,EAAE,CAAC,OAAO,EAAE,MAAM,KAAK,YAAY,EAC/C,YAAY,EAAE,MAAM,YAAY,EAChC,UAAU,EAAE,MAAM,MAAM,EAAE,EAC1B,MAAM,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC,EAC/B,SAAS,CAAC,EAAE,aAAa,EACzB,WAAW,CAAC,EAAE,WAAW,EACzB,aAAa,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC,GACrC,iBAAiB,CAInB"}
+{"version":3,"file":"background-indexer.d.ts","sourceRoot":"","sources":["../src/background-indexer.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;GAiBG;AAEH,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAChD,OAAO,KAAK,EAAiB,aAAa,EAAe,aAAa,EAAE,0BAA0B,EAAE,MAAM,YAAY,CAAC;AAKvH,OAAO,EAA2B,KAAK,aAAa,EAAE,MAAM,wBAAwB,CAAC;AAQrF,OAAO,EAAgC,KAAK,6BAA6B,EAAE,MAAM,sCAAsC,CAAC;AAExH,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,mBAAmB,CAAC;AA+CrD,MAAM,WAAW,YAAY;IAC3B,OAAO,EAAE,MAAM,CAAC;IAChB,iBAAiB,EAAE,MAAM,CAAC;IAC1B,cAAc,EAAE,MAAM,CAAC;IACvB,gBAAgB,EAAE,MAAM,CAAC;IACzB,aAAa,EAAE,MAAM,CAAC;IACtB,iBAAiB,EAAE,MAAM,CAAC;IAC1B,oFAAoF;IACpF,UAAU,EAAE,MAAM,CAAC;IACnB,oEAAoE;IACpE,yBAAyB,EAAE,MAAM,CAAC;IAClC,8EAA8E;IAC9E,4BAA4B,EAAE,MAAM,CAAC;IACrC,+EAA+E;IAC/E,6BAA6B,EAAE,MAAM,CAAC;IACtC,SAAS,EAAE,MAAM,CAAC;IAClB,wFAAwF;IACxF,kBAAkB,EAAE,MAAM,CAAC;CAC5B;AAED;;;;GAIG;AACH,MAAM,MAAM,aAAa,GAAG,CAAC,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,KAAK,OAAO,CAAC,aAAa,GAAG,IAAI,CAAC,CAAC;AAEnG,MAAM,WAAW,cAAc;IAC7B,OAAO,EAAE,MAAM,CAAC;IAChB,aAAa,EAAE,MAAM,CAAC;IACtB,SAAS,EAAE,MAAM,CAAC;CACnB;AA+XD,qBAAa,iBAAiB;IAmB1B,OAAO,CAAC,YAAY,CAAC;IACrB,OAAO,CAAC,YAAY,CAAC;IACrB,OAAO,CAAC,UAAU,CAAC;IACnB,OAAO,CAAC,SAAS,CAAC;IArBpB,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAgB;IACvC,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAyB;IACvD,OAAO,CAAC,QAAQ,CAAC,iBAAiB,CAAyC;IAC3E,OAAO,CAAC,QAAQ,CAAC,mBAAmB,CAAgC;IACpE,OAAO,CAAC,cAAc,CAA+C;IACrE,OAAO,CAAC,OAAO,CAAS;IACxB,OAAO,CAAC,WAAW,CAA4B;IAC/C,OAAO,CAAC,WAAW,CAAiC;IACpD,OAAO,CAAC,SAAS,CAAa;IAC9B,0EAA0E;IAC1E,OAAO,CAAC,mBAAmB,CAAa;IACxC,iFAAiF;IACjF,OAAO,CAAC,SAAS,CAAkB;IACnC,OAAO,CAAC,QAAQ,CAAC,0BAA0B,CAA6B;IACxE,0BAA0B,EAAE,0BAA0B,GAAG,IAAI,CAAQ;gBAGnE,MAAM,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC,EACvB,YAAY,CAAC,GAAE,CAAC,OAAO,EAAE,MAAM,KAAK,YAAY,aAAA,EAChD,YAAY,CAAC,GAAE,MAAM,YAAY,aAAA,EACjC,UAAU,CAAC,GAAE,MAAM,MAAM,EAAE,aAAA,EAC3B,SAAS,CAAC,EAAE,aAAa,YAAA,EACjC,aAAa,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC,EACtC,iBAAiB,CAAC,EAAE,OAAO,YAAY,EAAE,iBAAiB,EAC1D,mBAAmB,CAAC,EAAE,6BAA6B;IAmCrD;;;OAGG;IACH,cAAc,CAAC,EAAE,EAAE,WAAW,GAAG,IAAI;IAIrC;;OAEG;IACH,KAAK,IAAI,IAAI;IAkDb;;;;;;;OAOG;IACH,OAAO,CAAC,gBAAgB;IAkDxB;;;OAGG;IACH,OAAO,CAAC,oBAAoB;IAiB5B;;OAEG;IACH,IAAI,IAAI,IAAI;IAOZ;;OAEG;IACG,IAAI,IAAI,OAAO,CAAC,YAAY,EAAE,CAAC;IA4LrC;;;;;;;;;OASG;YACW,YAAY;IA0Q1B;;OAEG;IACH,OAAO,CAAC,oBAAoB;IA+B5B;;OAEG;IACH,OAAO,CAAC,uBAAuB;IAK/B;;OAEG;IACH,OAAO,CAAC,YAAY;IAsBpB;;OAEG;IACH,OAAO,CAAC,YAAY;IAWpB;;;OAGG;IACH,OAAO,CAAC,UAAU;IA8ClB;;OAEG;IACH,OAAO,CAAC,aAAa;IAarB;;;;;;;OAOG;IACG,sBAAsB,IAAI,OAAO,CAAC,IAAI,CAAC;IAgF7C;;OAEG;IACH,aAAa,CAAC,SAAS,EAAE,YAAY,GAAG,cAAc,EAAE;CAezD;AAID;;;GAGG;AACH,wBAAgB,aAAa,CAC3B,YAAY,EAAE,CAAC,OAAO,EAAE,MAAM,KAAK,YAAY,EAC/C,YAAY,EAAE,MAAM,YAAY,EAChC,UAAU,EAAE,MAAM,MAAM,EAAE,EAC1B,MAAM,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC,EAC/B,SAAS,CAAC,EAAE,aAAa,EACzB,WAAW,CAAC,EAAE,WAAW,EACzB,aAAa,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC,EACtC,iBAAiB,CAAC,EAAE,OAAO,YAAY,EAAE,iBAAiB,GACzD,iBAAiB,CAInB"}