@psiclawops/hypermem 0.5.4 → 0.5.6

package/README.md

@@ -116,7 +116,7 @@ send transcript to model                tool compression by turn age
 model responds → append again           keystone guard + hyperform profile
                                         composed prompt → model
      ┌──────────────────┐               model responds → afterTurn ingest
-     │  loop until full  │               → write back to all 4 layers
+     │ loop until full  │               → write back to all 4 layers
      └──────────────────┘
 
 When it fills:                          When budget is exceeded:
@@ -127,7 +127,7 @@ When it fills:                          When budget is exceeded:
 
 | | Standard | hypercompositor |
 |---|---|---|
-| Context source | Growing transcript | 4 independent storage layers |
+| Context source | Growing transcript only | Transcript + 3 additional storage layers |
 | When context fills | Trim + summarize (lossy) | Budget allocation (lossless storage) |
 | Old decisions | Lost after compaction | Retrievable via keystones + semantic recall |
 | Topic changes | All history competes equally | Scoped retrieval by active topic |
@@ -136,21 +136,45 @@ When it fills:                          When budget is exceeded:
 
 High-signal turns are marked as keystones and survive pressure trimming ahead of ordinary history.
 
+The compositor fills 9 slots in priority order (system prompt → identity → hyperform → history → facts → wiki → semantic recall → cross-session → action summary). Each slot consumes tokens from the remaining budget before the next slot runs. Slots that don't fit this turn stay in storage, not destroyed.
+
+For the full fill order, budget formula, and all configuration knobs, see **[Tuning](#tuning)** below and **[docs/TUNING.md](./docs/TUNING.md)**.
+
 ---
 
 ## hyperform
 
 Raw model output has two problems. It drifts from your standards (sycophancy, hedging, pagination, formatting) and it drifts from your facts (confabulation, contradiction, stale claims). hyperform handles both: normalization enforces consistency, confabulation resistance checks output against what's actually stored.
 
-**Normalization** shapes output to match a profile you define. Three presets ship with hypermem:
+Consistent output isn't just aesthetic. A model that paginates short answers, preambles with filler, or inflates lists uses more output tokens per turn. Over hundreds of turns, that compounds into real cost. hyperform directives compress output at the source: fewer tokens generated means lower API spend per session, and less context pressure for subsequent turns.
+
+### Behavior standards
 
-| Profile | Tokens | Covers |
+Behavior standards define how your agents write. Anti-sycophancy rules prevent filler openings. Density targets compress answers. Anti-pattern bans remove common AI markers (em dashes, AI vocabulary, inflated significance). These rules apply to all models equally.
+
+| Tier | Tokens | What it injects |
 |---|---|---|
-| `light` | ~100 | Anti-sycophancy, em dash ban, AI vocab ban, length targets, evidence calibration |
-| `standard` | ~250 | Full directive set plus pagination rules and hedging policy |
-| `full` | ~400 | Complete normalization with full directive set and model-specific calibration |
+| `light` | ~100 | 9 standalone directives: lead with answer, no sycophancy, no em dashes, AI vocab ban, length targets (simple/analysis/code), filler ban, no pagination of short answers, evidence calibration, numbers over adjectives. No database required. |
+| `standard` | ~250 | Full directive set from the `fleet_output_standard` table: structural rules, density targets per task type, anti-patterns, format rules, compression ratios, voice directives, and task-context overrides. Falls back to `light` directives if no record exists. |
+| `full` | ~250 + adaptation | Same directives as `standard`, plus model adaptation (see below). |
+
+### Model adaptation
+
+Different models have different default behaviors. GPT-5.4 tends toward 2x verbosity and long lists. Claude Opus defaults to hedging and preambles. Gemini produces bulleted summaries where prose would be more direct. Model adaptation corrects for these tendencies per model.
+
+Adaptation entries are stored in the `model_output_directives` table and matched by model ID using exact match, then glob pattern (longest wins), then wildcard fallback. Each entry contains:
+
+- **Calibration** — known model tendencies and specific adjustments (e.g., "2x verbosity: cut first drafts in half")
+- **Corrections** — hard/medium/soft severity rules applied in order (e.g., "No preamble before the answer")
+- **Task overrides** — per-task-type adjustments
+
+Model adaptation is only active at the `full` tier. At `light` and `standard`, model-specific corrections are suppressed.
+
+The `model_output_directives` table starts empty. You populate it with corrections for the models you run. See [docs/TUNING.md](./docs/TUNING.md#creating-custom-entries) for the schema and SQL examples.
 
-The same prompt, GPT-5.4, with and without `outputProfile: "light"`:
+### Before and after
+
+The same prompt, GPT-5.4, with and without `hyperformProfile: "light"`:
 
 ```
 Prompt: "How should I size my context window budget for a long-running agent session?"
@@ -172,11 +196,13 @@ Would you like me to go deeper on any of these?
 WITH outputProfile: "light":
 For a 128k window: reserve 14k for identity/system, target 46k for history, 10k for recent
 tool context, and leave ~30k as allocator reserve. hypermem handles slot competition
-automatically -- set contextWindowReserve to your preferred floor and let the compositor fill.
+automatically — set `reserveFraction` to your preferred floor and let the compositor fill.
 ```
 
 **Confabulation resistance** checks output against stored facts before claims are recorded. No LLM call. Pattern matching against the fact corpus, with confidence scoring and contradiction detection. Unsupported claims are flagged, contradictions surface in diagnostics, and a confabulation risk score is attached to the stored episode.
 
+Set `compositor.hyperformProfile` to `light`, `standard`, or `full`. For tier selection guidance, configuration details, and custom entry creation, see **[Tuning](#tuning)** below and **[docs/TUNING.md](./docs/TUNING.md)**.
+
 ---
 
 ## What it solves
@@ -197,17 +223,32 @@ OpenClaw 2026.4.7 ships memory wiki for structured storage. hypermem goes furthe
 
 Spawned subagents inherit a bounded context block: recent parent turns, session-scoped documents, and relevant facts. Scope is isolated from the shared library. Documents are cleaned up on completion.
 
+### Context that doesn't repeat itself
+
+Retrieval paths pull from four layers, trigger shortcuts, temporal indexes, open-domain FTS5, semantic recall, and cross-session summaries. Without dedup, the same fact surfaces through multiple paths and wastes budget on repetition.
+
+hypermem runs content fingerprint dedup across all compose-time retrieval. Every fact, temporal result, open-domain hit, and semantic recall entry is normalized and fingerprinted on a 120-char prefix. O(1) lookup in a shared set catches duplicates regardless of which retrieval path produced them, including rephrased near-duplicates that substring matching missed. Diagnostics track dedup counts and fingerprint collisions per compose call.
+
+Identity content (SOUL.md, USER.md, IDENTITY.md) and doc chunks already injected by OpenClaw's bootstrap are fingerprinted before retrieval runs, so the compositor never double-injects content the runtime already placed in the prompt.
+
+### Integrity under failure
+
+The background indexer runs a startup integrity check against `library.db` on every boot. If the schema is corrupt, tables are missing, or critical indexes are damaged, the indexer enters circuit-breaker mode: it logs the failure, skips indexing for the session, and avoids cascading writes into a broken database. The agent still runs with cached and in-memory data while the operator is notified.
+
+SQL queries that interpolate datetime values are fully parameterized. FTS5 trigger terms are quoted to prevent injection through crafted content. These aren't theoretical: agentic sessions ingest arbitrary user and tool output into the fact store, and unparameterized queries on that path were a real attack surface.
+
 ---
 
 ## Pressure management
 
-hypermem composes context fresh on every turn, but a long-running session still accumulates history in its JSONL transcript. When that grows large enough, incoming tool results have nowhere to land and get silently stripped. Three automatic paths handle this:
+hypermem composes context fresh on every turn, but a long-running session still accumulates history in its JSONL transcript. When that grows large enough, incoming tool results have nowhere to land and get silently stripped. Four automatic paths handle this:
 
 | Path | Trigger | Action |
 |---|---|---|
 | **Pressure-tiered tool-loop trim** | Any tool-loop turn | Measures projected occupancy before results land; trims large results at 80%+ and truncates the messages[] array for the current turn |
 | **AfterTurn trim** | Every turn at >80% | Pre-emptive headroom cut after the assistant replies, before the next turn arrives |
 | **Deep compaction** | compact() at >85% | Cuts in-memory cache to 25% budget and truncates JSONL to ~20% depth. Bypasses the normal reshape guard |
+| **Reshape guard** | Structured tool history on downshift | `canPersistReshapedHistory()` blocks a lower-context snapshot from overwriting the full JSONL history |
 
 **The one thing these paths cannot fix:** a session whose JSONL transcript on disk is already at 98% when the gateway restarts. The JSONL loads into runtime context before any compaction runs. Check `session_status` on startup. If you're above 85%, start a fresh session.
 
@@ -281,6 +322,8 @@ Retrieval follows a fixed pipeline on every compose call:
 
 FTS5 queries use compound indexes on `agentId + sort key` and prefix optimization (3+ chars, capped at 8 terms, OR queries). These indexes yielded a 25% read improvement over baseline despite a 47% increase in stored data.
 
+### Retrieval pipeline
+
 **L4: Library DB.** Per-agent storage can't hold shared knowledge. Facts established by one agent, wiki pages synthesized from cross-agent topics, shared registry state: these belong to the system, not one agent. One shared SQLite database:
 
 | Collection | What it holds |
@@ -311,17 +354,17 @@ Facts are ranked by `confidence × recencyDecay`, where decay is exponential wit
        │
   topic detection ──► scope retrieval to active thread
        │
-  ┌────┴────────────────────────────────────────────┐
-  │              query 4 layers (parallel)           │
-  │                                                  │
-  │  L1 in-memory  L2 History    L3 Vectors  L4 Library │
+  ┌────┴───────────────────────────────────────────────┐
+  │              query 4 layers (parallel)             │
+  │                                                    │
+  │  L1 in-memory  L2 History   L3 Vectors  L4 Library │
   │  hot state    durable       semantic    facts/wiki │
   │  0.1ms        0.16ms        0.29ms      0.08ms     │
-  └────┬────────────────────────────────────────────┘
+  └────┬───────────────────────────────────────────────┘
        │
   budget allocator ──► 10 slots, fixed token cap
        │
-  tool compression ──► clusters by age, T0 3 turns full → T1 6k → T2 800 → T3 150-char stub
+  tool compression ──► clusterNeutralMessages() → T0 full → T1 6k → T2 800 → T3 150-char stub
        │
   keystone guard ──► high-signal turns survive pressure
        │
@@ -340,14 +383,52 @@ Slot-level budget allocation is shown in the [hypercompositor diagram](#what-the
 
 ## Requirements
 
-**Current release: hypermem 0.5.4.** Topic-aware memory and compiled-knowledge system, optimized to run light by default and scale up when operators need richer context.
-
-What 0.5.4 includes:
-- Topic-aware context tracking
-- Compiled knowledge / wiki-like synthesis and recall
-- Metrics dashboard primitives
-- Obsidian import and export
-- Aligned runtime profiles: `light`, `standard`, `full`
+**Current release: hypermem 0.5.6.**
+
+### Release highlights
+
+**0.5.6** — Content fingerprint dedup and hardening
+- O(1) fingerprint dedup across all retrieval paths (temporal, open-domain, semantic, cross-session). Catches rephrased near-duplicates that substring matching missed.
+- Identity bootstrap pre-fingerprinting: SOUL.md, USER.md, IDENTITY.md content already in the prompt is never double-injected by retrieval.
+- Indexer circuit breaker with startup integrity check for library.db corruption. Graceful degradation, not cascading failure.
+- SQL parameterization hardening on datetime and FTS5 paths.
+
+**0.5.5** — Tuning collapse and config schema
+- Plugin config schema: all tuning knobs declarable in `openclaw.json`. No more manual config.json edits.
+- Tuning simplified to 4 primary knobs: `budgetFraction`, `reserveFraction`, `historyFraction`, `memoryFraction`.
+- Identity and doc chunk dedup against OpenClaw bootstrap injection.
+- Window cache with freshness diagnostics.
+
+**0.5.0** — Redis removal and context engine
+- Redis replaced with SQLite in-memory cache. Zero external services.
+- Context engine plugin: runs as an OpenClaw `contextEngine` slot, composing prompts per-turn.
+- Transform-first assembly: tool results compressed before budget allocation, not after.
+- Cluster-aware budget shaping: related tool turns grouped and trimmed together.
+- Hybrid FTS5 + KNN retrieval with Reciprocal Rank Fusion.
+- Workspace seeding: agents auto-ingest their workspace docs on bootstrap.
+- Runtime profiles: `light`, `standard`, `full`.
+- Obsidian import and export.
+- Metrics dashboard primitives.
+
+**0.4.0** — Eviction and migration
+- Image and heavy-content eviction pre-pass in assembly. Old screenshots and large tool outputs aged out before they compete for budget.
+- Engine version stamps in library.db. Schema migration runs automatically on version bump.
+- Migration guides and scripts for Cognee, QMD, Mem0, Zep, Honcho, and raw MEMORY.md files.
+
+**0.3.0** — Subagent context and retrieval
+- Subagent context inheritance: spawned subagents get bounded parent context, session-scoped docs, and relevant facts.
+- Tool Gradient v2: turn-age tiers with head+tail truncation on tool results.
+- Cursor-aware indexer with ghost message suppression.
+
+**0.2.0** — Retrieval access control
+- Trigger registry ownership and auditability.
+- Retrieval access control, trigger fallback paths, and history rebalance.
+
+**0.1.0** — Core architecture
+- Four-layer memory: in-memory cache, message history, vector search, structured library.
+- 8-level priority compositor with slot-based prompt assembly.
+- Cross-agent memory access with visibility-scoped permissions.
+- Knowledge graph with DAG traversal.
 
 | Requirement | Version | Notes |
 |---|---|---|
@@ -360,9 +441,8 @@ SQLite is a library, not a service. All four layers run in-process with no exter
 **Runtime version constants** (importable from the package):
 ```typescript
 import {
-  ENGINE_VERSION,        // '0.5.4'
+  ENGINE_VERSION,        // '0.5.6'
   MIN_NODE_VERSION,      // '22.0.0'
-  MIN_SQLITE_VERSION,    // '3.35.0'
   SQLITE_VEC_VERSION,    // '0.1.9'
   MAIN_SCHEMA_VERSION,   // 6  (hypermem.db)
   LIBRARY_SCHEMA_VERSION_EXPORT, // 12 (library.db)
@@ -407,49 +487,52 @@ If you prefer, hand the install to your OpenClaw agent:
 
 ### Tuning
 
-hypermem ships three aligned operating profiles: `light`, `standard`, and `full`. Pick one and set `outputProfile` in your config. Everything else follows.
-
-| Profile | Context window | Budget fraction | Best for |
-|---|---|---|---|
-| `light` | 64k | 0.50 | Single-agent installs, minimal parallel work |
-| `standard` | 128k | 0.65 | Normal OpenClaw deployments |
-| `full` | 200k+ | 0.55 | Large-context or multi-agent installs, maximum richness |
+Two independent surfaces: **context assembly** (what fills the context window) and **output shaping** (how the model writes). Pick a profile first — most deployments adjust one or two settings on top.
 
-**Start with `light`** on 64k models or single-agent systems. Move to `standard` once the system has stable latency and headroom. Use `full` only when you want maximum context richness and have the budget for it.
+| Profile | Target window | Best for |
+|---|---|---|
+| `light` | 64k | Single agent, small models, constrained resources |
+| `standard` | 128k | Normal deployments, small fleets |
+| `full` | 200k+ | Multi-agent fleets, large-context models |
 
-Primary tuning knobs:
+Start with `light`. Use `mergeProfile()` to adjust individual settings:
 
-- **`targetBudgetFraction`**: caps total non-history context weight. Lower values force lighter composition.
-- **`wikiTokenCap`**: caps compiled-knowledge/wiki contribution.
-- **`outputProfile`**: `light`, `standard`, or `full`. Controls how much hyperform guidance is injected per turn.
+```typescript
+import { mergeProfile } from '@psiclawops/hypermem';
+const config = mergeProfile('standard', { compositor: { maxFacts: 40 } });
+```
 
-Drop a `~/.openclaw/hypermem/config.json` to override compositor defaults. Takes effect on gateway restart:
+Drop a `~/.openclaw/hypermem/config.json` to override defaults (takes effect on gateway restart):
 
 ```json
 {
-  "deferToolPruning": true,
   "compositor": {
-    "defaultTokenBudget": 60000,
-    "maxFacts": 18,
-    "contextWindowReserve": 0.25,
-    "outputProfile": "standard"
+    "budgetFraction": 0.70,
+    "hyperformProfile": "standard"
   }
 }
 ```
 
-Additional compositor knobs: `maxCrossSessionContext`, `maxRecentToolPairs`, `maxProseToolPairs`, see INSTALL.md for full descriptions.
-
-`deferToolPruning: true` tells hypermem to skip its own T0/T1/T2/T3 gradient when OpenClaw's native `contextPruning` extension is active (Anthropic and Google providers). On those providers, OpenClaw's pruner handles tool result trimming: ratio-driven at >30% context fill, soft-trim head+tail for results over 4,000 chars, hard-clear above 50k total, with the last 3 assistant turns always protected. hypermem's gradient remains active as fallback for other providers (GPT-5.4, etc.). Default: `true` for Anthropic installs.
-
-`outputProfile` valid values: `"light"` (~100 tokens: anti-sycophancy, em dash ban, AI vocab ban, length targets, evidence calibration), `"standard"` (~250 tokens: full directive set plus pagination and hedging rules), `"full"` (~400 tokens: complete normalization with full directive set and model-specific calibration). Default: `"standard"`.
+Or configure through `openclaw.json` (preferred for managed deployments):
 
-Context presets ship as named profiles importable from the package:
-
-```typescript
-import { lightProfile, standardProfile, fullProfile } from '@psiclawops/hypermem';
+```json
+{
+  "plugins": {
+    "entries": {
+      "hypercompositor": {
+        "config": {
+          "compositor": { "budgetFraction": 0.70 },
+          "hyperformProfile": "standard"
+        }
+      }
+    }
+  }
+}
 ```
 
-Pass to `HyperMem.create()` as the base config. Full tuning notes are in INSTALL.md.
+Plugin config in `openclaw.json` takes precedence over `config.json`. Both sources are merged, with plugin config winning on overlap. The config schema is validated on gateway start and visible via `openclaw config get plugins.entries.hypercompositor.config`.
+
+Full reference: **[docs/TUNING.md](./docs/TUNING.md)**
 
 ---
 
@@ -545,6 +628,12 @@ Design guide: [PsiClawOps/AgenticCognitiveArchitecture](https://github.com/PsiCl
 
 ---
 
+## Acknowledgments
+
+The embedding-space fidelity threshold used in compaction validation was informed by the geometric preservation mathematics published by the [libravdb](https://github.com/xDarkicex/openclaw-memory-libravdb) project.
+
+---
+
 ## License
 
 Apache-2.0, [PsiClawOps](https://github.com/PsiClawOps)

package/dist/background-indexer.d.ts

@@ -56,6 +56,10 @@ export declare class BackgroundIndexer {
     private vectorStore;
     private synthesizer;
     private tickCount;
+    /** Circuit breaker: consecutive tick failure count. Resets on success. */
+    private consecutiveFailures;
+    /** True when the indexer is running in backoff mode due to repeated failures. */
+    private inBackoff;
     constructor(config?: Partial<IndexerConfig>, getMessageDb?: ((agentId: string) => DatabaseSync) | undefined, getLibraryDb?: (() => DatabaseSync) | undefined, listAgents?: (() => string[]) | undefined, getCursor?: CursorFetcher | undefined, dreamerConfig?: Partial<DreamerConfig>);
     /**
      * Set the vector store for embedding new facts/episodes at index time.
@@ -66,6 +70,20 @@ export declare class BackgroundIndexer {
      * Start periodic indexing.
      */
     start(): void;
+    /**
+     * Circuit breaker for tick failures.
+     *
+     * - Tracks consecutive failures.
+     * - After 3 failures, logs actionable recovery guidance once, then switches
+     *   the indexer to 10× backoff interval so it stops spamming the log.
+     * - On the next successful tick, resets state and restores normal interval.
+     */
+    private _handleTickError;
+    /**
+     * Reset the circuit breaker and restore normal interval after a successful tick.
+     * Called at the end of a successful tick().
+     */
+    private _resetCircuitBreaker;
     /**
      * Stop periodic indexing.
      */

package/dist/background-indexer.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"background-indexer.d.ts","sourceRoot":"","sources":["../src/background-indexer.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;GAiBG;AAEH,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAChD,OAAO,KAAK,EAAiB,aAAa,EAAe,aAAa,EAAE,MAAM,YAAY,CAAC;AAK3F,OAAO,EAA2B,KAAK,aAAa,EAAE,MAAM,wBAAwB,CAAC;AAOrF,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,mBAAmB,CAAC;AAuCrD,MAAM,WAAW,YAAY;IAC3B,OAAO,EAAE,MAAM,CAAC;IAChB,iBAAiB,EAAE,MAAM,CAAC;IAC1B,cAAc,EAAE,MAAM,CAAC;IACvB,gBAAgB,EAAE,MAAM,CAAC;IACzB,aAAa,EAAE,MAAM,CAAC;IACtB,iBAAiB,EAAE,MAAM,CAAC;IAC1B,oFAAoF;IACpF,UAAU,EAAE,MAAM,CAAC;IACnB,SAAS,EAAE,MAAM,CAAC;IAClB,wFAAwF;IACxF,kBAAkB,EAAE,MAAM,CAAC;CAC5B;AAED;;;;GAIG;AACH,MAAM,MAAM,aAAa,GAAG,CAAC,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,KAAK,OAAO,CAAC,aAAa,GAAG,IAAI,CAAC,CAAC;AAEnG,MAAM,WAAW,cAAc;IAC7B,OAAO,EAAE,MAAM,CAAC;IAChB,aAAa,EAAE,MAAM,CAAC;IACtB,SAAS,EAAE,MAAM,CAAC;CACnB;AA+XD,qBAAa,iBAAiB;IAW1B,OAAO,CAAC,YAAY,CAAC;IACrB,OAAO,CAAC,YAAY,CAAC;IACrB,OAAO,CAAC,UAAU,CAAC;IACnB,OAAO,CAAC,SAAS,CAAC;IAbpB,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAgB;IACvC,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAyB;IACvD,OAAO,CAAC,cAAc,CAA+C;IACrE,OAAO,CAAC,OAAO,CAAS;IACxB,OAAO,CAAC,WAAW,CAA4B;IAC/C,OAAO,CAAC,WAAW,CAAiC;IACpD,OAAO,CAAC,SAAS,CAAa;gBAG5B,MAAM,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC,EACvB,YAAY,CAAC,GAAE,CAAC,OAAO,EAAE,MAAM,KAAK,YAAY,aAAA,EAChD,YAAY,CAAC,GAAE,MAAM,YAAY,aAAA,EACjC,UAAU,CAAC,GAAE,MAAM,MAAM,EAAE,aAAA,EAC3B,SAAS,CAAC,EAAE,aAAa,YAAA,EACjC,aAAa,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC;IA8BxC;;;OAGG;IACH,cAAc,CAAC,EAAE,EAAE,WAAW,GAAG,IAAI;IAIrC;;OAEG;IACH,KAAK,IAAI,IAAI;IA0Bb;;OAEG;IACH,IAAI,IAAI,IAAI;IAOZ;;OAEG;IACG,IAAI,IAAI,OAAO,CAAC,YAAY,EAAE,CAAC;IAsIrC;;;;;;;;;OASG;YACW,YAAY;IA4M1B;;OAEG;IACH,OAAO,CAAC,oBAAoB;IA+B5B;;OAEG;IACH,OAAO,CAAC,uBAAuB;IAK/B;;OAEG;IACH,OAAO,CAAC,YAAY;IAsBpB;;OAEG;IACH,OAAO,CAAC,YAAY;IAWpB;;;OAGG;IACH,OAAO,CAAC,UAAU;IA8ClB;;OAEG;IACH,OAAO,CAAC,aAAa;IAarB;;;;;;;OAOG;IACG,sBAAsB,IAAI,OAAO,CAAC,IAAI,CAAC;IAgF7C;;OAEG;IACH,aAAa,CAAC,SAAS,EAAE,YAAY,GAAG,cAAc,EAAE;CAezD;AAID;;;GAGG;AACH,wBAAgB,aAAa,CAC3B,YAAY,EAAE,CAAC,OAAO,EAAE,MAAM,KAAK,YAAY,EAC/C,YAAY,EAAE,MAAM,YAAY,EAChC,UAAU,EAAE,MAAM,MAAM,EAAE,EAC1B,MAAM,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC,EAC/B,SAAS,CAAC,EAAE,aAAa,EACzB,WAAW,CAAC,EAAE,WAAW,EACzB,aAAa,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC,GACrC,iBAAiB,CAInB"}
+{"version":3,"file":"background-indexer.d.ts","sourceRoot":"","sources":["../src/background-indexer.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;GAiBG;AAEH,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAChD,OAAO,KAAK,EAAiB,aAAa,EAAe,aAAa,EAAE,MAAM,YAAY,CAAC;AAK3F,OAAO,EAA2B,KAAK,aAAa,EAAE,MAAM,wBAAwB,CAAC;AAOrF,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,mBAAmB,CAAC;AAuCrD,MAAM,WAAW,YAAY;IAC3B,OAAO,EAAE,MAAM,CAAC;IAChB,iBAAiB,EAAE,MAAM,CAAC;IAC1B,cAAc,EAAE,MAAM,CAAC;IACvB,gBAAgB,EAAE,MAAM,CAAC;IACzB,aAAa,EAAE,MAAM,CAAC;IACtB,iBAAiB,EAAE,MAAM,CAAC;IAC1B,oFAAoF;IACpF,UAAU,EAAE,MAAM,CAAC;IACnB,SAAS,EAAE,MAAM,CAAC;IAClB,wFAAwF;IACxF,kBAAkB,EAAE,MAAM,CAAC;CAC5B;AAED;;;;GAIG;AACH,MAAM,MAAM,aAAa,GAAG,CAAC,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,KAAK,OAAO,CAAC,aAAa,GAAG,IAAI,CAAC,CAAC;AAEnG,MAAM,WAAW,cAAc;IAC7B,OAAO,EAAE,MAAM,CAAC;IAChB,aAAa,EAAE,MAAM,CAAC;IACtB,SAAS,EAAE,MAAM,CAAC;CACnB;AA+XD,qBAAa,iBAAiB;IAe1B,OAAO,CAAC,YAAY,CAAC;IACrB,OAAO,CAAC,YAAY,CAAC;IACrB,OAAO,CAAC,UAAU,CAAC;IACnB,OAAO,CAAC,SAAS,CAAC;IAjBpB,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAgB;IACvC,OAAO,CAAC,QAAQ,CAAC,aAAa,CAAyB;IACvD,OAAO,CAAC,cAAc,CAA+C;IACrE,OAAO,CAAC,OAAO,CAAS;IACxB,OAAO,CAAC,WAAW,CAA4B;IAC/C,OAAO,CAAC,WAAW,CAAiC;IACpD,OAAO,CAAC,SAAS,CAAa;IAC9B,0EAA0E;IAC1E,OAAO,CAAC,mBAAmB,CAAa;IACxC,iFAAiF;IACjF,OAAO,CAAC,SAAS,CAAkB;gBAGjC,MAAM,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC,EACvB,YAAY,CAAC,GAAE,CAAC,OAAO,EAAE,MAAM,KAAK,YAAY,aAAA,EAChD,YAAY,CAAC,GAAE,MAAM,YAAY,aAAA,EACjC,UAAU,CAAC,GAAE,MAAM,MAAM,EAAE,aAAA,EAC3B,SAAS,CAAC,EAAE,aAAa,YAAA,EACjC,aAAa,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC;IA8BxC;;;OAGG;IACH,cAAc,CAAC,EAAE,EAAE,WAAW,GAAG,IAAI;IAIrC;;OAEG;IACH,KAAK,IAAI,IAAI;IAkDb;;;;;;;OAOG;IACH,OAAO,CAAC,gBAAgB;IAkDxB;;;OAGG;IACH,OAAO,CAAC,oBAAoB;IAiB5B;;OAEG;IACH,IAAI,IAAI,IAAI;IAOZ;;OAEG;IACG,IAAI,IAAI,OAAO,CAAC,YAAY,EAAE,CAAC;IA2IrC;;;;;;;;;OASG;YACW,YAAY;IA4M1B;;OAEG;IACH,OAAO,CAAC,oBAAoB;IA+B5B;;OAEG;IACH,OAAO,CAAC,uBAAuB;IAK/B;;OAEG;IACH,OAAO,CAAC,YAAY;IAsBpB;;OAEG;IACH,OAAO,CAAC,YAAY;IAWpB;;;OAGG;IACH,OAAO,CAAC,UAAU;IA8ClB;;OAEG;IACH,OAAO,CAAC,aAAa;IAarB;;;;;;;OAOG;IACG,sBAAsB,IAAI,OAAO,CAAC,IAAI,CAAC;IAgF7C;;OAEG;IACH,aAAa,CAAC,SAAS,EAAE,YAAY,GAAG,cAAc,EAAE;CAezD;AAID;;;GAGG;AACH,wBAAgB,aAAa,CAC3B,YAAY,EAAE,CAAC,OAAO,EAAE,MAAM,KAAK,YAAY,EAC/C,YAAY,EAAE,MAAM,YAAY,EAChC,UAAU,EAAE,MAAM,MAAM,EAAE,EAC1B,MAAM,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC,EAC/B,SAAS,CAAC,EAAE,aAAa,EACzB,WAAW,CAAC,EAAE,WAAW,EACzB,aAAa,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC,GACrC,iBAAiB,CAInB"}

package/dist/background-indexer.js

@@ -33,22 +33,22 @@ import { isSafeForSharedVisibility } from './secret-scanner.js';
 // domain-scoped retrieval (e.g. getActiveFacts({ domain: 'infrastructure' }))
 // returns results. New agents default to 'general'.
 const AGENT_DOMAIN_MAP = {
-    forge: 'infrastructure',
-    vigil: 'infrastructure',
-    pylon: 'infrastructure',
-    plane: 'infrastructure',
-    compass: 'product',
-    helm: 'product',
-    chisel: 'product',
-    facet: 'product',
-    sentinel: 'security',
-    bastion: 'security',
-    gauge: 'security',
+    alice: 'infrastructure',
+    jack: 'infrastructure',
+    hank: 'infrastructure',
+    irene: 'infrastructure',
+    bob: 'product',
+    eve: 'product',
+    frank: 'product',
+    grace: 'product',
+    dave: 'security',
+    leo: 'security',
+    kate: 'security',
     clarity: 'ux',
-    anvil: 'governance',
-    vanguard: 'strategy',
-    crucible: 'development',
-    relay: 'communications',
+    carol: 'governance',
+    oscar: 'strategy',
+    mike: 'development',
+    nancy: 'communications',
     main: 'general',
     'channel-mini': 'general',
 };
@@ -83,7 +83,7 @@ function extractFactCandidates(content) {
     // Preference patterns — medium confidence (0.60)
     const preferencePatterns = [
         /(?:prefer|always use|never use|don't use|avoid) (.{10,150})/gi,
-        /(?:ragesaq|operator) (?:wants|prefers|likes|hates|dislikes) (.{10,150})/gi,
+        /(?:operator) (?:wants|prefers|likes|hates|dislikes) (.{10,150})/gi,
     ];
     // Operational patterns: deployments, incidents, fixes — high confidence (0.70)
     const operationalPatterns = [
@@ -137,7 +137,7 @@ const OPERATIONAL_BOILERPLATE = [
     /still\s*waiting/i,
     /will\s*pick\s*(it\s*)?up\s*(on\s*(next|the))?/i,
     /message\s*is\s*in\s*(his|her|their|the)\s*queue/i,
-    /sent\s+to\s+(anvil|compass|clarity|sentinel|vanguard|forge)/i,
+    /sent\s+to\s+(carol|bob|clarity|dave|oscar|alice)/i,
     /dispatched\s+(it\s+)?to/i,
     /timed\s*out\s*after/i,
     /\bNO_REPLY\b/,
@@ -408,6 +408,10 @@ export class BackgroundIndexer {
     vectorStore = null;
     synthesizer = null;
     tickCount = 0;
+    /** Circuit breaker: consecutive tick failure count. Resets on success. */
+    consecutiveFailures = 0;
+    /** True when the indexer is running in backoff mode due to repeated failures. */
+    inBackoff = false;
     constructor(config, getMessageDb, getLibraryDb, listAgents, getCursor, dreamerConfig) {
         this.getMessageDb = getMessageDb;
         this.getLibraryDb = getLibraryDb;
@@ -457,9 +461,31 @@ export class BackgroundIndexer {
             return;
         if (this.intervalHandle)
             return;
+        // Startup integrity check — catch corruption before the first tick writes anything.
+        if (this.getLibraryDb) {
+            try {
+                const libDb = this.getLibraryDb();
+                if (libDb) {
+                    const row = libDb.prepare('PRAGMA quick_check').get();
+                    if (row?.integrity_check && row.integrity_check !== 'ok') {
+                        console.error('[indexer] ⚠️  library.db integrity check failed: ' + row.integrity_check + '\n' +
+                            '[indexer] Recovery: stop OpenClaw, run ' +
+                            '`sqlite3 ~/.openclaw/hypermem/library.db ".recover" | sqlite3 ~/.openclaw/hypermem/library_recovered.db`' +
+                            ', swap the files, and restart. If recovery fails, delete library.db — the indexer rebuilds from message history.');
+                        // Don't start the interval — nothing will succeed with a corrupt DB.
+                        return;
+                    }
+                }
+            }
+            catch (err) {
+                // If we can't even open the DB, log and bail — don't start the interval.
+                console.error('[indexer] Could not open library.db for integrity check:', err.message);
+                return;
+            }
+        }
         // Run once immediately
         this.tick().catch(err => {
-            console.error('[indexer] Initial tick failed:', err);
+            this._handleTickError(err, 'initial');
         });
         // Run episode vector backfill once at startup (no-op if already done)
         if (this.vectorStore && this.getLibraryDb) {
@@ -470,11 +496,83 @@ export class BackgroundIndexer {
         // Then periodically
         this.intervalHandle = setInterval(() => {
             this.tick().catch(err => {
-                console.error('[indexer] Periodic tick failed:', err);
+                this._handleTickError(err, 'periodic');
             });
         }, this.config.periodicInterval);
         console.log(`[indexer] Started with interval ${this.config.periodicInterval}ms, batchSize ${this.config.batchSize}, maxPerTick ${this.config.maxMessagesPerTick}`);
     }
+    /**
+     * Circuit breaker for tick failures.
+     *
+     * - Tracks consecutive failures.
+     * - After 3 failures, logs actionable recovery guidance once, then switches
+     *   the indexer to 10× backoff interval so it stops spamming the log.
+     * - On the next successful tick, resets state and restores normal interval.
+     */
+    _handleTickError(err, phase) {
+        this.consecutiveFailures++;
+        const msg = err instanceof Error ? err.message : String(err);
+        const isSqliteCorrupt = msg.includes('database disk image is malformed') ||
+            msg.includes('SQLITE_CORRUPT') ||
+            (err instanceof Error && 'code' in err && err.code === 'ERR_SQLITE_ERROR');
+        if (this.consecutiveFailures < 3) {
+            // First 1–2 failures: log normally.
+            console.error(`[indexer] ${phase === 'initial' ? 'Initial' : 'Periodic'} tick failed (attempt ${this.consecutiveFailures}/3):`, err);
+            return;
+        }
+        if (this.consecutiveFailures === 3) {
+            // Third failure: log once with recovery instructions, then enter backoff.
+            if (isSqliteCorrupt) {
+                console.error(`[indexer] ⛔ Tick failed 3 times consecutively — library.db appears corrupted. Entering backoff mode.\n` +
+                    `[indexer] Recovery steps:\n` +
+                    `[indexer]   1. Stop OpenClaw: openclaw gateway stop\n` +
+                    `[indexer]   2. Check damage: sqlite3 ~/.openclaw/hypermem/library.db "PRAGMA integrity_check"\n` +
+                    `[indexer]   3. Attempt recovery: sqlite3 ~/.openclaw/hypermem/library.db ".recover" | sqlite3 ~/.openclaw/hypermem/library_recovered.db\n` +
+                    `[indexer]   4. Swap: mv library.db library_corrupt.bak && mv library_recovered.db library.db\n` +
+                    `[indexer]   5. If recovery fails, delete library.db — the indexer rebuilds from message history on next start.\n` +
+                    `[indexer]   6. Restart: openclaw gateway start\n` +
+                    `[indexer] Indexer will retry every ${(this.config.periodicInterval * 10) / 60000} minutes until then.`);
+            }
+            else {
+                console.error(`[indexer] ⛔ Tick failed 3 times consecutively (${msg}). Entering backoff mode. ` +
+                    `Will retry every ${(this.config.periodicInterval * 10) / 60000} minutes.`);
+            }
+            // Switch to backoff interval.
+            this.inBackoff = true;
+            if (this.intervalHandle) {
+                clearInterval(this.intervalHandle);
+            }
+            this.intervalHandle = setInterval(() => {
+                this.tick().catch(backoffErr => {
+                    this._handleTickError(backoffErr, 'periodic');
+                });
+            }, this.config.periodicInterval * 10);
+            return;
+        }
+        // Beyond 3: silent (already logged, in backoff — don't spam).
+    }
+    /**
+     * Reset the circuit breaker and restore normal interval after a successful tick.
+     * Called at the end of a successful tick().
+     */
+    _resetCircuitBreaker() {
+        if (this.consecutiveFailures === 0)
+            return;
+        const wasInBackoff = this.inBackoff;
+        this.consecutiveFailures = 0;
+        this.inBackoff = false;
+        if (wasInBackoff) {
+            // Restore normal interval.
+            if (this.intervalHandle)
+                clearInterval(this.intervalHandle);
+            this.intervalHandle = setInterval(() => {
+                this.tick().catch(err => {
+                    this._handleTickError(err, 'periodic');
+                });
+            }, this.config.periodicInterval);
+            console.log('[indexer] Circuit breaker reset — tick succeeded, restored normal interval.');
+        }
+    }
     /**
      * Stop periodic indexing.
      */
@@ -494,6 +592,7 @@ export class BackgroundIndexer {
         }
         this.running = true;
         const results = [];
+        let tickSucceeded = false;
         try {
             if (!this.listAgents || !this.getMessageDb || !this.getLibraryDb) {
                 console.warn('[indexer] Missing database accessors — skipping');
@@ -601,8 +700,12 @@ export class BackgroundIndexer {
                     }
                 }
             }
+            // If we reach here, the tick completed without throwing.
+            tickSucceeded = true;
         }
         finally {
+            if (tickSucceeded)
+                this._resetCircuitBreaker();
             this.running = false;
         }
         return results;

package/dist/cache.d.ts

@@ -5,7 +5,7 @@
  * Same public interface, zero external dependencies, zero TCP overhead.
  */
 import { DatabaseSync } from 'node:sqlite';
-import type { CacheConfig, SessionMeta, SessionCursor, StoredMessage, NeutralMessage } from './types.js';
+import type { CacheConfig, ComposeDiagnostics, SessionMeta, SessionCursor, StoredMessage, NeutralMessage } from './types.js';
 export interface ModelState {
     model: string;
     tokenBudget: number;
@@ -13,6 +13,13 @@ export interface ModelState {
     historyDepth: number;
     reshapedAt?: string;
 }
+export interface WindowCacheMeta {
+    slots: Record<string, number>;
+    totalTokens: number;
+    warnings: string[];
+    diagnostics: ComposeDiagnostics;
+    composedAt: string;
+}
 export declare class CacheLayer {
     private db;
     private readonly config;
@@ -41,6 +48,7 @@ export declare class CacheLayer {
     private stmtEvictHistory;
     private stmtSetWindow;
     private stmtGetWindow;
+    private stmtGetFreshWindowBundle;
     private stmtDeleteWindow;
     private stmtEvictWindows;
     private stmtSetKv;
@@ -67,6 +75,21 @@ export declare class CacheLayer {
     setWindow(agentId: string, sessionKey: string, messages: NeutralMessage[], ttlSeconds?: number): Promise<void>;
     getWindow(agentId: string, sessionKey: string): Promise<NeutralMessage[] | null>;
     invalidateWindow(agentId: string, sessionKey: string): Promise<void>;
+    /**
+     * Returns the cached window + metadata only if a single read shows the cache
+     * and cursor still refer to the same composed window.
+     * Used for C4 window cache fast-exit in compositor.ts.
+     */
+    getFreshWindowBundle(agentId: string, sessionKey: string, lastMessageId: number): Promise<{
+        messages: NeutralMessage[];
+        meta: WindowCacheMeta;
+    } | null>;
+    /**
+     * Store compose result metadata alongside the window cache.
+     * Enables the C4 fast-exit to return a complete ComposeResult without re-running.
+     */
+    setWindowMeta(agentId: string, sessionKey: string, meta: WindowCacheMeta, ttl: number): Promise<void>;
+    getWindowMeta(agentId: string, sessionKey: string): Promise<WindowCacheMeta | null>;
     setCursor(agentId: string, sessionKey: string, cursor: SessionCursor): Promise<void>;
     getCursor(agentId: string, sessionKey: string): Promise<SessionCursor | null>;
     warmSession(agentId: string, sessionKey: string, slots: {