clawmem 0.7.2 → 0.8.1

package/AGENTS.md

@@ -96,6 +96,14 @@ curl http://host:8090/v1/models
 | `CLAWMEM_ENABLE_AMEM` | enabled | A-MEM note construction + link generation during indexing. |
 | `CLAWMEM_ENABLE_CONSOLIDATION` | disabled | Background worker backfills unenriched docs. Needs long-lived MCP process. |
 | `CLAWMEM_CONSOLIDATION_INTERVAL` | 300000 | Worker interval in ms (min 15000). |
+| `CLAWMEM_HEAVY_LANE` | disabled | **v0.8.0.** Enable the quiet-window heavy maintenance worker — a second, longer-interval consolidation lane with DB-backed `worker_leases` exclusivity, stale-first batching, and `maintenance_runs` journaling. Runs alongside the light lane; off by default. |
+| `CLAWMEM_HEAVY_LANE_INTERVAL` | 1800000 | **v0.8.0.** Heavy-lane tick interval in ms (min 30000, default 30 min). |
+| `CLAWMEM_HEAVY_LANE_WINDOW_START` | (none) | **v0.8.0.** Start hour (0-23) of the quiet window. Unset → no window. |
+| `CLAWMEM_HEAVY_LANE_WINDOW_END` | (none) | **v0.8.0.** End hour (0-23, exclusive) of the quiet window. Supports midnight wrap (22→6). |
+| `CLAWMEM_HEAVY_LANE_MAX_USAGES` | 30 | **v0.8.0.** Max `context_usage` rows in the last 10 min before the heavy lane skips with `reason='query_rate_high'`. |
+| `CLAWMEM_HEAVY_LANE_OBS_LIMIT` | 100 | **v0.8.0.** Phase 2 stale-first observation batch size. |
+| `CLAWMEM_HEAVY_LANE_DED_LIMIT` | 40 | **v0.8.0.** Phase 3 stale-first deductive candidate batch size. |
+| `CLAWMEM_HEAVY_LANE_SURPRISAL` | `false` | **v0.8.0.** When `true`, the heavy lane seeds Phase 2 with k-NN anomaly-ranked doc ids from `computeSurprisalScores` instead of stale-first ordering. Degrades to stale-first (`surprisal-fallback-stale` metric) on vaults without embeddings. |
 | `CLAWMEM_NUDGE_INTERVAL` | `15` | Prompts between lifecycle tool use before `<vault-nudge>` injection. 0 to disable. |
 | `CLAWMEM_MERGE_SCORE_NORMAL` | `0.93` | **v0.7.1.** Phase 2 merge-safety score threshold when candidate and existing anchors align. Merges above this normalized 3-gram cosine similarity are allowed. |
 | `CLAWMEM_MERGE_SCORE_STRICT` | `0.98` | **v0.7.1.** Strictest merge-safety score threshold (fallback when anchors are ambiguous). |
@@ -251,7 +259,7 @@ ClawMem hooks handle ~90% of retrieval automatically. Agent-initiated MCP calls
 
 | Hook | Trigger | Budget | Content |
 |------|---------|--------|---------|
-| `context-surfacing` | UserPromptSubmit | profile-driven (default 800) | retrieval gate → profile-driven hybrid search (vector if `useVector`, timeout from profile) → FTS supplement → file-aware supplemental search (E13) → snooze filter → noise filter → spreading activation (E11: co-activated doc boost) → memory type diversification (E10) → tiered injection (HOT/WARM/COLD snippets) → `<vault-context><instruction>…</instruction><facts>…</facts><relationships>…</relationships></vault-context>` (v0.7.1: instruction always prepended when context is returned; relationships block lists memory-graph edges where BOTH endpoints are in the surfaced set, truncated first when over budget) + optional `<vault-routing>` hint. Budget, max results, vector timeout, and min score all driven by `CLAWMEM_PROFILE`. |
+| `context-surfacing` | UserPromptSubmit | profile-driven (default 800) | retrieval gate → **multi-turn query construction** (v0.8.1: current prompt + up to 2 recent same-session priors from `context_usage.query_text`, 10-min max age, capped at 2000 chars with current-first preservation — used only for discovery: vector/FTS/expansion, NOT for rerank/scoring/snippet extraction) → profile-driven hybrid search (vector if `useVector`, timeout from profile) → FTS supplement → file-aware supplemental search (E13, raw current prompt) → snooze filter → noise filter → spreading activation (E11: co-activated doc boost) → memory type diversification (E10) → tiered injection (HOT/WARM/COLD snippets) → `<vault-context><instruction>…</instruction><facts>…</facts><relationships>…</relationships></vault-context>` (v0.7.1: instruction always prepended when context is returned; relationships block lists memory-graph edges where BOTH endpoints are in the surfaced set, truncated first when over budget) + optional `<vault-routing>` hint. Budget, max results, vector timeout, and min score all driven by `CLAWMEM_PROFILE`. Raw prompt persisted to `context_usage.query_text` for future multi-turn lookback — except on gated skip paths (slash commands, heartbeats, too-short prompts) where the text is withheld for privacy. |
 | `postcompact-inject` | SessionStart (compact) | 1200 tokens | re-injects authoritative context after compaction: precompact state (600) + recent decisions (400) + antipatterns (150) + vault context (200) → `<vault-postcompact>` |
 | `curator-nudge` | SessionStart | 200 tokens | surfaces curator report actions, nudges when report is stale (>7 days) |
 | `precompact-extract` | PreCompact | — | extracts decisions, file paths, open questions → writes `precompact-state.md` to auto-memory. Query-aware decision ranking. Reindexes auto-memory collection. |
@@ -503,8 +511,9 @@ The `memory_relations` table is populated by multiple independent sources:
 | `buildTemporalBackbone()` | temporal | `build_graphs` MCP tool (manual) | Creation-order edges between all active docs. |
 | `buildSemanticGraph()` | semantic | `build_graphs` MCP tool (manual) | Pure cosine similarity. PK collision: `INSERT OR IGNORE` means A-MEM semantic edges take precedence if they exist first. |
 | Entity co-occurrence graph | entity | A-MEM enrichment (indexing) | LLM entity extraction → quality filters (title/length/blocklist/location validation) → type-agnostic canonical resolution within compatibility buckets (person, org, location, tech=project/service/tool/concept) → `entity_mentions` + `entity_cooccurrences` tables. Entity edges use IDF-based specificity scoring. Feeds ENTITY intent queries and MPFP `[entity, semantic]` patterns. |
-| `consolidated_observations` | supporting, contradicts | Consolidation worker (background) | 3-tier consolidation: facts → observations → mental models. Observations track `proof_count`, `trend` (STABLE/STRENGTHENING/WEAKENING/STALE), and source links. **v0.7.1 safety gates:** name-aware merge gate uses entity-anchor comparison + 3-gram cosine similarity (dual-threshold `CLAWMEM_MERGE_SCORE_NORMAL`=0.93 / `_STRICT`=0.98) to prevent cross-entity merges ("Alice decided X" merging into "Bob decided X"). Merge-time contradiction gate runs deterministic heuristic + LLM check; blocked merges route to `CLAWMEM_CONTRADICTION_POLICY`=`link` (new row + `contradicts` edge, default) or `supersede` (old row `status='inactive'`, new row replaces). |
-| Deductive synthesis | supporting, contradicts | Consolidation worker Phase 3 (background, every ~15 min) | Combines 2-3 related recent observations (decision/preference/milestone/problem, last 7 days) into `content_type='deductive'` documents with `source_doc_ids` provenance. First-class searchable docs with ∞ half-life. **v0.7.1 anti-contamination wrapper:** every draft passes through deterministic pre-checks (empty conclusion, invalid source_indices, pool-only entity contamination via `entity_mentions` or lexical fallback) + LLM validator (fail-open with `validatorFallbackAccepts` counter) + dedupe. Per-reason rejection stats exposed via `DeductiveSynthesisStats` (contaminationRejects, invalidIndexRejects, unsupportedRejects, emptyRejects, dedupSkipped, validatorFallbackAccepts). Contradictory dedupe matches are linked via `contradicts` edges. |
+| `consolidated_observations` | supporting, contradicts | Consolidation worker (background, light + heavy lanes) | 3-tier consolidation: facts → observations → mental models. Observations track `proof_count`, `trend` (STABLE/STRENGTHENING/WEAKENING/STALE), and source links. **v0.7.1 safety gates:** name-aware merge gate uses entity-anchor comparison + 3-gram cosine similarity (dual-threshold `CLAWMEM_MERGE_SCORE_NORMAL`=0.93 / `_STRICT`=0.98) to prevent cross-entity merges ("Alice decided X" merging into "Bob decided X"). Merge-time contradiction gate runs deterministic heuristic + LLM check; blocked merges route to `CLAWMEM_CONTRADICTION_POLICY`=`link` (new row + `contradicts` edge, default) or `supersede` (old row `status='inactive'`, new row replaces). **v0.8.0:** the heavy lane calls `consolidateObservations(store, llm, { maxDocs, guarded: true, staleOnly: true })` — `guarded: true` forces merge-safety enforcement regardless of `CLAWMEM_MERGE_GUARD_DRY_RUN`, and `staleOnly: true` reorders candidates by `recall_stats.last_recalled_at ASC` so long-unseen docs bubble up first. Optional `candidateIds` filter plumbs k-NN anomaly ids from `computeSurprisalScores`. |
+| Deductive synthesis | supporting, contradicts | Consolidation worker Phase 3 (background, every ~15 min in the light lane; batched in the heavy lane) | Combines 2-3 related recent observations (decision/preference/milestone/problem, last 7 days) into `content_type='deductive'` documents with `source_doc_ids` provenance. First-class searchable docs with ∞ half-life. **v0.7.1 anti-contamination wrapper:** every draft passes through deterministic pre-checks (empty conclusion, invalid source_indices, pool-only entity contamination via `entity_mentions` or lexical fallback) + LLM validator (fail-open with `validatorFallbackAccepts` counter) + dedupe. Per-reason rejection stats exposed via `DeductiveSynthesisStats` (contaminationRejects, invalidIndexRejects, unsupportedRejects, emptyRejects, dedupSkipped, validatorFallbackAccepts). Contradictory dedupe matches are linked via `contradicts` edges. **v0.8.0:** heavy lane calls `generateDeductiveObservations(store, llm, { maxRecent, guarded: true, staleOnly: true })` for stale-first batching on large vaults. |
+| Heavy maintenance lane journal | — | `clawmem` process with `CLAWMEM_HEAVY_LANE=true` (v0.8.0) | Writes a row to the `maintenance_runs` table for every scheduled heavy-lane attempt (including skips). Columns: `lane` (`heavy`), `phase` (`gate`/`consolidate`/`deductive`), `status` (`started`/`completed`/`failed`/`skipped`), `reason` (`outside_window`/`query_rate_high`/`lease_unavailable`), per-phase `selected`/`processed`/`created`/`null_call` counts, and `metrics_json` with selector type (`stale-first`/`surprisal`/`surprisal-fallback-stale`) + full `DeductiveSynthesisStats` on completed Phase 3 runs. Exclusivity is enforced via the new `worker_leases` table with atomic `INSERT ... ON CONFLICT DO UPDATE ... WHERE expires_at <= ?` acquisition, random 16-byte fencing tokens, and TTL reclaim. Gate uses the existing `context_usage` table (no `query_activity` table) and `recall_stats.last_recalled_at` for stale-first ordering. Off by default — requires explicit opt-in. |
 | Conversation synthesis (`runConversationSynthesis`) | semantic, supporting, contradicts, causal, temporal, entity | `clawmem mine <dir> --synthesize` (opt-in, post-index) | **v0.7.2.** Two-pass LLM pipeline over freshly imported `content_type='conversation'` docs. Pass 1 extracts structured facts (decision/preference/milestone/problem) via `extractFactsFromConversation`, saves each via dedup-aware `saveMemory`, populates a local Set-based alias map. Pass 2 resolves cross-fact links against the local map first (fails closed on ambiguity — multi-candidate titles return unresolved), falls back to collection-scoped SQL lookup with LIMIT 2 ambiguity detection. Relations upsert via `ON CONFLICT DO UPDATE SET weight = MAX(weight, excluded.weight)` — idempotent on equal-weight reruns but monotonically accepts stronger later evidence. Synthesized fact paths are a pure function of `(sourceDocId, slug(title), short sha256(normalizedTitle))`, so reruns update in place instead of creating parallel rows. Counters split into `llmFailures` (null/thrown/invalid-JSON) vs `docsWithNoFacts` (valid-empty extraction). All failures non-fatal — never rolls back the mine import. |
 
 **Edge collision:** Both `generateMemoryLinks()` and `buildSemanticGraph()` insert `relation_type='semantic'`. PK is `(source_id, target_id, relation_type)` — first writer wins.

package/CLAUDE.md

@@ -96,6 +96,14 @@ curl http://host:8090/v1/models
 | `CLAWMEM_ENABLE_AMEM` | enabled | A-MEM note construction + link generation during indexing. |
 | `CLAWMEM_ENABLE_CONSOLIDATION` | disabled | Background worker backfills unenriched docs. Needs long-lived MCP process. |
 | `CLAWMEM_CONSOLIDATION_INTERVAL` | 300000 | Worker interval in ms (min 15000). |
+| `CLAWMEM_HEAVY_LANE` | disabled | **v0.8.0.** Enable the quiet-window heavy maintenance worker — a second, longer-interval consolidation lane with DB-backed `worker_leases` exclusivity, stale-first batching, and `maintenance_runs` journaling. Runs alongside the light lane; off by default. |
+| `CLAWMEM_HEAVY_LANE_INTERVAL` | 1800000 | **v0.8.0.** Heavy-lane tick interval in ms (min 30000, default 30 min). |
+| `CLAWMEM_HEAVY_LANE_WINDOW_START` | (none) | **v0.8.0.** Start hour (0-23) of the quiet window. Unset → no window. |
+| `CLAWMEM_HEAVY_LANE_WINDOW_END` | (none) | **v0.8.0.** End hour (0-23, exclusive) of the quiet window. Supports midnight wrap (22→6). |
+| `CLAWMEM_HEAVY_LANE_MAX_USAGES` | 30 | **v0.8.0.** Max `context_usage` rows in the last 10 min before the heavy lane skips with `reason='query_rate_high'`. |
+| `CLAWMEM_HEAVY_LANE_OBS_LIMIT` | 100 | **v0.8.0.** Phase 2 stale-first observation batch size. |
+| `CLAWMEM_HEAVY_LANE_DED_LIMIT` | 40 | **v0.8.0.** Phase 3 stale-first deductive candidate batch size. |
+| `CLAWMEM_HEAVY_LANE_SURPRISAL` | `false` | **v0.8.0.** When `true`, the heavy lane seeds Phase 2 with k-NN anomaly-ranked doc ids from `computeSurprisalScores` instead of stale-first ordering. Degrades to stale-first (`surprisal-fallback-stale` metric) on vaults without embeddings. |
 | `CLAWMEM_NUDGE_INTERVAL` | `15` | Prompts between lifecycle tool use before `<vault-nudge>` injection. 0 to disable. |
 | `CLAWMEM_MERGE_SCORE_NORMAL` | `0.93` | **v0.7.1.** Phase 2 merge-safety score threshold when candidate and existing anchors align. Merges above this normalized 3-gram cosine similarity are allowed. |
 | `CLAWMEM_MERGE_SCORE_STRICT` | `0.98` | **v0.7.1.** Strictest merge-safety score threshold (fallback when anchors are ambiguous). |
@@ -251,7 +259,7 @@ ClawMem hooks handle ~90% of retrieval automatically. Agent-initiated MCP calls
 
 | Hook | Trigger | Budget | Content |
 |------|---------|--------|---------|
-| `context-surfacing` | UserPromptSubmit | profile-driven (default 800) | retrieval gate → profile-driven hybrid search (vector if `useVector`, timeout from profile) → FTS supplement → file-aware supplemental search (E13) → snooze filter → noise filter → spreading activation (E11: co-activated doc boost) → memory type diversification (E10) → tiered injection (HOT/WARM/COLD snippets) → `<vault-context><instruction>…</instruction><facts>…</facts><relationships>…</relationships></vault-context>` (v0.7.1: instruction always prepended when context is returned; relationships block lists memory-graph edges where BOTH endpoints are in the surfaced set, truncated first when over budget) + optional `<vault-routing>` hint. Budget, max results, vector timeout, and min score all driven by `CLAWMEM_PROFILE`. |
+| `context-surfacing` | UserPromptSubmit | profile-driven (default 800) | retrieval gate → **multi-turn query construction** (v0.8.1: current prompt + up to 2 recent same-session priors from `context_usage.query_text`, 10-min max age, capped at 2000 chars with current-first preservation — used only for discovery: vector/FTS/expansion, NOT for rerank/scoring/snippet extraction) → profile-driven hybrid search (vector if `useVector`, timeout from profile) → FTS supplement → file-aware supplemental search (E13, raw current prompt) → snooze filter → noise filter → spreading activation (E11: co-activated doc boost) → memory type diversification (E10) → tiered injection (HOT/WARM/COLD snippets) → `<vault-context><instruction>…</instruction><facts>…</facts><relationships>…</relationships></vault-context>` (v0.7.1: instruction always prepended when context is returned; relationships block lists memory-graph edges where BOTH endpoints are in the surfaced set, truncated first when over budget) + optional `<vault-routing>` hint. Budget, max results, vector timeout, and min score all driven by `CLAWMEM_PROFILE`. Raw prompt persisted to `context_usage.query_text` for future multi-turn lookback — except on gated skip paths (slash commands, heartbeats, too-short prompts) where the text is withheld for privacy. |
 | `postcompact-inject` | SessionStart (compact) | 1200 tokens | re-injects authoritative context after compaction: precompact state (600) + recent decisions (400) + antipatterns (150) + vault context (200) → `<vault-postcompact>` |
 | `curator-nudge` | SessionStart | 200 tokens | surfaces curator report actions, nudges when report is stale (>7 days) |
 | `precompact-extract` | PreCompact | — | extracts decisions, file paths, open questions → writes `precompact-state.md` to auto-memory. Query-aware decision ranking. Reindexes auto-memory collection. |
@@ -503,8 +511,9 @@ The `memory_relations` table is populated by multiple independent sources:
 | `buildTemporalBackbone()` | temporal | `build_graphs` MCP tool (manual) | Creation-order edges between all active docs. |
 | `buildSemanticGraph()` | semantic | `build_graphs` MCP tool (manual) | Pure cosine similarity. PK collision: `INSERT OR IGNORE` means A-MEM semantic edges take precedence if they exist first. |
 | Entity co-occurrence graph | entity | A-MEM enrichment (indexing) | LLM entity extraction → quality filters (title/length/blocklist/location validation) → type-agnostic canonical resolution within compatibility buckets (person, org, location, tech=project/service/tool/concept) → `entity_mentions` + `entity_cooccurrences` tables. Entity edges use IDF-based specificity scoring. Feeds ENTITY intent queries and MPFP `[entity, semantic]` patterns. |
-| `consolidated_observations` | supporting, contradicts | Consolidation worker (background) | 3-tier consolidation: facts → observations → mental models. Observations track `proof_count`, `trend` (STABLE/STRENGTHENING/WEAKENING/STALE), and source links. **v0.7.1 safety gates:** name-aware merge gate uses entity-anchor comparison + 3-gram cosine similarity (dual-threshold `CLAWMEM_MERGE_SCORE_NORMAL`=0.93 / `_STRICT`=0.98) to prevent cross-entity merges ("Alice decided X" merging into "Bob decided X"). Merge-time contradiction gate runs deterministic heuristic + LLM check; blocked merges route to `CLAWMEM_CONTRADICTION_POLICY`=`link` (new row + `contradicts` edge, default) or `supersede` (old row `status='inactive'`, new row replaces). |
-| Deductive synthesis | supporting, contradicts | Consolidation worker Phase 3 (background, every ~15 min) | Combines 2-3 related recent observations (decision/preference/milestone/problem, last 7 days) into `content_type='deductive'` documents with `source_doc_ids` provenance. First-class searchable docs with ∞ half-life. **v0.7.1 anti-contamination wrapper:** every draft passes through deterministic pre-checks (empty conclusion, invalid source_indices, pool-only entity contamination via `entity_mentions` or lexical fallback) + LLM validator (fail-open with `validatorFallbackAccepts` counter) + dedupe. Per-reason rejection stats exposed via `DeductiveSynthesisStats` (contaminationRejects, invalidIndexRejects, unsupportedRejects, emptyRejects, dedupSkipped, validatorFallbackAccepts). Contradictory dedupe matches are linked via `contradicts` edges. |
+| `consolidated_observations` | supporting, contradicts | Consolidation worker (background, light + heavy lanes) | 3-tier consolidation: facts → observations → mental models. Observations track `proof_count`, `trend` (STABLE/STRENGTHENING/WEAKENING/STALE), and source links. **v0.7.1 safety gates:** name-aware merge gate uses entity-anchor comparison + 3-gram cosine similarity (dual-threshold `CLAWMEM_MERGE_SCORE_NORMAL`=0.93 / `_STRICT`=0.98) to prevent cross-entity merges ("Alice decided X" merging into "Bob decided X"). Merge-time contradiction gate runs deterministic heuristic + LLM check; blocked merges route to `CLAWMEM_CONTRADICTION_POLICY`=`link` (new row + `contradicts` edge, default) or `supersede` (old row `status='inactive'`, new row replaces). **v0.8.0:** the heavy lane calls `consolidateObservations(store, llm, { maxDocs, guarded: true, staleOnly: true })` — `guarded: true` forces merge-safety enforcement regardless of `CLAWMEM_MERGE_GUARD_DRY_RUN`, and `staleOnly: true` reorders candidates by `recall_stats.last_recalled_at ASC` so long-unseen docs bubble up first. Optional `candidateIds` filter plumbs k-NN anomaly ids from `computeSurprisalScores`. |
+| Deductive synthesis | supporting, contradicts | Consolidation worker Phase 3 (background, every ~15 min in the light lane; batched in the heavy lane) | Combines 2-3 related recent observations (decision/preference/milestone/problem, last 7 days) into `content_type='deductive'` documents with `source_doc_ids` provenance. First-class searchable docs with ∞ half-life. **v0.7.1 anti-contamination wrapper:** every draft passes through deterministic pre-checks (empty conclusion, invalid source_indices, pool-only entity contamination via `entity_mentions` or lexical fallback) + LLM validator (fail-open with `validatorFallbackAccepts` counter) + dedupe. Per-reason rejection stats exposed via `DeductiveSynthesisStats` (contaminationRejects, invalidIndexRejects, unsupportedRejects, emptyRejects, dedupSkipped, validatorFallbackAccepts). Contradictory dedupe matches are linked via `contradicts` edges. **v0.8.0:** heavy lane calls `generateDeductiveObservations(store, llm, { maxRecent, guarded: true, staleOnly: true })` for stale-first batching on large vaults. |
+| Heavy maintenance lane journal | — | `clawmem` process with `CLAWMEM_HEAVY_LANE=true` (v0.8.0) | Writes a row to the `maintenance_runs` table for every scheduled heavy-lane attempt (including skips). Columns: `lane` (`heavy`), `phase` (`gate`/`consolidate`/`deductive`), `status` (`started`/`completed`/`failed`/`skipped`), `reason` (`outside_window`/`query_rate_high`/`lease_unavailable`), per-phase `selected`/`processed`/`created`/`null_call` counts, and `metrics_json` with selector type (`stale-first`/`surprisal`/`surprisal-fallback-stale`) + full `DeductiveSynthesisStats` on completed Phase 3 runs. Exclusivity is enforced via the new `worker_leases` table with atomic `INSERT ... ON CONFLICT DO UPDATE ... WHERE expires_at <= ?` acquisition, random 16-byte fencing tokens, and TTL reclaim. Gate uses the existing `context_usage` table (no `query_activity` table) and `recall_stats.last_recalled_at` for stale-first ordering. Off by default — requires explicit opt-in. |
 | Conversation synthesis (`runConversationSynthesis`) | semantic, supporting, contradicts, causal, temporal, entity | `clawmem mine <dir> --synthesize` (opt-in, post-index) | **v0.7.2.** Two-pass LLM pipeline over freshly imported `content_type='conversation'` docs. Pass 1 extracts structured facts (decision/preference/milestone/problem) via `extractFactsFromConversation`, saves each via dedup-aware `saveMemory`, populates a local Set-based alias map. Pass 2 resolves cross-fact links against the local map first (fails closed on ambiguity — multi-candidate titles return unresolved), falls back to collection-scoped SQL lookup with LIMIT 2 ambiguity detection. Relations upsert via `ON CONFLICT DO UPDATE SET weight = MAX(weight, excluded.weight)` — idempotent on equal-weight reruns but monotonically accepts stronger later evidence. Synthesized fact paths are a pure function of `(sourceDocId, slug(title), short sha256(normalizedTitle))`, so reruns update in place instead of creating parallel rows. Counters split into `llmFailures` (null/thrown/invalid-JSON) vs `docsWithNoFacts` (valid-empty extraction). All failures non-fatal — never rolls back the mine import. |
 
 **Edge collision:** Both `generateMemoryLinks()` and `buildSemanticGraph()` insert `relation_type='semantic'`. PK is `(source_id, target_id, relation_type)` — first writer wins.

package/README.md

@@ -43,6 +43,7 @@ ClawMem turns your markdown notes, project docs, and research dumps into persist
 - **Manages document lifecycle** — policy-driven archival sweeps with restore capability
 - **Auto-routes queries** via `memory_retrieve` — classifies intent and dispatches to the optimal search backend
 - **Syncs project issues** from Beads issue trackers into searchable memory
+- **Runs a quiet-window heavy maintenance lane** — a second consolidation worker, off by default behind `CLAWMEM_HEAVY_LANE=true`, that runs on a longer interval only inside a configurable hour window. Gated by `context_usage` query-rate so it never competes for CPU/GPU with interactive sessions, scoped exclusively via DB-backed `worker_leases`, stale-first by default with an optional surprisal selector, and journals every attempt in `maintenance_runs` for operator visibility (v0.8.0)
 
 Runs fully local with no API keys and no cloud services. Integrates via Claude Code hooks and MCP tools, as an OpenClaw ContextEngine plugin, or as a Hermes Agent MemoryProvider plugin. All modes share the same vault for cross-runtime memory. Works with any MCP-compatible client.
 
@@ -80,6 +81,33 @@ Opt-in LLM pass that runs **after** `clawmem mine` finishes indexing an imported
 
 Adds +63 tests (46 unit + 5 integration + 12 regression) on top of the v0.7.1 baseline.
 
+### v0.8.0 Quiet-Window Heavy Maintenance Lane
+
+A second, longer-interval consolidation worker that keeps Phase 2 + Phase 3 running on large vaults without starving interactive sessions. Off by default — set `CLAWMEM_HEAVY_LANE=true` to enable. The existing 5-minute light-lane worker is unchanged. See [heavy maintenance lane](docs/concepts/architecture.md#heavy-maintenance-lane-v080) for the architectural walkthrough.
+
+- **Quiet-window gating** — the heavy lane only runs inside the hours set by `CLAWMEM_HEAVY_LANE_WINDOW_START` / `CLAWMEM_HEAVY_LANE_WINDOW_END` (0-23). Supports midnight wraparound (e.g., 22→6). Null on either bound means "always in window".
+- **Query-rate gating via `context_usage`** — counts hook injections in the last 10 minutes and skips the tick when the rate exceeds `CLAWMEM_HEAVY_LANE_MAX_USAGES` (default 30). No new `query_activity` table; reuses v0.7.0 telemetry.
+- **DB-backed worker leases** — exclusivity enforced via a new `worker_leases` table with atomic `INSERT ... ON CONFLICT DO UPDATE ... WHERE expires_at <= ?` acquisition, random 16-byte fencing tokens, and TTL reclaim. Safe under multi-process contention; any SQLite error translates to a `lease_unavailable` skip rather than a thrown exception.
+- **Stale-first selection** — Phase 2 and Phase 3 reorder their candidate sets by `COALESCE(recall_stats.last_recalled_at, documents.last_accessed_at, documents.modified_at) ASC` so long-unseen docs bubble up first. Empty `recall_stats` falls through to access-time without erroring.
+- **Optional surprisal selector** — `CLAWMEM_HEAVY_LANE_SURPRISAL=true` plumbs k-NN anomaly-ranked doc ids (via the existing `computeSurprisalScores`) into Phase 2 as an explicit `candidateIds` filter. Degrades to stale-first on vaults without embeddings and logs `selector: 'surprisal-fallback-stale'` in the journal.
+- **`maintenance_runs` journal** — every scheduled attempt writes a row: `status` (`started`/`completed`/`failed`/`skipped`), `reason` for skips, selected/processed/created/null_call counts, and a `metrics_json` payload with selector type and full `DeductiveSynthesisStats` breakdown. Operators can reconstruct any lane decision without reading worker logs.
+- **Force-enforce merge gate** — the heavy lane passes `guarded: true` to `consolidateObservations`, which overrides `CLAWMEM_MERGE_GUARD_DRY_RUN` inside `findSimilarConsolidation` so experimenting operators cannot weaken heavy-lane enforcement via env flag.
+
+Adds +56 tests (13 worker-lease + 35 maintenance unit + 8 maintenance integration) on top of the v0.7.2 baseline.
+
+### v0.8.1 Multi-Turn Prior-Query Lookback
+
+`context-surfacing` now builds its retrieval query from the current prompt plus up to two recent same-session prior prompts, so a short follow-up turn ("do the same for X", "explain the rationale") can still inherit the vocabulary of earlier turns. The raw prompt is persisted in a new nullable `context_usage.query_text` column so future hook ticks can reconstitute the multi-turn query from the DB. See [multi-turn lookback](docs/concepts/architecture.md#multi-turn-prior-query-lookback-v081) for the full walkthrough.
+
+- **Additive schema migration** — new nullable `query_text TEXT` column on `context_usage`, guarded by `PRAGMA table_info`. Pre-v0.8.1 stores get the column added on first open; ad-hoc stores that skip the migration path degrade transparently via a feature-detect WeakMap so `insertUsageFn` never writes a column that doesn't exist.
+- **Discovery path only** — the multi-turn query feeds vector search, BM25, and query expansion. Cross-encoder reranking continues to use the RAW current prompt so relevance scoring is not diluted by older turns, and composite scoring / snippet extraction / dedupe / routing-hint detection all remain on the raw prompt as well.
+- **Privacy-conscious persistence split** — gated skip paths (slash commands, `MIN_PROMPT_LENGTH`, `shouldSkipRetrieval`, heartbeat dedupe) do NOT persist their raw text because those turns are not meaningful user questions and carry a higher sensitivity profile. Post-retrieval empty paths (empty result set, threshold blocked, budget blocked) DO persist so a follow-up turn can still inherit the intent even when the current turn surfaced nothing.
+- **Current-first truncation** — the combined query is clamped to 2000 chars with the current prompt preserved verbatim at the head. Older priors are dropped first when the budget runs out. If the current prompt alone already exceeds the cap, priors are omitted entirely and the current prompt is truncated.
+- **SQL-level self-match guard** — duplicate submits of the same prompt are filtered out of the lookback SELECT via `AND query_text != ?` so a retry burst cannot eat into the 2-prior budget and leave the lookback window underfilled.
+- **10-minute max age, session-scoped** — priors older than 10 minutes or from a different `session_id` are invisible to the lookback. All fallback paths (missing column, DB error, no matching rows) return the current prompt unchanged — the hook never throws on lookback failures.
+
+Adds +27 tests (22 unit + 5 integration) on top of the v0.8.0 baseline.
+
 ## Architecture
 
 <p align="center">
@@ -1172,6 +1200,7 @@ Built on the shoulders of:
 - [QMD](https://github.com/tobi/qmd) — search backend (BM25 + vectors + RRF + reranking)
 - [SAME](https://github.com/sgx-labs/statelessagent) — agent memory concepts (recency decay, confidence scoring, session tracking)
 - [supermemory](https://github.com/supermemoryai/clawdbot-supermemory) — hook patterns and context surfacing ideas
+- [Thoth](https://github.com/siddsachar/Thoth) — anti-contamination deductive synthesis, contradiction-aware + name-aware merge gates, post-import conversation fact extraction, quiet-window heavy maintenance lane with worker leases, context instruction framing, relationship snippets, multi-turn prior-query lookback
 
 ## Roadmap
 

package/SKILL.md

@@ -92,6 +92,12 @@ curl http://host:8090/v1/models
 | `CLAWMEM_MERGE_GUARD_DRY_RUN` | `false` | **v0.7.1.** When `true`, Phase 2 merge-safety rejections are logged but not enforced — use for calibration. |
 | `CLAWMEM_CONTRADICTION_POLICY` | `link` | **v0.7.1.** How the merge-time contradiction gate handles a blocked merge. `link` (default) keeps both rows + inserts `contradicts` edge. `supersede` marks the old row `status='inactive'`. |
 | `CLAWMEM_CONTRADICTION_MIN_CONFIDENCE` | `0.5` | **v0.7.1.** Minimum combined heuristic+LLM confidence required before the contradiction gate blocks a merge. |
+| `CLAWMEM_HEAVY_LANE` | disabled | **v0.8.0.** Enable the quiet-window heavy maintenance worker — a second, longer-interval consolidation lane with DB-backed `worker_leases` exclusivity, stale-first batching, and `maintenance_runs` journaling. Runs alongside the light lane. |
+| `CLAWMEM_HEAVY_LANE_INTERVAL` | 1800000 | **v0.8.0.** Heavy-lane tick interval in ms (min 30000, default 30 min). |
+| `CLAWMEM_HEAVY_LANE_WINDOW_START` / `_END` | (none) | **v0.8.0.** Start/end hours (0-23) of the quiet window. Supports midnight wrap (22→6). Null on either bound = always in window. |
+| `CLAWMEM_HEAVY_LANE_MAX_USAGES` | 30 | **v0.8.0.** Max `context_usage` rows in the last 10 min before the heavy lane skips with `reason='query_rate_high'`. |
+| `CLAWMEM_HEAVY_LANE_OBS_LIMIT` / `_DED_LIMIT` | 100 / 40 | **v0.8.0.** Phase 2 / Phase 3 stale-first batch sizes. |
+| `CLAWMEM_HEAVY_LANE_SURPRISAL` | `false` | **v0.8.0.** When `true`, the heavy lane seeds Phase 2 with k-NN anomaly-ranked doc ids from `computeSurprisalScores` instead of stale-first ordering. Degrades to stale-first on vaults without embeddings. |
 
 **Note:** The `bin/clawmem` wrapper sets all endpoint defaults. Always use the wrapper — never `bun run src/clawmem.ts` directly.
 
@@ -184,7 +190,7 @@ Hooks handle ~90% of retrieval. Zero agent effort.
 
 | Hook | Trigger | Budget | Content |
 |------|---------|--------|---------|
-| `context-surfacing` | UserPromptSubmit | profile-driven (default 800) | retrieval gate -> profile-driven hybrid search (vector if `useVector`, timeout from profile) -> FTS supplement -> file-aware search (E13) -> snooze filter -> noise filter -> spreading activation (E11) -> memory type diversification (E10) -> tiered injection (HOT/WARM/COLD) -> `<vault-context><instruction>...</instruction><facts>...</facts><relationships>...</relationships></vault-context>` (v0.7.1: instruction always prepended; relationships list memory-graph edges where BOTH endpoints are in the surfaced set; relationships truncated first when over budget) + optional `<vault-routing>` hint. Budget, max results, vector timeout, min score all driven by `CLAWMEM_PROFILE`. |
+| `context-surfacing` | UserPromptSubmit | profile-driven (default 800) | retrieval gate -> **multi-turn query** (v0.8.1: current + up to 2 recent same-session priors from `context_usage.query_text`, 10-min max age, 2000-char cap with current-first, used only for discovery — not rerank/scoring/snippet) -> profile-driven hybrid search (vector if `useVector`, timeout from profile) -> FTS supplement -> file-aware search (E13, raw current) -> snooze filter -> noise filter -> spreading activation (E11) -> memory type diversification (E10) -> tiered injection (HOT/WARM/COLD) -> `<vault-context><instruction>...</instruction><facts>...</facts><relationships>...</relationships></vault-context>` (v0.7.1: instruction always prepended; relationships list memory-graph edges where BOTH endpoints are in the surfaced set; relationships truncated first when over budget) + optional `<vault-routing>` hint. Budget, max results, vector timeout, min score all driven by `CLAWMEM_PROFILE`. Raw prompt persisted to `context_usage.query_text` for future lookback — gated skip paths (slash commands, heartbeats, too-short prompts) withhold the text for privacy. |
 | `postcompact-inject` | SessionStart (compact) | 1200 tokens | re-injects authoritative context after compaction: precompact state (600) + decisions (400) + antipatterns (150) + vault context (200) -> `<vault-postcompact>` |
 | `curator-nudge` | SessionStart | 200 tokens | surfaces curator report actions, nudges when report is stale (>7 days) |
 | `precompact-extract` | PreCompact | — | extracts decisions, file paths, open questions -> writes `precompact-state.md`. Query-aware ranking. Reindexes auto-memory. |
@@ -526,9 +532,10 @@ mcp__clawmem__vsearch(query, collection="name", compact=true)   # vector
 | Beads `syncBeadsIssues()` | causal, supporting, semantic | `beads_sync` MCP or watcher | Queries `bd` CLI (Dolt backend). |
 | `buildTemporalBackbone()` | temporal | `build_graphs` MCP (manual) | Creation-order edges. |
 | `buildSemanticGraph()` | semantic | `build_graphs` MCP (manual) | Pure cosine similarity. A-MEM edges take precedence (first-writer wins). |
-| `consolidated_observations` | supporting, contradicts | Consolidation worker (background) | **v0.7.1 safety gates:** Phase 2 name-aware merge gate (entity anchors + 3-gram cosine, dual-threshold `CLAWMEM_MERGE_SCORE_NORMAL`=0.93 / `_STRICT`=0.98) blocks cross-entity merges. Merge-time contradiction gate (heuristic + LLM) routes blocked merges to `link` (default, inserts `contradicts` edge) or `supersede` (old row `status='inactive'`) via `CLAWMEM_CONTRADICTION_POLICY`. |
-| Deductive synthesis | supporting, contradicts | Consolidation worker Phase 3 (every ~15 min) | Combines 2-3 related observations (decision/preference/milestone/problem, last 7 days) into `content_type='deductive'` docs. **v0.7.1 anti-contamination:** deterministic pre-checks (empty/invalid_indices/pool-only entity contamination) + LLM validator (fail-open, `validatorFallbackAccepts` counter) + dedupe. Per-reason rejection stats via `DeductiveSynthesisStats`. Contradictory dedupe matches linked via `contradicts` edges. |
+| `consolidated_observations` | supporting, contradicts | Consolidation worker (background, light + heavy lanes) | **v0.7.1 safety gates:** Phase 2 name-aware merge gate (entity anchors + 3-gram cosine, dual-threshold `CLAWMEM_MERGE_SCORE_NORMAL`=0.93 / `_STRICT`=0.98) blocks cross-entity merges. Merge-time contradiction gate (heuristic + LLM) routes blocked merges to `link` (default, inserts `contradicts` edge) or `supersede` (old row `status='inactive'`) via `CLAWMEM_CONTRADICTION_POLICY`. **v0.8.0:** heavy lane calls `consolidateObservations(store, llm, { maxDocs, guarded: true, staleOnly: true, candidateIds? })` — `guarded: true` forces enforcement regardless of `CLAWMEM_MERGE_GUARD_DRY_RUN`, `staleOnly: true` reorders by `recall_stats.last_recalled_at ASC`, and optional `candidateIds` plumbs surprisal-selector output. |
+| Deductive synthesis | supporting, contradicts | Consolidation worker Phase 3 (every ~15 min in light lane; batched in heavy lane) | Combines 2-3 related observations (decision/preference/milestone/problem, last 7 days) into `content_type='deductive'` docs. **v0.7.1 anti-contamination:** deterministic pre-checks (empty/invalid_indices/pool-only entity contamination) + LLM validator (fail-open, `validatorFallbackAccepts` counter) + dedupe. Per-reason rejection stats via `DeductiveSynthesisStats`. Contradictory dedupe matches linked via `contradicts` edges. **v0.8.0:** heavy lane calls `generateDeductiveObservations(store, llm, { maxRecent, guarded: true, staleOnly: true })` for stale-first batching. |
 | Conversation synthesis | semantic, supporting, contradicts, causal, temporal, entity | `clawmem mine <dir> --synthesize` (opt-in, post-index) | **v0.7.2.** Two-pass LLM pipeline over freshly imported `content_type='conversation'` docs. Pass 1 extracts structured decision/preference/milestone/problem facts + aliases + cross-fact links, saves via dedup-aware `saveMemory`, populates ambiguity-aware local Set map. Pass 2 resolves links (local first, SQL fallback with `LIMIT 2` ambiguity detection), upserts relations via `ON CONFLICT DO UPDATE SET weight = MAX(weight, excluded.weight)`. Synthesized paths are a pure function of `(sourceDocId, slug(title), short sha256(normalizedTitle))` so reruns update in place. All failures non-fatal. Counters split: `llmFailures` (LLM/parse error) vs `docsWithNoFacts` (valid empty extraction). |
+| Heavy maintenance lane journal | — | `CLAWMEM_HEAVY_LANE=true` (v0.8.0) | Writes a row to `maintenance_runs` for every scheduled heavy-lane attempt (including skips). Columns: `lane` (`heavy`), `phase` (`gate`/`consolidate`/`deductive`), `status` (`started`/`completed`/`failed`/`skipped`), `reason` (`outside_window`/`query_rate_high`/`lease_unavailable`), per-phase counts, and `metrics_json` with selector type (`stale-first`/`surprisal`/`surprisal-fallback-stale`) + full `DeductiveSynthesisStats`. Exclusivity via new `worker_leases` table (atomic `INSERT ... ON CONFLICT DO UPDATE ... WHERE expires_at <= ?`, 16-byte fencing tokens, TTL reclaim). Gate reuses `context_usage` (no `query_activity` table). |
 
 **Graph traversal asymmetry:** `adaptiveTraversal()` traverses all edge types outbound (source->target) but only `semantic` and `entity` inbound.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "clawmem",
-  "version": "0.7.2",
+  "version": "0.8.1",
   "description": "On-device context engine and memory for AI agents. Claude Code and OpenClaw. Hooks + MCP server + hybrid RAG search.",
   "type": "module",
   "bin": {

package/src/consolidation.ts

@@ -114,6 +114,50 @@ interface ObservationCluster {
   collection: string;
 }
 
+/**
+ * Phase 2 consolidation option bag (v0.8.0 Ext 5). All fields optional;
+ * omitting the bag reproduces pre-Ext-5 behavior exactly.
+ *
+ *  - `maxDocs`       override for the observation batch size (default 50)
+ *  - `guarded`       when true, force merge-safety enforcement regardless of
+ *                    the `CLAWMEM_MERGE_GUARD_DRY_RUN` env var. Heavy-lane
+ *                    callers pass this so experimenting operators cannot
+ *                    weaken the heavy-lane gate by toggling an env flag.
+ *  - `staleOnly`     when true, order observations by
+ *                    `recall_stats.last_recalled_at ASC` (with
+ *                    `documents.last_accessed_at` fallback) so least-recalled
+ *                    documents are processed first instead of most-recent.
+ *  - `candidateIds`  when non-empty, restricts the Phase 2 candidate set to
+ *                    exactly these document ids. Heavy lane uses this to
+ *                    plumb surprisal-selector output into consolidation so
+ *                    `useSurprisalSelector: true` actually feeds anomaly-first
+ *                    candidates instead of just relabeling the default batch.
+ */
+export interface ConsolidateOptions {
+  maxDocs?: number;
+  guarded?: boolean;
+  staleOnly?: boolean;
+  candidateIds?: number[];
+}
+
+/**
+ * Phase 3 deductive synthesis option bag (v0.8.0 Ext 5). All fields optional;
+ * omitting the bag reproduces pre-Ext-5 behavior exactly.
+ *
+ *  - `maxRecent`  override for the recent-observation window size (default 20)
+ *  - `guarded`    forward-compat marker; currently a no-op because the Phase 3
+ *                 deductive guardrails always enforce their deterministic
+ *                 pre-checks + LLM validator regardless of this flag
+ *  - `staleOnly`  when true, order candidate observations by
+ *                 `recall_stats.last_recalled_at ASC` with
+ *                 `documents.last_accessed_at` fallback
+ */
+export interface DeductiveOptions {
+  maxRecent?: number;
+  guarded?: boolean;
+  staleOnly?: boolean;
+}
+
 // =============================================================================
 // Worker State
 // =============================================================================
@@ -247,17 +291,71 @@ async function backfillAmem(store: Store, llm: LlamaCpp): Promise<void> {
 /**
  * Find clusters of related observations and synthesize into consolidated observations.
  * Runs per-collection to prevent cross-vault false merges.
+ *
+ * `opts` (v0.8.0 Ext 5) lets the heavy-maintenance lane override batch size,
+ * force merge-safety enforcement, and switch to stale-first ordering. The
+ * zero-arg call path (internal light-lane tick) preserves pre-Ext-5 behavior.
  */
-async function consolidateObservations(store: Store, llm: LlamaCpp): Promise<void> {
-  console.log("[consolidation] Starting observation consolidation");
+export async function consolidateObservations(
+  store: Store,
+  llm: LlamaCpp,
+  opts: ConsolidateOptions = {},
+): Promise<void> {
+  const maxDocs = opts.maxDocs && opts.maxDocs > 0 ? opts.maxDocs : 50;
+  const staleOnly = opts.staleOnly === true;
+  const guarded = opts.guarded === true;
+  const candidateIds = opts.candidateIds && opts.candidateIds.length > 0
+    ? opts.candidateIds
+    : null;
+
+  // Early exit when caller passed candidateIds: [] explicitly. An empty
+  // array means "the selector found nothing" — not "select everything".
+  if (opts.candidateIds && opts.candidateIds.length === 0) {
+    console.log("[consolidation] Empty candidateIds array — nothing to consolidate");
+    return;
+  }
+
+  console.log(
+    `[consolidation] Starting observation consolidation (maxDocs=${maxDocs}, ` +
+      `staleOnly=${staleOnly}, guarded=${guarded}, ` +
+      `candidateIds=${candidateIds ? candidateIds.length : "null"})`,
+  );
+
+  // Base SELECT — observation-type docs not yet consolidated.
+  // Stale-first ordering joins recall_stats.last_recalled_at ASC with a
+  // documents.last_accessed_at fallback so long-unseen docs bubble up first.
+  // Default ordering (modified_at DESC) preserves pre-Ext-5 light-lane semantics.
+  const orderBy = staleOnly
+    ? `ORDER BY
+         d.collection,
+         COALESCE(rs.last_recalled_at, d.last_accessed_at, d.modified_at) ASC,
+         d.modified_at ASC`
+    : `ORDER BY d.collection, d.modified_at DESC`;
+
+  const joinClause = staleOnly
+    ? `LEFT JOIN recall_stats rs ON rs.doc_id = d.id`
+    : ``;
+
+  // When candidateIds is provided, restrict the SELECT to those docs. This
+  // is how the heavy lane plumbs surprisal-selector output into Phase 2:
+  // select anomaly-first IDs via computeSurprisalScores, then ask
+  // consolidateObservations to limit its pattern detection to that subset.
+  const candidateFilter = candidateIds
+    ? `AND d.id IN (${candidateIds.map(() => "?").join(",")})`
+    : ``;
+
+  const sqlParams: (number | string)[] = candidateIds
+    ? [...candidateIds, maxDocs]
+    : [maxDocs];
 
-  // Find observation-type documents not yet consolidated
   const observations = store.db.prepare(`
     SELECT d.id, d.title, d.facts, d.amem_context as context, d.modified_at, d.collection
     FROM documents d
+    ${joinClause}
     WHERE d.active = 1
       AND d.content_type = 'observation'
       AND d.facts IS NOT NULL
+      ${candidateFilter}
       AND d.id NOT IN (
         SELECT value FROM (
           SELECT json_each.value as value
@@ -265,9 +363,9 @@ async function consolidateObservations(store: Store, llm: LlamaCpp): Promise<voi
           WHERE co.status = 'active'
         )
       )
-    ORDER BY d.collection, d.modified_at DESC
-    LIMIT 50
-  `).all() as { id: number; title: string; facts: string; context: string; modified_at: string; collection: string }[];
+    ${orderBy}
+    LIMIT ?
+  `).all(...sqlParams) as { id: number; title: string; facts: string; context: string; modified_at: string; collection: string }[];
 
   if (observations.length === 0) {
     console.log("[consolidation] No unconsolidated observations found");
@@ -288,7 +386,7 @@ async function consolidateObservations(store: Store, llm: LlamaCpp): Promise<voi
     if (cluster.docs.length < 2) continue; // Need at least 2 observations to consolidate
 
     try {
-      await synthesizeCluster(store, llm, cluster);
+      await synthesizeCluster(store, llm, cluster, { guarded });
     } catch (err) {
       console.error(`[consolidation] Failed to consolidate cluster for ${collection}:`, err);
     }
@@ -302,11 +400,15 @@ async function consolidateObservations(store: Store, llm: LlamaCpp): Promise<voi
 
 /**
  * Synthesize a cluster of observations into consolidated observations using LLM.
+ *
+ * `opts.guarded` (v0.8.0 Ext 5) forces merge-safety enforcement inside
+ * `findSimilarConsolidation` regardless of `CLAWMEM_MERGE_GUARD_DRY_RUN`.
  */
 async function synthesizeCluster(
   store: Store,
   llm: LlamaCpp,
-  cluster: ObservationCluster
+  cluster: ObservationCluster,
+  opts: { guarded?: boolean } = {},
 ): Promise<void> {
   const docsText = cluster.docs.map((d, i) =>
     `${i + 1}. [${d.modified_at}] "${d.title}"\n   Facts: ${d.facts?.slice(0, 300) || 'none'}\n   Context: ${d.context?.slice(0, 200) || 'none'}`
@@ -364,11 +466,13 @@ Return ONLY the JSON array. /no_think`;
 
     // Check for existing similar consolidated observation (avoid duplicates).
     // Two-stage gate: Jaccard shortlist + name-aware merge safety (Ext 3).
+    // `guarded` forces gate enforcement when the heavy lane calls us.
     const existing = findSimilarConsolidation(
       store,
       pattern.observation,
       cluster.collection,
-      sourceDocIds
+      sourceDocIds,
+      opts.guarded === true,
     );
     if (existing) {
       // Ext 2: contradiction gate. Before merging into an existing
@@ -524,7 +628,8 @@ export function findSimilarConsolidation(
   store: Store,
   observation: string,
   collection: string,
-  candidateSourceDocIds: number[]
+  candidateSourceDocIds: number[],
+  forceEnforce: boolean = false,
 ): { id: number; observation: string; source_doc_ids: string } | null {
   // ORDER BY id ASC makes "first shortlist hit" deterministic across
   // SQLite plan changes — the dry-run legacy parity case relies on
@@ -553,7 +658,12 @@ export function findSimilarConsolidation(
 
   if (shortlist.length === 0) return null;
 
-  const dryRun = process.env.CLAWMEM_MERGE_GUARD_DRY_RUN === "true";
+  // `forceEnforce` (v0.8.0 Ext 5) lets the heavy lane override the env
+  // `CLAWMEM_MERGE_GUARD_DRY_RUN` so operators can experiment with dry-run
+  // in the light lane without weakening heavy-lane guarantees.
+  const dryRun = forceEnforce
+    ? false
+    : process.env.CLAWMEM_MERGE_GUARD_DRY_RUN === "true";
 
   // Dry-run: preserve EXACT legacy behavior — return the first shortlist
   // hit (the pre-Ext-3 code iterated the SELECT rows in order and returned
@@ -723,18 +833,38 @@ function updateTrends(store: Store): void {
  *
  * Only considers decision/preference/milestone/problem observations from the
  * last 7 days that haven't already been used as sources for deductions.
+ *
+ * `opts` (v0.8.0 Ext 5):
+ *  - `maxRecent`  override batch size (default 20)
+ *  - `staleOnly`  order by recall_stats.last_recalled_at ASC with
+ *                 documents.last_accessed_at fallback instead of
+ *                 modified_at DESC
+ *  - `guarded`    forward-compat marker — no-op in v0.8.0 because the Phase 3
+ *                 guardrails always enforce their gates regardless
  */
-async function generateDeductiveObservations(
+export async function generateDeductiveObservations(
   store: Store,
-  llm: LlamaCpp
+  llm: LlamaCpp,
+  opts: DeductiveOptions = {},
 ): Promise<DeductiveSynthesisStats> {
+  const maxRecent = opts.maxRecent && opts.maxRecent > 0 ? opts.maxRecent : 20;
+  const staleOnly = opts.staleOnly === true;
   const stats = emptyDeductiveStats();
   // Find recent high-value observations not yet used in deductions
   const DEDUCTIVE_TYPES = ['decision', 'preference', 'milestone', 'problem'];
+
+  const orderBy = staleOnly
+    ? `ORDER BY COALESCE(rs.last_recalled_at, d.last_accessed_at, d.modified_at) ASC, d.modified_at ASC`
+    : `ORDER BY d.modified_at DESC`;
+  const joinClause = staleOnly
+    ? `LEFT JOIN recall_stats rs ON rs.doc_id = d.id`
+    : ``;
+
   const recentObs = store.db.prepare(`
     SELECT d.id, d.title, d.facts, d.narrative, d.observation_type, d.content_type,
            d.collection, d.path, d.modified_at
     FROM documents d
+    ${joinClause}
     WHERE d.active = 1
       AND d.content_type IN (${DEDUCTIVE_TYPES.map(() => '?').join(',')})
       AND d.observation_type IS NOT NULL
@@ -747,9 +877,9 @@ async function generateDeductiveObservations(
           WHERE dd.content_type = 'deductive' AND dd.active = 1
         )
       )
-    ORDER BY d.modified_at DESC
-    LIMIT 20
-  `).all(...DEDUCTIVE_TYPES) as {
+    ${orderBy}
+    LIMIT ?
+  `).all(...DEDUCTIVE_TYPES, maxRecent) as {
     id: number; title: string; facts: string; narrative: string;
     observation_type: string; content_type: string; collection: string;
     path: string; modified_at: string;