claude_memory 0.12.1 → 0.13.1

data/docs/dashboard.md

@@ -116,6 +116,29 @@ sqlite-vec coverage. Each surfaces an actionable fix string (e.g.,
 "Run `claude-memory init` to install the standard hook set"). Status
 escalates to the worst individual check (error > warning > healthy).
 
+### Observations (episodic layer, 0.13.0+)
+
+The episodic counterpart to the fact-based panels. Facts answer "what is
+true"; **observations** are an append-only log of "what happened" in your
+sessions. Surfaced both as a first-class sidebar panel (headline numbers)
+and an Advanced → Observations tab (full detail):
+
+- **Counts by status / kind / priority** — active vs. consolidated vs.
+  expired; decision / preference / event; 🔴 important / 🟡 maybe / 🟢 info.
+- **Corroboration + promotion readiness** — how many observations have been
+  seen enough times (≥2, the corroboration gate) to be promotable to facts,
+  and the highest corroboration count seen. Promotion is the
+  anti-hallucination gate: a one-off mention never becomes a fact.
+- **Compression ratio** — source content tokens ÷ observation tokens, the
+  Mastra-style measure of how much the episodic log condenses raw sessions.
+- **Recent timeline** — the latest observations, newest first, with their
+  priority markers.
+
+Promote a corroborated observation to a fact with `memory.promote_observation`
+(or `claude-memory observations promote`), merge related ones with
+`memory.consolidate_observations`, or run the `/reflect` skill for a guided
+survey → consolidate → promote pass.
+
 ### Activity drill-down
 
 Clicking any moment opens a modal with the parsed payload, prettified JSON,
@@ -190,3 +213,8 @@ WAL writer lock open across page loads.
   flags as stale.
 - `claude-memory dedupe-conflicts` / `reclassify-references` — one-shot
   cleanups for what the Conflicts and Knowledge → References panels surface.
+- `claude-memory observations [list|promote|consolidate]` *(0.13.0+)* — the
+  CLI mirror of the Observations panel: list/inspect the episodic log
+  (`--kind`, `--status`, `--scope`, `--json`), promote a corroborated
+  observation to a fact, or consolidate related ones. `claude-memory stats
+  --observations` prints the counts summary.

data/docs/improvements.md

@@ -1,6 +1,6 @@
 # Improvements to Consider
 
-*Updated: 2026-05-23 - Added AI Memory Systems Landscape Analysis (Nakajima/Opus 4.6 Research article, 2026-03-26) — meta-study of 7 benchmarks + ~12 systems. Four High Priority items: graph traversal as third RRF source (#64), temporal-aware retrieval (#65), bi-temporal schema cleanup (#66), LongMemEval integration (#67). One promotion: improvement #57 (provenance-strength ranking) Medium → High, validated as the "soft epistemic separation" pattern. See `docs/influence/ai-memory-systems-2026.md`. Previously: 2026-05-01 - Added Strands Agent SOPs study (article, not repo) — one M-priority item (parameter blocks in skill frontmatter); rest already implemented or deferred. See `docs/influence/strands-agent-sops.md`. Previously: 2026-04-28 (post-0.10.0) - Restructured 1.0 punchlist around milestone versions. **0.11.0 "Trust & Cost"** ships #47 (token budget), #48 (hallucination rate), #51 (claude-memory show), #53 (first-week ROI nudge — moved up from post-1.0), and a 3-scenario prototype of #49 (harm benchmark). **0.12.0 "Release Discipline"** ships #49 full corpus, #50 (CLAUDE.md baseline), #52 (benchmark scoreboard). **1.0.0** lands soak-validated #54/#55/#56 if time + new #59 API stability audit. See `docs/1_0_punchlist.md` for the full plan with calendar targets. Also added 2026-04-28: two ranking-signal gaps surfaced by the Mercury / "Why Karpathy's Second Brain Breaks" article (Zaid, 2026-04-28) — provenance-strength-aware ranking (#57) and reinforcement/decay scoring (#58). Earlier 2026-04-28 updates: opened the 1.0 punchlist track + added cq study. Previously: 2026-03-30 - Re-studied all 7 influencer repos. New recommendations: CLAUDE_CONFIG_DIR support (#26, from episodic-memory), Usage Stats / ROI Tracking (#27, from grepai v0.35.0). New Features to Avoid: AST-Aware Code Chunking (QMD), Custom Instructions via Env Var (lossless-claw v0.5.2), OpenClaw Context Injection (claude-mem v10.6.0). Repos with no changes: kbs (v0.2.1), claude-supermemory (v2.0.1), episodic-memory (v1.0.15). Previously: 14 features implemented through 2026-03-24.*
+*Updated: 2026-06-17 - Added #70 (recall-preserving fact precision on real transcripts — live obs-experiment found Layer-1 fact noise from prose/comparisons/English-word collisions; claim-context gating was measured to crater the distillation benchmark Fact F1 0.958→0.64, so the lever is downstream: wire ReferenceMaterialDetector into the ingest path / Layer-2. Observation extraction was tightened in-branch; facts left at baseline). Earlier 2026-06-16 - Added #69 (self-heal the FTS rank index after concurrent ingest — live incident: hook-vs-MCP write contention leaves `ORDER BY rank` malformed while data stays intact, silently degrading recall until a manual `compact`). Earlier 2026-06-16 - Added Mastra Observational Memory study — one High Priority item (#68, episodic observation layer: Observer + Reflector + observation→fact promotion bridge) and one Medium item (compression/cache telemetry + LongMemEval episodic suite). Key insight: ClaudeMemory has no episodic layer; observations ("what happened") complement facts ("what is true"). See `docs/influence/mastra-observational-memory.md`. Previously: 2026-05-23 - Added AI Memory Systems Landscape Analysis (Nakajima/Opus 4.6 Research article, 2026-03-26) — meta-study of 7 benchmarks + ~12 systems. Four High Priority items: graph traversal as third RRF source (#64), temporal-aware retrieval (#65), bi-temporal schema cleanup (#66), LongMemEval integration (#67). One promotion: improvement #57 (provenance-strength ranking) Medium → High, validated as the "soft epistemic separation" pattern. See `docs/influence/ai-memory-systems-2026.md`. Previously: 2026-05-01 - Added Strands Agent SOPs study (article, not repo) — one M-priority item (parameter blocks in skill frontmatter); rest already implemented or deferred. See `docs/influence/strands-agent-sops.md`. Previously: 2026-04-28 (post-0.10.0) - Restructured 1.0 punchlist around milestone versions. **0.11.0 "Trust & Cost"** ships #47 (token budget), #48 (hallucination rate), #51 (claude-memory show), #53 (first-week ROI nudge — moved up from post-1.0), and a 3-scenario prototype of #49 (harm benchmark). **0.12.0 "Release Discipline"** ships #49 full corpus, #50 (CLAUDE.md baseline), #52 (benchmark scoreboard). **1.0.0** lands soak-validated #54/#55/#56 if time + new #59 API stability audit. See `docs/1_0_punchlist.md` for the full plan with calendar targets. Also added 2026-04-28: two ranking-signal gaps surfaced by the Mercury / "Why Karpathy's Second Brain Breaks" article (Zaid, 2026-04-28) — provenance-strength-aware ranking (#57) and reinforcement/decay scoring (#58). Earlier 2026-04-28 updates: opened the 1.0 punchlist track + added cq study. Previously: 2026-03-30 - Re-studied all 7 influencer repos. New recommendations: CLAUDE_CONFIG_DIR support (#26, from episodic-memory), Usage Stats / ROI Tracking (#27, from grepai v0.35.0). New Features to Avoid: AST-Aware Code Chunking (QMD), Custom Instructions via Env Var (lossless-claw v0.5.2), OpenClaw Context Injection (claude-mem v10.6.0). Repos with no changes: kbs (v0.2.1), claude-supermemory (v2.0.1), episodic-memory (v1.0.15). Previously: 14 features implemented through 2026-03-24.*
 *Sources:*
 - *[thedotmack/claude-mem](https://github.com/thedotmack/claude-mem) - Memory compression system (v10.6.3, re-studied 2026-03-30)*
 - *[obra/episodic-memory](https://github.com/obra/episodic-memory) - Semantic conversation search (v1.0.15, re-studied 2026-03-30 — no changes)*
@@ -9,6 +9,7 @@
 - *[tobi/qmd](https://github.com/tobi/qmd) - On-device hybrid search engine (v2.0.1+unreleased, re-studied 2026-03-30)*
 - *[MadBomber/kbs](https://github.com/MadBomber/kbs) - Knowledge-Based System with RETE inference (v0.2.1, studied 2026-03-30 — no changes)*
 - *[martian-engineering/lossless-claw](https://github.com/martian-engineering/lossless-claw) - DAG-based lossless context management (v0.5.2, re-studied 2026-03-30)*
+- *[Mastra Observational Memory](https://mastra.ai/blog/observational-memory) - Text-based dual-agent episodic memory (studied 2026-06-16)*
 
 This document contains only unimplemented improvements. Completed items are removed.
 
@@ -291,6 +292,61 @@ Source: 2026-04-28 1.0 readiness review (`docs/1_0_punchlist.md` #6)
 
 ---
 
+### 69. Self-Heal the FTS Rank Index After Concurrent Ingest
+
+Source: 2026-06-16 live incident on `claude/observational-layer-design-7662r9` — observed first-hand, not a study.
+
+**Gap.** The contentless FTS5 index (`content_fts`) silently drifts into a broken state under concurrent writers: the ingest hook (`claude-memory hook ingest`) and the MCP server (`store_extraction` → `Index::LexicalFTS#index_content_item`) both write the same WAL DB, and a large ingest produces `"Database busy, retrying"` (`Store::RetryHandler`) followed by an FTS index where **plain `MATCH` works but `... ORDER BY rank` raises `database disk image is malformed`**. `integrity_check` passes and all rows are intact — so `recall`/`recall_index` ranking is silently degraded (the rank query throws or returns nothing) while nothing looks wrong. The only fix today is the user manually running `claude-memory compact` (which `rebuild_fts` + vacuums). Documented in `docs/influence/...` gotchas and surfaced reactively in the dashboard (`lib/claude_memory/dashboard/api.rb:338`), but never repaired automatically. Severe form (btree corruption, plain `MATCH` also failing) was separately seen when **two** memory MCP servers ran concurrently — de-duping to a single server removed that, but the benign rank-artifact still recurs from hook-vs-MCP contention alone.
+
+**Implementation.**
+
+- **Cheap probe + self-heal in the sweep tail.** In `Sweep::Maintenance`, add a `repair_fts_rank` step: run a bounded `SELECT rowid FROM content_fts WHERE content_fts MATCH 'a' ORDER BY rank LIMIT 1`; on `malformed`, call `Index::LexicalFTS#rebuild!` (already contentless-safe) instead of requiring a manual `compact`. Sweep already runs on PreCompact/SessionEnd, so recall ranking self-repairs within the session that broke it. Guard with a time budget so a huge index doesn't blow the hook timeout.
+- **Reduce the contention that triggers it.** Raise the SQLite `busy_timeout` and use `BEGIN IMMEDIATE` for the FTS-writing transactions so the ingest hook and MCP server serialize cleanly instead of racing (the retry-loop WARN is the symptom). Confirm both the hook path and `ManagementHandlers#store_extraction` open connections with the same pragmas via `Store::RetryHandler`.
+- **Proactive detection.** Add a `doctor` check that runs the rank probe and reports "FTS rank index needs rebuild — run `claude-memory compact`" (or auto-heals if the sweep step lands). Optionally a `roi_nudge`-style one-liner.
+- **Option (larger).** Evaluate external-content FTS5 over `content_items` instead of contentless — more robust to `rebuild` and avoids the auxiliary-index drift entirely. Note as a follow-up, not part of the first fix.
+
+**Acceptance.**
+
+- An integration test that interleaves a large `hook ingest` with an MCP `store_extraction` against the same DB leaves `MATCH ... ORDER BY rank` working (or self-healed by the next sweep) — no manual `compact` required.
+- `doctor` flags the rank-artifact when present.
+- `"Database busy, retrying"` WARN frequency drops under the contention test.
+
+**Effort.** Medium (~2 days). Self-heal step + pragmas are small; the integration test reproducing concurrent writers is the bulk.
+
+**Why high priority.** It silently degrades `recall` — a core feature — and the user has no signal except empty/unranked results until they happen to run `compact`. Recurs on normal usage (hit live 2026-06-16). Relates to the WAL/connection-release discipline already noted for the dashboard.
+
+---
+
+### 70. Recall-Preserving Fact Precision on Real Transcripts (downstream, not regex)
+
+Source: 2026-06-17 obs-experiment live session — observed first-hand.
+
+**Gap.** Layer-1 `NullDistiller` fact extraction is noisy on real (large, mixed-content) transcripts. A live session on a tiny **Ruby + SQLite** project produced `uses_database = postgres/mysql`, `uses_framework = rails/express`, `uses_language = go`, `deployment_platform = docker` — all from prose mentions, comparisons, negations, instruction/skill text, and English-word collisions (`go` in "want this to go", `express` in "expressive"). This is the documented #48 hallucination problem, now characterized with live data.
+
+**What was ruled out (with data).** Gating fact emission on a usage/claim verb near the entity (`using X`, `deployed on X`, `written in X`) was implemented and measured against `spec/benchmarks/distillation/extraction_spec.rb`:
+
+| | Fact Precision | Fact Recall | F1 |
+|---|---|---|---|
+| baseline | 0.919 | 1.0 | 0.958 |
+| + claim-context | 0.615 | 0.667 | **0.64** |
+
+Regex can't separate terse legit claims (`MongoDB database`, `on AWS`, `Dockerized`) from terse prose mentions (`Postgres/MySQL buy you…`) — claim-context trades recall ~1:1 and even loses precision on the clean corpus. **Reverted.**
+
+**Why distiller-level tightening is mostly off the table (2026-06-17 finding).** The benchmark *enforces* high-recall: case `ext_ent_negative` — "We looked at MongoDB but **decided against it**" — still **expects** `uses_database: mongodb`. The fact distiller is deliberately "extract every mention, filter downstream." So negation/comparison **exclusion** also regresses recall (it would drop the rejected-MongoDB case). The genuine downstream precision lever is Layer-2 / the observation→fact promotion gate, which already prevents Layer-1 noise from being *committed* as corroborated facts.
+
+**Done (recall-safe slice):**
+- ✅ **English-word collision fix** — `go` is matched case-sensitively (`Go`/`golang`) so the verb "go" / "go-to" no longer fires `uses_language=go`. Benchmark Fact Precision **0.919 → 0.935**, Recall held at **1.0** (it removed a real false positive — "my go-to database"). `react`/`rust`/`express` are the same collision class and can follow the same `(?-i:)` pattern, each verified against the benchmark.
+
+**Deferred / not worth it:** wiring `ReferenceMaterialDetector` into the ingest path is a no-op for current Layer-1 (it produces stack predicates, not `convention` facts the detector targets) — skip until Layer-1 emits conventions.
+
+**Acceptance (revised).** Distiller-level work is bounded to recall-safe English-word-collision fixes (benchmark Fact F1 ≥ baseline). Broader fact precision is owned by Layer-2 + the promotion gate, not regex.
+
+**Effort.** Medium. Wiring the existing detector is small; extending its heuristics + a real-transcript precision fixture is the bulk.
+
+**Why this priority.** Fact noise predates the observational layer (it's #48) and is mitigated downstream (promotion gate), so it's not a blocker — but it's the most-visible remaining quality gap on real sessions. The observational layer's Layer-1 observation extraction was tightened in this branch (high-precision/low-recall); facts were deliberately left at baseline pending this recall-preserving approach.
+
+---
+
 ## cq Study (2026-04-28)
 
 Source: docs/influence/cq.md — usefulness-focused study (not internals)
@@ -411,6 +467,122 @@ Source: `docs/influence/ai-memory-systems-2026.md` — meta-study of the Nakajim
 
 ---
 
+## Mastra Observational Memory Study (2026-06-16)
+
+Source: `docs/influence/mastra-observational-memory.md` — architecture study of Mastra's Observational Memory (OM), a text-based dual-agent (Observer + Reflector) episodic memory that compresses raw messages into an append-only, dated observation log living in the context window. SOTA on LongMemEval (84–95%) at 3–6× compression, cache-stable by design.
+
+**Headline finding.** In OM's taxonomy ClaudeMemory is the thing it positions against: a structured *semantic* store injected *dynamically per query*. The gap OM exposes is not retrieval quality — it's that **ClaudeMemory has no episodic layer at all.** Facts answer "what is true"; observations answer "what happened." OM is purely episodic, we are purely semantic. The two are complementary, and we already own analogues of OM's Observer (distillation pipeline) and Reflector (Resolve + Sweep) — they just emit facts, not a narrative log.
+
+### High Priority Recommendations
+
+- [x] **68. Episodic Observation Layer (Observer + Reflector + promotion bridge)** ⭐ — ✅ **Shipped 2026-06-16/17** (phases 1–4)
+  - Value: Adds the missing episodic half of memory (narrative "what happened" log) and a cache-stable injection mode, on top of the existing semantic fact store. The promotion bridge (observation→fact on corroboration) doubles as an anti-hallucination gate for the documented reject-churn problem (distiller commits `uses_database`/`uses_framework` facts from one-off doc example text).
+  - Evidence: `docs/influence/mastra-observational-memory.md`. Our distill pipeline (`lib/claude_memory/distill/`) is already an Observer that emits facts; `resolve/` + `sweep/` is already a Reflector over facts. No episodic store exists.
+  - Implementation (phased):
+    1. ✅ **Shipped 2026-06-16** (schema v19): `observations` table (`body`, `kind`, `priority` 🔴/🟡/🟢, `scope`, `source_content_item_id`, `consolidated_into` lineage, `token_count`); `Domain::Observation`; NullDistiller emits observation rows; Resolver persists them; `memory.observations` read tool. **Append-only with tombstoning, not lossy drop** — preserves provenance.
+    2. ✅ **Shipped 2026-06-16**: two-block SessionStart injection via `ContextInjector` — Block 1 = observation log (🔴 marked, 🟡/🟢 stripped as Mastra does for the actor) ahead of Block 2 = undistilled tail; `Observe::ObservationsRenderer` shared with the published `.claude/rules/claude_memory.observations.md` snapshot; `observation_count` added to the `hook_context` activity event for token/compression measurement.
+    3. ✅ **Shipped 2026-06-17**: `Observe::Reflector` — deterministic, free (no LLM) GC. Dedupes near-identical active observations into the newest (tombstone via `consolidated_into`) and expires stale info-level (🟢) ones past a TTL (`observation_info_ttl_days`, default 30); 🔴/🟡 never expire. Provenance-preserving (rows tombstoned, never deleted). Wired into `Sweep::Maintenance#reflect_observations` → `Sweeper#run!`, so it runs on the existing `PreCompact`/`SessionEnd` sweep — context-pressure-triggered, the analog of Mastra's ~40k-token threshold. (Semantic "merge related/surface patterns" deferred to phase 4 — needs the LLM.)
+    4. ✅ **Shipped 2026-06-17** (schema v20): the observation→fact **promotion bridge**. Dedup folds duplicates' `corroboration_count` into the keeper; once an observation crosses `Domain::Observation::PROMOTION_THRESHOLD` (2) sightings it becomes a promotion candidate. `memory.promote_observation` creates the fact via the resolver, links provenance, marks the observation promoted, and **refuses uncorroborated observations server-side** — the anti-hallucination gate. `ContextInjector` surfaces candidates in a SessionStart "## Observation Reflection" section instructing Claude to promote inline (automatic semantic reflection, no extra API cost); a manual `/reflect` skill drives deep on-demand passes. (Trigger is SessionStart rather than PreCompact — the already-wired free injection point; a PreCompact context hook is a possible future refinement.)
+    5. ✅ **Shipped 2026-06-17** (branch `claude/observational-layer-complete`): the LLM half. **Layer-2 Claude-as-observer** — the SessionStart extraction prompt asks Claude to emit episodic observations in the `observations` field of `memory.store_extraction` (coerced/validated at the handler border, persisted via the resolver), making the log rich where Layer-1 regex is high-precision/low-recall. **Semantic reflection** — `memory.consolidate_observations` merges related-but-differently-worded observations into one synthesized row with *combined* corroboration (which can tip it over the promotion gate), tombstoning the sources; surfaced in the reflection section + `/reflect`. (Also fixed a latent Liskov bug: `ReferenceMaterialDetector#reclassify` dropped observations when a fact was present.)
+    6. ✅ **Shipped 2026-06-17** (branch `claude/observational-layer-complete`): observability + measurement. `Dashboard::Observations` panel (`/api/observations`, Advanced → Observations tab) — counts by status/kind/priority, corroboration + promotion readiness, recent timeline, and a **compression ratio** (source content tokens ÷ observation tokens, Mastra-style). The compression metric is the measurement half of design rec E; a full LongMemEval-style episodic benchmark remains (overlaps #67).
+    7. ✅ **Shipped 2026-06-18** (branch `claude/observational-layer-complete`): polish. **PreCompact reflection trigger** — `claude-memory hook context` injects only the reflection nudge (`ContextInjector#reflection_context`) on PreCompact (context pressure, the Mastra token-threshold analog), wired into the standard PreCompact hook set; not the full snapshot. **Observation↔fact provenance** — `memory.observations` exposes status/corroboration_count/promoted_fact_id/consolidated_into; `memory.explain(fact_id)` shows `promoted_from_observations` (reverse link via `observations_for_fact`). Full observational layer complete; remaining: LongMemEval episodic benchmark (#67).
+  - Effort: Large, phased. Phase 1 ~2-3 days; full arc ~2 weeks. Reuses distill/resolve/sweep/publish/context-hook machinery and `context_tokens` telemetry.
+  - Trade-off: reflection is automatic on *lifecycle events* (compaction/session boundaries), not a wall clock — Claude Code has no timer/cron hook, and Routines/subagents incur separate token budgets (rejected). Observer/Reflector reuse the existing session (no extra API cost). **Augments dynamic recall, does not replace it (user-confirmed 2026-06-16).** See claude-code-guide consultation in the influence doc.
+
+### Medium Priority
+
+- [ ] **Compression / cache telemetry + LongMemEval episodic suite** (see influence doc rec E)
+  - Value: Report compression ratio and token reduction on Trust/Health panels using existing `context_tokens` events (0.11.0). Add a LongMemEval-style long-session suite to DevMemBench to score the episodic layer. Overlaps with existing item #67 (LongMemEval integration) — coordinate.
+  - Effort: Medium. Depends on #68 phase 1-2.
+
+### Features to Avoid (from this study)
+
+- **Two always-on background LLM agents** — violates the no-separate-API-call convention. Observer = context-hook injection; Reflector = deterministic shell-side GC + `PreCompact`-injected semantic consolidation (rides the existing session).
+- **Claude Code Routines / subagents for recurring reflection** — Routines run as a separate scheduled cloud session; subagents run in their own context (~7× tokens). Both incur extra spend; reserve only for an explicitly opted-in one-off backfill.
+- **Lossy drop on reflection** ("never forgives") — we tombstone via `consolidated_into` and retain raw `content_items`; provenance is non-negotiable.
+- **Replacing dynamic recall with a wholesale-loaded log** — augment, don't replace; keep `memory.recall` for targeted lookups.
+
+---
+
+### 71. Exclude the project DB from the published gem (gem is 28MB, ~96MB of it the dogfooding DB)
+
+Source: 2026-06-18 live observation while building the 0.13.0 release gem.
+
+**Problem.** `claude_memory.gemspec` builds its file list from `git ls-files` and rejects `bin/ Gemfile .gitignore .rspec spec/ .github/ .standard.yml` — but **not** `.claude/memory.sqlite3`, which is tracked (per the "always commit the project DB" convention). So the published gem *ships the repo's own dogfooding memory database*: the working-tree DB is ~96MB, compressing to a **28MB gem** (v0.6.0 was 280KB; the gem has been silently growing — 0.9.1 was 19MB — as the DB accumulates). Gem users get nothing from it (they init their own empty DB on install), it bloats every download, and it's trending toward RubyGems' 100MB ceiling.
+
+**Fix.** Add `.claude/` (or at least `.claude/memory.sqlite3` + WAL/SHM siblings) to the gemspec reject filter. Verify with `gem build` that the gem drops to <1MB and that nothing in the gem actually requires the file at runtime (it shouldn't — runtime opens the *user's* DB path via `Configuration`). Add a spec asserting `Gem::Specification.load(...).files` excludes `.claude/memory.sqlite3` so it can't regress.
+
+**Why High.** Low effort, high impact: ~28MB → <1MB published gem, and it removes a slow-growing landmine before it actually exceeds the RubyGems size limit and blocks a release. Not introduced by 0.13.0 — pre-existing and compounding.
+
+**Note on the convention.** This does *not* conflict with "always commit `.claude/memory.sqlite3`" — that's about repo reproducibility for collaborators. Shipping it *in the gem* is a separate, unintended consequence of the `git ls-files` manifest.
+
+---
+
+> **Observational-layer audit (2026-06-23).** A critical examination of every observation in this project's DB found the episodic layer is, in practice, producing ~no useful observations and is injecting noise into sessions. Four root causes (#72–#75), each backed by the live data below. Snapshot at audit time: **113 active observations, 0 consolidated, 0 expired, 0 promoted, every `corroboration_count = 1`**; only `decision`/`preference` kinds; every row traces to a `claude_code` transcript on the `observational-layer-*` branches. The count grew 99 → 105 → 113 *during* the audit session — the dogfooding loop is live and compounding. These supersede the optimistic framing of #68; the mechanism is sound, the inputs and the matching are not.
+
+### 72. Layer-2 (Claude-as-observer) produces **zero** observations — the quality source is silent ⭐
+
+Source: 2026-06-23 observational-layer audit.
+
+**🟡 Structural fix shipped 2026-06-23** (option a). Decoupled the observation-capture ask from the buried, rarely-fired deep-distill paragraph into its own prominent SessionStart section (`ContextInjector#format_observation_capture_prompt` — "## Log What Happened"). This maximizes the chance Claude authors observations, but persistence still rides a `store_extraction` tool call, so **effectiveness is not yet proven** — whether Layer-2 now actually fires is measurable via the `mcp_extraction` content-item source and needs real-session validation (and ultimately the #75 eval). Not closing this until that signal turns positive.
+
+**Problem.** The design's quality observations were always meant to come from **Layer-2** (Claude-as-observer): the SessionStart prompt (`ContextInjector#format_distillation_prompt`) asks Claude to populate the `observations` field of its `memory.store_extraction` call. **The vehicle is dormant.** Evidence (sharpened 2026-06-23 with `mcp_tool_calls.called_at` + `activity_events`):
+- `store_extraction` was invoked **4 times ever — all on 2026-04-17 to 2026-04-30**, i.e. *six-plus weeks before the observational layer (and the `observations` parameter) shipped on 2026-06-16/18*. Those calls **could not** have carried observations; the field didn't exist yet.
+- **Since the layer shipped, `store_extraction` has fired ZERO times.** Layer-2 has never run, not once, in the feature's entire life. (Corroborating: `store_extraction` creates a synthetic `source: "mcp_extraction"` content_item; **0 of the 113 observations trace to `mcp_extraction`** — all to `claude_code` ingest.)
+- **Layer-1 auto-ingest dominates ~100:1**: `activity_events` shows **409 `hook_ingest` vs 4 `store_extraction` total**. Content flows in automatically on every Stop/SessionEnd hook without Claude's involvement, so the Layer-2 deep-distill path — gated on a *fresh session* with *undistilled ≥200-char* items that Claude must *choose* to act on — essentially never triggers.
+
+**Why this is the highest-leverage finding.** Layer-1 (regex over raw transcript) *cannot* produce episodic narrative — it only scrapes fragments (see #74; #74 makes it high-precision, not narrative). The design delegated quality to Layer-2, but Layer-2 is structurally dormant: it isn't a prompt-wording problem (the `store_extraction` schema *does* expose `observations` with a good description, and the prompt *does* ask) — it's that **the path the observations ride on doesn't fire** in normal operation. So the episodic log is, and will remain, 100% Layer-1 scrapes until observation authoring is moved onto a path that actually runs.
+
+**Fix (design fork — not a one-shot code change).** Options:
+- **(a) Author observations on the Layer-1 hook path, but with an LLM.** The hook can't call Claude (no API budget), so this means: have the *next* SessionStart context inject the raw undistilled tail and ask Claude to emit observations directly as part of the normal turn (not gated behind a voluntary `store_extraction` deep-distill). Rides the session, no extra cost — same mechanism the fact-injection uses.
+- **(b) Make Layer-2 fire reliably** — lower the fresh-session/≥200-char gate, or make observation emission a first-class, early, non-optional instruction. Risk: effectiveness is unmeasurable without real A/B sessions, and the headless-recall gap (`project_headless_retrieval_gap.md`) says Claude often won't call MCP tools at all.
+- **(c) Derive observations from what Claude already produces** — the `decisions`/`facts` it extracts are higher-signal than regex scrapes; synthesize observations from those deterministically.
+- Regardless: **add telemetry distinguishing Layer-1 vs Layer-2 observation provenance** so "is Layer-2 firing?" is a dashboard number, not a forensic dig.
+
+**Cross-links.** Blocks the value premise of #68; #74/#73 only make the Layer-1 floor less bad and the loop functional — neither makes the log *good*. This is the one that does.
+
+### 73. Observation dedup/corroboration is normalized-**exact**, so the promotion loop can never fire ⭐
+
+**✅ Shipped 2026-06-23.** Replaced exact-string grouping with greedy clustering over an injected similarity matcher (`Reflector#dedupe_scope`). Default matcher is `Observe::TokenOverlapMatcher` — lexical Jaccard token-overlap (deterministic, free, no embedding dependency), threshold 0.5. Folds the common case (one event re-observed with slightly different wording → corroboration now accumulates and can cross the promotion gate) while keeping unrelated statements apart (Jaccard ~0). **Deliberate limitation, data-driven:** measured that tfidf cosine (0.32) can't separate pure synonym paraphrases from unrelated pairs (0.13) on short bodies, so neither the default lexical matcher nor tfidf folds "use SQLite" vs "chose SQLite" — that needs real embeddings, which can be injected via the `matcher:` seam (any object responding to `similar?(a, b)`) when fastembed is configured.
+
+Source: 2026-06-23 observational-layer audit.
+
+**Problem.** `Observe::Reflector#dedupe` folds observations by `group_by { [scope, normalize(body)] }` where `normalize` is just `downcase` + whitespace-collapse + strip. Two observations corroborate **only if their bodies are byte-identical after lowercasing**. Real captures of "the same thing" are never byte-identical — e.g. the four stored variants "PreCompact hook set.", "PreCompact hook set — the design's Mastra-token-threshold analog.", "PreCompact set alongside ingest + sweep." describe one event but never fold. Result, confirmed in the data: **every observation has `corroboration_count = 1`; 0 consolidated; 0 promoted.** The corroboration gate — the layer's headline anti-hallucination feature — is **dead by construction** on any varied text. It can only fire if the *exact same string* recurs, which regex fragments from different transcript chunks essentially never do.
+
+**Fix.** Corroboration/dedup must be **semantic**, not exact: reuse the existing embedding stack (`Embeddings` + sqlite-vec) to fold observations above a similarity threshold, or fold on a normalized *subject+kind* key rather than the full body. Until then, the promotion gate provides no value and the "graduate after 2 sightings" story is unsupported. Add a spec that two paraphrases of one event corroborate.
+
+**Cross-links.** Without this, #72's quality observations still wouldn't promote.
+
+### 74. Layer-1 Observer ingests code/doc/transcript fragments; `noise_body?` lets ~⅓ through
+
+**✅ Shipped 2026-06-23** (commit `d81a684`). Strengthened `NOISE_BODY_SIGNATURE` (code/JSON `key: "value"`, method calls, spaced table pipes, box-drawing glyphs, `(vector)` labels, JSONL fields) and added a prose-start requirement. Verified against the audit corpus: all five real noise samples now rejected, clean prose kept. **Residual (not a regression):** *truncated-prose* fragments with no code signature (e.g. "encompasses how to use fr…") can still slip — that's the greedy `.+` capture, and ultimately the Layer-2 question (#72), not the noise filter.
+
+Source: 2026-06-23 observational-layer audit.
+
+**Problem.** The Layer-1 Observer runs `decided to (.+)` / `we always|never (.+)` over **raw transcript text**, which on this repo (and any repo whose sessions discuss code) is saturated with trigger phrases inside source, specs, docs, and tool output. The `noise_body?` filter (`NOISE_BODY_SIGNATURE = /\bdef\s|\bclass\s|\bmodule\s|=>|::|","|":\s*"|[{}]|\$\(|&&|\|\|/`) is tuned for code-*syntax* and misses prose/table/transcript fragments. Measured against the live 113: the filter catches **39**, but **38 obvious-noise rows slip through** (≈44% look like noise by a conservative heuristic; manual review puts it higher). Concrete slipped examples actually sitting in the injected log:
+- `[89] decided to use SQLite", kind: "decision", priority: 1) expect(id).to be_a(Integer)…` — a **spec fixture line**.
+- `[104] decided to gate promotion on corroboration" | | Changes | Explicitly…` — a **CHANGELOG table row** (`| |` ≠ `||`, so it dodges the filter).
+- `[48]–[55] / first-person `we always|never`)…` — fragments of the **distiller's own source-code comment**.
+- `[99] · (vector) 78 ├─ How frozen_string_literal…` — **benchmark tree output**.
+
+These are priority-1 `decision` rows, so they *are* injected into Block 1 of SessionStart (observed live in this session's own context) — spending context budget on garbage and risking misdirection (e.g. `[46] decided to use Postgres.`, a fixture string, implying a stack the project doesn't use).
+
+**Fix.** Make Layer-1 high-precision-or-silent: reject bodies that look like code/markdown/transcript (leading `-`/`#`/`|`, table pipes, `key: "value"` shapes, tree glyphs `├─└─`, `(vector)`, backtick-dense spans, JSONL artifacts) — invert the default from high-recall to high-precision, since the recall here is ~all noise. Pair with the `ContentSanitizer`/Observer border. (This is the P1 item from the 2026-06-18 quality review, now empirically confirmed and worse than estimated.)
+
+**Cross-links.** Even fully fixed, Layer-1 is a stopgap until #72; together they decide whether the log is signal or noise.
+
+### 75. The episodic layer has no fair test — this repo is a pathological self-pollution case
+
+Source: 2026-06-23 observational-layer audit.
+
+**Problem.** Every observation traces to this project's *own* `claude_code` design transcripts, whose specs literally contain `insert_observation(body: "decided to use SQLite")` and whose docs are full of "decided to…" prose. claude_memory dogfooding on its own repo is the **worst possible self-test** for the Observer — it maximizes trigger-text density and self-ingestion. So the audit above measures *self-pollution*, not the design's ceiling; a normal Rails/Django app would look very different. We currently have **no measurement of the layer's value on a representative project**, and the optimistic compression/promotion story in #68 was never validated.
+
+**Fix.** Stand up the deferred **LongMemEval-style episodic suite** (#67/#68 medium item) and/or capture a real non-claude_memory project trace as a fixture, and report observation precision (signal vs noise), corroboration/promotion rates, and compression on *that*. Treat "episodic layer adds value" as **unproven** in public materials until this exists (the 0.13.0 blog draft already hedges accordingly). Until then, the self-pollution makes the dashboard Observations panel actively misleading on this repo.
+
+**Cross-links.** Gates any future episodic value claim; depends on #72–#74 being fixed first to be worth measuring.
+
+---
+
 ## Medium Priority
 
 ### ~~18. Shell Completion for CLI~~ ✅ Implemented 2026-03-20

data/docs/influence/mastra-observational-memory.md

@@ -0,0 +1,198 @@
+# Mastra Observational Memory — Influence Study
+
+*Analysis Date: 2026-06-16*
+*Source: Mastra "Observational Memory" (announcement + docs + research, Feb 2026)*
+*Type: Architecture study (feature/paradigm, not a full repo clone)*
+*Status: Design exploration — no code yet. Branch `claude/observational-layer-design-7662r9`.*
+
+*Sources:*
+- *[Announcing Observational Memory (Mastra blog)](https://mastra.ai/blog/observational-memory)*
+- *[Observational Memory docs](https://mastra.ai/docs/memory/observational-memory)*
+- *[Observational Memory research / LongMemEval](https://mastra.ai/research/observational-memory)*
+- *[VentureBeat: "Observational memory cuts AI agent costs 10x..."](https://venturebeat.com/data/observational-memory-cuts-ai-agent-costs-10x-and-outscores-rag-on-long)*
+- *[The Decoder: traffic-light priority system](https://the-decoder.com/mastras-open-source-ai-memory-uses-traffic-light-emojis-for-more-efficient-compression/)*
+
+---
+
+## Executive Summary
+
+### What this is
+
+Mastra Observational Memory (OM) is a **text-based, dual-agent episodic memory** for long-running agents. It compresses raw message history into a structured, append-only log of dated **observations** that lives entirely in the LLM context window — no vector or graph DB. It reports state-of-the-art LongMemEval scores (84.23% with gpt-4o; 94.87% with gpt-5-mini) at 3–6× token compression.
+
+### Why ClaudeMemory cares
+
+In Mastra's taxonomy, ClaudeMemory is the thing OM positions *against*: a structured **semantic** store (subject-predicate-object facts with scope, validity windows, supersession, provenance) injected **dynamically per query** via `memory.recall` and SessionStart fact injection.
+
+The key realization from this study: **ClaudeMemory has no episodic layer at all.** Facts answer "what is true." Observations answer "what happened." OM is purely episodic; ClaudeMemory is purely semantic. An observational layer is not redundant with distillation — it is the missing half.
+
+We already own two of OM's four moving parts in spirit:
+- The **distillation pipeline** (NullDistiller + Layer-2 Claude-as-distiller) is an Observer that emits *facts* instead of a *narrative log*.
+- **Resolve + Sweep** is a Reflector that operates on *facts* instead of *observations*.
+
+The work is therefore: add a narrative episodic store, point the existing Observer/Reflector machinery at it, add a cache-stable injection mode, and — uniquely to us — bridge observations into facts via corroboration.
+
+---
+
+## How Mastra OM Works
+
+### Two-block context window
+
+1. **Observation block** — a compressed, append-only log of dated observations (decisions, key events, distilled facts from older messages). Reads like a log of decisions and actions, not documentation.
+2. **Raw tail** — recent messages not yet compressed.
+
+### The Observer
+
+Fires when raw message tokens cross ~30k (configurable). A separate background agent compresses messages into new dated observations appended to the observation block. Each observation captures one discrete event: a user statement, an agent action, a tool-call result, or a preference expressed in passing. 3–6× compression.
+
+### The Reflector
+
+Fires when observations cross ~40k tokens (configurable). A separate background agent garbage-collects: combines related items, reflects on overarching patterns, and drops context that no longer matters.
+
+### Traffic-light priority
+
+Observations carry 🔴 (important) / 🟡 (maybe important) / 🟢 (info only). The priority is **internal** to the Observer/Reflector pipeline. When observations are presented to the main "Actor" agent, 🟡 and 🟢 are stripped — only 🔴 survives — because the priority emojis serve the memory pipeline and are visual noise to the actor.
+
+### Prompt-cache stability (the headline win)
+
+Because the observation block is **append-only between reflections**, the prompt prefix stays stable and every turn gets a full cache hit. Cache invalidates only on a reflection, which is infrequent. This is explicitly contrasted with RAG-style memory that re-retrieves and rewrites the prompt every turn, busting the cache and producing a variable cost curve.
+
+### Storage
+
+Plain text in a standard backend (Postgres / LibSQL / MongoDB), loaded directly into the context window — not pulled through embedding search.
+
+---
+
+## Comparative Analysis vs ClaudeMemory
+
+| Dimension | Mastra OM | ClaudeMemory today |
+|-----------|-----------|--------------------|
+| Memory type | Episodic (narrative log) | Semantic (SPO facts) |
+| Storage | Plain text in context window | Normalized SQLite + FTS5 + vec0 |
+| Retrieval | None — log loaded wholesale | Dynamic per-query (FTS + vector RRF) |
+| Compression | Observer (LLM), 3–6× | NullDistiller + Claude-as-distiller → facts |
+| Consolidation | Reflector (LLM), lossy drop | Resolve (supersession) + Sweep (TTL/GC) |
+| Provenance | Weak — compression is lossy | Strong — provenance receipts, lineage |
+| Cache behavior | Stable append-only prefix | Per-query injection (cache-busting) |
+| Cost | Two background LLM agents (extra API $) | Claude-as-distiller, zero extra API $ |
+
+**The two systems are complementary, not competing.** OM's weakness is exactly ClaudeMemory's strength (provenance, truth maintenance) and vice versa (episodic recall, cache-stable injection).
+
+---
+
+## Adoption Opportunities (prioritized)
+
+### High Priority
+
+**A. Episodic observation store + Layer-1 Observer.** New `observations` table (schema v19 — v18 was taken by OTel telemetry); NullDistiller emits observation rows alongside facts; `memory.observations` read tool. Append-only with `consolidated_into` lineage (mirrors `fact_links`) rather than Mastra's lossy drop — preserves our provenance guarantee. Zero behavior change to facts.
+
+**B. Cache-stable injection.** Publish `.claude/rules/claude_memory.observations.md` (append-only, dated, 🔴+plain only — 🟡/🟢 stripped as Mastra does for the actor). SessionStart injects a two-block context: Block 1 = consolidated observations (stable, cache-friendly), Block 2 = recent undistilled tail. Front-loading a stable block reduces the per-turn `memory.recall` churn that busts caching. *Honest limit:* we influence Claude Code's cache via a stable `additionalContext` prefix within a session; we don't control it. Cross-session caching remains Claude Code's domain.
+
+**C. The observation→fact promotion bridge (unique to us).** The Reflector promotes *corroborated* observations into structured facts. An observation is low-commitment; a fact is committed truth. Requiring repeated, corroborated sightings before promotion is a natural confidence gate — and directly mitigates the documented hallucination problem where the distiller commits `uses_database`/`uses_framework` facts from one-off example text in docs (today producing reject churn). Observation-first, fact-on-corroboration makes premature hallucinated facts never commit.
+
+### Medium Priority
+
+**D. Automatic Reflector (free) — confirmed feasible.** A consultation with the claude-code-guide agent (2026-06-16) confirms automatic reflection is achievable with zero extra API cost, in two tiers:
+- **Deterministic tier (fully autonomous, no model):** dedupe near-identical observations, drop stale 🟢 past a TTL, merge by entity/time window — pure Ruby, run shell-side inside the `PreCompact` and `SessionEnd` hooks (and the existing Sweep). This needs no model and fires automatically.
+- **Semantic tier (autonomous-on-next-turn, rides the session):** at `PreCompact`, the hook injects a reflection instruction via `additionalContext` ("consolidate the observation log: combine related items, surface patterns, drop the irrelevant"). Claude Code itself performs the consolidation on its next turn, inside the existing session — no separate paid call.
+
+See the dedicated section below for why `PreCompact` is the right trigger and what the constraints are. This **supersedes** the earlier "manual `/reflect` only" recommendation: `/reflect` remains as a manual on-demand deep pass, but reflection is now primarily automatic.
+
+**E. Compression / cache telemetry.** Reuse the `context_tokens` telemetry on `hook_context` events (0.11.0) and the Trust/Health panels to report compression ratio and token reduction. Add a LongMemEval-style episodic/long-session suite to DevMemBench alongside the existing retrieval and truth-maintenance suites.
+
+### Features to Avoid (from this study)
+
+- **Two always-on background LLM agents.** Violates the standing convention against features requiring separate Anthropic API calls. Our Observer = context-hook injection (Claude-as-distiller); our Reflector = deterministic shell-side GC + `PreCompact`-injected semantic consolidation that rides the existing session (see automatic-reflection section).
+- **Claude Code Routines / subagents for reflection.** Routines run as a separate scheduled cloud session (separate token budget); subagents run in their own context window (~7× token burn). Both incur extra spend — rejected for recurring reflection. Reserve them, if ever, for a one-off heavy backfill the user explicitly opts into.
+- **Lossy drop on reflection.** Mastra truly discards observations ("never forgives"). We tombstone via `consolidated_into` and retain raw `content_items` — provenance is non-negotiable.
+- **Replacing dynamic recall.** Augment, don't replace. Observations become a front-loaded episodic block; `memory.recall` stays for targeted lookups.
+
+---
+
+## Proposed Data Model (sketch)
+
+```
+observations  (schema v19)
+  id, ts (event time), session_id
+  body            -- dense narrative text, the observation itself
+  kind            -- user_statement | agent_action | tool_result | preference | decision | event
+  priority        -- 1=🔴 important, 2=🟡 maybe, 3=🟢 info  (internal pipeline signal)
+  scope, project_path
+  source_content_item_id   -- provenance back to the raw transcript chunk
+  consolidated_into        -- Reflector lineage (mirrors fact_links supersession)
+  token_count              -- for budget / compression math
+  status, created_at, reflected_at
+```
+
+## Proposed Pipeline Integration
+
+```
+Transcripts → Ingest → Index (FTS5)
+                   ↓
+   ┌─────────────── Distill ───────────────┐
+   │                                         │
+ Facts (SPO, semantic)            Observations (narrative, episodic)  ← NEW
+   │                                         │
+ Resolve (truth maint.)          Reflect (consolidate / GC / pattern) ← NEW
+   │                                         │
+ Store (facts)                   Store (observations)                 ← NEW
+   │                                         │
+   └──────────── Promotion bridge ──────────┘
+           (Reflector promotes corroborated observations → facts)
+                   ↓
+   Publish: stable observation block (cache-friendly) + fact snapshot
+```
+
+## Automatic Reflection in Claude Code (consultation findings, 2026-06-16)
+
+Source: claude-code-guide agent consultation. Citations: [Hooks reference](https://code.claude.com/docs/en/hooks.md), [Subagents](https://code.claude.com/docs/en/subagents.md), [Routines / scheduled tasks](https://code.claude.com/docs/en/web-scheduled-tasks).
+
+**What does not exist:** There is no timer-, cron-, or idle-based hook event. Hook events are lifecycle-driven only — `SessionStart`, `SessionEnd`, `UserPromptSubmit`, `Stop`/`StopFailure`, `PreCompact`/`PostCompact`, `PreToolUse`/`PostToolUse(Failure)`, plus async signals (`FileChanged`, etc.). No hook can force a model turn or enqueue a prompt; a hook can only inject `additionalContext` that the model acts on at its *next* invocation.
+
+**What this unlocks anyway:** `PreCompact` is the right reflection trigger because it fires precisely when the context window is filling — i.e. on *context pressure*. That is conceptually the same signal Mastra uses (Reflector fires at a ~40k-token observation threshold). So "reflect when memory gets big" maps cleanly onto "reflect when Claude Code is about to compact."
+
+**The free automatic pattern (recommended):**
+- `PreCompact` + `SessionEnd` hooks run the **deterministic** Reflector shell-side in Ruby (dedupe / TTL-drop 🟢 / merge) — fully autonomous, no model, no cost.
+- `PreCompact` injects an `additionalContext` instruction that makes Claude perform the **semantic** consolidation (pattern-finding, observation→fact promotion) on its next turn, inside the existing session — no separate paid call.
+- `SessionStart` injects the consolidated two-block observation log (already in recommendation B).
+
+**Where extra cost is unavoidable (and therefore rejected):** truly autonomous *between-session* reflection on a wall clock. That requires Claude Code Routines (separate paid cloud session) or a headless `claude -p` call or a subagent (~7× tokens) — all separate spend. We accept the tradeoff: our reflection is automatic on *lifecycle events* (compaction, session boundaries), not on a wall-clock timer. For our single-developer, local-first scale this is sufficient.
+
+## Suggested Phasing
+
+1. Schema + Layer-1 Observer (table, NullDistiller rows, `memory.observations`).
+2. Stable two-block injection; measure token/compression deltas.
+3. **Automatic Reflector**: deterministic GC shell-side in `PreCompact` + `SessionEnd`/Sweep.
+4. **Automatic semantic reflection**: `PreCompact` `additionalContext` consolidation instruction + observation→fact promotion bridge. Keep a manual `/reflect` skill for on-demand deep passes.
+
+Phase 4 is where this stops being "Mastra-on-Ruby" and becomes a hybrid episodic+semantic system stronger than either alone.
+
+## Decisions for ClaudeMemory (memory-convention format)
+
+Per the `/study-repo` memory discipline, the following are decisions about **claude_memory itself** derived from this study — to be stored via `memory.store_extraction` (`subject=claude_memory`, `decision`/`architecture` predicate, reason clause embedded) once the memory MCP server is connected. External facts about Mastra stay in this influence doc, not in memory.
+
+- **Decision:** claude_memory will add an episodic observation layer that *augments* (does not replace) the dynamic-recall semantic fact store — because facts answer "what is true" and observations answer "what happened," and we currently have no episodic half; recall stays for targeted lookups while observations provide a stable front-loaded narrative. (User-confirmed "augment" on 2026-06-16.)
+- **Decision:** observation reflection will be automatic via the `PreCompact` and `SessionEnd` hooks rather than a manual-only skill — because Claude Code exposes no timer/cron hook, but `PreCompact` fires on context pressure (the analog of Mastra's token-threshold trigger) and rides the existing session at no extra API cost.
+- **Decision:** the Reflector's deterministic GC runs shell-side in Ruby and its semantic consolidation runs via `PreCompact` `additionalContext` (Claude-as-reflector inline) — to keep automatic reflection within the no-extra-API-cost convention, explicitly rejecting Claude Code Routines and subagents because each incurs a separate token budget.
+- **Decision:** reflection will tombstone superseded observations via a `consolidated_into` link rather than hard-deleting them (unlike Mastra's lossy drop) — to preserve claude_memory's provenance guarantee while still bounding context size.
+- **Decision:** an observation is promoted to a structured fact only after corroboration across multiple observations — because requiring repeated sightings before commitment doubles as an anti-hallucination gate against the documented reject-churn from one-off doc/example text.
+
+> **Pending memory ingestion (deferred 2026-06-16).** These were NOT written to the project DB in the remote design session: `.claude/memory.sqlite3` is git-LFS-backed and the container had only the pointer (no `git-lfs`, real DB not materialized), so writing would have clobbered it. Store them in a local session via `memory.store_extraction` with this payload:
+>
+> ```json
+> {
+>   "scope": "project",
+>   "facts": [
+>     {"subject": "claude_memory", "predicate": "decision", "object": "Add an episodic observation layer that augments (does not replace) the dynamic-recall semantic fact store, because facts answer 'what is true' and observations answer 'what happened' and we currently lack the episodic half; recall stays for targeted lookups."},
+>     {"subject": "claude_memory", "predicate": "decision", "object": "Make observation reflection automatic via the PreCompact and SessionEnd hooks rather than a manual-only skill, because Claude Code exposes no timer/cron hook but PreCompact fires on context pressure (the analog of Mastra's token threshold) and rides the existing session at no extra API cost."},
+>     {"subject": "claude_memory", "predicate": "decision", "object": "Run the Reflector's deterministic GC shell-side in Ruby and its semantic consolidation via PreCompact additionalContext (Claude-as-reflector inline), to keep automatic reflection within the no-extra-API-cost convention, explicitly rejecting Claude Code Routines and subagents because each incurs a separate token budget."},
+>     {"subject": "claude_memory", "predicate": "decision", "object": "Tombstone superseded observations via a consolidated_into link rather than hard-deleting them (unlike Mastra's lossy drop), to preserve claude_memory's provenance guarantee while still bounding context size."},
+>     {"subject": "claude_memory", "predicate": "decision", "object": "Promote an observation to a structured fact only after corroboration across multiple observations, because requiring repeated sightings before commitment doubles as an anti-hallucination gate against reject-churn from one-off doc/example text."}
+>   ]
+> }
+> ```
+
+## Open Questions
+
+- **Augment vs replace recall?** Resolved: **augment** (user-confirmed 2026-06-16). Observations become a front-loaded episodic block; `memory.recall` stays for targeted lookups.
+- **Automatic vs manual reflection?** Resolved: **automatic** via `PreCompact`/`SessionEnd` (deterministic GC shell-side + semantic consolidation injected for the next turn), with `/reflect` retained for manual deep passes. The only thing we forgo is wall-clock between-session reflection, which would cost extra (Routines/subagents) — deliberately rejected.