claude_memory 0.12.1 → 0.13.1

data/docs/influence/strands-agent-sops.md

@@ -0,0 +1,163 @@
+# Strands Agent SOPs Analysis
+
+*Analysis Date: 2026-05-01*
+*Source: AWS Open Source Blog — "Introducing Strands Agent SOPs: Natural Language Workflows for AI Agents"*
+*URL: https://aws.amazon.com/blogs/opensource/introducing-strands-agent-sops-natural-language-workflows-for-ai-agents/*
+*Type: Article (not a repo). PyPI package `strands-agents-sops`, GitHub `strands-agents/agent-sop`.*
+
+---
+
+## Executive Summary
+
+**Agent SOPs** are markdown-based "Standard Operating Procedures" that wrap an AI agent's instructions in a parameterized, RFC-2119-keyworded, chain-able format. Amazon teams use thousands of them internally; the open-source release ships four reference SOPs (`codebase-summary`, `pdd`, `code-task-generator`, `code-assist`) and tooling to author/load them via Strands Agents, MCP prompts, Claude Skills, or raw model calls.
+
+**Verdict for ClaudeMemory**: ClaudeMemory already implements most of what SOPs propose, under different names. Skills (`/distill-transcripts`, `/release`, `/study-repo`) *are* SOPs. The hook context injection pipeline already chains stages (ingest → distill → resolve → publish). The current distillation prompt already uses RFC-2119-ish "MUST" language for the reason-clause requirement.
+
+The genuinely novel ideas — and the only ones worth a closer look — are:
+1. **Explicit parameter contracts** at SOP entry (Required/Optional with defaults), versus our skills that take freeform `$ARGUMENTS`.
+2. **Progress checkpoints + resumability** for long-running workflows (`✅ Step 1 complete` style markers the agent emits).
+3. **Self-describing format spec** (`strands-agents-sops rule` command) that lets Claude author new SOPs from a description.
+
+The rest is old news for us. **Recommendation: do not adopt the Strands library or format. Borrow two narrow ideas (resumability + explicit parameters) into `/distill-transcripts` if and only if real distillation runs are large enough to fail mid-batch.**
+
+## What an SOP Actually Is
+
+Per the article, an SOP is markdown with these conventions:
+
+- **RFC 2119 keywords** (MUST / SHOULD / MAY) for behavioral control.
+- **Required/Optional parameters block** with defaults — gathered from the user via natural-language dialogue at invocation time.
+- **Numbered steps** the agent executes sequentially.
+- **Progress annotations** the agent prints as it goes (`✅ Validated codebase path exists`).
+- **Output artifacts** in a conventional `.sop/<name>/` directory, used as handoff between chained SOPs.
+
+Example parameter declaration (the only verbatim format snippet in the article):
+
+```
+Required Parameters:
+• codebase_path: Path to the codebase to analyze
+Optional Parameters:
+• output_dir: Directory where documentation will be stored (default: ".sop/summary")
+```
+
+Invocation surfaces:
+
+- **Strands Agents (Python)**: `Agent(system_prompt=code_assist, tools=[editor, shell])` — SOP becomes the system prompt.
+- **MCP**: SOPs registered as MCP *prompts* (the `prompts/list` + `prompts/get` channel), invoked with `@codebase-summary` in Kiro CLI / `/prompts` listing.
+- **Claude Skills**: A CLI converts SOPs to Anthropic Skill format.
+- **Direct LLM**: paste into a model's message and run.
+
+Composition is **sequential chaining via artifact handoff** — `codebase-summary` writes docs, `pdd` reads them. No nesting/include directive is shown.
+
+## How This Maps to ClaudeMemory Today
+
+| Strands concept | Our equivalent | Status |
+|---|---|---|
+| Markdown SOP | `lib/claude_memory/commands/skills/*.md` (Anthropic Skills) | ✅ Have it |
+| MCP prompts surface | `MCP::QueryGuide` registers `memory_guide` via `prompts/list`+`prompts/get` | ✅ Have it |
+| RFC-2119 "MUST" in instructions | `distill-transcripts.md:38-43` uses MUST for reason-clause embed | ✅ Have it |
+| SessionStart prompt injection | `hook_command.rb:213` writes `hookSpecificOutput.additionalContext` | ✅ Have it |
+| SOP chaining via artifacts | `Ingest → Distill → Resolve → Store → Publish` (CLAUDE.md L72-79) | ✅ Have it (DB rows are the artifacts) |
+| AI-assisted SOP authoring | `/skill-creator` skill | ✅ Have it |
+| Format spec exposable to Claude | `strands-agents-sops rule` CLI | ⚠️ Partial — our distillation prompt is in `distill-transcripts.md`, not exposed as a tool |
+| Required/Optional parameter contract | We pass `$ARGUMENTS` as freeform text | ❌ Missing |
+| Progress checkpoints + resumability | `/distill-transcripts` runs end-to-end; no mid-batch checkpoint | ❌ Missing |
+| Reference SOPs (`codebase-summary` etc.) | N/A — wrong domain | ❌ Not applicable |
+
+The pattern is clear: we independently arrived at the same architecture. The two bullets in the "Missing" rows are the only candidates worth thinking about for adoption.
+
+## Where SOPs Could Improve Distillation
+
+### 1. Resumability for `/distill-transcripts`
+
+**Current state.** `/distill-transcripts --limit 10` calls `memory.undistilled`, processes items one-by-one, calls `memory.mark_distilled` after each. If the run aborts mid-batch (rate limit, context exhaustion, user Ctrl-C), the items processed before the abort are marked, the rest are not. There is no explicit checkpoint file, but the DB itself is the checkpoint.
+
+**SOPs angle.** SOPs add visible progress markers (`✅ Item 4/10 complete`) and a resume contract (`if .sop/distill/state.json exists, skip processed items`).
+
+**Honest verdict.** Our DB-as-checkpoint already handles resumability. The visible-progress angle is a UX win for big runs but not a correctness improvement. **Do this only if we add a `--limit 100`+ workflow that users actually run.** Today nobody runs that.
+
+### 2. Required/Optional Parameter Block in Skills
+
+**Current state.** `/distill-transcripts` accepts `--limit N` parsed implicitly inside the skill body. Other skills accept `$ARGUMENTS` as a freeform blob. Users discover parameters by reading the skill markdown.
+
+**SOPs angle.** Declared parameter blocks let the agent prompt the user before running ("what's the codebase_path?"), and let tooling (an SOP registry, MCP prompt list) introspect the contract.
+
+**Honest verdict.** Anthropic Skills allow YAML frontmatter that already does this (`argument-hint`, parameter docs). We are under-using that frontmatter. **Cheap, safe improvement.** Adding a `Parameters:` block to the top of `distill-transcripts.md`, `release.md`, `study-repo.md` (and friends) costs ~10 minutes per skill and makes them self-documenting to both humans and any agent reading them.
+
+### 3. Format-Spec-As-Tool for Authoring
+
+**Current state.** `/skill-creator` exists. It has the format knowledge in its prompt body.
+
+**SOPs angle.** `strands-agents-sops rule` is a CLI command that prints the SOP format spec to stdout, so any agent can `Bash` it and learn how to author one. This is a small but real ergonomic win — the spec lives in one place, not duplicated into every "make a new skill" prompt.
+
+**Honest verdict.** Marginal. We don't have a sprawl of skill-authoring locations to consolidate. **Defer indefinitely** unless we start writing many more skills.
+
+## What NOT to Adopt
+
+- **The Python package itself.** Strands is a Python agent framework; we're a Ruby gem. No reuse path.
+- **The `.sop/<name>/` artifact directory convention.** We persist via DB rows + `claude_memory.generated.md`. Adding a parallel filesystem artifact tree would just add cleanup burden and an out-of-DB state to reconcile.
+- **The four reference SOPs (`codebase-summary` etc.).** Wrong domain — they're for code-workflow agents, not memory pipelines. Nothing to lift.
+- **Renaming "skills" to "SOPs" in our docs.** Anthropic's term is *Skills*; that's the term Claude Code users know. Adopting Amazon's term creates confusion for zero gain.
+- **Sequential-only chaining as an enforced pattern.** Our pipeline already chains, but we should keep room for parallel work (e.g., NullDistiller layer 1 runs synchronously in the ingest hook regardless of layer 2/3). SOP chaining is sequential by construction.
+
+## Adoption Opportunities
+
+### Medium Priority
+
+#### 1. Parameter blocks in skill frontmatter
+
+- **Value**: Self-documenting skills; Claude can prompt the user for missing parameters instead of guessing from `$ARGUMENTS`. Better intro-spectability for any future skill registry UI.
+- **Evidence**: Article's `Required Parameters / Optional Parameters` block — only structural snippet quoted verbatim; Anthropic Skills format already supports `argument-hint` and similar fields we under-use.
+- **Implementation**: Add `## Parameters` section near the top of `lib/claude_memory/commands/skills/distill-transcripts.md`, `release.md`, `study-repo.md`, `quality-update.md`, `improve.md`. Format: bullet list with `name: description (default: …)`.
+- **Effort**: ~30 minutes total across all skills.
+- **Trade-off**: Tiny doc maintenance burden; no runtime cost.
+- **Recommendation**: ADOPT (low-cost, high-clarity).
+
+### Low Priority
+
+#### 2. Progress markers + explicit checkpoint file in `/distill-transcripts`
+
+- **Value**: Better UX on long runs (users see progress); cleaner resume after mid-batch failure.
+- **Evidence**: Article shows `✅ Validated codebase path exists` style output; SOPs document progress to support resumability.
+- **Implementation**: Have `/distill-transcripts` print `[N/M] item <docid> → K facts` after each `memory.mark_distilled`. Optionally, write `.claude/distill_state.json` with `last_processed_content_id` so a re-run can resume.
+- **Effort**: ~1 hour for stdout markers; ~3 hours including a state file with safe-resume semantics.
+- **Trade-off**: State file adds another moving piece; DB already handles correctness, so this is purely UX. Not worth doing until somebody actually runs `/distill-transcripts --limit 100+` regularly.
+- **Recommendation**: DEFER. Revisit if dashboard/usage data shows multi-hundred-item distillation runs.
+
+#### 3. SOP-style format spec exposed as MCP prompt
+
+- **Value**: Lets a future "make me a new skill" agent fetch our skill format spec via MCP `prompts/get` instead of duplicating it.
+- **Evidence**: Article's `strands-agents-sops rule` command — same idea.
+- **Implementation**: Add a `skill_authoring_guide` prompt to `MCP::QueryGuide` alongside `memory_guide`.
+- **Effort**: ~1 hour.
+- **Trade-off**: Solves a problem we don't yet have. We have one skill-authoring location (`/skill-creator`).
+- **Recommendation**: DEFER until skill sprawl is a real problem.
+
+### Features to Avoid
+
+- **Generic "SOP runtime" abstraction** layered over our skills: pure ceremony. Anthropic Skills already give us the runtime.
+- **`.sop/<name>/` artifact filesystem** parallel to our DB: doubles state, doubles cleanup, halves the value of having a curated SQLite store.
+- **Adopting the term "SOP" anywhere user-facing**: term collision with Skills.
+
+## Implementation Recommendations
+
+**Phase 1 (do this in any 0.12.x release).** Add `## Parameters` blocks to the existing skill markdowns. ~30 minutes. Closes the only meaningful gap from this study.
+
+**Phase 2 (defer).** Progress markers + `.claude/distill_state.json` checkpoint, only after we see real users running large distillation batches.
+
+**Phase 3 (avoid unless triggered).** MCP-prompt-exposed skill format spec, only after we have ≥3 skill-authoring locations to consolidate.
+
+## Architecture Decisions
+
+**Preserve.** Our DB-as-checkpoint substrate, our use of Anthropic Skills as the SOP equivalent, our `additionalContext` injection on SessionStart, our distillation prompt's explicit reason-clause requirement.
+
+**Adopt.** Explicit parameter declarations in skill frontmatter (Phase 1).
+
+**Reject.** Strands Python package, `.sop/` artifact tree, generic SOP runtime abstraction, terminology adoption ("SOP" → user-facing).
+
+## Key Takeaways
+
+1. **We are already doing this.** Strands describes a class of patterns — markdown instructions, MCP prompts, parameterized invocation, sequential chaining, RFC-2119 vocabulary — that ClaudeMemory has independently. The existence of Strands is *validation*, not a roadmap.
+2. **Anthropic Skills ≈ Strands SOPs.** Same idea, different label, different ecosystem. Don't refactor toward Strands; we'd just be renaming Skills.
+3. **One narrow win.** Explicit parameter declarations in skill frontmatter cost ~30 minutes and make our skills self-documenting. Worth doing.
+4. **One narrow defer.** Progress markers + checkpoint files in `/distill-transcripts` are real UX improvements *if* anyone runs distillation at scale; today nobody does. Revisit when the data says to.
+5. **No deep architectural shifts.** Nothing in the article justifies a redesign of our distillation, storage, or prompting pipelines.

data/docs/quality_review.md

@@ -9,6 +9,51 @@
 
 ---
 
+## Observational Layer — Pre-Merge Review (2026-06-18)
+
+**Review Date:** 2026-06-18
+**Previous Review:** 2026-04-28 (51 days ago)
+**Scope:** the observational-layer branch (`claude/observational-layer-design-7662r9`, 22 commits ahead of `origin/main`, ~57 files). Two parallel expert-lens reviews — core/data layer (migrations 019/020, `Domain::Observation`, `SQLiteStore` observation methods, `Resolver`) and pipeline layer (`NullDistiller` extraction, renderer, `Reflector`, `ContextInjector`, MCP handlers, dashboard panel).
+
+**Verdict:** No hard merge-blockers. The append-only/tombstone discipline is consistent, `Domain::Observation` is a clean immutable value object, border validation (`coerce_observation`) is textbook, and test coverage on this surface is above the repo average. The items below are latent correctness edge-cases + cleanups; none break the system as shipped (the layer is experimental and observations are project-scoped only today).
+
+### High — address or consciously accept before merge
+
+- **H1 · `consolidate_observations` read-modify-write race** — `lib/claude_memory/store/sqlite_store.rb:805-826` (Evans/Bernhardt). The source `SELECT` and `combined = sources.sum{…}` run *outside* the `@db.transaction` block, and the tombstone `UPDATE` (822) doesn't re-assert `status: "active"`. Two reflectors firing close together (PreCompact + SessionEnd) could double-count corroboration or re-tombstone an already-consolidated source. SQLite's single-writer lock narrows the window but doesn't close the gap. **Fix:** move the read inside the transaction and re-filter `status: "active"` on the update. Mechanical, ~1h incl. spec.
+- **P1 · `noise_body?` over-broad — drops legit prose** — `lib/claude_memory/distill/null_distiller.rb:53,169` (Grimm/Bernhardt). `NOISE_BODY_SIGNATURE` matches `::`, `{}`, `=>` anywhere, so `"decided to adopt ClaudeMemory::Observation as the model"` is silently dropped — common in Ruby prose. Confirmed at runtime. **This is a precision-tuning *design* change to extraction behavior, not a mechanical fix** — per the project's data-driven-design convention it should be surveyed against real corpus data before retuning, not changed blind. Candidate: narrow to strong structural markers (`def `/`class `/`module `, JSON `","`/`":\s*"`, `$(`, `&&`, `||`), drop bare `::`/`{}`/`=>`, add a false-negative spec corpus.
+- **P2 · cross-scope promote nudge — latent wrong-DB landmine** — `lib/claude_memory/hook/context_injector.rb:159-194` + `lib/claude_memory/mcp/handlers/management_handlers.rb:88` (Evans/Beck). `fetch_promotion_candidates` flat-maps project+global stores; the reflection block emits `[obs #<id>]` (a *per-DB* autoincrement id) with no scope; `promote_observation`/`consolidate_observations` default `scope: "project"`. A global-store candidate would route the promote call to the wrong DB. **Dead today** (nothing writes observations to the global DB), but a genuine landmine if global observations ever appear. **Fix:** restrict reflection candidates to the project store + document, or scope-tag the nudge line (mirror `emitted_facts_by_scope`). ~1-2h.
+- **M1 · `consolidate_observations` has zero test coverage** — the most complex method in `sqlite_store.rb` is the only observation method with no specs (the `< 2 → nil` guard, summed-corroboration-tips-threshold, multi-row tombstone are all load-bearing and untested). Add specs alongside the H1 fix. ~1h.
+
+### Medium
+
+- **M2/P7 · token-estimate `/4.0` heuristic duplicated 3×** — `sqlite_store.rb:714,819` + `dashboard/observations.rb:98` (Metz DRY). The compression-ratio correctness depends on both halves using the same divisor. Extract `Core::TokenEstimate.from_chars`/`.from_bytes`. ~1h.
+- **M3/P10 · `recent_observations` `min_priority` name inverted** — `sqlite_store.rb:731` (Beck revealing-names). Filters `priority <= min_priority`, but priority is inverted (1=important), so a higher "minimum" returns *more* rows. Rename `max_priority_value`/`importance_floor`. ~30m + callers.
+- **M4 · `Resolver#apply` `@return` rdoc stale** — `resolver.rb:29` omits the `:observations_created` and `:fact_ids` keys this PR adds. ~10m.
+- **M5 · `fact_ids` array silently contains `nil`s** — `resolver.rb:45,61` (Grimm meaningful-returns). Comment promises positional alignment with `extraction.facts`, but `:discard` contributes `nil`; the sole consumer already `.compact.first`s. Pick one contract and document it (or compact at source). ~20m.
+- **P3 · `clean_observation_body` is 6 chained gsubs, brittle** — `null_distiller.rb:178` (Bernhardt). Pure text logic buried as a private method; extract to a tested `Observe::BodyCleaner` with an input→output spec table. ~2h.
+- **P4 · `extract_decisions`/`extract_observations` double-scan `DECISION_PATTERNS`** — `null_distiller.rb:105,138` (Metz DRY). Two full regex passes per chunk on the P95<5ms hot path; titles and bodies also diverge, complicating later corroboration. ~2-3h.
+- **P5 · `consolidate_observations` reuses `coerce_observation(args)` on the whole tool-args hash** — `management_handlers.rb:151` (Grimm border). Couples the consolidation tool's param names to the observation schema and pulls in `kind`/`priority` defaults the caller may not intend. Pass a narrowed hash. ~1h.
+- **P6 · dashboard N+1 across stores** — `dashboard/observations.rb:48-99` (Evans, bounded). 8-12 small aggregate queries per load; acceptable at store-count 2 but the `.where(status: "active")` predicate repeats ~6×. One `group_and_count(:status)` per store. ~2h.
+
+### Low (fast-follow cleanups)
+
+- **L1** `persist_observations` reaches into raw hashes (`obs[:body]`…) — coerce through `Domain::Observation` at the border; defaults are triplicated. `resolver.rb:81`. ~1-2h.
+- **L2** `respond_to?(:observations)` guard is dead defensiveness — `Extraction` always defines it. `resolver.rb:82`. ~5m.
+- **L3/P8** status strings (`"active"`/`"consolidated"`/`"expired"`) and `2`/`3` literals scattered — add `STATUSES`/reuse `PROMOTION_THRESHOLD`/`INFO` from `Domain::Observation`. ~30m.
+- **L4** `increment_corroboration` returns void while sibling mutators return `updated > 0` — make symmetric. `sqlite_store.rb:772`. ~10m.
+- **L5** migration index DDL uses raw `CREATE INDEX` rather than Sequel's `index` DSL — idiomatic-only. ~30m, optional.
+- **L6** `consolidate_observations` doesn't thread `session_id` (synthesized rows get NULL) — document the intent or thread it. ~5m.
+
+### What's done well
+
+Append-only/tombstone discipline honored end-to-end with "row preserved, not deleted" specs; `Domain::Observation` immutable/frozen/self-validating with intention-revealing predicates; `corroborated?(threshold)` kept a total function (threshold injected, not hard-coded); resolver change genuinely additive (observations persist *inside* the extraction transaction, "no observations → fact behavior unchanged" tested); pure Sequel datasets throughout (no raw SQL except index DDL); promotion-gate tests pin the anti-hallucination invariant; `coerce_observation` border validation with `filter_map` drops invalids without aborting the batch; the Go-language case-sensitivity fix is clean and well-specced.
+
+### Recommended pre-merge action
+
+Fix the mechanical items — **H1** (read-inside-transaction), **P2** (defuse cross-scope), **M1** (the missing consolidation spec), **M4/M5** (document this PR's own new `apply` surface). Flag **P1** (regex retune) for a data-driven decision — do not change blind. Everything else is tracked here as fast-follow.
+
+---
+
 ## Post-0.11 Investigation: Hallucination Rate Metric Calibration (2026-04-30)
 
 When #48 (hallucination-rate metric) was first run against this project's real DB, it surfaced numbers that *looked* alarming:

data/lib/claude_memory/audit/checks.rb

@@ -226,6 +226,155 @@ module ClaudeMemory
         end
       end
 
+      # Scopes whose stores carry an observations table. Observation checks
+      # iterate both DBs because observations may be project- or global-scoped.
+      OBSERVATION_SCOPES = %i[project global].freeze
+
+      # Valid observation lifecycle states. Anything else means a writer or
+      # migration stamped a status the resolver/reflector never produce.
+      OBSERVATION_STATUSES = %w[active consolidated expired].freeze
+
+      def observation_stores(manager)
+        OBSERVATION_SCOPES
+          .map { |scope| [scope, manager.store_if_exists(scope.to_s)] }
+          .reject { |_, store| store.nil? }
+      end
+
+      # C011 — Orphaned observations (provenance points at a missing content item).
+      def orphaned_observations(manager)
+        observation_stores(manager).flat_map do |scope, store|
+          content_ids = store.content_items.select(:id)
+          orphans = store.observations
+            .exclude(source_content_item_id: nil)
+            .exclude(source_content_item_id: content_ids)
+            .select(:id)
+            .all
+          next [] if orphans.empty?
+
+          [Finding.new(
+            id: "C011",
+            severity: :warn,
+            title: "#{orphans.size} observation(s) in #{scope} DB reference a missing content item",
+            detail: "An observation's source_content_item_id should point at the content_items row it was distilled from. A dangling pointer means the source row was pruned or never existed, so the observation's provenance can no longer be explained.",
+            suggestion: "Inspect with memory.observations. These rows are append-only; if the provenance is unrecoverable, consolidate or expire them via the Reflector (PreCompact/SessionEnd) rather than deleting.",
+            fact_ids: orphans.map { |r| r[:id] }
+          )]
+        end
+      end
+
+      # C012 — Promotion consistency (promoted_at ⇔ promoted_fact_id, fact must exist + be active).
+      def observation_promotion_consistency(manager)
+        observation_stores(manager).flat_map do |scope, store|
+          active_fact_ids = store.facts.where(status: "active").select(:id)
+
+          missing_fact_id = store.observations
+            .exclude(promoted_at: nil)
+            .where(promoted_fact_id: nil)
+            .select(:id).all
+          dangling_fact = store.observations
+            .exclude(promoted_fact_id: nil)
+            .exclude(promoted_fact_id: store.facts.select(:id))
+            .select(:id, :promoted_fact_id).all
+          inactive_fact = store.observations
+            .exclude(promoted_fact_id: nil)
+            .exclude(promoted_fact_id: active_fact_ids)
+            .exclude(promoted_fact_id: dangling_fact.map { |r| r[:promoted_fact_id] })
+            .select(:id, :promoted_fact_id).all
+          missing_timestamp = store.observations
+            .exclude(promoted_fact_id: nil)
+            .where(promoted_at: nil)
+            .select(:id).all
+
+          obs_ids = (missing_fact_id + dangling_fact + inactive_fact + missing_timestamp).map { |r| r[:id] }.uniq
+          next [] if obs_ids.empty?
+
+          problems = []
+          problems << "#{missing_fact_id.size} promoted but missing promoted_fact_id" unless missing_fact_id.empty?
+          problems << "#{dangling_fact.size} promoted_fact_id pointing at a non-existent fact" unless dangling_fact.empty?
+          problems << "#{inactive_fact.size} promoted into a non-active fact" unless inactive_fact.empty?
+          problems << "#{missing_timestamp.size} have promoted_fact_id but no promoted_at" unless missing_timestamp.empty?
+
+          [Finding.new(
+            id: "C012",
+            severity: :error,
+            title: "#{obs_ids.size} observation(s) in #{scope} DB have inconsistent promotion state",
+            detail: "Promotion must be atomic: a promoted observation has both promoted_at set and promoted_fact_id pointing at an existing, active fact. Violations (#{problems.join("; ")}) mean mark_observation_promoted ran partially or the target fact was later rejected/superseded, leaving the observation pointing at nothing usable.",
+            suggestion: "Inspect the fact with claude-memory explain <fact_id>. If the fact was intentionally rejected, the observation should be re-opened for re-promotion via memory.promote_observation; if mark_observation_promoted half-ran, re-run promotion.",
+            fact_ids: obs_ids
+          )]
+        end
+      end
+
+      # C013 — Tombstone-chain validity (consolidated_into must point to a real,
+      # non-self row and a consolidated observation must not stay active).
+      def observation_tombstone_chain(manager)
+        observation_stores(manager).flat_map do |scope, store|
+          obs_ids = store.observations.select(:id)
+
+          dangling = store.observations
+            .exclude(consolidated_into: nil)
+            .exclude(consolidated_into: obs_ids)
+            .select(:id, :consolidated_into).all
+          self_link = store.observations
+            .exclude(consolidated_into: nil)
+            .where(Sequel[:consolidated_into] => Sequel[:id])
+            .select(:id).all
+          active_but_tombstoned = store.observations
+            .exclude(consolidated_into: nil)
+            .where(status: "active")
+            .select(:id).all
+          consolidated_without_link = store.observations
+            .where(status: "consolidated", consolidated_into: nil)
+            .select(:id).all
+
+          flagged = (dangling + self_link + active_but_tombstoned + consolidated_without_link).map { |r| r[:id] }.uniq
+          next [] if flagged.empty?
+
+          problems = []
+          problems << "#{dangling.size} consolidated_into → missing observation" unless dangling.empty?
+          problems << "#{self_link.size} consolidated_into self-link" unless self_link.empty?
+          problems << "#{active_but_tombstoned.size} active yet have a consolidated_into target" unless active_but_tombstoned.empty?
+          problems << "#{consolidated_without_link.size} status=consolidated with no consolidated_into keeper" unless consolidated_without_link.empty?
+
+          [Finding.new(
+            id: "C013",
+            severity: :error,
+            title: "#{flagged.size} observation(s) in #{scope} DB have a broken tombstone chain",
+            detail: "Tombstoning is append-only: a superseded observation gets status=consolidated and consolidated_into pointing at the surviving keeper. Violations (#{problems.join("; ")}) corrupt the lineage — recall could surface a tombstoned row, or a consolidated row could orphan its history.",
+            suggestion: "Inspect with memory.observations. Re-run the deterministic Reflector (fires on PreCompact/SessionEnd) to re-derive consolidation; a self-link or active+tombstoned row indicates a Reflector bug — file it rather than hand-editing the append-only table.",
+            fact_ids: flagged
+          )]
+        end
+      end
+
+      # C014 — Status / corroboration sanity (known status set, corroboration ≥ 1).
+      def observation_status_corroboration(manager)
+        observation_stores(manager).flat_map do |scope, store|
+          bad_status = store.observations
+            .exclude(status: OBSERVATION_STATUSES)
+            .select(:id).all
+          bad_corroboration = store.observations
+            .where { corroboration_count < 1 }
+            .select(:id).all
+
+          flagged = (bad_status + bad_corroboration).map { |r| r[:id] }.uniq
+          next [] if flagged.empty?
+
+          problems = []
+          problems << "#{bad_status.size} with status outside #{OBSERVATION_STATUSES.inspect}" unless bad_status.empty?
+          problems << "#{bad_corroboration.size} with corroboration_count < 1" unless bad_corroboration.empty?
+
+          [Finding.new(
+            id: "C014",
+            severity: :warn,
+            title: "#{flagged.size} observation(s) in #{scope} DB have invalid status/corroboration",
+            detail: "Every observation should carry a known lifecycle status (#{OBSERVATION_STATUSES.join("/")}) and at least one sighting (corroboration_count ≥ 1; a fresh insert counts as 1). Violations (#{problems.join("; ")}) break the promotion gate (which keys off corroboration) and the recall filters (which key off status).",
+            suggestion: "Inspect with memory.observations. A corroboration_count < 1 means increment_corroboration math went negative; an unknown status means a migration or external writer bypassed insert_observation. Re-derive via the Reflector if possible.",
+            fact_ids: flagged
+          )]
+        end
+      end
+
       def normalize_convention(text)
         text.to_s
           .downcase

data/lib/claude_memory/audit/runner.rb

@@ -20,6 +20,10 @@ module ClaudeMemory
         bare_conclusion_rate
         project_starvation
         auto_memory_unimported
+        orphaned_observations
+        observation_promotion_consistency
+        observation_tombstone_chain
+        observation_status_corroboration
       ].freeze
 
       Result = Data.define(:findings, :stats) do

data/lib/claude_memory/commands/census_command.rb

@@ -138,7 +138,7 @@ module ClaudeMemory
 
         predicates = db[:facts].select(:predicate, :status).group_and_count(:predicate, :status).all
           .each_with_object(Hash.new { |h, k| h[k] = Hash.new(0) }) do |row, acc|
-            acc[row[:predicate].to_s][row[:status].to_s] += row[:count].to_i
+          acc[row[:predicate].to_s][row[:status].to_s] += row[:count].to_i
         end
 
         entity_types = db[:entities].group_and_count(:type).all.each_with_object(Hash.new(0)) do |row, acc|

data/lib/claude_memory/commands/hook_command.rb

@@ -205,14 +205,26 @@ module ClaudeMemory
 
         t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
         injector = ClaudeMemory::Hook::ContextInjector.new(manager, source: source)
-        context_text = injector.generate_context
+        # On PreCompact (context pressure) inject only the reflection nudge, not
+        # the full snapshot; everywhere else inject the full SessionStart context.
+        context_text = if payload["hook_event_name"] == "PreCompact"
+          injector.reflection_context
+        else
+          injector.generate_context
+        end
         duration_ms = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0) * 1000).round
 
         if context_text
           response = {
             hookSpecificOutput: {
               hookEventName: "SessionStart",
-              additionalContext: context_text
+              # Wrap in <claude-memory-context> so a later ingest strips our own
+              # injected snapshot back out (ContentSanitizer lists this tag in
+              # SYSTEM_TAGS). Without the wrapper, memory's injected facts and
+              # observation log leak into the transcript and get re-distilled —
+              # a self-ingestion feedback loop. Claude still reads the content;
+              # only the re-ingestion path treats it as strippable.
+              additionalContext: "<claude-memory-context>\n#{context_text}\n</claude-memory-context>"
             }
           }
           stdout.puts JSON.generate(response)
@@ -243,7 +255,8 @@ module ClaudeMemory
           top_fact_ids: injector.emitted_fact_ids.first(10),
           top_facts_by_scope: (by_scope if by_scope.any?),
           top_subjects: injector.emitted_subjects.uniq.first(10),
-          fact_count: injector.emitted_fact_ids.size
+          fact_count: injector.emitted_fact_ids.size,
+          observation_count: injector.emitted_observation_count
         }.compact
 
         ClaudeMemory::ActivityLog.record(store,

data/lib/claude_memory/commands/initializers/hooks_configurator.rb

@@ -126,7 +126,9 @@ module ClaudeMemory
                   {"type" => "command", "command" => ingest_cmd, "timeout" => 30,
                    "statusMessage" => "Saving memory..."},
                   {"type" => "command", "command" => sweep_cmd, "timeout" => 30,
-                   "statusMessage" => "Sweeping memory..."}
+                   "statusMessage" => "Sweeping memory..."},
+                  {"type" => "command", "command" => context_cmd, "timeout" => 5,
+                   "statusMessage" => "Reflecting on memory..."}
                 ]
               }],
               "SessionEnd" => [{

data/lib/claude_memory/commands/install_skill_command.rb

@@ -15,6 +15,10 @@ module ClaudeMemory
         "distill-transcripts" => {
           file: "distill-transcripts.md",
           description: "Distill transcripts — extract facts/entities/decisions from undistilled content"
+        },
+        "reflect" => {
+          file: "reflect.md",
+          description: "Reflect on observations — consolidate the episodic log and promote corroborated observations to facts"
         }
       }.freeze