agenr 1.3.0 → 1.5.0

package/CHANGELOG.md

@@ -1,5 +1,52 @@
 # Changelog
 
+## [1.5.0] - 2026-04-01
+
+Temporal validity, claim extraction, surgeon supersession pass, and OpenClaw auth improvements.
+
+### Added
+
+- **Temporal validity schema.** New `valid_from`, `valid_to`, `claim_key`, `supersession_kind`, and `supersession_reason` columns on entries. Entries can now express when a fact is true in the world, how facts replace each other, and why. All columns are nullable and backward-compatible.
+- **Claim key extraction.** Automatic `claim_key` assignment at store time via LLM classification. Identifies the `entity/attribute` slot an entry occupies (e.g., `jim/home_city`, `agenr/default_model`). Runs as a best-effort pipeline stage — entries store normally if extraction fails or is disabled. Supports entity hint seeding from existing claim keys to prevent namespace drift. Configurable confidence threshold (default 0.8), eligible types (fact, preference, decision), and model override.
+- **Claim extraction in bulk ingest.** `agenr ingest` now runs batch claim extraction with configurable concurrency. Entries that already have a `claim_key` are skipped.
+- **Claim extraction model in CLI setup.** `agenr setup` and `agenr init` now include a "Claim extraction" stage in the model customization flow, alongside extraction, dedup, episode, and surgeon.
+- **Explicit supersession in store pipeline.** `agenr_store` accepts optional `supersedes`, `claim_key`, `valid_from`, and `valid_to` fields. When `supersedes` is provided, the old entry is atomically linked to the new one in a transaction.
+- **Surgeon supersession pass.** New `agenr surgeon run --pass supersession` finds entries that should be linked via `superseded_by` and creates those links with kind and reason. Candidate generation groups by shared claim key (highest confidence) or shared subject+type (medium confidence). Four new tools: `query_supersession_candidates`, `link_supersession`, `assign_claim_key`, `set_validity`. Hard rules enforced in code: no cross-type links, no milestone supersession, no core-expiry supersession.
+- **`claimExtractionModel` in OpenClaw plugin config.** Model override for claim extraction at store time, using OpenClaw's auth system.
+- **Lightweight OpenClaw LLM client.** New `createOpenClawLlmClient` factory resolves credentials from OpenClaw's `modelAuth.resolveApiKeyForProvider` and calls pi-ai's `completeSimple()` directly — no embedded agent runner overhead.
+
+### Changed
+
+- **OpenClaw continuity and episode summaries use lightweight LLM client.** Migrated from the heavy embedded agent runner (`runEmbeddedPiAgent` with temp directories and full agent lifecycle) to the new `createOpenClawLlmClient` for direct `completeSimple()` calls. Same auth, dramatically less overhead.
+- **OpenClaw claim extraction uses OpenClaw auth.** Store-time claim extraction now resolves credentials through OpenClaw's `modelAuth` system instead of agenr's CLI credentials. Fixes silent failures when agenr API keys aren't configured in OpenClaw-hosted contexts.
+- **Surgeon is now pass-aware.** The agent loop, prompt selection, tool selection, and completion guards are all driven by the pass type. Shared infrastructure (budget, audit, trace) remains pass-agnostic.
+- **Multi-surface session continuity.** Improved continuity resolution across different chat surfaces and session types.
+- **Architecture cleanup.** Internal refactoring across multiple phases for cleaner module boundaries.
+
+### Documentation
+
+- **Model config auth boundary.** New architecture doc and updated code comments clarifying the split between CLI-context models (agenr auth) and OpenClaw-context models (OpenClaw auth).
+- **Updated README.md and SURGEON.md** with supersession pass documentation.
+
+## [1.4.0] - 2026-03-30
+
+Configurable summary models, surgeon personal knowledge protection, and documentation overhaul.
+
+### Added
+
+- **Configurable continuity and episode summary models.** New `continuityModel` and `episodeModel` fields in the OpenClaw plugin config (`plugins.entries.agenr.config`) allow overriding the model used for continuity and episode summary generation independently. Falls back to the agent's primary model when unset. Use a fast model like `openai/gpt-5.4-mini` for these structured extraction tasks instead of burning Opus tokens.
+- **Personal knowledge protection in surgeon.** The surgeon retirement pass now has explicit guidance that personal facts (family, pets, hardware, contacts, identity, physical environment) are durable by nature. Only retires personal entries when contradicted or clearly duplicated — not for low recall or moderate importance.
+- **Corpus age awareness in surgeon.** The `_meta` table now tracks `last_bulk_ingest_at`, surfaced via `get_health_stats`. The surgeon heavily discounts `recall_count = 0` when the corpus was rebuilt within 30 days, preventing mass retirements of freshly ingested entries.
+- **New documentation: `docs/EPISODES.md`.** Comprehensive episodic memory docs covering lifecycle, CLI usage, recall modes, temporal window parser, search modes, embeddings, session discovery, and architecture.
+- **New documentation: `docs/SURGEON.md`.** Comprehensive surgeon docs covering tools, CLI commands, dry-run vs apply, budget governance, configuration, protection thresholds, and audit history.
+
+### Changed
+
+- **Continuity summary timeout increased.** Inner timeout bumped from 15s to 30s, read-time wrapper from 20s to 35s. Prevents timeout failures when using slower models for continuity summaries.
+- **Updated `docs/RECALL.md`.** Added unified recall mode routing (`auto`/`entries`/`episodes`), auto-routing rules, temporal window parser reference, and episode search pipeline documentation.
+- **Updated `docs/INGEST.md`.** Added episode ingest section with full flag documentation, behavior differences from entry ingest, session discovery, surface reconstruction, and practical examples.
+- **Updated `README.md`.** Added episodic memory and surgeon to features list, CLI commands table, and new "How Episodes Work" and "How the Surgeon Works" sections with doc links.
+
 ## [1.3.0] - 2026-03-30
 
 Episodic memory — session-level temporal recall for the brain.

package/README.md

@@ -14,18 +14,20 @@ Local-first, durable memory infrastructure for AI agents.
 
 ## What is agenr?
 
-agenr gives agents a persistent brain: a local SQLite database of durable knowledge that survives across sessions, tools, and agent restarts. Instead of relying on fragile prompt state or file-based scratch memory, agents can ingest transcripts, extract decisions and lessons, store them as typed entries, and recall them later with semantic search and memory-aware ranking.
+agenr gives agents a persistent brain: a local SQLite database of durable knowledge that survives across sessions, tools, and agent restarts. Instead of relying on fragile prompt state or file-based scratch memory, agents can ingest transcripts, extract decisions and lessons, store them as typed entries, generate episodic summaries of what happened, and recall them later with semantic search and memory-aware ranking.
 
-It exists because most agent runtimes forget everything important between sessions. Even when a tool has a built-in memory feature, it is often lossy, file-based, or tightly coupled to one surface. agenr keeps memory structured and queryable: facts, decisions, preferences, lessons, todos, events, and relationships live in one local store instead of getting flattened into prompt text.
+It exists because most agent runtimes forget everything important between sessions. Even when a tool has a built-in memory feature, it is often lossy, file-based, or tightly coupled to one surface. agenr keeps memory structured and queryable: facts, decisions, preferences, lessons, milestones, relationships, and session-level episodes live in one local store instead of getting flattened into prompt text.
 
-What makes agenr different is the combination of local-first storage, semantic embeddings, hybrid recall, and adapter-friendly architecture. The core is hexagonal, so multiple agent systems can share the same brain over time. Today the production adapter is the OpenClaw memory plugin, published separately as `@agenr/openclaw-plugin`, and the CLI provides offline ingest and recall against that same database.
+What makes agenr different is the combination of local-first storage, semantic embeddings, hybrid recall, episodic temporal memory, and adapter-friendly architecture. The core is hexagonal, so multiple agent systems can share the same brain over time. Today the production adapter is the OpenClaw memory plugin, published separately as `@agenr/openclaw-plugin`, and the CLI provides offline ingest, recall, and maintenance against that same database.
 
 ## Features
 
-- Hybrid recall: vector similarity, lexical FTS, temporal awareness, recency decay, and importance weighting.
+- Hybrid recall for durable knowledge: vector similarity, lexical FTS, temporal awareness, recency decay, and importance weighting.
+- Episodic memory: session-level summaries with temporal filtering and optional semantic episode search for questions like "what happened yesterday?"
 - LLM-powered knowledge extraction from conversation transcripts.
 - Semantic deduplication using exact hashes, normalized hashes, embeddings, and within-run clustering.
-- Session continuity with predecessor resolution, recent transcript tails, and LLM-generated session summaries.
+- Session continuity with predecessor resolution, recent transcript tails, and LLM-generated continuity summaries.
+- Surgeon maintenance passes for corpus health: run retirement cleanup for stale entries or supersession review for same-slot lineage, both with audit history.
 - Agent tools for `store`, `recall`, `retire`, `update`, and `trace` through the OpenClaw plugin.
 - Native OpenClaw memory plugin that replaces OpenClaw's built-in memory slot.
 - Local-first storage with SQLite/libSQL. Memory stays on your machine; only model and embedding calls leave it.
@@ -46,7 +48,7 @@ It walks through:
 - model selection filtered by the auth method you chose
 - OpenAI embedding key setup
 - OpenClaw detection and optional plugin installation
-- session scanning and optional bulk ingestion of existing transcripts
+- session scanning and optional bulk ingestion of existing transcripts into durable entries and episodic summaries
 
 Run `agenr init` again any time you want to re-run onboarding, reinstall the plugin, or ingest another batch of existing sessions.
 
@@ -112,13 +114,18 @@ Key config fields:
 | `auth`                           | Authentication method: `openai-api-key`, `openai-subscription`, `anthropic-api-key`, `anthropic-oauth`, or `anthropic-token`.                                                     |
 | `provider` / `model`             | Default LLM provider and model used for extraction tasks unless overridden.                                                                                                       |
 | `credentials`                    | Stored manual credentials. Today that can include `openaiApiKey`, `anthropicApiKey`, and `anthropicOauthToken`. The config file is written with locked-down permissions (`0600`). |
-| `credentials.openaiApiKey`       | OpenAI key used for embeddings, and also for extraction when `auth` is `openai-api-key`. Older configs may still rely on legacy `embeddingApiKey` or `apiKey` fallback fields.    |
+| `credentials.openaiApiKey`       | OpenAI key used for embeddings, and also for extraction when `auth` is `openai-api-key`. Agenr no longer reads legacy `embeddingApiKey` or `apiKey` fields.                       |
 | `embeddingModel`                 | Embedding model. Defaults to `text-embedding-3-small`.                                                                                                                            |
 | `extractionModel` / `dedupModel` | Optional per-pipeline overrides so extraction and dedup can use different provider/model pairs.                                                                                   |
 | `extractionContext`              | Optional user context injected into extraction prompts to help the model decide what is worth remembering.                                                                        |
 | `dbPath`                         | Knowledge database location. Defaults to `~/.agenr/knowledge.db` unless overridden.                                                                                               |
 
-Important: when agenr is running as an OpenClaw plugin, session summaries use OpenClaw's configured LLM and auth, not agenr's. Agenr's config is still required for embeddings and for CLI-based ingestion/recall.
+Important: when agenr is running as an OpenClaw plugin, session summaries and store-time claim extraction use OpenClaw's configured LLM auth, not agenr's. Agenr's config is still required for embeddings and for CLI-based ingestion/recall.
+
+Compatibility policy:
+
+- Agenr only supports the current `config.json` shape. Move any legacy `apiKey` value into `credentials.openaiApiKey` or `credentials.anthropicApiKey`, move any `embeddingApiKey` value into `credentials.openaiApiKey`, then remove the legacy fields.
+- Agenr only supports fresh databases and databases already at the current schema version. Older schema versions are no longer auto-migrated at startup.
 
 ## What You Need
 
@@ -128,17 +135,21 @@ Important: when agenr is running as an OpenClaw plugin, session summaries use Op
 
 ## CLI Commands
 
-The current CLI surface is intentionally small. Today the `db` group only exposes `reset`.
-
-| Command                        | What it does                                                                                                             |
-| ------------------------------ | ------------------------------------------------------------------------------------------------------------------------ |
-| `agenr init`                   | Interactive first-run wizard: auth, model selection, OpenClaw detection, plugin install, and optional initial ingestion. |
-| `agenr setup`                  | Configure auth, model defaults, embeddings, and the agenr database path.                                                 |
-| `agenr recall <query>`         | Run the hybrid recall pipeline with optional temporal and type/tag filters.                                              |
-| `agenr ingest <path>`          | Default durable-entry ingest shorthand. Equivalent to `agenr ingest entries <path>`.                                     |
-| `agenr ingest entries <path>`  | Bulk-ingest one file or directory of OpenClaw transcript files into durable knowledge entries.                           |
-| `agenr ingest episodes <path>` | Backfill episodic summaries from OpenClaw session transcripts, including rotated `.reset.*` and `.deleted.*` files.      |
-| `agenr db reset`               | Delete and recreate the knowledge database.                                                                              |
+The CLI surface is still intentionally compact, but it now covers setup, recall, ingest, and corpus maintenance.
+
+| Command                         | What it does                                                                                                                                                      |
+| ------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `agenr init`                    | Interactive first-run wizard: auth, model selection, OpenClaw detection, plugin install, and optional initial ingestion.                                          |
+| `agenr setup`                   | Configure auth, model defaults, embeddings, and the agenr database path.                                                                                          |
+| `agenr recall <query>`          | Run the hybrid recall pipeline with optional temporal and type/tag filters.                                                                                       |
+| `agenr ingest <path>`           | Default durable-entry ingest shorthand. Equivalent to `agenr ingest entries <path>`.                                                                              |
+| `agenr ingest entries <path>`   | Bulk-ingest one file or directory of OpenClaw transcript files into durable knowledge entries.                                                                    |
+| `agenr ingest episodes [path]`  | Backfill episodic summaries from OpenClaw session transcripts, including rotated `.reset.*` and `.deleted.*` files.                                               |
+| `agenr surgeon run`             | Execute a surgeon maintenance pass. Defaults to retirement; use `--pass supersession` for lineage review. Dry-run by default; add `--apply` to mutate the corpus. |
+| `agenr surgeon status`          | Show corpus health plus the latest surgeon run summary.                                                                                                           |
+| `agenr surgeon history`         | Show recent surgeon runs.                                                                                                                                         |
+| `agenr surgeon actions <runId>` | Show the audit trail for one surgeon run.                                                                                                                         |
+| `agenr db reset`                | Delete and recreate the knowledge database.                                                                                                                       |
 
 The OpenClaw plugin also gives the agent five tools directly inside the runtime: `agenr_store`, `agenr_recall`, `agenr_retire`, `agenr_update`, and `agenr_trace`.
 
@@ -148,12 +159,18 @@ Examples:
 # Recall knowledge
 agenr recall "what decisions did we make about the API?"
 
-# Ingest transcripts
+# Ingest transcripts into durable entries
 agenr ingest ~/.openclaw/agents/main/sessions/
 
-# Backfill episode summaries
+# Backfill episodic summaries
 agenr ingest episodes ~/.openclaw/agents/main/sessions/ --recent 30d
 
+# Run the surgeon retirement pass (dry-run by default)
+agenr surgeon run --budget 2.00
+
+# Run the surgeon supersession pass
+agenr surgeon run --pass supersession --budget 2.00
+
 # Reset the database
 agenr db reset
 ```
@@ -172,7 +189,20 @@ Recall is a hybrid pipeline. Agenr embeds the query, retrieves candidates throug
 
 ## How Ingestion Works
 
-Ingestion is transcript-to-memory: parse OpenClaw JSONL transcripts, normalize them, choose whole-file or chunked extraction, run LLM extraction, validate the structured entries, deduplicate them within the ingest run, generate embeddings, and then persist surviving entries through the store pipeline. The current CLI ingest path is OpenClaw-transcript-specific. Details: [docs/INGEST.md](./docs/INGEST.md) and [docs/STORE.md](./docs/STORE.md).
+Agenr has two ingest pipelines over the same transcript corpus:
+
+- `agenr ingest entries <path>` extracts durable typed knowledge such as facts, decisions, preferences, lessons, milestones, and relationships.
+- `agenr ingest episodes [path]` generates one narrative summary per session so the brain can answer temporal questions like "what happened last week?"
+
+Both paths parse OpenClaw transcripts first, but they optimize for different outputs: entry ingest distills durable knowledge and runs semantic dedup across the whole ingest batch, while episode ingest does a session-by-session preflight pass, uses `sessions.json` metadata when available, reconstructs missing surface metadata for rotated files, and writes episodic summaries. Details: [docs/INGEST.md](./docs/INGEST.md) and [docs/STORE.md](./docs/STORE.md).
+
+## How Episodes Work
+
+Episodes are session-level memory artifacts stored separately from durable entries. They preserve temporal narrative: what happened in a session, when it happened, which agent/session it belonged to, and optionally an embedding for semantic episode search. Recall can route narrative or time-bounded questions toward episodes automatically, or combine episode and entry results in mixed mode. For implementation details and the episode recall model, see [docs/EPISODES.md](./docs/EPISODES.md).
+
+## How the Surgeon Works
+
+The surgeon is a maintenance system for the durable-memory corpus. Today it supports two passes: retirement, which reviews stale entries for conservative cleanup, and supersession, which links older entries to newer replacements without deleting history. `agenr surgeon run` is safe by default because it starts in dry-run mode; `--pass supersession` switches the workflow, and `--apply` is the explicit mutation switch. For runtime details, governance, and audit behavior, see [docs/SURGEON.md](./docs/SURGEON.md).
 
 ## Development