pi-memory-evolution 0.2.0

package/docs/core-quality.md

@@ -0,0 +1,224 @@
+# Core memory quality: evidence, aging, feedback and second lookup
+
+This is the next **unreleased 0.2.0** development step. It improves the three core
+requirements without introducing an approval queue, vector service or another model
+provider. The default path remains automatic capture/evolution and per-user-turn recall.
+It is an evidence-policy implementation, not a claim of independently verified memory.
+
+## Evidence and automatic updates
+
+New records have an optional, host-assigned `evidence` object:
+
+```text
+basis:    summary | user_statement | tool_observation | manual_correction
+method:   local | model | manual
+sourceId: source event or manual action identifier
+at:       evidence timestamp
+```
+
+Summary extraction is labeled `summary/local`; model consolidation from a summary is
+`summary/model`, not user testimony. Claims from explicit user learning statements are
+`user_statement/model`: the source is the user, but the extraction is still inferred.
+Completed-work observations are `tool_observation/model`, only for nominated project
+states. A failure observation is still an observation, never automatic proof of success.
+Literal `/memory correct` creates `manual_correction/manual` evidence. Manual action
+identifiers and their actual before/after states are retained in event history; they do
+not imply a separate raw-summary row.
+
+The model's allowed output fields remain kind/content/replaces/searchTerms. It cannot
+set evidence, confidence, verification, utility or feedback. Existing evidence/feedback
+is supplied to the background model as historical data, and store guards enforce the
+write policy regardless of its interpretation.
+
+### Conservative replacement
+
+All existing subject/origin, pin, generation, source-time, nomination and transactional
+checks still apply. A replacement cannot change the memory kind to escape its evidence
+policy. Ordinal source priorities are:
+
+| Evidence | Priority |
+|---|---:|
+| Unknown | 0 |
+| Summary | 1 |
+| User statement about a fact/project state | 2 |
+| User statement about a preference/decision | 3 |
+| Tool observation about project state | 3 |
+| Literal manual correction | 4 |
+
+`confirmed` records and explicit `accurate` assessments are also protected at priority 4
+for replacement. These are source appropriateness rules, **not truth probabilities**.
+A current explicit user learning/correction statement may supersede earlier testimony.
+A fresh, linked tool observation can update project progress even if its earlier state
+was manually corrected: correcting “pending” must not freeze progress forever. Neither
+exception overrides pinning, source-time or origin checks.
+
+A lower-priority proposed replacement does not overwrite the stronger old record. Its
+new variant (including a variant locally extracted from that same source) is quarantined
+as `conflicted`; duplicate additions in the same model batch cannot bypass this. Existing
+unrelated variants are not silently quarantined. History records how many weaker
+replacements were withheld. Original evidence remains recallable. The transaction is
+undoable and the processed job is not repeatedly billed as a failure.
+
+An explicit incorporated user reaffirmation or fresh progress observation can refresh
+unchanged content's evidence date. Pure aliases and repeated summaries cannot. If a
+replacement reuses an already stored target value, incorporated newer evidence is
+attached instead of leaving the target's old date/source unchanged. New content does
+not inherit old utility/accuracy feedback. A literal correction clears old aliases and
+feedback; undo restores the actual prior metadata.
+
+**Limit:** conflict detection still depends on the model identifying a `replaces` target
+in its bounded same-origin candidate set. Arbitrary contradictory additions, paraphrases
+and cross-origin identities are not automatically resolved. Multiple source events are
+not treated as independent corroboration; repeated summaries may share the same root
+observation. There is no reinforcement count or model-generated confidence score.
+
+## Gradual aging and relevance-first ranking
+
+Aging is computed at read time; no periodic writes, deletion or artificial timestamp
+refresh is required. All times are numeric and negative ages clamp to zero.
+
+```text
+freshness = floor + (1 - floor) * 2 ** (-ageDays / halfLifeDays)
+```
+
+| Kind | Half-life of the decaying portion | Floor | Hard recall expiry |
+|---|---:|---:|---|
+| project_state | 3 days | 0.50 | after 7 days |
+| fact | 90 days | 0.75 | none |
+| decision | 180 days | 0.85 | none |
+| preference | 365 days | 0.95 | none |
+
+Pinning sets freshness to 1 and exempts age expiry, but adds no credibility, cannot
+revive a conflict, and cannot bypass relevance. Facts/decisions/preferences never vanish
+just because they are old. The project-state cap remains unchanged so upgrading does not
+revive already expired project claims. This first step classifies by memory kind, not
+semantic subtypes such as completed/blocked tasks or volatile configuration facts.
+
+The existing lexical score and subject/literal/coverage/relative-cutoff gates run first.
+The 75% relative cutoff is based on **raw relevance**, not quality-adjusted scores.
+Only eligible matches are ordered by:
+
+```text
+rankScore = relevance * freshness * evidenceWeight * utility * accuracy
+
+evidenceWeight = 1 + 0.04 * sourcePriority
+utility       = useful: 1.05, unhelpful: 0.90, otherwise: 1
+accuracy      = accurate: 1.05, incorrect: 0.50, otherwise: 1
+```
+
+Incorrect assessments are excluded altogether by lifecycle guards; the defensive 0.50
+factor never makes them eligible. Pin/date/ID resolve remaining equal rank scores. The
+constants are conservative policy choices, not statistically calibrated accuracy or an
+online-learned ranker. Changing utility cannot increase source priority. Old records
+without evidence metadata get the neutral evidence factor 1, not fabricated authority.
+
+Short named-attribute queries additionally require the named subject and requested
+attributes. For example, after SQLite authentication is suppressed, PostgreSQL
+authentication and SQLite timeout cannot substitute as answers. This rule applies to
+queries with up to four normalized features, one nonnumeric nonconcept subject and an
+explicit attribute; generic status/progress words are not mandatory answer tokens.
+Explicit origin names may identify a subject, but attribute evidence must be in the body
+or aliases. Numeric replacement values are not mistaken for subject names; Chinese
+`多少` is query framing, not an entity. This is a bounded grammatical heuristic, not
+general entity/coreference recognition.
+
+`/memory explain` exposes raw score, rankScore, source basis/method, freshness, utility,
+accuracy and exclusion reasons. Diagnostics still omit memory bodies and are capped at
+8 KB in memory. `/memory show` includes persisted evidence/feedback and current quality.
+The digest reserves its trust guidance, stays within three claims / 2048 UTF-8 bytes,
+and adds evidence basis/method and `aging` (freshness below 0.85). Its source label points
+to the current evidence when known; legacy records fall back to their original source.
+An `accurate` label means a user assessment, not independent verification.
+
+## Explicit feedback without an approval workflow
+
+Optional controls:
+
+```text
+/memory feedback <id> useful
+/memory feedback <id> unhelpful
+/memory feedback <id> accurate
+/memory feedback <id> incorrect
+```
+
+Two independent last-verdict slots store utility and accuracy, each with a source ID and
+timestamp. Usefulness is a global modest utility preference, not a query-specific
+ranking model. `unhelpful` must not imply the fact is false. `accurate` is an explicit
+user attestation, not a test result. Neither refreshes the evidence clock.
+
+`incorrect` quarantines the selected claim and active exact duplicates within its origin;
+it never touches equal text from another origin. Known queued repeats/observers retire.
+It is reversible through actual event undo, literal correction or explicit resolution.
+A conflicted/forgotten record cannot be revived by positive feedback. Resolution
+preserves the evidence date, preventing stale states from appearing fresh again.
+
+Feedback is local and has no paid learning call. A whole user-role message of the form
+`记忆 <24-hex-id> 有用。` or `memory <24-hex-id> is incorrect.` uses the same path. Chinese
+有用/没用/正确/错误 map to the four verdicts. The parser does not accept examples, quotes,
+questions, multiple statements or vague “wrong”; assistant/tool text is never scanned as
+user feedback. IDs outside this conservative syntax remain addressable by the command.
+Natural-language correction prose continues through the existing model path, not an
+inferred utility score. User-role input has the same trust boundary as existing learning
+cues; this is not a separate identity/authentication system for external RPC clients.
+
+Receipts are keyed by source ID plus memory ID. Replaying after restart or undo cannot
+reapply the old event. Repeating the same verdict with a new event adds no weight or
+memory revision, but its receipt timestamp still prevents an older intervening verdict
+from winning. Feedback older than the current content's evidence timestamp is ignored.
+There is no retrieval/use count and no credit from assistant success claims or citation
+frequency. Feedback changes invalidate stale in-flight model writes transactionally.
+Receipts are audit/deduplication data; forget/undo does not securely erase them.
+
+## Read-only mid-task recall
+
+`memory_recall({ query })` registers through Pi's public tool API. It is available unless
+a user tool allowlist disables it; the extension never changes that allowlist. The query
+is an explicit topic of 1–512 characters. The tool uses the same global relevance,
+lifecycle and quality policy, returning at most three claims within 2048 UTF-8 bytes.
+A context-free continuation returns no arbitrary recent fallback. Cancellation and store
+failures produce safe errors, not invented empty-success results.
+
+No model call is made by the retrieval function, and it neither mutates memory/history
+nor persists queries in the memory DB. Pi still stores normal tool calls/results in its
+transcript, and the surrounding agent model turns incur normal usage. The agent decides
+whether a second lookup is needed; this does not guarantee autonomous gap detection.
+Per-user-turn automatic injection remains enabled even when this tool is disabled.
+
+## Schema and activation
+
+Schema 2/3/4 upgrades transactionally to **5**. Missing retry fields are added as before,
+plus `feedback_receipts(source_id, memory_id, verdict, at)`. Existing memory/source/event
+JSON is not rewritten; IDs, timestamps, tombstones, aliases, source jobs and history are
+preserved. Missing optional evidence stays unknown. No JSONL re-import, evidence-date
+reset, automatic state revival or fabricated verification occurs.
+
+Reads validate evidence/feedback shape and tool-observation kind, and return independent
+copies of nested metadata. Status validates receipt fields in addition to existing
+record/job/history integrity. Unsupported schema versions fail before DDL. Old builds
+reject schema 5: do not manually downgrade its marker.
+
+Stop all Pi processes sharing the state directory and back up the complete state before
+activation. Update/restart all those processes together (or reload after a consistent
+backup); do not mix old writers with the new schema. A rollback requires a matching
+backup. Development tests use temporary directories, not the production database.
+
+## Validation
+
+- This step passed 194 tests, strict typecheck and package dry-run inspection. The later
+  [progress-pipeline follow-up](progress-pipeline.md) records the current validation and
+  the fixes for long tasks, natural requirement capture and interrupted work.
+- New tests cover source/method labels, model metadata forgery rejection, weaker
+  replacement quarantine, same-batch duplicate/replacement bypasses, conflict-clock preservation, fresh progress after manual
+  correction, reused replacements, alias-only stability, aging floors/expiry/pins,
+  relevance-first ordering, source-kind appropriateness, feedback replay/restart/undo,
+  late/repeated feedback, cross-origin isolation, queued observers, nested-copy safety,
+  stale model rejection, malformed metadata, schema-4 migration with unchanged raw JSON
+  and history, named-attribute negatives, and bounded evidence injection.
+- The real installed Pi/Bun test uses a loopback fake model. It exercises actual
+  `memory_recall` schema loading and tool-result payloads, cross-directory lookup without
+  memory/history writes, evidence labels, exact-ID feedback without paid learning,
+  quarantine and no wrong-subject fallback. Existing work-observation, failed-push,
+  startup/timer recovery, model/authentication and conversational tests remain.
+- No live paid provider, production-memory migration, or multi-day natural-usage accuracy
+  evaluation was performed. Test counts show regression coverage, not real-world recall
+  precision or proof of truth. See [design.md](design.md) for unchanged invariants.

package/docs/design.md

@@ -0,0 +1,328 @@
+# Memory evolution v0.2
+
+## Goal
+
+Automatically improve memory with Pi's active model, then recall relevant claims across
+sessions and directories. Directory placement must not determine which memories a
+conversation can use. No approval workflow, secondary agent, external retrieval service
+or automatic changes to project files, system configuration, skills or extension code.
+
+## Runtime
+
+1. `session_start`: open/migrate lazily, check persisted pending/failed work across all
+   origins and start a bounded recovery timer (one source per check, every 15 seconds
+   after the previous check/call ends).
+2. `session_compact`: atomically capture a sanitized session-qualified source and bounded
+   local claims; enqueue semantic consolidation.
+3. `agent_end`: capture explicit cues and natural user requirements/preferences,
+   distinguishing recall questions from new statements. For work with linked operation
+   results, capture a bounded `progress` source, including operations completed before an
+   interrupted final response. Mixed statements/work use separate serialized sources;
+   assistant/tool text never becomes a user preference. Pure memory lookup/inspection
+   without a recognized work operation cannot trigger progress learning.
+4. `before_agent_start`: resolve the current topic, search the whole memory database and
+   append a bounded, source-labeled digest to this turn's system prompt. No model call.
+5. `session_shutdown`: stop polling, abort work, return cancelled jobs to pending without
+   increasing their failure count, drain the serial task chain and close SQLite.
+
+Factories do not write state or start background work. Processes with a nonempty
+`PI_SUBAGENT_AGENT_ID` are skipped. Recovery polls only indexed persisted job state;
+there is no full-ledger backfill or scanning of arbitrary historical Pi session files.
+The timer is unreferenced so it cannot hold a print process open and is never started
+from the factory. Session-scoped context getters resolve the current model/auth at each
+attempt; no stale model snapshot or foreground turn cancellation controls recovery.
+
+## Conversation-aware recall
+
+`adapter/session-context.ts` uses the public `buildContextEntries()` facade, not session
+files or `getEntries()` across branches. It examines at most 4096 trailing active entries
+and at most 4096 messages in total (including retained tails), selecting at most 6 user
+texts of 2,048 UTF-8 bytes each. Consecutive topic-less continuations share a slot; reset
+and unknown-topic barriers remain. Retained user tails survive compaction. Assistant/tool/custom and
+injected messages are excluded, as are raw compaction summaries. Context is transient:
+it is never re-captured as a source. Missing/invalidated context leaves direct-query
+recall available rather than poisoning the hook.
+
+`memory/query.ts` separates conversational recall framing from the subject, without
+rewriting literal paths/filenames or stored evidence. Its query-only discourse vocabulary
+handles Chinese/English asking/remembering phrases; technical memory/recall questions
+retain those concepts. Unknown single-character query subjects remain unmatched barriers,
+not permission to inherit an old topic or generate CJK fragment matches.
+
+The bounded user history is replayed oldest first. Topic-less follow-ups inherit the last
+resolved subject/focus. Related or attribute-only follow-ups carry a **structured** plan:
+current query plus supporting subject context. Thus `SQLite → port? → auth? → continue`
+retains SQLite without treating old port matches as answers about authentication. The
+context does not grow by concatenating every earlier facet. Explicit new subjects stand
+alone, even when unknown to the database; reset phrases stop inheritance. A fresh session
+saying only `continue` identifies no topic and injects nothing.
+
+All stored claims, including `legacy` imports, are candidates. Retrieval is local:
+`Intl.Segmenter` words, exact path/filename identifiers, and a small Chinese/English
+concept map. Synonyms contribute one feature rather than duplicated votes. Model-derived
+`searchTerms` extend the vocabulary; old records need no reprocessing for the bootstrap
+concepts. Paths such as `/work/pi-memory-evolution` do not imply the topic `memory`.
+Common/filler words and generic configuration words cannot qualify a record.
+
+For eligible records, a seen query feature weighs `1 + log((N+1)/(df+1))`; an unseen
+feature weighs **1**, not the maximum IDF. Literals multiply weight by 2. Strongest field
+factors are **assertion body 1, aliases 0.8, quoted question mention 0.25, explicit origin
+identifier 0.2**. Quoted questions (`“...?”`, `「...？」`, `"...?"`) cannot qualify alone:
+a replay note repeating a user's question is not evidence of its answer. Other quoted
+facts remain ordinary evidence. Concept words in origins are excluded. Source IDs, legacy
+labels and cwd have no authority bonus.
+
+Current-focus coverage must be >=45%; at least 3 focus features still require 2 matches.
+A single match cannot qualify alongside unknown non-attribute words. All explicit literal
+constraints must match, including qualified paths rather than only shared basenames.
+Supporting context contributes at 0.35 weight and cannot replace current-focus evidence.
+Named context subject features must match; a concept-only contextual subject needs 60%
+weighted subject coverage. Generic attributes are not subject anchors. Evidence gets mild
+length normalization `0.8 + 0.2 * min(1, 12 / max(1, bodyFeatureCount))`. This cannot bypass
+the subject/coverage gates. Scores below 75% of the best eligible result are rejected.
+After these relevance gates, scores are multiplied by separate evidence/freshness/feedback
+factors; pin/date/ID break remaining ties. These are policy weights, not truth probabilities.
+Short queries with one nonnumeric named subject and explicit attributes require both, with
+generic status/progress excluded from the mandatory-attribute check. Explicit origin names
+can still help a named-origin query; attributes require body/alias evidence. Facet redundancy is tracked per origin **and kind**,
+so a project-state mention cannot suppress a factual/preference answer. Exact same-content
+same-origin duplicates are still removed, and distinct origins remain separate.
+There is **no arbitrary recency fallback** or minimum result count.
+
+One evaluation provides selection and bounded diagnostics. `/memory explain` shows the
+last automatic snapshot, including actual digest count/bytes; `/memory explain <query>`
+previews an explicit query without user-history inheritance. Snapshots hold normalized
+focus/context (up to 32 features each), lifecycle counts and up to 10 scored candidates
+(up to 16 matched features each), with no memory bodies. Serialized diagnostics are
+sanitized/capped at 8,000 bytes plus an ellipsis, held only in the extension instance,
+replaced on every automatic attempt (including empty/error), and never persisted as
+learning input. Selection is not a guarantee of model understanding.
+
+These are precision-oriented heuristics, not semantic verification or universal
+translation. Word segmentation can vary with the runtime's ICU version; uncommon
+languages, short/ambiguous queries and unannotated old records may still be missed.
+
+The digest contains at most 3 claims within 2,048 UTF-8 bytes, with historical-data trust
+guidance reserved first and an explicit warning that selected matches are not the complete
+inventory. Each JSON row includes ID, kind, status, origin, source ID,
+stored update date, evidence basis/method, aging warning, optional explicit accuracy
+assessment and a matching excerpt of up to 400 bytes. The source label uses the current
+evidence source when known (including manual corrections), otherwise the original source ID. Oversized metadata labels
+are clipped with an ellipsis/hash suffix. Origins are provenance hints, not evidence
+that another project's fact applies here. Identical content is deduplicated only within
+one origin: equal port/path text from different contexts can mean different facts.
+
+Forgotten/conflicted claims never recall. Unpinned project-state claims expire from recall
+after 7 days; facts/preferences/decisions have no automatic age deletion. All kinds have
+bounded gradual freshness decay, with separate half-lives/floors and pin exemption.
+Pin/unpin, legacy annotation, explicit feedback and conflict resolution preserve the evidence
+date, and undo restores the prior date. Event history separately records when an operation occurred.
+
+See [core-quality.md](core-quality.md) for the evidence contract, exact ranking policy,
+weaker-replacement guard, replay-safe feedback and read-only mid-task `memory_recall` tool.
+Neither automatic injection nor tool lookup is counted as evidence/usefulness feedback.
+
+## Completed-work observations
+
+The old cue/compaction-only input loop could retain “not committed” even when a normal
+work turn later committed/pushed: that turn was never an evolution source.
+`progress-observation.ts` requires a work request, linked call/result IDs, and at least
+one recognized work operation. It scans up to 4096 current-turn messages, retains at most
+8 observations by operation importance, and preserves chronological order. Commit/push
+and test/process results outrank late routine inspection. Each stored operation <=1024
+bytes, output <=2048 bytes; serialized evidence including request/report, bounded resource
+hints and omitted-count/completion flags stays <=28,000 bytes. Head/tail previews preserve
+failure endings. Internal memory tools and observations referencing the owned state directory
+are excluded. An error/aborted final response uses `completion=interrupted` and no assistant
+report: observed operations are evidence, not proof the entire task finished.
+
+`memory/progress-targets.ts` separately nominates at most 8 active, unpinned project
+states using user topics and explicit operation resources. It does not use answer-recall
+literal gates, per-path top-2, relative cutoffs or facet deduplication. Explicit cd/git -C
+and real checkout roots from file operations can match qualified paths or explicit bare
+project names; capture origin alone supplies no evidence. Pending states receive nomination
+priority. Conflicting absolute paths of the same basename cannot qualify through a topic
+fallback. States expired from ordinary recall may receive new evidence, without reviving
+forgotten/conflicted records. Tool output cannot nominate targets. No tracked related state means no call. `progress` sources are not parsed as local
+summary claims: the model must return `project_state` plus `replaces` naming an eligible
+host-nominated ID. Store guards enforce these restrictions even for a malformed model
+batch. No new preference/fact, unrelated target or cross-origin overwrite is allowed.
+The prompt requires evidence for each outcome and warns that commit/test success is not
+push success, assistant reports are not proof, and failures/unfinished clauses must
+remain. Outputs stay **provisional**: this is not independent success verification.
+
+One qualified work source may add one background call. A mixed statement/work turn can
+add a separate user-source call, serialized with progress to retain their distinct authority.
+There is no assistant-only/ordinary-chat polling or startup transcript replay. Interrupted
+work that was durably captured can recover with the existing retry mechanism. Hard kills
+before agent_end, unknown commands and sources outside the scan/selection budgets remain
+limits; there is no new disk-backed per-tool work journal. `/memory learning` reports
+capture/nomination decisions, while status/history expose processed-versus-changed outcomes.
+See [progress-pipeline.md](progress-pipeline.md) for policy details and validation.
+Existing stale records are not guessed complete on upgrade. New observations/compactions
+can retire them; exact-ID correction remains available. An incorporated new observation
+may refresh an unchanged pending state's evidence date. Alias-only enrichment cannot.
+
+## Model boundary and conservative writes
+
+`adapter/pi-api.ts` calls Pi 0.85's public
+`ctx.modelRegistry.complete(ctx.model, context, options)`, preserving model/provider/auth
+resolution. Model identity is captured before awaiting completion, so switching models
+or invalidating a context cannot mislabel provenance. No credentials are copied to state.
+
+Each input contains a sanitized source (at most 32,000 bytes) and up to 32 recently updated
+active claims **from that source origin**, each capped at 1,440 bytes. This deliberately
+limits automatic replacement authority, **not recall eligibility**. One origin can cover
+multiple projects. The prompt requires an explicitly identifiable same subject/fact and
+preservation of project/resource qualifications; matching cwd alone is not identity.
+
+Output is validated JSON (an outer Markdown fence is tolerated), at most 64,000 bytes
+and 16 claims of 4–480 UTF-16 code units each. Fields are restricted to `kind`, `content`,
+optional `replaces` and `searchTerms`. Aliases are at most 8 sanitized strings of 2–64
+characters, with total JSON <=1024 bytes. Malformed claims/aliases reject the batch.
+The prompt asks for concise Chinese/English aliases, never added facts. Existing text
+can gain aliases without changing its provenance/evidence date; correction clears stale
+aliases and undo restores the actual prior metadata. Unknown, cross-origin, pinned,
+stale, duplicate-target and cyclic replacements are rejected transactionally. Only normal
+`stop` completion is accepted, never truncated/tool/error output. Model paths are not
+used for file operations, and model claims remain `provisional`, not awaiting approval. Host-assigned evidence types
+cannot be supplied by model output. A weaker proposed replacement is withheld; only that
+source's new variant is quarantined and recorded, never an unrelated existing stronger claim.
+A fresh explicit user statement or linked project-state tool observation can still supersede
+older evidence. Unsupported semantic contradictions without a model `replaces` link are
+not detected globally.
+
+Each attempt uses at most one model call, no tools, an 8,192-output-token cap (clamped
+against a smaller model limit), a fresh request session ID and `cacheRetention: "none"`.
+A 120-second outer deadline bounds waiting
+even when a provider ignores abort; remote computation/billing cannot be guaranteed to
+stop. Failed calls retain local summary claims. User-cue prose is saved but needs a
+successful model attempt to become claims; it has no local extraction fallback.
+
+Global recall is **not global rewriting**. Cross-origin variants remain separate instead
+of guessing which project they describe. Exact-ID correction/forget works from any
+session, affecting the target and exact duplicates within its origin, not identical text
+from unrelated origins. Suppression hashes and same-origin conflict controls retain that
+boundary. Some truly equivalent cross-origin corrections will consequently coexist;
+explicit controls are available without becoming approval gates for automatic learning.
+
+## Persistence and lifecycle
+
+One SQLite database, WAL + FULL synchronous mode and private file permissions.
+`sqlite.ts` selects built-in Bun or Node SQLite, with no external database dependency.
+
+- `memories`: claims, optional search aliases, revision/status/layer, source ID, capture origin (`scope`) and hash;
+  optional `suppressedHashes` carries correction history through legacy annotation;
+  optional host-assigned `evidence` and last explicit utility/accuracy `feedback` describe
+  provenance and user assessments, never model-generated confidence.
+- `sources`: sanitized evidence and durable job state/lease/attempt, consecutive failure
+  count, next retry timestamp, last failure timestamp and a fixed error category;
+  `progress` evidence additionally carries a bounded, unique target-ID list.
+- `blocked`: origin-qualified exact-content hashes for forgotten/superseded claims.
+- `events`: actual before/after states, actor, operation timestamp and source/model reason.
+- `feedback_receipts`: exact source-ID/memory-ID idempotency keys, verdict and numeric
+  timestamp, including redundant/late feedback receipts; undo never reopens them.
+- `metadata`: schema/import marker.
+
+The `scope` field records canonical cwd, not an inferred repository/branch/subject identity
+(or an explicit annotation of a legacy record). It is no longer a recall boundary. Reads
+are cached globally or for an explicitly requested origin, invalidated by local commits
+and SQLite `data_version` across connections. New IDs hash a JSON tuple of origin/kind/
+content; existing IDs remain valid. Low-level origin filters are exact, including literal
+`*` values; automatic recall uses the unfiltered reader.
+
+Batches use `BEGIN IMMEDIATE`; no transaction spans network I/O. Source-origin generation
+and job attempt are checked before completion can commit. A lease lasts the attempt
+budget plus 30 seconds (150 seconds by default), preventing another process from stealing
+work at the old 60-second boundary. The timer converts expired leases to an `interrupted`
+failure with backoff; attempt/state checks prevent late results or failures from changing
+a new owner's job. Model waiting and recurring recovery use the same serial task chain;
+queued capture/manual work prevents the timer from piling up duplicate tasks.
+
+Automatic selection/claim both enforce persisted due time and failure budget, ordered by
+retry time then oldest source. Each actual failure schedules 1 minute, 5 minutes, 15 minutes,
+then 1 hour of backoff. Five consecutive failures pause that source with a warning and
+status diagnostics; repeated crashes also consume the budget. Shutdown cancellation does
+not. Successful completion resets the failure fields. Other eligible work continues;
+there is no unbounded per-source model loop. `/memory evolve` selects one newest eligible
+source and can override the delay/cap for one explicit attempt (not reset the budget).
+Completed/retired jobs are never forced to run again. A source resumed in another directory
+retains its original provenance. While Pi is closed no polling occurs.
+
+Only allowlisted error codes are persisted, never exception strings, provider error bodies,
+model response text or credentials. Stage categories distinguish provider/unavailable,
+output limit, invalid output, write rejection, stale output, timeout and interrupted work.
+Status reports retrying/paused counts and up to five failed-job details with next due times;
+normal structural validation is still separate from model/job health.
+
+Capture is idempotent. Raw summaries are evidence only, never a parent recall fallback.
+Exact forgotten content cannot be re-added within its origin under another source/kind.
+Suppression also retires known repeating pending/failed sources, including formatted
+claims beyond the normal 16-claim ingestion quota, except the current valid replacement
+transaction's source. This conservatively skips the whole source's pending model pass;
+unrelated local claims remain, but unlearned prose may need a new source.
+
+Memory reads validate indexed identity/origin/hash against JSON. Undo validates paired,
+unique before/after IDs and only succeeds when the current records still equal the event's
+after state. New records become tombstones rather than being physically erased. Status
+also validates source jobs/history and the schema marker. Unsupported schema versions
+are rejected before DDL. Schemas 2/3/4 upgrade transactionally to 5 without rewriting
+claims/history or resetting evidence timestamps; missing retry columns/indexes and the
+feedback receipt table are added. Old evidence metadata stays absent/unknown.
+Old failures below the cap are due immediately, with unknown cause/time explicitly labeled.
+The source/alias/retry contract is validated on read.
+These detect structural corruption, not all well-formed edits by an owner of the database.
+Undo does not clear suppression hashes or reopen jobs. Forget/undo is not secure erasure.
+
+## Existing data and commands
+
+Earlier 0.2 SQLite records, IDs, histories and origin labels stay intact and become
+eligible for global relevance-based recall, including existing `legacy` claims. The
+schema-2/3/4-to-5 upgrade requires no data copying, JSONL re-import or manual reset.
+Stop/back up before upgrading, reload all processes sharing the DB, and restore a matching
+backup for rollback; older code must not be pointed at a manually downgraded marker.
+
+Original JSONL memory/action ledgers are imported once, without rewriting originals.
+Invalid JSON/actions halt import rather than losing corrections or reviving forgotten
+facts. Parent corrections preserve unchanged children and derive new facts without
+reintroducing explicitly suppressed child content. Missing origins retain the `legacy`
+label. Adoption is an optional annotation, not a recall prerequisite. Completed imports
+are not replayed; earlier discarded revision information is not automatically reconstructed.
+Old signals/proposals/execution plans remain historical files, never automatic actions.
+
+List defaults to all origins, 20 per page sorted by update time then ID; `all` is an alias.
+`here` and `legacy` are optional inspection filters, not recall settings. Search uses only
+its explicit query and returns up to 10 global recallable matches. History shows the latest
+10 global events. `show` includes provenance. Status identifies capture origin and global
+recall mode. There is no full export command; concurrent writes can shift page boundaries.
+
+## Validation and limits
+
+Temporary-directory tests cover lifecycle sequences, real SQLite multi-process writes,
+replay/migration/corruption/undo/timeout, global recall, topic switching, weak matches,
+context tails, provenance and cross-origin write guards. The real-Pi RPC test uses a
+loopback fake OpenAI-compatible model: two automatic updates in one directory, then a
+fresh Pi process/session in another directory to verify recall, contextual follow-ups,
+topic changes, bilingual aliases and exact-ID forget. A real temporary Git repository
+also exercises commit success + push failure through actual tool events and the
+constrained progress-update path. It also verifies model/auth reuse, no approval, startup
+recovery of persisted failures, and recovery after malformed model output via the real
+15-second timer without user activity. Unit tests cover persisted backoff/caps, migration
+from the actual schema-3 table shape, long leases, competing owners, cancellation, timeout,
+late results, suppressed sources, backlog draining, and fixed-code diagnostics.
+The [quality validation record](quality-validation.md) records the three-issue follow-up
+and distinguishes synthetic/real-host checks from real-data read-only replay.
+
+Model inference and sanitization are not perfect. Provisional labels, pin/correct/undo
+are recovery controls, not proof of truth. Lexical matching can miss semantic or cross-
+language equivalence. Missing retained user context after compaction can leave a vague
+follow-up unresolved. Cross-origin semantic identity is not inferred reliably. This is
+one user's agent database, not a multi-user access-control boundary. Raw evidence/history
+grows until deliberately managed; no automatic purge or physical secret erasure is claimed.
+Background model usage is not added to Pi's normal session token accounting.
+
+Node/npm development requires 22.19+ to match Pi 0.85's engine; the standalone Bun host is
+also tested. There is no installed GitHub Actions workflow: `npm run check` and
+`npm run test:pi` run locally. Fake-provider validation is not live-provider accuracy or a
+multi-day TUI trial. See [README.md](../README.md) for commands/recovery, and the historical
+[follow-up review](review-0.2.md) for previously reproduced defects and validation limits.