AbstractMemory 0.3.2__tar.gz → 0.5.0__tar.gz

{abstractmemory-0.3.2 → abstractmemory-0.5.0}/CHANGELOG.md

@@ -5,9 +5,345 @@ All notable changes to this package will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) (pre-1.0).
 
-## Unreleased
+## [0.5.0] - 2026-11-10
+
+Every memory now carries an authored title and summary, a home keeps a persistent queue of what still needs
+understanding, a record's understanding is rewritten in place with every earlier reading kept, the home's topics are
+formed during sleep, and a host can fence every memory write behind its Stop (backlogs 0059, 0060, 0061;
+[ADR 0001](docs/adr/0001-altered-memories-ripple.md),
+[ADR 0003](docs/adr/0003-one-graph-ledger-provenance-and-revisions.md) decision 6a).
+
+### Upgrading
+
+- **Open homes that form records with `MemorySystem(..., task_queue=True)`.** A missing title or summary is always
+  stored beside the task that writes it, so these refuse a home without a task queue, before writing anything:
+  `world_model_pass`, `dream_pass`, `mine_candidates_pass`, `consolidation_pass`, `wake_cue_dedup_pass`,
+  `sleep_pass`, `communities_pass`, `embed_missing`, the ripple verbs, a `remember_many` record with `pending=...`,
+  and an `engram` whose unnamed spark items get no authored title (`report_only` reads still run). `declare_pending`
+  and `rearm_pending` answer `no_task_queue`. The queue adds its tables (`memq_*`) to the home's file on first open.
+- **Pass an `author` callable.** Memory calls no model: `world_model_pass` / `world_model_update`, `dream_pass`,
+  `mine_candidates_pass`, `consolidation_pass`, `wake_cue_dedup_pass`, `sleep_pass`, `communities_pass` and `engram`
+  take `author(kind, body, context) -> {title, summary, author, reader, sources}`. Without it, or when it fails, the
+  records still form, pending, and every result's `authoring = {authored, failures}` names the failures
+  (`no_author_supplied`, `author_raised`, `invalid_result`).
+- **Treat a title as optional while it is pending.** `MemoryRecordInput.title` may be `None` with
+  `pending=("title",)`, and every read serves a pending title as `None` (`MemoryHandle.title` is `Optional[str]`,
+  display blocks and `record_understanding` carry `pending`), never a placeholder or words from the body. A host's
+  drainer completes `title.author` / `digest.author` tasks with `retitle` / `rewrite_digest`; records stored before
+  0.5.0 without an authored title or summary are brought into the same state with `declare_pending`.
+  `MemoryRecordInput` raises `ValueError` for an empty title without `pending=("title",)`, and for a title given
+  with it.
+- **Memory's passes no longer store their own titles or summaries.** Cards, dreams, offers, consolidation candidates
+  and the dedup day summary form authored or pending; their bookkeeping text ("World model: ...", "Dream: ...") is
+  only the provisional body their tasks author from. `world_model_pass` no longer queues `card.author` for new cards
+  (`digest.author` owns the body; `card.author` remains for legacy cards).
+- **Hosts that fence writes** pass `commit_guard=` and wrap the moment they flip it in `system.write_barrier()`.
+  Two residual windows remain: a LanceDB store takes no write lock (a LanceDB write whose guard passed before the
+  flip may land after it, at most one per concurrent writer; `BarrierHold.uncovered` names it with a `#FALLBACK`
+  warning), and the two SQLite autocommit commit points (`SQLiteTripleStore._write_pin` without a cursor, the
+  off-file graph's `mark_journal_applied`) are covered only by a barrier taken on the same handle.
+- **Community records are never embedded and never recalled**: a store's embedding paths, recall shelves, keyword
+  indexes and memory scans skip kind `community` (`canonical_text.NON_EMBEDDED_KINDS`).
+- `CANONICAL_TEXT_VERSION` is 3 (stored text is never cut). Record digests render as before; a plain assertion
+  with an `original_context` longer than 400 characters keeps its earlier text and vector until `reembed_store`
+  re-derives it.
+- `version_count` is 1 for a record that was never rewritten (was 0); `digest_version` stays 0.
+- `apply_redigestion` is deprecated (it forms a new record with zero usage); use `rewrite_digest`.
+
+### Added: titles and summaries are always authored
+
+- `MemoryRecordInput(pending=(), pending_failure=None)`: `pending` is a subset of `PENDING_FIELDS`
+  (`{"title", "summary"}`). A pending record carries `attributes.pending = {title, summary, since, attempts}` (plus
+  `failure` when authoring failed) and `remember_many` queues its authoring task (`PENDING_TASK_KINDS`:
+  `title.author` / `digest.author`, capability `summaries`, `FORMATION_PRIORITY`). A pending summary is refused on
+  the entity's own words (`UNDERSTANDING_PROTECTED_KINDS`), except a public diary projection's summary.
+- The host's `author` callable on every pass (`formation_authoring`): authored fields are stored with their
+  provenance (`title_provenance`, `understanding`, version 0, cause `formation`) and owe no task; a standing record is
+  never authored again.
+- `engram(..., author=None)`: a spark item without a `name` has its title authored before it forms (it was
+  `<section>-<ordinal>`, such as `trait-0`), else it forms pending; a `name` stays the title. `EngramResult.authoring`.
+- `retitle(record_id, *, title, author, cause, expected_version, ...)`: changes only a record's title, in place, for
+  every kind; the body and the reading stay byte-identical, the previous title is kept as a version, `title_version`
+  is the compare-and-set. Refusals: `empty_title`, `hash_chained` (an owner-direct diary entry whose `entry_hash`
+  attests its title), `closed`, `stale`, `embedder_unavailable`. A digest or card reading that changes the title
+  writes the title version too.
+- `declare_pending(record_id, *, fields=("title", "summary"), cause="repair", reason=None)`: a stored record whose
+  title or summary was never authored gets exactly the state of a record formed pending (marker and tasks in one
+  transaction; a declared title is stored `None`, its old title kept as a version). Idempotent (`already_pending`).
+- `rearm_pending(record_id, ...)` re-queues the authoring task of a pending field whose tasks all ran out, and
+  `pending_records(*, scopes=None, live_task=False)` lists the records still pending, with their live and unowned
+  fields.
+- A public diary entry's summary is authored like any record's (at formation, or declared pending for an existing
+  entry); its body, the book's entry and every hash chain are untouched. Owner-direct, hash-chained and private
+  entries stay protected (`diary.diary_summary_refusal`).
+- `rekey_world_model(old_key, new_key, *, scope, owner_id, author, cause, ...)`: moves a standing card to a new key
+  in place and keeps the old key as an alias, so a retitled interest's next `world_model_pass` forms no second card.
+  An untitled interest's card is keyed `topic:<graph id>` and `retitle` re-keys it when the title lands
+  (`card_rekey` in the result).
+- `REWRITE_CAUSES` gains `repair` (a correction of what a process wrote wrong).
+
+### Added: understanding rewritten in place (0059)
+
+- `rewrite_digest`: replaces a record's digest, title and keywords in place. The record keeps its graph id, digest
+  assertion id, edges, usage counts and journal; its keyword entry and vector are replaced in the same transaction.
+  `expected_version` is a compare-and-set; each reading records `author` (`AUTHOR_KINDS`), `reader`
+  (`{provider, model, thinking}`), `sources` (mandatory for a model author) and `cause`. Refusals are named results
+  that write nothing. Results carry `cosine_distance` between the old and new vectors.
+- `narrate_dream` (a narration beside a dream's summary), `revert_digest`, `versions`, `current_of`,
+  `record_understanding`, `understanding_fields`; `author_world_model` rewrites a card in place by default (accepts
+  `reader`, `sources`, `expected_fingerprint`; `in_place=False` keeps the revision path).
+- The journal's version family `UnderstandingVersion`, and the opt-in replay family `revision`
+  (`OPT_IN_FAMILIES`; the default six families are unchanged). Display blocks carry the reading's provenance
+  (`digest_method`, `digest_version`, `version_count`, `authored_by`, `reader`, `understood_at`, `cause`).
+- Every record says who wrote its words, when and why, even when its writer stamped nothing: a row without a stamp
+  reads as its formation reading, derived from its origin (`understanding.reading_provenance`); `remember_many`
+  records that reading at formation.
+
+### Added: the memory task queue (0060) and ripple v1 (0061)
+
+- `MemorySystem(task_queue=True)` keeps its queue in the home's file (`memq_tasks`, `memq_log`, `memq_windows`,
+  `memq_ripples`, `memq_ripple_gates`, `memq_maintenance`; a private in-memory database for the in-memory store). `system.tasks` (`MemoryTaskQueue`): `enqueue`, `select_for_window(budget_seconds, sensitivity)` (activation
+  through ADR 0001's logistic gate, strongest first, bounded by the window's time; every live task selected or left
+  out with a reason), `claim` / `ack` / `release` (leases, attempts, defer without spending an attempt), `left_out`.
+  `understanding_debt`, `enqueue_debt` (the backfill, dry-run by default) and `task_packet` on `MemorySystem`. The
+  understanding verbs accept `task_id=` and settle the task in their own transaction.
+- A formed record and the tasks it owes commit together (`memory_tasks.form_with_tasks`): one transaction on the
+  SQLite home; on the in-memory pair a failure rolls back exactly that attempt's rows, bindings and tasks.
+- `maintenance_due(..., tasks=)` reports pending tasks and is due when ripple or night-only work is pending.
+- Ripple v1: `seed_ripple`, `settle_reconsideration`, `form_consolidation` (night only), `refresh_ripples`,
+  `revert_ripple`, and the reads `ripples` / `ripple`. Candidates by relation, by the cards built on a memory and by
+  vector similarity; impact decays per hop through the logistic gate (floor 0.1), at most 2 hops and 20
+  reconsiderations per ripple and per window; protected kinds are never reconsidered; every gate decision is stored
+  with its potential.
+
+### Added: communities (topics)
+
+- A community is a formed record of kind `community` (`abstractmemory.communities.COMMUNITY_KIND`), one record per
+  version, with a stable `attributes.community_key` across versions. `communities_pass(system, *, scopes, owner_id,
+  author, cancelled, ...)`, which a host runs during its sleep, detects them (deterministic warm-started Louvain over
+  the vectored records, at most 5 rounds), attaches vector-less records through their engraved relations, lays them
+  out, authors the changed ones and writes the version in one guarded transaction. `embed_missing(...)` embeds the
+  formed records that have no vector with the store's own embedder.
+- Placement at formation: on a `task_queue=True` home, `remember_many` places every record it forms in its
+  provisional community inside the formation's transaction (nearest centroid, else the majority community of its
+  formation edges) and writes one `placed` journal event (an audit kind, never scored).
+- Reads (`abstractmemory.community_index`): `communities(system, *, scopes, owner_id, version=None)` (the payload
+  version 2 a host serves, with each community's `facts`: span, activity, hub, bridges, paths,
+  `changed_since_last_sleep`) and `community_of(system, record_id)`. A community record replays like any formed
+  record; its binding envelope draws only `refines`.
+- Topic titles come first: a community's `title.author` / `digest.author` tasks are queued at `COMMUNITY_PRIORITY`
+  (3, ahead of every other pending record's `FORMATION_PRIORITY` 5), so a drainer with a small budget names the
+  topics before a long repair tail. `communities_pass`'s `budget` goes to its topics, largest first, one call per
+  topic for both its title and its summary (`None` authors every topic); the receipt carries `budget` and
+  `budget_spent`. A topic's briefing keeps its member lines within 1500 tokens, the first line too.
+
+### Added: sleep cadence
+
+- `records_since_maintenance(system, *, scopes=None, after=None) -> {count, last_maintenance_at, since,
+  boundary_source, fragmented}`: the host's idle → sleep threshold. `sleep_pass` records when a whole night ran to
+  its end (a cancelled, `report_only` or anchored night records nothing; a cycle window is recorded as kind `cycle`
+  and does not reset the count); `after` is the host's own last maintenance, and the later of the two is the
+  boundary. `fragmented = {count, share, provisional, unvectored, pending}` says how much of the live home a sleep
+  would make whole. `MemoryTaskQueue.record_maintenance` / `last_maintenance`.
+- `sleep_pass(..., cancelled=None)`: the host's Stop check, read at every sub-phase boundary and immediately before
+  the completion marker; a stopped night records no completion and carries `cancelled_after`.
+
+### Added: the Stop fence hooks
+
+- `MemorySystem(..., commit_guard=None)`: the host's `guard(where)` is installed on the store and journal and called
+  inside every write transaction immediately before its COMMIT, once per outermost transaction, whoever writes (the
+  facade, `MemoryGraph`, the task queue, the sleep passes). Raising rolls the write back; nothing of it lands and the
+  exception reaches the caller unchanged. A store or journal type whose writes do not call a guard is refused
+  (`TypeError`), and so is a second, different guard (`ValueError`).
+- `MemorySystem.write_barrier(*, timeout=None)` / `write_barrier(system, *, timeout=None)`: a context manager the
+  host wraps around its flip of the guard. It holds this home's in-process locks in the writers' own order and one
+  `BEGIN IMMEDIATE` per distinct SQLite file, so every transaction whose guard passed commits before the block and
+  every later one calls its guard after it. It writes nothing; `WriteBarrierTimeout` names the lock or file still
+  held.
 
-*No unreleased changes.*
+### Changed
+
+- Stored text is never cut: a world-model card quotes its evidence whole, a dream signal's `fragment` is kept whole
+  (`dream_signals.FRAGMENT_CAP` is gone), an assertion's `original_context` renders whole in the canonical text, a
+  plain assertion's "s p o" display title and an imported file's prose are kept whole. Readers shorten text at
+  display time (`canonical_text.short_title`).
+- Titles are explicit and whole: a card formed from an interest keeps the interest's whole subject head as its
+  target; lesson-candidate, interest-candidate and group-offer titles are no longer bounded.
+- `world_model_pass`: an interest whose title heads to an alias of a card counts as an election of the card's
+  primary key.
+- `dream_pass` returns `signals` and `signals_omitted`.
+- `journal_cold_cut` copies the version family and the task queue.
+- The identity card's `lessons` section also counts the lessons written in the diary (`diary_type='lesson'`), once
+  each when a lesson record derives from the entry.
+- The identity card's record briefs carry the reading fields (`authored_by`, `understood_at`, `cause`,
+  `digest_version`, `version_count`); diary-redacted replay blocks carry `authored_by`, `understood_at` and `cause`,
+  never the entry's words.
+
+### Fixed
+
+- `records_since_maintenance(...)["fragmented"]["count"]` counts each live record once: a record both vector-less
+  and pending, or placed and pending, is one fragmented record, and a placement whose record has been closed since
+  counts in `provisional` only, so `share` never exceeds 1.0. The per-kind counts are unchanged.
+- Identical authored words settle a pending field: an in-place write whose words equal the provisional ones clears
+  the marker, completes the task and records the reading (it answered `unchanged` and left the task leased).
+- On the in-memory pair, formation and rewriting share one lock (they could deadlock), and a failed formation
+  removes only its own rows and bindings, never a concurrent writer's.
+- `form_with_tasks` calls the commit guard immediately before its COMMITs, after the rows, bindings and tasks.
+- A completed task no longer carries the error of an earlier failed attempt (the failure stays in the task log).
+- The wake-cue dedup pass no longer tokenizes a missing title as "None".
+
+### Deprecated
+
+- `apply_redigestion` (emits `DeprecationWarning`): use `rewrite_digest`.
+
+## [0.4.0] - 2026-10-27
+
+The memory graph ([ADR 0003](docs/adr/0003-one-graph-ledger-provenance-and-revisions.md), backlogs 0056-0058):
+`abstractmemory.graph.MemoryGraph` is the only manipulation surface of a mind; every change is a revision, every
+memory points back to the ledger, and every database is a connector behind the graph (SQLite the default). The home
+store holds a notebook-sized corpus. The API is additive: every 0.3.x call keeps working (`close_assertion` stays;
+`ClosureRecord` gains only the `merged` kind). Operator decisions adopted (2026-10-27): visibility is strict
+ownership (lineage never crosses owners; ADR 0003 decision 2); stores are connectors behind `MemoryGraph`, SQLite the
+default (decision 3); the vector re-encoding below is a documented one-way migration with a `vector_encoding` marker.
+
+### Upgrade notes: ONE-WAY migration
+
+- Opening a SQLite home with 0.4.0 rewrites its JSON-text vectors as float32 BLOBs. A 0.3.x reader decodes JSON
+  vectors only: on a home a 0.4.0 process has written it silently finds no vectors (zero semantic hits, no error).
+  Every process sharing a home or a plane must move to 0.4.0 together; an entity home copied to a 0.3.x install
+  loses semantic recall. A 0.4.0 home declares it: `<table>_meta` key `vector_encoding` = `float32-blob`
+  (`abstractmemory.vector_matrix.VECTOR_ENCODING_KEY`), written by the first write or by the upgrade, never by a
+  plain open.
+- Graph homes add four tables (`memg_revisions`, `memg_provenance`, `memg_derivations`, `memg_history`) on first
+  use; nothing else in an existing file changes shape.
+
+### Added
+
+- **Connectors** (ADR 0003 decision 3, operator decision 2026-10-27): `MemoryGraph` is the only manipulation
+  surface of a mind and every store is a connector behind it: `abstractmemory.graph.backend` (`GraphBackend`,
+  `GraphTx` Protocols with their invariants; `register_backend`, `backend_for`, `registered_backends`). SQLite (the
+  default for planes and entity homes) and in-memory are declared implementations; a new LanceDB connector
+  (`abstractmemory.graph.lancedb_backend`, over `LanceDBTripleStore`: explicit Arrow schemas, a staged transaction,
+  a log-first commit, `repair()` re-derives a half-projected revision from the log). One conformance suite,
+  `tests/test_graph_conformance.py`, runs every connector; the LanceDB connector cannot promise serialized writers
+  across processes nor invisibility of a half-projected revision before `repair()` (strict xfails).
+- **`abstractmemory.graph`**, the one write path of a mind:
+  - `SourceRef` (`run`, `step` with the ledger row address `(run_id, seq)`, `seq` 1-based, `artifact`, `message`, `event`,
+    `diary_entry`; optional `runtime_id`; one canonical `key()`), validated per kind (`InvalidSourceRef`); a
+    `run` / `step` ref naming a synthetic run id (`SYNTHETIC_RUN_ID_PREFIXES`: `chat-`, `diary:`,
+    `session_memory_`, `global_memory`) is refused.
+  - `Provenance(sources, derived_from, actor, reader, reason, extra)`, strict about its shape;
+    `Provenance.from_legacy(...)` lifts the old flat keys (`run_id`, `span_id`, `payload_ref`, `entry_id`, ...)
+    in one place, never fabricates a reference (a synthetic run id stays in `extra`), and reads the mixed shape
+    the runtime stamps (flat keys plus `sources`: flat keys into `extra`).
+  - `Revision` (`revision_id`, `scope`, `owner_id`, `cause`, `occurred_at`, `recorded_at`, `ops`, `provenance`,
+    `summary`) with the ops `assert`, `close` (`retract` / `supersede` / `reinterpreted` / `merged`, replacement
+    ids, ripple source and hops) and `narrow`. Every time a revision carries is parsed and normalized to one
+    UTC text form; a non-timestamp is refused. A close ends the row at the earlier of its current end and the
+    close (so `TripleQuery(active_at=)` and ranked recall agree after every close); a close or narrow before the
+    claim's `valid_from` is refused.
+  - `MemoryGraph(store, journal).apply_revision(rev)`: refuses a memory that is neither sourced nor derived from a
+    grounded memory (`ProvenanceRequired`; chains that never reach a source and cycles included); applies all ops
+    in ONE SQLite transaction (triples and their keyword rows, window changes, the journal closure when the
+    journal shares the file, the revision, provenance, derivation and history rows), or not at all; idempotent on
+    the revision id; deterministic ids so the logged revision replays identically. A store row written outside
+    the graph is adopted only by an `import` / `migration` revision that carries it as stored (logged with its
+    previous provenance, so a replay reproduces it); a different claim under an id the graph holds is refused;
+    a close/narrow-only revision must name a cause other than the memories it changes; a `derived_from` across
+    owners is refused (other owners' knowledge enters as a source). Re-observation, not duplication: an open
+    assert of a claim the graph holds open records its sources on the held memory (`re_observed`, history
+    `re-observed`) instead of a second row. Record time is the store's: a fresh apply stamps `recorded_at` with
+    its clock (`apply_revision(rev, replay=True)`, used by `replay_into`, keeps the logged value). A revision
+    already in the log returns its logged outcome. Every memory is also indexed by the sources of the revision
+    that wrote it (`via`). `SourceRef.digest`: an optional sha256 of the verbatim, beside the address.
+  - Reads, all bounded index reads: `history(memory_id)` (asserted / closed / narrowed, by which revision, with
+    its cause), `sources_of(memory_id, transitive=True)`, `derived_from_source(ref)` (a `run` ref matches
+    everything stamped with that run), the lossless change feed `revisions(after=cursor)` (a cursor beyond the
+    log's head is refused), `replay_into(target)` (an empty home replays to the same file, adoptions included),
+    `names_ref(ref, owner_ids)` (the owner gate in one index probe), and `repair()` for closures a crash left
+    unwritten when the journal lives in another file (never needed for same-file plane and entity homes).
+  - Four tables in the home file: `memg_revisions`, `memg_provenance`, `memg_derivations`, `memg_history`.
+- **`narrow_assertion(assertion_id, valid_until)`** on `SQLiteTripleStore` and `InMemoryTripleStore`: moves an
+  already-closed validity window earlier and never widens it (`ValueError` for an open window or a value that is
+  not a timestamp; `False` for an unknown id or a date at or after the current end, compared as instants). `LanceDBTripleStore.narrow_assertion` raises
+  `NotImplementedError` (on LanceDB a window changes through the graph's LanceDB connector). AbstractNote no longer
+  needs to write the store's private table.
+- **Closure kind `merged`** (`abstractmemory.journal.CLOSURE_KINDS`): the memory was merged into another, named in
+  `replacement_ids`. `close_assertions` and `close_record` accept it.
+- Docs: `api.md` (the graph section and its tables), `memory-system.md` ("Time axes", "Provenance").
+- Measured at 50,000 assertions (`tests/bench/bench_graph.py`): `history` 0.02 ms, `sources_of` 0.04 ms,
+  `derived_from_source` by step 0.05 ms and by run 0.8 ms, a change-feed page of 100 revisions 4 ms; the 0.3
+  alternative (scanning provenance JSON) took 18 ms on vectorless rows and 38 ms with 384-d vectors in the rows.
+
+### Performance (0058)
+
+Backlog 0058 (ADR 0003 decisions 3 and 12): the SQLite home store holds a
+notebook-sized corpus. Measured with `tests/bench/bench_store.py`, `main`
+versus this release on the same machine, back to back: subject + owner query
+at 50k rows 14.8 ms -> 0.33 ms; scope "latest 100" 377 ms -> 0.9 ms; scope
+full scan 2.6 s -> 0.5 s; vector query at 50k 5.3 s -> 1.4 ms (first query of
+a handle 0.44 s, it builds the matrix); file 449 MB -> 122 MB; MemorySystem
+`reconstruct` at 20k records 2.7 s -> 0.2 s; `maintenance_due` at 20k 1.9 s ->
+0.07 ms when nothing is new, 0.1 s after new records. Full table: backlog
+0058's Report.
+
+- **Owner-filtered reads use the indexes.** The owner filter is now
+  `owner_id = ?` instead of `COALESCE(owner_id, '') = ?` (same rows: the
+  filter only applies to a non-empty owner, which a NULL or empty stored owner
+  never equals). A new index `idx_<table>_scope_owner_observed (scope,
+  owner_id, observed_at, assertion_id)` serves "latest N in a (scope, owner)",
+  as-of windows and per-pair scans in query order without sorting the
+  partition; reads anchored on ids, a subject or an object use their own index
+  and check scope/owner per row. Existing homes gain the index when the store
+  is opened; no row is rewritten. `SQLiteTripleStore.query_plan(q)` returns
+  SQLite's `EXPLAIN QUERY PLAN` lines for a query (tests pin the plans).
+- **Structured reads no longer fetch the vector column.** `query` (without
+  `query_text`/`query_vector`) and `query_keywords` select the assertion
+  columns only, instead of `SELECT *` dragging the stored embedding (several KB
+  per row) and the canonical text through every scan. They no longer read the
+  embedding pin either (it guards semantic queries only). Semantic queries and
+  `stored_vector` still read the vector; `TripleAssertion` is unchanged.
+- **Vectors are float32 BLOBs.** `SQLiteTripleStore` stores vectors as float32
+  little-endian BLOBs (4 bytes per dimension instead of ~20 bytes of JSON
+  text). Homes holding JSON vectors are upgraded in place when the store opens:
+  rows above the `vector_blob_high_water` cursor are rewritten in bounded
+  transactions (idempotent; an interrupted upgrade resumes; ~5 s once for 50k
+  rows of 384-d vectors). Stored precision is float32: a score differs from
+  the float64 value an embedder returned by about 1e-7. A value float32 cannot
+  hold raises `ValueError` at write (zero rows written) instead of being
+  stored as infinity; `stored_vector` returns the float32 values. One-way: see
+  "Upgrade notes" above.
+- **Opening a home is a read.** `SQLiteTripleStore` and `SQLiteJournal` take no write lock when opening an
+  up-to-date home (the keyword-index and vector backfills run only when there is work; every insert path keeps
+  their cursors at the head; the journal checks its schema in a deferred transaction), so a reader never waits
+  behind a writer (before: up to `busy_timeout`, then "database is locked").
+- **Matrix cache footprint.** A (re)build reserves exactly the table's vector rows (one allocation); appends
+  grow by at most 10 % (`store_tuning.VECTOR_MATRIX_GROWTH`; no doubling). Per process that queries a home:
+  rows x dims x 4 B plus about 170 B per row (`store_tuning.py`). Measured at 100k x 1024: 424 MB held (552 MB
+  with doubling), build peak 460 MB (833 MB).
+- **Vector queries from an in-process matrix cache.** With numpy installed
+  (new extra `abstractmemory[fast]`; also in `all`, `all-apple`, `all-gpu`
+  and `test`), vector queries are served from a float32 matrix built at a
+  handle's first semantic query, appended with rows written since by any
+  handle or process, and rebuilt when the vectors are rewritten
+  (`replace_vectors`, the upgrade; `vector_generation` in the meta sidecar).
+  Results are the reference scan's (`vector_scoring.rank_by_cosine`): same
+  rows, same order (ties in rowid order), bit-identical scores — a float32
+  pass picks the candidates with a margin above its error bound, and they are
+  re-scored with the reference arithmetic. Without numpy, or when a stored
+  vector cannot be scored exactly (mixed dimensions, non-finite values), the
+  Python scan answers, labelled once per handle with a `#FALLBACK`
+  `RuntimeWarning`. `SQLiteTripleStore.vector_engine` reports the path.
+- **`maintenance_due` bounded by a cursor.** For the SQLite and in-memory
+  stores the answer is folded from the new store change feed
+  `assertions_after(cursor, *, scope, owner_id, limit)`: a check reads only the
+  rows written since the previous check of the same (store, journal, scopes)
+  and re-reads the journal's formation bindings only when the journal moved.
+  `last_maintenance_seq` uses the same fold. The answer is the full scan's
+  (other stores keep the scan).
+- Tuning constants of the store (margins, batch and page sizes) live in
+  `abstractmemory.store_tuning`, each with its bound or measurement.
+- The benchmark lives in the repository: `tests/bench/bench_store.py`.
 
 ## [0.3.2] - 2026-10-20
 

{abstractmemory-0.3.2 → abstractmemory-0.5.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: AbstractMemory
-Version: 0.3.2
+Version: 0.5.0
 Summary: AbstractMemory: temporal, provenance-aware semantic memory primitives for AbstractFramework.
 Project-URL: AbstractFramework (monorepo), https://github.com/lpalbou/abstractframework
 Author: Laurent-Philippe Albou
@@ -19,18 +19,24 @@ Classifier: Programming Language :: Python :: 3.13
 Requires-Python: >=3.10
 Provides-Extra: all
 Requires-Dist: lancedb; extra == 'all'
+Requires-Dist: numpy>=1.24; extra == 'all'
 Provides-Extra: all-apple
 Requires-Dist: lancedb; extra == 'all-apple'
+Requires-Dist: numpy>=1.24; extra == 'all-apple'
 Provides-Extra: all-gpu
 Requires-Dist: lancedb; extra == 'all-gpu'
+Requires-Dist: numpy>=1.24; extra == 'all-gpu'
 Provides-Extra: apple
 Provides-Extra: dev
 Requires-Dist: pytest>=7.0.0; extra == 'dev'
+Provides-Extra: fast
+Requires-Dist: numpy>=1.24; extra == 'fast'
 Provides-Extra: gpu
 Provides-Extra: lancedb
 Requires-Dist: lancedb; extra == 'lancedb'
 Provides-Extra: test
 Requires-Dist: lancedb; extra == 'test'
+Requires-Dist: numpy>=1.24; extra == 'test'
 Requires-Dist: pytest>=7.0.0; extra == 'test'
 Description-Content-Type: text/markdown
 
@@ -49,6 +55,15 @@ Storage never decays and nothing is ever deleted; only retrieval strength change
 - The authoritative export list is [`src/abstractmemory/__init__.py`](src/abstractmemory/__init__.py); [`docs/api.md`](docs/api.md) documents it.
 - Requires Python 3.10+.
 
+## What's new in 0.5.0
+
+- Every memory's title and summary is authored: memory's passes take a host `author` callable, and a record whose authoring failed forms pending, with its task, until a reader completes it (`retitle`, `rewrite_digest`, `declare_pending`, `rearm_pending`).
+- A persistent task queue per home (`MemorySystem(task_queue=True)`), ripple v1, and understanding rewritten in place with every earlier reading kept as a version.
+- Topics: `community` records formed during sleep, with provisional placement of each new record at formation.
+- The sleep cadence read `records_since_maintenance`, and the Stop fence hooks `commit_guard` and `write_barrier`.
+
+Upgrading from 0.4.x: see the "Upgrading" notes in [`CHANGELOG.md`](CHANGELOG.md).
+
 ## Ecosystem (AbstractFramework)
 
 AbstractMemory is a component of the **AbstractFramework** ecosystem. It has no dependency on AbstractCore or AbstractRuntime; embeddings for semantic retrieval can come from any OpenAI-compatible `/embeddings` endpoint (`OpenAICompatTextEmbedder`), from an AbstractGateway deployment (`AbstractGatewayTextEmbedder`), or from your own `TextEmbedder` implementation.
@@ -86,6 +101,13 @@ Optional LanceDB backend:
 python -m pip install -e ".[lancedb]"
 ```
 
+Fast vector search on the SQLite store (numpy; without it the store serves the
+same results from a labelled Python scan):
+
+```bash
+python -m pip install -e ".[fast]"
+```
+
 PyPI (packaged release):
 
 ```bash
@@ -93,7 +115,7 @@ python -m pip install AbstractMemory
 python -m pip install "AbstractMemory[lancedb]"
 ```
 
-The distribution name is `AbstractMemory` (pip is case-insensitive); the import name is `abstractmemory`. The `[apple]`/`[gpu]` extras are no-op compatibility aliases; `[all]`, `[all-apple]`, and `[all-gpu]` install the LanceDB backend.
+The distribution name is `AbstractMemory` (pip is case-insensitive); the import name is `abstractmemory`. The `[apple]`/`[gpu]` extras are no-op compatibility aliases; `[fast]` installs numpy for the SQLite store's vector matrix cache; `[all]`, `[all-apple]`, and `[all-gpu]` install the LanceDB backend and numpy.
 
 ## Quick example — layer 1 (triples)
 
@@ -141,6 +163,10 @@ result = system.reconstruct(Stimulus(cue_text="pool outage"), scopes=[("session"
 system.commit_selection(result.trace_id, [h.record_id for h in result.handles[:2]])
 ```
 
+## The memory graph: revisions and provenance
+
+`abstractmemory.graph` is the write path of a mind (0.4.0): every change to the graph is one `Revision` (a cause such as `ingest`, `reinterpret` or `maintenance`, plus `assert` / `close` / `narrow` operations applied together and atomically, once per revision id), and every memory must name where it came from in AbstractRuntime's ledger (a `SourceRef` to a run, a step `(run_id, seq)`, an artifact, a message, an event or a diary entry) or the memories it was derived from; a write with neither is refused with `ProvenanceRequired`. `MemoryGraph(store, journal)` then answers `history(memory_id)` (asserted, closed, narrowed: by which revision and why), `sources_of(memory_id)`, `derived_from_source(ref)`, a lossless change feed `revisions(after=cursor)`, and `replay_into(...)`. The database is a connector behind the graph (`abstractmemory.graph.backend`): SQLite by default, LanceDB and in-memory held to the same conformance suite. See [`docs/api.md`](docs/api.md#the-memory-graph-revisions-and-provenance) and the "Time axes" and "Provenance" sections of [`docs/memory-system.md`](docs/memory-system.md).
+
 ## Documentation
 
 - Getting started: [`docs/getting-started.md`](docs/getting-started.md)
@@ -164,7 +190,7 @@ system.commit_selection(result.trace_id, [h.record_id for h in result.handles[:2
 
 ## Design principles
 
-- **Append-only, no deletion**: updates are new assertions; belief revision is closure records (retract/supersede/reinterpreted); forgetting is decay of retrieval strength plus closures and silencing — the substrate is lossless.
+- **Append-only, no deletion**: updates are new assertions; belief revision is closure records (retract/supersede/reinterpreted/merged); forgetting is decay of retrieval strength plus closures and silencing — the substrate is lossless. The one mutable cell is a record's understanding: `rewrite_digest` and `retitle` rewrite a digest or a title in place and keep every earlier reading as a version.
 - **Reads are pure**: reconstruction, inspection, replay, and the entity card deposit nothing. `commit_selection` is the only strengthening path.
 - **One seq axis**: the journal assigns a monotonic `seq` to every record; any past state is reproducible by anchoring reads at `as_of`.
 - **Works-or-loud**: degraded paths are labeled `#FALLBACK` in result warnings; invalid inputs raise actionable errors instead of silently meaning something else.

{abstractmemory-0.3.2 → abstractmemory-0.5.0}/README.md

@@ -13,6 +13,15 @@ Storage never decays and nothing is ever deleted; only retrieval strength change
 - The authoritative export list is [`src/abstractmemory/__init__.py`](src/abstractmemory/__init__.py); [`docs/api.md`](docs/api.md) documents it.
 - Requires Python 3.10+.
 
+## What's new in 0.5.0
+
+- Every memory's title and summary is authored: memory's passes take a host `author` callable, and a record whose authoring failed forms pending, with its task, until a reader completes it (`retitle`, `rewrite_digest`, `declare_pending`, `rearm_pending`).
+- A persistent task queue per home (`MemorySystem(task_queue=True)`), ripple v1, and understanding rewritten in place with every earlier reading kept as a version.
+- Topics: `community` records formed during sleep, with provisional placement of each new record at formation.
+- The sleep cadence read `records_since_maintenance`, and the Stop fence hooks `commit_guard` and `write_barrier`.
+
+Upgrading from 0.4.x: see the "Upgrading" notes in [`CHANGELOG.md`](CHANGELOG.md).
+
 ## Ecosystem (AbstractFramework)
 
 AbstractMemory is a component of the **AbstractFramework** ecosystem. It has no dependency on AbstractCore or AbstractRuntime; embeddings for semantic retrieval can come from any OpenAI-compatible `/embeddings` endpoint (`OpenAICompatTextEmbedder`), from an AbstractGateway deployment (`AbstractGatewayTextEmbedder`), or from your own `TextEmbedder` implementation.
@@ -50,6 +59,13 @@ Optional LanceDB backend:
 python -m pip install -e ".[lancedb]"
 ```
 
+Fast vector search on the SQLite store (numpy; without it the store serves the
+same results from a labelled Python scan):
+
+```bash
+python -m pip install -e ".[fast]"
+```
+
 PyPI (packaged release):
 
 ```bash
@@ -57,7 +73,7 @@ python -m pip install AbstractMemory
 python -m pip install "AbstractMemory[lancedb]"
 ```
 
-The distribution name is `AbstractMemory` (pip is case-insensitive); the import name is `abstractmemory`. The `[apple]`/`[gpu]` extras are no-op compatibility aliases; `[all]`, `[all-apple]`, and `[all-gpu]` install the LanceDB backend.
+The distribution name is `AbstractMemory` (pip is case-insensitive); the import name is `abstractmemory`. The `[apple]`/`[gpu]` extras are no-op compatibility aliases; `[fast]` installs numpy for the SQLite store's vector matrix cache; `[all]`, `[all-apple]`, and `[all-gpu]` install the LanceDB backend and numpy.
 
 ## Quick example — layer 1 (triples)
 
@@ -105,6 +121,10 @@ result = system.reconstruct(Stimulus(cue_text="pool outage"), scopes=[("session"
 system.commit_selection(result.trace_id, [h.record_id for h in result.handles[:2]])
 ```
 
+## The memory graph: revisions and provenance
+
+`abstractmemory.graph` is the write path of a mind (0.4.0): every change to the graph is one `Revision` (a cause such as `ingest`, `reinterpret` or `maintenance`, plus `assert` / `close` / `narrow` operations applied together and atomically, once per revision id), and every memory must name where it came from in AbstractRuntime's ledger (a `SourceRef` to a run, a step `(run_id, seq)`, an artifact, a message, an event or a diary entry) or the memories it was derived from; a write with neither is refused with `ProvenanceRequired`. `MemoryGraph(store, journal)` then answers `history(memory_id)` (asserted, closed, narrowed: by which revision and why), `sources_of(memory_id)`, `derived_from_source(ref)`, a lossless change feed `revisions(after=cursor)`, and `replay_into(...)`. The database is a connector behind the graph (`abstractmemory.graph.backend`): SQLite by default, LanceDB and in-memory held to the same conformance suite. See [`docs/api.md`](docs/api.md#the-memory-graph-revisions-and-provenance) and the "Time axes" and "Provenance" sections of [`docs/memory-system.md`](docs/memory-system.md).
+
 ## Documentation
 
 - Getting started: [`docs/getting-started.md`](docs/getting-started.md)
@@ -128,7 +148,7 @@ system.commit_selection(result.trace_id, [h.record_id for h in result.handles[:2
 
 ## Design principles
 
-- **Append-only, no deletion**: updates are new assertions; belief revision is closure records (retract/supersede/reinterpreted); forgetting is decay of retrieval strength plus closures and silencing — the substrate is lossless.
+- **Append-only, no deletion**: updates are new assertions; belief revision is closure records (retract/supersede/reinterpreted/merged); forgetting is decay of retrieval strength plus closures and silencing — the substrate is lossless. The one mutable cell is a record's understanding: `rewrite_digest` and `retitle` rewrite a digest or a title in place and keep every earlier reading as a version.
 - **Reads are pure**: reconstruction, inspection, replay, and the entity card deposit nothing. `commit_selection` is the only strengthening path.
 - **One seq axis**: the journal assigns a monotonic `seq` to every record; any past state is reproducible by anchoring reads at `as_of`.
 - **Works-or-loud**: degraded paths are labeled `#FALLBACK` in result warnings; invalid inputs raise actionable errors instead of silently meaning something else.

{abstractmemory-0.3.2 → abstractmemory-0.5.0}/docs/README.md

@@ -19,6 +19,7 @@ AbstractMemory is a Python library for durable, append-only agent memory: tempor
 - FAQ: [`faq.md`](faq.md) — common questions and current limits.
 - Troubleshooting: [`troubleshooting.md`](troubleshooting.md) — symptom-oriented fixes for setup, retrieval, embedding-space, durability, and maintenance problems.
 - Development: [`development.md`](development.md) — local setup, tests, and the design backlog.
+- Architecture decisions: [`adr/README.md`](adr/README.md) — durable rules, including ADR 0001 (an altered memory ripples through the memories connected to it) and ADR 0002 (AbstractMemory owns the semantic temporal memory graph and all of its processing; applications build on it).
 
 ## Related