AbstractMemory 0.5.0__tar.gz → 0.5.1__tar.gz

{abstractmemory-0.5.0 → abstractmemory-0.5.1}/CHANGELOG.md

@@ -5,6 +5,47 @@ All notable changes to this package will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html) (pre-1.0).
 
+## [0.5.1] - 2026-11-10
+
+### Added: `community_keys_of`, the membership-only read for recall stamping
+
+The runtime stamps every recalled record with its topic by reading the whole `communities()` payload per recall, which
+computes every topic's facts: 19 ms at 2,000 records and 190 ms at 20,000 on its synthetic home.
+
+- `community_index.community_keys_of(system, record_ids, *, scopes, owner_id) -> {record_id: key | None}` and
+  `community_memberships_of(...)` -> `{record_id: {key, provisional} | None}`: the answer `communities()` gives for
+  those records (member, provisional since the version, or none), read from the in-memory community index. The hot
+  path is the formation's catch-up compare (`MAX(rowid)` on the open connection against the index's mark) and a dict
+  lookup per id: no fold, no facts, no file opened, stat'ed or locked. Measured for 50 ids (54 topics, SQLite home):
+  20 µs per call at 2,000 records, 22 µs at 20,000 (`communities()` on the same homes: 12 ms and 65 ms). After a
+  formation, the `placed` events above the last read are fetched once per scope pair; a new version reloads the
+  index. A dissolve-only version is seen at the next version, as at placement (backlog 0066).
+- The runtime switches its recall stamps to it: backlog 0069.
+
+Upgrading from 0.5.0: a host that stamps recalled records with their topic should call `community_keys_of` (or
+`community_memberships_of`) with the records' graph ids instead of reading the whole `communities()` payload per
+recall; nothing else changes. Known limits, carried in the backlog: a version that only dissolves communities (no
+new community record) is seen by the membership reads and by placement at the next version that writes one (0066);
+`communities()` read while a version is being written may serve one empty read (0067).
+
+### Fixed: a standing topic still pending is authored, and its old tasks move up
+
+Castor's second topics night (2026-11-10): 10 of 11 topics got titles; the 11th stood unchanged, its title and
+summary tasks queued on the first night at the old priority 5 behind ~267 repair tasks, so the pass skipped it with 6
+calls of its budget unused, and its receipt said `pending: 0` while the topics route showed it pending.
+
+- `communities_pass` spends what its `budget` leaves after the new and changed topics on the STANDING topics still
+  pending (unchanged tonight), largest first, one call each for the fields still pending, from tonight's briefing.
+  Each is written in place on its current record after the version (`rewrite_digest`, with the title when both are
+  pending, or `retitle`), which completes its live `title.author` / `digest.author` tasks; the record keeps its
+  `title_basis` (with the sample its author saw) and the author's verdict. The receipt's `pending` now counts every
+  topic still without a title or summary after the pass, read back; new keys `authored_in_place`, `reprioritized`
+  and `ids.authored_in_place`; a refused in-place write is a failure line with reason `write_refused`.
+- Every standing topic's live authoring task that waits at a later priority than `COMMUNITY_PRIORITY` (3) is moved up
+  to it, in the version's transaction (its own when no version is written): a topic's legacy priority-5 tasks no
+  longer wait behind the repair tail. New `MemoryTaskQueue.prioritize(target_ids, *, kinds, priority, reason,
+  dry_run=False)`: one UPDATE, never demotes a task already ahead, one `prioritized` log row each.
+
 ## [0.5.0] - 2026-11-10
 
 Every memory now carries an authored title and summary, a home keeps a persistent queue of what still needs

{abstractmemory-0.5.0 → abstractmemory-0.5.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: AbstractMemory
-Version: 0.5.0
+Version: 0.5.1
 Summary: AbstractMemory: temporal, provenance-aware semantic memory primitives for AbstractFramework.
 Project-URL: AbstractFramework (monorepo), https://github.com/lpalbou/abstractframework
 Author: Laurent-Philippe Albou
@@ -55,6 +55,11 @@ Storage never decays and nothing is ever deleted; only retrieval strength change
 - The authoritative export list is [`src/abstractmemory/__init__.py`](src/abstractmemory/__init__.py); [`docs/api.md`](docs/api.md) documents it.
 - Requires Python 3.10+.
 
+## What's new in 0.5.1
+
+- `community_keys_of` / `community_memberships_of`: the membership-only topic read for every recalled record, tens of microseconds per call from the in-memory community index (the runtime's recall stamps use it).
+- A standing topic still pending its title or summary is authored in place by `communities_pass` with the budget the changed topics leave, and its queued tasks move up to the topics' priority.
+
 ## What's new in 0.5.0
 
 - Every memory's title and summary is authored: memory's passes take a host `author` callable, and a record whose authoring failed forms pending, with its task, until a reader completes it (`retitle`, `rewrite_digest`, `declare_pending`, `rearm_pending`).

{abstractmemory-0.5.0 → abstractmemory-0.5.1}/README.md

@@ -13,6 +13,11 @@ Storage never decays and nothing is ever deleted; only retrieval strength change
 - The authoritative export list is [`src/abstractmemory/__init__.py`](src/abstractmemory/__init__.py); [`docs/api.md`](docs/api.md) documents it.
 - Requires Python 3.10+.
 
+## What's new in 0.5.1
+
+- `community_keys_of` / `community_memberships_of`: the membership-only topic read for every recalled record, tens of microseconds per call from the in-memory community index (the runtime's recall stamps use it).
+- A standing topic still pending its title or summary is authored in place by `communities_pass` with the budget the changed topics leave, and its queued tasks move up to the topics' priority.
+
 ## What's new in 0.5.0
 
 - Every memory's title and summary is authored: memory's passes take a host `author` callable, and a record whose authoring failed forms pending, with its task, until a reader completes it (`retitle`, `rewrite_digest`, `declare_pending`, `rearm_pending`).

{abstractmemory-0.5.0 → abstractmemory-0.5.1}/docs/api.md

@@ -383,7 +383,7 @@ Enqueue sites: `remember_many` (a `title.author` / `digest.author` per pending f
 - `ack(task_id, *, holder, result, now=None) -> {acked, reason?, task?}` — settle a leased task as done (for example `{"outcome": "unchanged", "seconds": 41.2, "tokens": 3120, "run_id": ...}`); a task a verb already settled returns `reason="already_done"`.
 - `release(task_id, *, holder, outcome, reason, until=None, detail=None, now=None) -> {released, task?}` — `outcome="retry"` (attempt spent; failed at `max_attempts`), `"defer"` (no attempt spent; not before `until`, for an unreachable endpoint or embedder), `"skip"` (terminal, for example `reason="over_context"` with the measured tokens in `detail`).
 - `left_out(*, holder=None, window=None) -> list[dict]` — what the last claimed window of each `(holder, window)` left out, and why.
-- `task(task_id)`, `tasks(*, status=None, kind=None, target_id=None, ripple_id=None, limit=0)`, `log(task_id)`, `counts()`, `cost_of(task)`, `supersede_stale(system)`, `supersede_target(target_id, *, reason)`.
+- `task(task_id)`, `tasks(*, status=None, kind=None, target_id=None, ripple_id=None, limit=0)`, `log(task_id)`, `counts()`, `cost_of(task)`, `supersede_stale(system)`, `supersede_target(target_id, *, reason)`, `prioritize(target_ids, *, kinds, priority, reason, dry_run=False)` (moves the live tasks of `kinds` on these targets that wait at a later priority up to `priority`, never demoting one already ahead, one `prioritized` log row each; returns the task ids).
 
 On `MemorySystem` (and as module functions taking the system first):
 
@@ -539,7 +539,8 @@ Communities are eventually consistent. A record formed after the last sleep is p
 - `communities(system, *, scopes, owner_id, version=None) -> dict` — PURE READ, the payload a host serves (version 2): `{version: 2, version_seq, first_sleep_pending, algorithm, communities: [...], links: [{a, b, weight}]}`. Each community is `{id (this version's record id), key (stable across versions), title, summary, pending, verdict, size, kinds, members: [{id, via}], provisional_ids, cohesion, layout: {xy, radius, seats}, centroid_stale, facts}`, largest first; `links` name community keys. `title` and `summary` are `None` while pending (`pending` names the fields still being written). `provisional_ids` are the records placed at formation since the version (their newest `placed` event), not yet members; `size`, `kinds` and `members` are the version's. `version` reads an earlier version (the newest record of each community at or before that seq, viewed up to the next version); a seq older than the first version raises `ValueError`. A community record missing a field the read serves raises `CommunitySchemaError`.
 - `facts` (read from the store and journal as they are now, one query each): `span: {first, last}` (when its first and newest members formed), `activity` (its members' uses, `selected` events, in the last `FACTS_WINDOW_DAYS` = 7 days), `hub` (the member used most, all time; `None` when none was ever used), `bridges` (members with an engraved edge to a member of another community), `paths` (the `FACTS_PATHS_TOP` = 5 member pairs most often used together, `co_selected`, in the last 7 days: `[{from, to, count}]`), `changed_since_last_sleep: {added, moved, split_from}` (members that joined and left at this version, and the community it was split from, when its version is the newest; zeros otherwise). Not served yet: which community a dissolved one merged into.
 - `community_of(system, record_id) -> dict` — `{record_id, community_key, community_id, provisional, via, how, score, version_seq, first_sleep_pending}`: a member of the current version (`provisional: false`, `via` as the version names it), a provisional placement (`provisional: true`, `via: "provisional"`, `how`, `score`), or no community (`community_key: None`). An unknown record or a community record raises `ValueError`.
-- `communities_pass(system, *, scopes, owner_id, author, cancelled, budget=None, seed_previous=True, detect_kinds=None) -> dict` — one night's communities. It reads the graph (the formed records, their engraved edges and the store's vectors), detects communities over every vectored record (a deterministic Louvain warm-started from the previous version, at most `MAX_DETECTION_ROUNDS` = 5 rounds; `written_amid` never drives detection), attaches the vector-less records through their engraved neighbours, carries each community's `community_key` across versions by membership overlap, computes the layout, has the changed communities' titles and summaries written by `author` (the same callable as the other passes, see [the `author` callable](#authored-with-the-record-the-author-callable); `budget` = the most author calls, all of them for the topics, largest first: one call per new or changed topic writes both its title and its summary, so the pass spends at most `min(budget, n)` and never more than one call per topic; `None` authors every topic; the topics past the budget form pending, their tasks at `COMMUNITY_PRIORITY`, ahead of every other pending record), calls `cancelled()` once, and writes the new versions, their bindings, the closures of the previous versions and their tasks in ONE commit-guarded transaction. An unchanged home writes nothing. A home opened without `task_queue=True` is refused; without numpy (`abstractmemory[fast]`) the pass is skipped with its reason. The receipt carries `version_seq`, `new` / `kept` / `renamed` / `dissolved` / `unchanged`, `written`, `unvectored`, `unplaced` and `unplaced_closed` (the vector-less records whose every live engraved relation names a closed record: no edge can place them), `authored` (the topics whose title and summary this pass wrote), `pending` (the topics written with a field still pending, their tasks queued), `failed`, `budget` (as given) and `budget_spent` (the author calls made: a host that shares one authoring budget between this pass and its drainer runs the pass first and gives the drainer `budget - budget_spent`), `authoring` (`{authored, failures}`; a topic left pending by the budget is no failure), `cancelled`, `skipped` and per-step `timings_ms`. What the author reads is the topic's briefing: its size, the kinds, the previous title, the sibling titles it must differ from (at most 40), and a stable kind-mixed sample of whole member lines within 1500 tokens (a line that does not fit is left out): under 2.3k tokens in all, about 1.5-1.9k for a home of a dozen topics.
+- `community_keys_of(system, record_ids, *, scopes, owner_id) -> dict` and `community_memberships_of(system, record_ids, *, scopes, owner_id) -> dict` — the membership-only read a host stamps on every recalled record: `{record_id: key | None}` and `{record_id: {key, provisional} | None}` for record graph ids (the ids members carry: a recalled record's `provenance.record_id`). Each answer is the one `communities(system, scopes=, owner_id=)` gives for that record: a member of a current community of these pairs (`provisional: false`), a record placed at formation since the version (`provisional: true`), else `None` (unplaced, unknown, a community record, another owner's record, no version yet). It is served from the in-memory community index under the store's lock: one `MAX(rowid)` compare with the index's mark (the formation's catch-up) and a dict lookup per id, no facts, no layout, no vector, no file opened or stat'ed; measured 20 µs per call for 50 ids at 2,000 records and 22 µs at 20,000 (54 topics; `communities()` on the same homes: 12 ms and 65 ms). After a formation the `placed` events above the last read are fetched once per scope pair; a new version reloads the index (once a night). Like placement, it sees a version that only dissolves communities (no new community record) at the next version. An unreadable version raises `CommunitySchemaError`; a store other than SQLite or in-memory raises `TypeError`.
+- `communities_pass(system, *, scopes, owner_id, author, cancelled, budget=None, seed_previous=True, detect_kinds=None) -> dict` — one night's communities. It reads the graph (the formed records, their engraved edges and the store's vectors), detects communities over every vectored record (a deterministic Louvain warm-started from the previous version, at most `MAX_DETECTION_ROUNDS` = 5 rounds; `written_amid` never drives detection), attaches the vector-less records through their engraved neighbours, carries each community's `community_key` across versions by membership overlap, computes the layout, has the changed communities' titles and summaries written by `author` (the same callable as the other passes, see [the `author` callable](#authored-with-the-record-the-author-callable); `budget` = the most author calls, all of them for the topics, largest first: one call per new or changed topic writes both its title and its summary, so the pass spends at most `min(budget, n)` and never more than one call per topic; `None` authors every topic; the topics past the budget form pending, their tasks at `COMMUNITY_PRIORITY`, ahead of every other pending record; what the budget leaves goes to the STANDING topics, unchanged tonight but still pending, largest first, one call each for the fields still pending), calls `cancelled()` once, and writes the new versions, their bindings, the closures of the previous versions and their tasks in ONE commit-guarded transaction, which also moves every standing topic's live `title.author` / `digest.author` task that waits at a later priority (queued before topics came first) up to `COMMUNITY_PRIORITY`. The standing pending topics are then written IN PLACE on their current record (`rewrite_digest`, with the title when both are pending, or `retitle`; each its own guarded transaction), which completes their live tasks. An unchanged home with no pending topic to author and no task to move writes nothing. A home opened without `task_queue=True` is refused; without numpy (`abstractmemory[fast]`) the pass is skipped with its reason. The receipt carries `version_seq`, `new` / `kept` / `renamed` / `dissolved` / `unchanged`, `written`, `unvectored`, `unplaced` and `unplaced_closed` (the vector-less records whose every live engraved relation names a closed record: no edge can place them), `authored` (the topics whose title and summary this pass wrote, in a new version or in place), `authored_in_place`, `pending` (every topic with a field still pending after the pass, written tonight or standing, read back: what the topics route serves as pending), `failed` (a refused in-place write is reason `write_refused`), `reprioritized` (the standing topics' tasks moved up), `budget` (as given) and `budget_spent` (the author calls made: a host that shares one authoring budget between this pass and its drainer runs the pass first and gives the drainer `budget - budget_spent`), `authoring` (`{authored, failures}`; a topic left pending by the budget is no failure), `cancelled`, `skipped` and per-step `timings_ms`. What the author reads is the topic's briefing: its size, the kinds, the previous title, the sibling titles it must differ from (at most 40), and a stable kind-mixed sample of whole member lines within 1500 tokens (a line that does not fit is left out): under 2.3k tokens in all, about 1.5-1.9k for a home of a dozen topics.
 - `embed_missing(system, *, scopes, owner_id, record_ids=None, cancelled) -> dict` — embeds the open formed records of these scopes that have no vector (`record_ids=None`: every one of them, which needs numpy; a list: those only) with the store's own embedder, one small commit-guarded commit per vector, so partial progress stands. `cancelled` is checked before each record. An embedder failure is never raised: that record and the rest are skipped (`embedder_unavailable`; `no_embedder` without one); a vector of the wrong dimension is `failed` (`dimension_mismatch`) and never written. Community records are never embedded. Returns `{pass_name, embedded, skipped, failed: [{id, reason}], reasons, cancelled}`.
 - `place_record(system, prepared, *, cur) -> dict | None` with `PreparedPlacement(record_id, assertion_id, scope, owner_id, vector=None, neighbours=())` — the per-record step formation runs inside its transaction (`cur`: the open transaction on the journal's file, or `None`). Returns the event's provenance plus `event_id`, `seq` and `stored`, or `None` (no version, already placed or a member, nothing to place by). Idempotent: the event id derives from the record and the version.
 - `COMMUNITIES_PAYLOAD_VERSION` (2), `COMMUNITY_KIND` (`"community"`), `PLACED_EVENT_KIND` (`"placed"`), `FACTS_WINDOW_DAYS` (7), `FACTS_PATHS_TOP` (5).

{abstractmemory-0.5.0 → abstractmemory-0.5.1}/docs/backlog/overview.md

@@ -9,7 +9,8 @@ inspect the current code first, then update the item if reality has changed.
 
 ## Current Status (counts recounted on disk 2026-11-10, after the 0.5.0 closure; first reconciled 2026-07-20 — the shipped tree audited item by item)
 
-- Planned: 9 — `planned/knowledge_graph_convergence/` 0050-0058 (0050-0055 added
+- Planned: 10 — `planned/0069` (runtime recall stamps use `community_keys_of`, added 2026-11-10) and
+  `planned/knowledge_graph_convergence/` 0050-0058 (0050-0055 added
   2026-10-27, ADR 0002; 0056-0058, ADR 0003; 0056 in progress, first slice
   on `feat/memory-graph-provenance`; 0053 has its communities slice shipped in
   0.5.0). The track folder also holds three one-line pointer files (0059-0061,
@@ -173,6 +174,10 @@ The 2026-07-20 reconciliation moved every earlier planned item to
 extension_wave 0037-0039) or `deprecated/` (001/003/006, each with the
 superseding pointer).
 
+| Item | Priority | Notes |
+| --- | --- | --- |
+| [`0069_runtime_recall_stamps_use_community_keys_of.md`](planned/0069_runtime_recall_stamps_use_community_keys_of.md) | P1 | Memory side shipped on `main` (0.5.1 material); the runtime's `_stamp_recall_topics` switches from the full `communities()` read (190 ms at 20k records) to `community_keys_of` (22 µs). |
+
 ### Knowledge graph convergence track (`planned/knowledge_graph_convergence/`, 0050-0061)
 
 Operator ruling 2026-10-27 ([ADR 0002](../adr/0002-abstractmemory-owns-the-memory-graph.md)); see the

abstractmemory-0.5.1/docs/backlog/planned/0069_runtime_recall_stamps_use_community_keys_of.md

@@ -0,0 +1,51 @@
+# Planned: the runtime's recall stamps use `community_keys_of` instead of the full topics read
+
+## Metadata
+- Created: 2026-11-10
+- Status: Planned
+- Completed: N/A
+- Target: abstractmemory 0.5.1 (the read, shipped on `main`) + the next AbstractRuntime release (the switch)
+- Priority: P1 (a hidden per-recall cost that grows with the home)
+- Owner: AbstractRuntime (`abstractruntime/src/abstractruntime/integrations/abstractmemory/seam_handlers.py`)
+
+## ADR status
+- Governing ADRs: None (communities are a slice of planned
+  [0053](knowledge_graph_convergence/0053_self_organisation_communities_consolidation_reflection.md))
+- ADR impact: None
+
+## Context
+AbstractRuntime `1afbf68` stamps every recalled handle of a visit with its topic (`result.handles[i].topic`, the
+community key, read by the Entity app's live activity panel, root backlog 1196).
+`seam_handlers._stamp_recall_topics` reads memory's whole `communities()` payload once per recall
+(`identity.topics.entity_communities`) and looks each handle up by `provenance.record_id`. That payload computes
+every topic's `facts`, so the read is linear in the home: 19 ms at 2,000 records and 190 ms at 20,000 on the
+runtime's synthetic home (54 topics, 44 handles), on the passive recall of every turn.
+
+## Current code reality (2026-11-10)
+- Memory now serves the membership-only read: `community_index.community_keys_of(system, record_ids, *, scopes,
+  owner_id) -> {record_id: key | None}` (and `community_memberships_of`, with `provisional`), the same answer as
+  `communities()` for those records, from the in-memory community index. Measured for 50 ids, 54 topics, SQLite
+  home: 20 µs per call at 2,000 records, 22 µs at 20,000 (`tests/test_community_keys_of.py`, run with `-s`).
+- The runtime still calls `entity_communities` in `_stamp_recall_topics`.
+
+## Problem or opportunity
+The stamp costs up to a fifth of a second per recall on a 20k home, for a key a dict lookup can give.
+
+## Proposed direction
+In `_stamp_recall_topics`, collect the handles' graph ids (`provenance.record_id`) and call
+`community_keys_of(memory_system, ids, scopes=scopes, owner_id=owner_id)` once; `handle["topic"] =
+keys.get(graph_id)`. Raise the runtime's `abstractmemory` floor to the release that ships it (0.5.1). Keep the
+existing failure path (a read that raises leaves the recall whole, without `topic`, with its `#FALLBACK` line).
+
+## Promotion criteria
+abstractmemory 0.5.1 published with `community_keys_of`.
+
+## Validation ideas
+The runtime's existing stamp tests unchanged in outcome; its synthetic-home measurement repeated (expect the stamp
+under 1 ms at 20,000 records); a counter test that the stamp no longer calls `communities()`.
+
+## Non-goals
+Changing what `topic` means or the prompt.
+
+## Guidance for future agents
+Pass graph ids, not digest assertion ids: the read does no store lookup per id, so an assertion id answers None.

{abstractmemory-0.5.0 → abstractmemory-0.5.1}/docs/memory-system.md

@@ -93,7 +93,7 @@ One full night (`sleep_pass`) runs FOUR sub-phases in canonical order — **reso
 
 ### Topics: communities
 
-The home's topics are records of kind `community`, formed by `communities_pass`, which the host runs during its sleep: it detects communities over the vectored records (a deterministic, warm-started Louvain), attaches vector-less records through their engraved relations, keeps each topic's stable `community_key` across versions, lays the topics out for an app to draw, and has each changed topic's title and summary authored. Each version is a new record that `refines` the previous one. A record formed between two sleeps is placed in its nearest topic at formation, provisionally, and the next sleep confirms or moves it. Communities are engine state: they are never embedded and never recalled. `communities(...)` and `community_of(...)` read them (see [`api.md`](api.md#communities-topics)).
+The home's topics are records of kind `community`, formed by `communities_pass`, which the host runs during its sleep: it detects communities over the vectored records (a deterministic, warm-started Louvain), attaches vector-less records through their engraved relations, keeps each topic's stable `community_key` across versions, lays the topics out for an app to draw, and has each changed topic's title and summary authored. Each version is a new record that `refines` the previous one. A record formed between two sleeps is placed in its nearest topic at formation, provisionally, and the next sleep confirms or moves it. Communities are engine state: they are never embedded and never recalled. `communities(...)` and `community_of(...)` read them, and `community_keys_of(...)` answers which topic each of a few records is in, cheaply enough for every recall (see [`api.md`](api.md#communities-topics)).
 
 ## Understanding: a reading can change, its history stays
 

{abstractmemory-0.5.0 → abstractmemory-0.5.1}/llms-full.txt

@@ -39,6 +39,11 @@ Storage never decays and nothing is ever deleted; only retrieval strength change
 - The authoritative export list is [`src/abstractmemory/__init__.py`](src/abstractmemory/__init__.py); [`docs/api.md`](docs/api.md) documents it.
 - Requires Python 3.10+.
 
+## What's new in 0.5.1
+
+- `community_keys_of` / `community_memberships_of`: the membership-only topic read for every recalled record, tens of microseconds per call from the in-memory community index (the runtime's recall stamps use it).
+- A standing topic still pending its title or summary is authored in place by `communities_pass` with the budget the changed topics leave, and its queued tasks move up to the topics' priority.
+
 ## What's new in 0.5.0
 
 - Every memory's title and summary is authored: memory's passes take a host `author` callable, and a record whose authoring failed forms pending, with its task, until a reader completes it (`retitle`, `rewrite_digest`, `declare_pending`, `rearm_pending`).
@@ -585,7 +590,7 @@ One full night (`sleep_pass`) runs FOUR sub-phases in canonical order — **reso
 
 ### Topics: communities
 
-The home's topics are records of kind `community`, formed by `communities_pass`, which the host runs during its sleep: it detects communities over the vectored records (a deterministic, warm-started Louvain), attaches vector-less records through their engraved relations, keeps each topic's stable `community_key` across versions, lays the topics out for an app to draw, and has each changed topic's title and summary authored. Each version is a new record that `refines` the previous one. A record formed between two sleeps is placed in its nearest topic at formation, provisionally, and the next sleep confirms or moves it. Communities are engine state: they are never embedded and never recalled. `communities(...)` and `community_of(...)` read them (see [`api.md`](docs/api.md#communities-topics)).
+The home's topics are records of kind `community`, formed by `communities_pass`, which the host runs during its sleep: it detects communities over the vectored records (a deterministic, warm-started Louvain), attaches vector-less records through their engraved relations, keeps each topic's stable `community_key` across versions, lays the topics out for an app to draw, and has each changed topic's title and summary authored. Each version is a new record that `refines` the previous one. A record formed between two sleeps is placed in its nearest topic at formation, provisionally, and the next sleep confirms or moves it. Communities are engine state: they are never embedded and never recalled. `communities(...)` and `community_of(...)` read them, and `community_keys_of(...)` answers which topic each of a few records is in, cheaply enough for every recall (see [`api.md`](docs/api.md#communities-topics)).
 
 ## Understanding: a reading can change, its history stays
 
@@ -1028,7 +1033,7 @@ Enqueue sites: `remember_many` (a `title.author` / `digest.author` per pending f
 - `ack(task_id, *, holder, result, now=None) -> {acked, reason?, task?}` — settle a leased task as done (for example `{"outcome": "unchanged", "seconds": 41.2, "tokens": 3120, "run_id": ...}`); a task a verb already settled returns `reason="already_done"`.
 - `release(task_id, *, holder, outcome, reason, until=None, detail=None, now=None) -> {released, task?}` — `outcome="retry"` (attempt spent; failed at `max_attempts`), `"defer"` (no attempt spent; not before `until`, for an unreachable endpoint or embedder), `"skip"` (terminal, for example `reason="over_context"` with the measured tokens in `detail`).
 - `left_out(*, holder=None, window=None) -> list[dict]` — what the last claimed window of each `(holder, window)` left out, and why.
-- `task(task_id)`, `tasks(*, status=None, kind=None, target_id=None, ripple_id=None, limit=0)`, `log(task_id)`, `counts()`, `cost_of(task)`, `supersede_stale(system)`, `supersede_target(target_id, *, reason)`.
+- `task(task_id)`, `tasks(*, status=None, kind=None, target_id=None, ripple_id=None, limit=0)`, `log(task_id)`, `counts()`, `cost_of(task)`, `supersede_stale(system)`, `supersede_target(target_id, *, reason)`, `prioritize(target_ids, *, kinds, priority, reason, dry_run=False)` (moves the live tasks of `kinds` on these targets that wait at a later priority up to `priority`, never demoting one already ahead, one `prioritized` log row each; returns the task ids).
 
 On `MemorySystem` (and as module functions taking the system first):
 
@@ -1184,7 +1189,8 @@ Communities are eventually consistent. A record formed after the last sleep is p
 - `communities(system, *, scopes, owner_id, version=None) -> dict` — PURE READ, the payload a host serves (version 2): `{version: 2, version_seq, first_sleep_pending, algorithm, communities: [...], links: [{a, b, weight}]}`. Each community is `{id (this version's record id), key (stable across versions), title, summary, pending, verdict, size, kinds, members: [{id, via}], provisional_ids, cohesion, layout: {xy, radius, seats}, centroid_stale, facts}`, largest first; `links` name community keys. `title` and `summary` are `None` while pending (`pending` names the fields still being written). `provisional_ids` are the records placed at formation since the version (their newest `placed` event), not yet members; `size`, `kinds` and `members` are the version's. `version` reads an earlier version (the newest record of each community at or before that seq, viewed up to the next version); a seq older than the first version raises `ValueError`. A community record missing a field the read serves raises `CommunitySchemaError`.
 - `facts` (read from the store and journal as they are now, one query each): `span: {first, last}` (when its first and newest members formed), `activity` (its members' uses, `selected` events, in the last `FACTS_WINDOW_DAYS` = 7 days), `hub` (the member used most, all time; `None` when none was ever used), `bridges` (members with an engraved edge to a member of another community), `paths` (the `FACTS_PATHS_TOP` = 5 member pairs most often used together, `co_selected`, in the last 7 days: `[{from, to, count}]`), `changed_since_last_sleep: {added, moved, split_from}` (members that joined and left at this version, and the community it was split from, when its version is the newest; zeros otherwise). Not served yet: which community a dissolved one merged into.
 - `community_of(system, record_id) -> dict` — `{record_id, community_key, community_id, provisional, via, how, score, version_seq, first_sleep_pending}`: a member of the current version (`provisional: false`, `via` as the version names it), a provisional placement (`provisional: true`, `via: "provisional"`, `how`, `score`), or no community (`community_key: None`). An unknown record or a community record raises `ValueError`.
-- `communities_pass(system, *, scopes, owner_id, author, cancelled, budget=None, seed_previous=True, detect_kinds=None) -> dict` — one night's communities. It reads the graph (the formed records, their engraved edges and the store's vectors), detects communities over every vectored record (a deterministic Louvain warm-started from the previous version, at most `MAX_DETECTION_ROUNDS` = 5 rounds; `written_amid` never drives detection), attaches the vector-less records through their engraved neighbours, carries each community's `community_key` across versions by membership overlap, computes the layout, has the changed communities' titles and summaries written by `author` (the same callable as the other passes, see [the `author` callable](#authored-with-the-record-the-author-callable); `budget` = the most author calls, all of them for the topics, largest first: one call per new or changed topic writes both its title and its summary, so the pass spends at most `min(budget, n)` and never more than one call per topic; `None` authors every topic; the topics past the budget form pending, their tasks at `COMMUNITY_PRIORITY`, ahead of every other pending record), calls `cancelled()` once, and writes the new versions, their bindings, the closures of the previous versions and their tasks in ONE commit-guarded transaction. An unchanged home writes nothing. A home opened without `task_queue=True` is refused; without numpy (`abstractmemory[fast]`) the pass is skipped with its reason. The receipt carries `version_seq`, `new` / `kept` / `renamed` / `dissolved` / `unchanged`, `written`, `unvectored`, `unplaced` and `unplaced_closed` (the vector-less records whose every live engraved relation names a closed record: no edge can place them), `authored` (the topics whose title and summary this pass wrote), `pending` (the topics written with a field still pending, their tasks queued), `failed`, `budget` (as given) and `budget_spent` (the author calls made: a host that shares one authoring budget between this pass and its drainer runs the pass first and gives the drainer `budget - budget_spent`), `authoring` (`{authored, failures}`; a topic left pending by the budget is no failure), `cancelled`, `skipped` and per-step `timings_ms`. What the author reads is the topic's briefing: its size, the kinds, the previous title, the sibling titles it must differ from (at most 40), and a stable kind-mixed sample of whole member lines within 1500 tokens (a line that does not fit is left out): under 2.3k tokens in all, about 1.5-1.9k for a home of a dozen topics.
+- `community_keys_of(system, record_ids, *, scopes, owner_id) -> dict` and `community_memberships_of(system, record_ids, *, scopes, owner_id) -> dict` — the membership-only read a host stamps on every recalled record: `{record_id: key | None}` and `{record_id: {key, provisional} | None}` for record graph ids (the ids members carry: a recalled record's `provenance.record_id`). Each answer is the one `communities(system, scopes=, owner_id=)` gives for that record: a member of a current community of these pairs (`provisional: false`), a record placed at formation since the version (`provisional: true`), else `None` (unplaced, unknown, a community record, another owner's record, no version yet). It is served from the in-memory community index under the store's lock: one `MAX(rowid)` compare with the index's mark (the formation's catch-up) and a dict lookup per id, no facts, no layout, no vector, no file opened or stat'ed; measured 20 µs per call for 50 ids at 2,000 records and 22 µs at 20,000 (54 topics; `communities()` on the same homes: 12 ms and 65 ms). After a formation the `placed` events above the last read are fetched once per scope pair; a new version reloads the index (once a night). Like placement, it sees a version that only dissolves communities (no new community record) at the next version. An unreadable version raises `CommunitySchemaError`; a store other than SQLite or in-memory raises `TypeError`.
+- `communities_pass(system, *, scopes, owner_id, author, cancelled, budget=None, seed_previous=True, detect_kinds=None) -> dict` — one night's communities. It reads the graph (the formed records, their engraved edges and the store's vectors), detects communities over every vectored record (a deterministic Louvain warm-started from the previous version, at most `MAX_DETECTION_ROUNDS` = 5 rounds; `written_amid` never drives detection), attaches the vector-less records through their engraved neighbours, carries each community's `community_key` across versions by membership overlap, computes the layout, has the changed communities' titles and summaries written by `author` (the same callable as the other passes, see [the `author` callable](#authored-with-the-record-the-author-callable); `budget` = the most author calls, all of them for the topics, largest first: one call per new or changed topic writes both its title and its summary, so the pass spends at most `min(budget, n)` and never more than one call per topic; `None` authors every topic; the topics past the budget form pending, their tasks at `COMMUNITY_PRIORITY`, ahead of every other pending record; what the budget leaves goes to the STANDING topics, unchanged tonight but still pending, largest first, one call each for the fields still pending), calls `cancelled()` once, and writes the new versions, their bindings, the closures of the previous versions and their tasks in ONE commit-guarded transaction, which also moves every standing topic's live `title.author` / `digest.author` task that waits at a later priority (queued before topics came first) up to `COMMUNITY_PRIORITY`. The standing pending topics are then written IN PLACE on their current record (`rewrite_digest`, with the title when both are pending, or `retitle`; each its own guarded transaction), which completes their live tasks. An unchanged home with no pending topic to author and no task to move writes nothing. A home opened without `task_queue=True` is refused; without numpy (`abstractmemory[fast]`) the pass is skipped with its reason. The receipt carries `version_seq`, `new` / `kept` / `renamed` / `dissolved` / `unchanged`, `written`, `unvectored`, `unplaced` and `unplaced_closed` (the vector-less records whose every live engraved relation names a closed record: no edge can place them), `authored` (the topics whose title and summary this pass wrote, in a new version or in place), `authored_in_place`, `pending` (every topic with a field still pending after the pass, written tonight or standing, read back: what the topics route serves as pending), `failed` (a refused in-place write is reason `write_refused`), `reprioritized` (the standing topics' tasks moved up), `budget` (as given) and `budget_spent` (the author calls made: a host that shares one authoring budget between this pass and its drainer runs the pass first and gives the drainer `budget - budget_spent`), `authoring` (`{authored, failures}`; a topic left pending by the budget is no failure), `cancelled`, `skipped` and per-step `timings_ms`. What the author reads is the topic's briefing: its size, the kinds, the previous title, the sibling titles it must differ from (at most 40), and a stable kind-mixed sample of whole member lines within 1500 tokens (a line that does not fit is left out): under 2.3k tokens in all, about 1.5-1.9k for a home of a dozen topics.
 - `embed_missing(system, *, scopes, owner_id, record_ids=None, cancelled) -> dict` — embeds the open formed records of these scopes that have no vector (`record_ids=None`: every one of them, which needs numpy; a list: those only) with the store's own embedder, one small commit-guarded commit per vector, so partial progress stands. `cancelled` is checked before each record. An embedder failure is never raised: that record and the rest are skipped (`embedder_unavailable`; `no_embedder` without one); a vector of the wrong dimension is `failed` (`dimension_mismatch`) and never written. Community records are never embedded. Returns `{pass_name, embedded, skipped, failed: [{id, reason}], reasons, cancelled}`.
 - `place_record(system, prepared, *, cur) -> dict | None` with `PreparedPlacement(record_id, assertion_id, scope, owner_id, vector=None, neighbours=())` — the per-record step formation runs inside its transaction (`cur`: the open transaction on the journal's file, or `None`). Returns the event's provenance plus `event_id`, `seq` and `stored`, or `None` (no version, already placed or a member, nothing to place by). Idempotent: the event id derives from the record and the version.
 - `COMMUNITIES_PAYLOAD_VERSION` (2), `COMMUNITY_KIND` (`"community"`), `PLACED_EVENT_KIND` (`"placed"`), `FACTS_WINDOW_DAYS` (7), `FACTS_PATHS_TOP` (5).

{abstractmemory-0.5.0 → abstractmemory-0.5.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "AbstractMemory"
-version = "0.5.0"
+version = "0.5.1"
 description = "AbstractMemory: temporal, provenance-aware semantic memory primitives for AbstractFramework."
 readme = "README.md"
 requires-python = ">=3.10"

abstractmemory-0.5.1/release-notes.md

@@ -0,0 +1,38 @@
+### Added: `community_keys_of`, the membership-only read for recall stamping
+
+The runtime stamps every recalled record with its topic by reading the whole `communities()` payload per recall, which
+computes every topic's facts: 19 ms at 2,000 records and 190 ms at 20,000 on its synthetic home.
+
+- `community_index.community_keys_of(system, record_ids, *, scopes, owner_id) -> {record_id: key | None}` and
+  `community_memberships_of(...)` -> `{record_id: {key, provisional} | None}`: the answer `communities()` gives for
+  those records (member, provisional since the version, or none), read from the in-memory community index. The hot
+  path is the formation's catch-up compare (`MAX(rowid)` on the open connection against the index's mark) and a dict
+  lookup per id: no fold, no facts, no file opened, stat'ed or locked. Measured for 50 ids (54 topics, SQLite home):
+  20 µs per call at 2,000 records, 22 µs at 20,000 (`communities()` on the same homes: 12 ms and 65 ms). After a
+  formation, the `placed` events above the last read are fetched once per scope pair; a new version reloads the
+  index. A dissolve-only version is seen at the next version, as at placement (backlog 0066).
+- The runtime switches its recall stamps to it: backlog 0069.
+
+Upgrading from 0.5.0: a host that stamps recalled records with their topic should call `community_keys_of` (or
+`community_memberships_of`) with the records' graph ids instead of reading the whole `communities()` payload per
+recall; nothing else changes. Known limits, carried in the backlog: a version that only dissolves communities (no
+new community record) is seen by the membership reads and by placement at the next version that writes one (0066);
+`communities()` read while a version is being written may serve one empty read (0067).
+
+### Fixed: a standing topic still pending is authored, and its old tasks move up
+
+Castor's second topics night (2026-11-10): 10 of 11 topics got titles; the 11th stood unchanged, its title and
+summary tasks queued on the first night at the old priority 5 behind ~267 repair tasks, so the pass skipped it with 6
+calls of its budget unused, and its receipt said `pending: 0` while the topics route showed it pending.
+
+- `communities_pass` spends what its `budget` leaves after the new and changed topics on the STANDING topics still
+  pending (unchanged tonight), largest first, one call each for the fields still pending, from tonight's briefing.
+  Each is written in place on its current record after the version (`rewrite_digest`, with the title when both are
+  pending, or `retitle`), which completes its live `title.author` / `digest.author` tasks; the record keeps its
+  `title_basis` (with the sample its author saw) and the author's verdict. The receipt's `pending` now counts every
+  topic still without a title or summary after the pass, read back; new keys `authored_in_place`, `reprioritized`
+  and `ids.authored_in_place`; a refused in-place write is a failure line with reason `write_refused`.
+- Every standing topic's live authoring task that waits at a later priority than `COMMUNITY_PRIORITY` (3) is moved up
+  to it, in the version's transaction (its own when no version is written): a topic's legacy priority-5 tasks no
+  longer wait behind the repair tail. New `MemoryTaskQueue.prioritize(target_ids, *, kinds, priority, reason,
+  dry_run=False)`: one UPDATE, never demotes a task already ahead, one `prioritized` log row each.