zaxy-memory 2.4.0__tar.gz → 2.4.2__tar.gz

{zaxy_memory-2.4.0 → zaxy_memory-2.4.2}/CHANGELOG.md

@@ -2,6 +2,38 @@
 
 All notable Zaxy release changes are recorded here.
 
+## 2.4.2 - 2026-06-15
+
+- **Memory Checkout latency fix — warm `memory_checkout` ~19s → ~1.6s on a
+  78k-event session.** Two retrieval costs that previously scaled with the
+  whole log on *every* checkout (the BM25 verbatim index and the verified
+  replay) are now cached per session and extended incrementally, so they scale
+  with newly appended events instead of the full corpus. Every append —
+  including checkout's own salience-reinforcement write — used to invalidate
+  both caches and force a full rebuild; on large, active sessions that pushed
+  checkout past the MCP stdio call timeout (the "checkout hangs over MCP"
+  symptom). Steady-state checkout is now well under the timeout.
+  - `VerbatimIndex.append_chunks()` extends the index reusing prior
+    tokenization; results are byte-identical to a full rebuild.
+  - `EventLog.read_from_offset()` tail-reads only newly appended bytes;
+    `verify_event_chain()` verifies the appended tail against the
+    already-verified prefix. The tail verification is also a consistency
+    guard — any offset skew or log rewrite falls back to a full verified
+    replay, so the fast path never silently misses, duplicates, or trusts a
+    bad event. The hash-chain integrity guarantee is preserved, not dropped.
+  - No API or behavior change; same cited facts are surfaced.
+
+## 2.4.1 - 2026-06-14
+
+- Fixed MCP-directory installability (Glama listed the server as "cannot be
+  installed"). The package shipped only a `zaxy` console script, but directory
+  auto-installers assume the script name matches the package name — so
+  `uvx zaxy-memory serve` failed with "executable `zaxy-memory` is not provided".
+  Added a `zaxy-memory` console-script alias (same entrypoint), declared the
+  launch in `server.json` (`runtimeHint: uvx`, `packageArguments: ["serve"]`), and
+  added `glama.json` (maintainer claim). `uvx zaxy-memory serve` now starts the
+  stdio MCP server directly. No code/runtime changes.
+
 ## 2.4.0 - 2026-06-14
 
 - **Portable signed memory export — EXPERIMENTAL / UNAUDITED, opt-in.** New

{zaxy_memory-2.4.0 → zaxy_memory-2.4.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: zaxy-memory
-Version: 2.4.0
+Version: 2.4.2
 Summary: Coordinator memory for auditable multi-agent projects
 Project-URL: Homepage, https://syndicalt.github.io/zaxy/
 Project-URL: Documentation, https://syndicalt.github.io/zaxy/docs/getting-started.html

zaxy_memory-2.4.2/docs/announcements/zaxy-v2.4-x-article.md

@@ -0,0 +1,102 @@
+# X Article Draft: Zaxy 2.4.0
+
+![Zaxy 2.4 release header](../assets/zaxy-v2.4-header.png)
+
+Zaxy 2.4.0 ships the first piece of **portable, signed agent memory** — and the
+honest headline is that it is **experimental, unaudited, and we are asking for
+review before anyone trusts it.**
+
+The short version: agent memory is locked inside vendor runtimes, and the moment
+you want to move it — between models, between agents, across a dispute — you need
+a way to prove *what* the memory is and *who* vouches for it, while revealing only
+the parts you choose. 2.4 adds that, built to the shape the [W3C "AI Agent Memory
+Interoperability" Community Group](https://www.w3.org/community/ai-agent-memory-interop/)
+(proposed weeks ago) and the *Portable Agent Memory* paper are converging on. It
+is opt-in (`pip install zaxy-memory[export]`), it warns loudly on import, and it
+makes no security claims it has not earned.
+
+## What shipped
+
+`zaxy.portable` — a self-sovereign, verifiable memory-export layer on top of
+Eventloom's existing hash-chained integrity:
+
+- **Signed export bundles.** Post-quantum **ML-DSA-65** (NIST FIPS-204) signatures,
+  with Ed25519 as a classical fallback — using vetted `pyca/cryptography`
+  primitives, never homegrown ones. Self-sovereign: you hold the key, the public
+  key *is* the identity, no certificate authority.
+- **Verifiable partial disclosure.** A domain-separated Merkle tree lets you hand
+  someone a *subset* of your memory plus an inclusion proof, and they can verify
+  it belongs to the signed set **without seeing the rest**.
+- **Injection-resistant rehydration.** Recalled content (tool output, transcripts)
+  is treated as data, not instructions — guarded fencing + flagging — so resurfaced
+  memory can't hijack the next model. Defense-in-depth, not a guarantee.
+- **Encryption envelope + cryptographic erasure.** Per-cell AES-256-GCM keys, and
+  GDPR-Article-17 erasure by destroying the key (the immutable ciphertext stays,
+  permanently undecryptable) — with the hard invariant that key material must never
+  enter the append-only log.
+- **Capability-scoped sharing** and a pluggable **public anchor** (offline stub
+  today; an OpenTimestamps hook for operator-independent verification).
+
+This sits on the 2.3 line that made memory *stick*: per-turn memory injection
+(2.3.2, extended to Codex in 2.3.3) that closed the recall-persistence gap, and
+opt-in tool-I/O offload provenance (2.3.4).
+
+## Why it matters
+
+Memory you can't move is a lock-in. Memory you *can* move but can't *verify* is a
+liability. The interesting problem isn't storage — it's a transfer format that is
+**authentic** (signed), **minimally disclosed** (prove a slice, not the whole
+history), **forgettable** (cryptographic erasure in an append-only world), and
+**safe to rehydrate** into a different model. That's an open frontier — a W3C group
+and fresh research are forming around it right now — and Zaxy already had the
+right substrate for it: an immutable, hash-chained event log where provenance is
+the product, not an afterthought.
+
+## The research behind it
+
+This release is the end of an honest chain of experiments, not a leap of faith:
+
+- We tested whether a **dense symbolic encoding** of memory would help models obey
+  it. It didn't: an invented glyph notation *failed comprehension* on both Claude
+  and GPT-4.1, and bought ~0% token savings over plain prose. The lesson — reuse a
+  notation the model already knows; don't invent one — shaped everything after.
+- We found the real lever was **injection, not format**: putting declarative state
+  in front of the model lifted adherence from ~0.25 to ~1.0, even buried in a long
+  context. That became the 2.3.2 recall fix.
+- Hardening tool-I/O provenance (2.3.4) surfaced that our own capture *truncated*
+  tool output — a gap in the audit trail. Closing it is what pointed at a proper
+  signed-export design.
+- For 2.4 we built to the emerging standard's choices — PQ signatures, Merkle
+  provenance, crypto-erasure — rather than a bespoke format, so it can interoperate
+  instead of fork.
+
+## The honest part — and the ask
+
+We are a small project. We **cannot commission a formal cryptographic audit**, and
+we will not pretend a clean self-review is one. What we *have* done is the
+proportionate thing: vetted primitives only, an adversarial self-review of the
+protocol composition (which already caught and fixed three real issues), opt-in
+packaging, a loud import warning, and a written threat model. The self-review is a
+*layer*, not a substitute for independent human eyes.
+
+So this is a genuine **call for review**. If you do cryptography or agent-memory
+work, please poke holes:
+
+- The composition questions are on
+  [crypto.stackexchange](https://crypto.stackexchange.com/) — does binding the
+  Merkle root + all metadata under one signature prevent swap attacks; are the
+  (non-position-bound) subset proofs forgeable; what breaks crypto-erasure beyond
+  "ensure no other key copy survives."
+- Design, threat model, and conformance mapping:
+  [`docs/experimental/portable-export-security.md`](https://github.com/syndicalt/zaxy/blob/v2.4.0/docs/experimental/portable-export-security.md)
+  and the [`zaxy.portable` source](https://github.com/syndicalt/zaxy/tree/v2.4.0/src/zaxy/portable)
+  (pinned to v2.4.0).
+- If you're in the W3C interop CG: tell us where we diverge from where the spec is
+  heading.
+
+Try it: `pip install "zaxy-memory[export]"`, then `zaxy export-keygen`, `zaxy
+export`, `zaxy verify-export`. The import warning is intentional — keep it in mind.
+
+Until real review comes back, it stays experimental and unadvertised as anything
+more. That's the deal, and we'd rather say it out loud than ship a quiet "trust
+me." Findings welcome — break it.

zaxy_memory-2.4.2/docs/assets/zaxy-v2.4-header.svg

@@ -0,0 +1,65 @@
+<svg width="1600" height="640" viewBox="0 0 1600 640" xmlns="http://www.w3.org/2000/svg" font-family="Inter, system-ui, sans-serif">
+  <defs>
+    <pattern id="grid" width="86" height="86" patternUnits="userSpaceOnUse">
+      <path d="M86 0H0V86" fill="none" stroke="#dce6e2" stroke-opacity="0.05" stroke-width="1"/>
+    </pattern>
+    <radialGradient id="vig" cx="64%" cy="40%" r="80%">
+      <stop offset="0%" stop-color="#16212b"/>
+      <stop offset="72%" stop-color="#0b1014"/>
+      <stop offset="100%" stop-color="#070b0e"/>
+    </radialGradient>
+    <filter id="glow" x="-60%" y="-60%" width="220%" height="220%"><feGaussianBlur stdDeviation="3"/></filter>
+    <g id="zmark">
+      <rect x="-19" y="-19" width="38" height="38" rx="9" fill="#f8faf7"/>
+      <path d="M-9,-7 H9 L-9,7 H10" fill="none" stroke="#0b1014" stroke-width="3.6" stroke-linejoin="round" stroke-linecap="round"/>
+    </g>
+  </defs>
+
+  <rect width="1600" height="640" fill="url(#vig)"/>
+  <rect width="1600" height="640" fill="url(#grid)"/>
+
+  <!-- LEFT: title block -->
+  <text x="96" y="150" font-family="JetBrains Mono, monospace" font-size="22" letter-spacing="5" fill="#2eb7a4">RELEASE · v2.4.0</text>
+  <text x="92" y="248" font-family="Georgia, 'Times New Roman', serif" font-size="84" font-weight="700" fill="#f8faf7">Portable, signed</text>
+  <text x="92" y="332" font-family="Georgia, 'Times New Roman', serif" font-size="84" font-weight="700" fill="#f8faf7">memory.</text>
+  <text x="96" y="392" font-family="Inter, system-ui, sans-serif" font-size="25" fill="#9fb0aa">Self-sovereign export · post-quantum signatures ·</text>
+  <text x="96" y="426" font-family="Inter, system-ui, sans-serif" font-size="25" fill="#9fb0aa">verifiable subset disclosure · injection-resistant rehydration</text>
+
+  <!-- experimental / review pill (honest, even on the card) -->
+  <g transform="translate(96,470)">
+    <rect x="0" y="0" width="340" height="44" rx="22" fill="none" stroke="#c58b2c" stroke-width="2"/>
+    <circle cx="26" cy="22" r="6" fill="#c58b2c"/>
+    <text x="44" y="29" font-family="JetBrains Mono, monospace" font-size="17" fill="#e0c08a">EXPERIMENTAL · UNAUDITED · review welcome</text>
+  </g>
+
+  <!-- RIGHT: Merkle tree with a signature seal at the root -->
+  <g transform="translate(1230,300)" stroke="#2eb7a4" fill="none">
+    <!-- edges -->
+    <g stroke-width="2.5" stroke-opacity="0.7">
+      <line x1="0" y1="-92" x2="-90" y2="0"/><line x1="0" y1="-92" x2="90" y2="0"/>
+      <line x1="-90" y1="0" x2="-140" y2="92"/><line x1="-90" y1="0" x2="-46" y2="92"/>
+      <line x1="90" y1="0" x2="46" y2="92"/><line x1="90" y1="0" x2="140" y2="92"/>
+    </g>
+    <!-- leaves -->
+    <g fill="#0e1620" stroke-width="2.5">
+      <rect x="-160" y="78" width="40" height="28" rx="5"/><rect x="-66" y="78" width="40" height="28" rx="5"/>
+      <rect x="26" y="78" width="40" height="28" rx="5"/><rect x="120" y="78" width="40" height="28" rx="5"/>
+    </g>
+    <!-- internal nodes -->
+    <circle cx="-90" cy="0" r="15" fill="#0e1620" stroke-width="2.5"/>
+    <circle cx="90" cy="0" r="15" fill="#0e1620" stroke-width="2.5"/>
+    <!-- root + seal -->
+    <circle cx="0" cy="-92" r="40" fill="#0e1620" stroke-width="3" filter="url(#glow)"/>
+    <circle cx="0" cy="-92" r="40" fill="none" stroke="#2eb7a4" stroke-width="3"/>
+    <path d="M-16,-92 l10,11 l20,-22" stroke="#2eb7a4" stroke-width="4" stroke-linecap="round" stroke-linejoin="round"/>
+    <circle cx="0" cy="-92" r="54" stroke="#2eb7a4" stroke-opacity="0.35" stroke-width="1.5"/>
+  </g>
+  <text x="1230" y="470" text-anchor="middle" font-family="JetBrains Mono, monospace" font-size="15" fill="#66747a">ML-DSA · Merkle root · signed</text>
+
+  <!-- footer brand -->
+  <g transform="translate(98,560)">
+    <use href="#zmark" transform="scale(0.85)"/>
+    <text x="32" y="7" font-family="Georgia, 'Times New Roman', serif" font-size="27" font-weight="700" fill="#f8faf7">Zaxy</text>
+    <text x="104" y="7" font-family="JetBrains Mono, monospace" font-size="15" fill="#66747a">auditable agent memory · MIT</text>
+  </g>
+</svg>

{zaxy_memory-2.4.0 → zaxy_memory-2.4.2}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "zaxy-memory"
-version = "2.4.0"
+version = "2.4.2"
 description = "Coordinator memory for auditable multi-agent projects"
 readme = "README.md"
 requires-python = ">=3.11"
@@ -105,6 +105,9 @@ dev = [
 
 [project.scripts]
 zaxy = "zaxy.__main__:main"
+# Alias so directory auto-installers that assume the console-script name matches
+# the package name can launch it: `uvx zaxy-memory serve`.
+zaxy-memory = "zaxy.__main__:main"
 
 [project.urls]
 Homepage = "https://syndicalt.github.io/zaxy/"

zaxy_memory-2.4.2/site/docs/announcements/zaxy-v2.4-x-article.html

@@ -0,0 +1,63 @@
+<!doctype html>
+<html lang="en">
+<head>
+  <meta charset="utf-8" />
+  <meta name="viewport" content="width=device-width, initial-scale=1" />
+  <title>X Article Draft: Zaxy 2.4.0 - Zaxy Docs</title>
+  <link rel="preconnect" href="https://fonts.googleapis.com" />
+  <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
+  <link href="https://fonts.googleapis.com/css2?family=Fraunces:opsz,wght@9..144,300;9..144,600&family=Inter:wght@400;550;650;750;850&family=JetBrains+Mono:wght@500;700&display=swap" rel="stylesheet" />
+  <link rel="stylesheet" href="../../style.css" />
+</head>
+<body class="doc-page">
+  <nav class="nav" aria-label="Documentation">
+    <a class="brand" href="../../index.html"><span class="brand-mark" aria-hidden="true"></span>Zaxy</a>
+    <div class="nav-links">
+      <a href="../../index.html">Overview</a>
+      <a href="../../docs/coordinate-roadmap.html">Coordinate</a>
+      <a href="../../docs/why-zaxy.html">Why Zaxy</a>
+      <a href="../../docs/benchmarks.html">Benchmarks</a>
+      <a href="../getting-started.html">Start</a>
+      <a href="https://github.com/syndicalt/zaxy" class="nav-cta">GitHub</a>
+    </div>
+  </nav>
+  <main class="doc-layout">
+    <article class="doc-content">
+      <h1 id="x-article-draft-zaxy-240">X Article Draft: Zaxy 2.4.0</h1>
+      <p><img src="../../assets/zaxy-v2.4-header.png" alt="Zaxy 2.4 release header" /></p>
+      <p>Zaxy 2.4.0 ships the first piece of <strong>portable, signed agent memory</strong> — and the honest headline is that it is <strong>experimental, unaudited, and we are asking for review before anyone trusts it.</strong></p>
+      <p>The short version: agent memory is locked inside vendor runtimes, and the moment you want to move it — between models, between agents, across a dispute — you need a way to prove <em>what</em> the memory is and <em>who</em> vouches for it, while revealing only the parts you choose. 2.4 adds that, built to the shape the <a href="https://www.w3.org/community/ai-agent-memory-interop/">W3C &quot;AI Agent Memory Interoperability&quot; Community Group</a> (proposed weeks ago) and the <em>Portable Agent Memory</em> paper are converging on. It is opt-in (<code>pip install zaxy-memory[export]</code>), it warns loudly on import, and it makes no security claims it has not earned.</p>
+      <h2 id="what-shipped">What shipped</h2>
+      <p><code>zaxy.portable</code> — a self-sovereign, verifiable memory-export layer on top of Eventloom&#x27;s existing hash-chained integrity:</p>
+      <ul>
+        <li><strong>Signed export bundles.</strong> Post-quantum <strong>ML-DSA-65</strong> (NIST FIPS-204) signatures, with Ed25519 as a classical fallback — using vetted <code>pyca/cryptography</code> primitives, never homegrown ones. Self-sovereign: you hold the key, the public key <em>is</em> the identity, no certificate authority.</li>
+        <li><strong>Verifiable partial disclosure.</strong> A domain-separated Merkle tree lets you hand someone a <em>subset</em> of your memory plus an inclusion proof, and they can verify it belongs to the signed set <strong>without seeing the rest</strong>.</li>
+        <li><strong>Injection-resistant rehydration.</strong> Recalled content (tool output, transcripts) is treated as data, not instructions — guarded fencing + flagging — so resurfaced memory can&#x27;t hijack the next model. Defense-in-depth, not a guarantee.</li>
+        <li><strong>Encryption envelope + cryptographic erasure.</strong> Per-cell AES-256-GCM keys, and GDPR-Article-17 erasure by destroying the key (the immutable ciphertext stays, permanently undecryptable) — with the hard invariant that key material must never enter the append-only log.</li>
+        <li><strong>Capability-scoped sharing</strong> and a pluggable <strong>public anchor</strong> (offline stub today; an OpenTimestamps hook for operator-independent verification).</li>
+      </ul>
+      <p>This sits on the 2.3 line that made memory <em>stick</em>: per-turn memory injection (2.3.2, extended to Codex in 2.3.3) that closed the recall-persistence gap, and opt-in tool-I/O offload provenance (2.3.4).</p>
+      <h2 id="why-it-matters">Why it matters</h2>
+      <p>Memory you can&#x27;t move is a lock-in. Memory you <em>can</em> move but can&#x27;t <em>verify</em> is a liability. The interesting problem isn&#x27;t storage — it&#x27;s a transfer format that is <strong>authentic</strong> (signed), <strong>minimally disclosed</strong> (prove a slice, not the whole history), <strong>forgettable</strong> (cryptographic erasure in an append-only world), and <strong>safe to rehydrate</strong> into a different model. That&#x27;s an open frontier — a W3C group and fresh research are forming around it right now — and Zaxy already had the right substrate for it: an immutable, hash-chained event log where provenance is the product, not an afterthought.</p>
+      <h2 id="the-research-behind-it">The research behind it</h2>
+      <p>This release is the end of an honest chain of experiments, not a leap of faith:</p>
+      <ul>
+        <li>We tested whether a <strong>dense symbolic encoding</strong> of memory would help models obey it. It didn&#x27;t: an invented glyph notation <em>failed comprehension</em> on both Claude and GPT-4.1, and bought ~0% token savings over plain prose. The lesson — reuse a notation the model already knows; don&#x27;t invent one — shaped everything after.</li>
+        <li>We found the real lever was <strong>injection, not format</strong>: putting declarative state in front of the model lifted adherence from ~0.25 to ~1.0, even buried in a long context. That became the 2.3.2 recall fix.</li>
+        <li>Hardening tool-I/O provenance (2.3.4) surfaced that our own capture <em>truncated</em> tool output — a gap in the audit trail. Closing it is what pointed at a proper signed-export design.</li>
+        <li>For 2.4 we built to the emerging standard&#x27;s choices — PQ signatures, Merkle provenance, crypto-erasure — rather than a bespoke format, so it can interoperate instead of fork.</li>
+      </ul>
+      <h2 id="the-honest-part-and-the-ask">The honest part — and the ask</h2>
+      <p>We are a small project. We <strong>cannot commission a formal cryptographic audit</strong>, and we will not pretend a clean self-review is one. What we <em>have</em> done is the proportionate thing: vetted primitives only, an adversarial self-review of the protocol composition (which already caught and fixed three real issues), opt-in packaging, a loud import warning, and a written threat model. The self-review is a <em>layer</em>, not a substitute for independent human eyes.</p>
+      <p>So this is a genuine <strong>call for review</strong>. If you do cryptography or agent-memory work, please poke holes:</p>
+      <ul>
+        <li>The composition questions are on <a href="https://crypto.stackexchange.com/">crypto.stackexchange</a> — does binding the Merkle root + all metadata under one signature prevent swap attacks; are the (non-position-bound) subset proofs forgeable; what breaks crypto-erasure beyond &quot;ensure no other key copy survives.&quot;</li>
+        <li>Design, threat model, and conformance mapping: <a href="https://github.com/syndicalt/zaxy/blob/v2.4.0/docs/experimental/portable-export-security.md"><code>docs/experimental/portable-export-security.md</code></a> and the <a href="https://github.com/syndicalt/zaxy/tree/v2.4.0/src/zaxy/portable"><code>zaxy.portable</code> source</a> (pinned to v2.4.0).</li>
+        <li>If you&#x27;re in the W3C interop CG: tell us where we diverge from where the spec is heading.</li>
+      </ul>
+      <p>Try it: <code>pip install &quot;zaxy-memory[export]&quot;</code>, then <code>zaxy export-keygen</code>, <code>zaxy export</code>, <code>zaxy verify-export</code>. The import warning is intentional — keep it in mind.</p>
+      <p>Until real review comes back, it stays experimental and unadvertised as anything more. That&#x27;s the deal, and we&#x27;d rather say it out loud than ship a quiet &quot;trust me.&quot; Findings welcome — break it.</p>
+    </article>
+  </main>
+</body>
+</html>

{zaxy_memory-2.4.0 → zaxy_memory-2.4.2}/site/index.html

@@ -62,7 +62,7 @@
     </section>
 
     <section class="proof-strip" aria-label="Project status">
-      <span>PyPI 2.4.0</span>
+      <span>PyPI 2.4.2</span>
       <span>47 MCP tools, 8-tool core profile</span>
       <span>cognitive ranking default</span>
       <span>embedded local runtime</span>
@@ -334,7 +334,7 @@ zaxy coordinate checkout --mission auth-main</code></pre>
   </main>
 
   <footer>
-    <span>Zaxy v2.4.0 - MIT</span>
+    <span>Zaxy v2.4.2 - MIT</span>
     <span><a href="README.html">README</a> <a href="docs/runbook.html">Runbook</a></span>
   </footer>
 </body>

{zaxy_memory-2.4.0 → zaxy_memory-2.4.2}/src/zaxy/core.py

@@ -54,7 +54,12 @@ from zaxy.context_refresh import (
 )
 from zaxy.documents import collect_document_events
 from zaxy.embedding import build_embedding_provider, embed_extraction
-from zaxy.event import EventLog, ReplayResult  # noqa: F401 - compatibility for existing tests
+from zaxy.event import (  # noqa: F401 - ReplayResult re-export for existing tests
+    EventLog,
+    IntegrityReport,
+    ReplayResult,
+    verify_event_chain,
+)
 from zaxy.evidence import select_checkout_evidence
 from zaxy.extract import extract
 from zaxy.inference import build_inferred_edge_events
@@ -138,7 +143,7 @@ from zaxy.synthesis_artifact import (
 from zaxy.synthesis_packet import synthesis_packet_from_items
 from zaxy.trace import MemoryTracer
 from zaxy.transcripts import collect_transcript_events
-from zaxy.verbatim import VerbatimIndex
+from zaxy.verbatim import VerbatimIndex, _chunks_from_events
 from zaxy.working_set import build_working_set, format_working_set
 from zaxy.workspace import (
     WorkspaceProfile,
@@ -404,7 +409,8 @@ class MemoryFabric:
             packet_memory_enabled=resolved_settings.context_packet_memory_enabled,
             packet_memory_slots=resolved_settings.context_packet_memory_slots,
         )
-        self._verbatim_index_cache: dict[str, tuple[tuple[int, int], VerbatimIndex]] = {}
+        self._verbatim_index_cache: dict[str, tuple[tuple[int, int], VerbatimIndex, int]] = {}
+        self._replay_cache: dict[str, tuple[tuple[int, int], ReplayResult, int]] = {}
         self._event_ref_index_cache: dict[str, tuple[tuple[int, int], dict[int, tuple[str, str]]]] = {}
         self._session_cue_index_cache: dict[str, tuple[tuple[int, int], dict[int, frozenset[str]]]] = {}
         self._query_page_cache: dict[
@@ -2156,14 +2162,30 @@ class MemoryFabric:
         return contexts
 
     def _verbatim_index(self, session_id: str) -> VerbatimIndex:
-        """Return a cached verbatim index for the current Eventloom file state."""
+        """Return a verbatim index for the current Eventloom file state.
+
+        The index is cached per session and extended incrementally: when the
+        append-only log has only grown, just the newly appended events are read
+        and tokenized (:meth:`VerbatimIndex.append_chunks`) instead of
+        rebuilding the BM25 index over the whole corpus on every change. The
+        stored cursor is the exact byte offset that was indexed, so concurrent
+        appends during a build never cause missed or duplicated events. A full
+        rebuild only happens on a cold cache or if the log shrank / was
+        rewritten (e.g. compaction).
+        """
         eventlog = self.session_manager.get(session_id).eventlog
         signature = _eventlog_file_signature(eventlog)
         cached = self._verbatim_index_cache.get(session_id)
         if cached is not None and cached[0] == signature:
             return cached[1]
-        index = VerbatimIndex.from_event_logs([eventlog])
-        self._verbatim_index_cache[session_id] = (signature, index)
+        if cached is not None and signature[1] > cached[2]:
+            new_events, end_offset = eventlog.read_from_offset(cached[2])
+            index = cached[1].append_chunks(_chunks_from_events(new_events))
+            self._verbatim_index_cache[session_id] = (signature, index, end_offset)
+            return index
+        events, end_offset = eventlog.read_from_offset(0)
+        index = VerbatimIndex.from_events(events)
+        self._verbatim_index_cache[session_id] = (signature, index, end_offset)
         return index
 
     def _session_event_ref_index(self, session_id: str) -> dict[int, tuple[str, str]]:
@@ -2389,9 +2411,77 @@ class MemoryFabric:
     async def replay(self, from_seq: int = 1, session_id: str = "default") -> ReplayResult:
         """Replay events from the log starting at a sequence number.
 
-        Returns the full replay result including integrity verification.
+        Returns the full replay result including integrity verification. The
+        verified replay is cached per session and extended incrementally as the
+        append-only log grows: only newly appended events are parsed and
+        integrity-checked (their seals plus the chain link to the cached,
+        already-verified prefix) instead of re-reading and re-hashing the entire
+        log on every call. A full re-verify happens on a cold cache, if the log
+        shrank / was rewritten, or if incremental verification detects a break.
+        """
+        result = self._cached_full_replay(session_id)
+        if from_seq <= 1:
+            return result
+        filtered = [event for event in result.events if event.seq >= from_seq]
+        return ReplayResult.model_construct(events=filtered, integrity=result.integrity)
+
+    def _cached_full_replay(self, session_id: str) -> ReplayResult:
+        """Return the full verified replay for a session, cached + incremental.
+
+        The cold/full path delegates to ``session_manager.replay`` (the
+        authoritative read + full integrity verify). When the cached log has
+        only grown, the appended tail is read and verified against the cached
+        prefix instead. The tail verification doubles as a consistency guard:
+        any offset skew (a concurrent append during the cold read, a rewrite)
+        surfaces as a seq/hash mismatch and falls back to a full replay, so the
+        fast path can never silently miss, duplicate, or trust a bad event.
+        """
+        eventlog = self.session_manager.get(session_id).eventlog
+        signature = _eventlog_file_signature(eventlog)
+        cached = self._replay_cache.get(session_id)
+        if cached is not None and cached[0] == signature:
+            return cached[1]
+        if (
+            cached is not None
+            and isinstance(cached[2], int)
+            and isinstance(signature[1], int)
+            and signature[1] > cached[2]
+            and cached[1].integrity is not None
+            and cached[1].integrity.ok
+        ):
+            new_events, end_offset = eventlog.read_from_offset(cached[2])
+            extended = self._extend_replay(cached[1], new_events)
+            if extended is not None:
+                self._replay_cache[session_id] = (signature, extended, end_offset)
+                return extended
+        result = cast(ReplayResult, self.session_manager.replay(session_id, from_seq=1))
+        offset = _eventlog_file_signature(eventlog)[1]
+        self._replay_cache[session_id] = (signature, result, offset)
+        return result
+
+    @staticmethod
+    def _extend_replay(cached: ReplayResult, new_events: list[Any]) -> ReplayResult | None:
+        """Extend a verified replay with appended events, or None to rebuild.
+
+        Verifies only the new tail against the cached prefix's last event.
+        Returns ``None`` (signalling a full re-verify) when the tail fails
+        verification, so a tampered or reordered append never silently passes.
         """
-        return cast(ReplayResult, self.session_manager.replay(session_id, from_seq=from_seq))
+        if not new_events:
+            return cached
+        last = cached.events[-1] if cached.events else None
+        tail = verify_event_chain(
+            new_events,
+            first_seq=(last.seq + 1) if last is not None else 1,
+            prev_hash=last.hash if last is not None else None,
+        )
+        if not tail.ok:
+            return None
+        combined = [*cached.events, *new_events]
+        return ReplayResult(
+            events=combined,
+            integrity=IntegrityReport(ok=True, total_events=len(combined)),
+        )
 
     async def propose_consolidation_candidates(
         self,

{zaxy_memory-2.4.0 → zaxy_memory-2.4.2}/src/zaxy/event.py

@@ -149,6 +149,62 @@ class ReplayResult(BaseModel):
     projection: dict[str, Any] = Field(default_factory=dict)
 
 
+def verify_event_chain(
+    events: list[Event],
+    *,
+    first_seq: int = 1,
+    prev_hash: str | None = None,
+) -> IntegrityReport:
+    """Verify a contiguous run of events: per-event seals + hash-chain links.
+
+    ``first_seq`` and ``prev_hash`` let a caller verify a tail segment against
+    an already-verified prefix (incremental replay): pass the sequence number
+    and hash of the event immediately preceding ``events[0]``. With the
+    defaults this verifies a whole log from the genesis event. ``total_events``
+    counts only the events passed here; a caller stitching a verified prefix to
+    a verified tail computes the combined total itself.
+    """
+    total = len(events)
+    if total == 0:
+        return IntegrityReport(ok=True, total_events=0)
+
+    expected_seq = first_seq
+    for ev in events:
+        if ev.seq != expected_seq:
+            return IntegrityReport(
+                ok=False,
+                total_events=total,
+                broken_at_seq=ev.seq,
+                broken_reason=f"Event sequence expected {expected_seq} but found {ev.seq}",
+            )
+        if not ev.verify():
+            return IntegrityReport(
+                ok=False,
+                total_events=total,
+                broken_at_seq=ev.seq,
+                broken_reason=f"Event {ev.seq} hash mismatch",
+            )
+        if expected_seq == 1:
+            if ev.prev_hash is not None:
+                return IntegrityReport(
+                    ok=False,
+                    total_events=total,
+                    broken_at_seq=ev.seq,
+                    broken_reason="First event has prev_hash set",
+                )
+        elif ev.prev_hash != prev_hash:
+            return IntegrityReport(
+                ok=False,
+                total_events=total,
+                broken_at_seq=ev.seq,
+                broken_reason=f"Event {ev.seq} prev_hash does not link to previous",
+            )
+        prev_hash = ev.hash
+        expected_seq += 1
+
+    return IntegrityReport(ok=True, total_events=total)
+
+
 class EventLog:
     """Append-only event log backed by a JSONL file.
 
@@ -313,47 +369,7 @@ class EventLog:
 
     def verify(self) -> IntegrityReport:
         """Verify the entire log: hash chain + individual event seals."""
-        events = self.read_all()
-        total = len(events)
-
-        if total == 0:
-            return IntegrityReport(ok=True, total_events=0)
-
-        prev_hash: str | None = None
-        for i, ev in enumerate(events, start=1):
-            if ev.seq != i:
-                return IntegrityReport(
-                    ok=False,
-                    total_events=total,
-                    broken_at_seq=ev.seq,
-                    broken_reason=f"Event sequence expected {i} but found {ev.seq}",
-                )
-            if not ev.verify():
-                return IntegrityReport(
-                    ok=False,
-                    total_events=total,
-                    broken_at_seq=ev.seq,
-                    broken_reason=f"Event {ev.seq} hash mismatch",
-                )
-            if i == 1:
-                if ev.prev_hash is not None:
-                    return IntegrityReport(
-                        ok=False,
-                        total_events=total,
-                        broken_at_seq=ev.seq,
-                        broken_reason="First event has prev_hash set",
-                    )
-            else:
-                if ev.prev_hash != prev_hash:
-                    return IntegrityReport(
-                        ok=False,
-                        total_events=total,
-                        broken_at_seq=ev.seq,
-                        broken_reason=f"Event {ev.seq} prev_hash does not link to previous",
-                    )
-            prev_hash = ev.hash
-
-        return IntegrityReport(ok=True, total_events=total)
+        return verify_event_chain(self.read_all())
 
     def replay(
         self,
@@ -402,6 +418,42 @@ class EventLog:
                 self._unlock(fh.fileno())
         return [_event_from_json_line(line) for line in lines]
 
+    def read_from_offset(self, offset: int) -> tuple[list[Event], int]:
+        """Read events appended at or after byte ``offset``.
+
+        Returns ``(events, end_offset)`` where ``events`` are parsed from
+        ``offset`` to the current end of file and ``end_offset`` is the new
+        byte length. ``offset`` must fall on a record boundary (e.g. a value
+        previously returned here, or a file size); ``offset=0`` reads the
+        whole log, so this is a superset of :meth:`read_all` that also reports
+        the cursor. A shared lock guarantees no concurrent append is observed
+        mid-record, and the append-only hash chain guarantees the prefix
+        before ``offset`` is immutable, so incremental readers never miss or
+        duplicate events.
+        """
+        if not self.path.exists():
+            return [], 0
+        if offset < 0:
+            offset = 0
+        with open(self.path, "rb") as fh:
+            self._lock(fh.fileno(), exclusive=False)
+            try:
+                fh.seek(0, os.SEEK_END)
+                end = fh.tell()
+                if offset >= end:
+                    return [], end
+                fh.seek(offset)
+                data = fh.read(end - offset)
+            finally:
+                self._unlock(fh.fileno())
+        events: list[Event] = []
+        for raw in data.split(b"\n"):
+            line = raw.strip()
+            if not line:
+                continue
+            events.append(_event_from_json_line(line.decode("utf-8")))
+        return events, end
+
     # ------------------------------------------------------------------
     # Handoff & Summaries
     # ------------------------------------------------------------------