memleaf 0.2.30__tar.gz → 0.2.32__tar.gz

{memleaf-0.2.30 → memleaf-0.2.32}/CHANGELOG.md

@@ -2,6 +2,19 @@
 
 All notable changes to memleaf are documented here.
 
+## 0.2.32 — 2026-09-07
+
+- Add bounded `unknown_unit` Gate diagnostics that identify the exact response field and retain only allowlisted type/length/digest and expected-set summaries; raw invalid values, legal ID lists and model output remain excluded from normal logs and failed state.
+- Give bounded correction attempts the failed constraint, field path and immutable legal-ID inventory used by validation, so coverage and evidence-binding references are repaired by regeneration rather than host-side ID substitution or relaxed evidence authority.
+- Keep `invalid_evidence` compatibility, source-neutral physical evidence boundaries, and watermark/cleanup safety unchanged: a persistently failed Gate does not advance the watermark or trigger cleanup.
+
+## 0.2.31 — 2026-09-07
+
+- Unify Core and the copied Hermes provider on one idempotent UTF-8 evidence budget: at most 64 source records, 32 KiB per body, and 128 KiB of aggregate body text. Complete matched sources above the former 2,000-character/eight-record limits remain available within those bounds.
+- Keep loss diagnostics bounded to 64 marker identities plus one aggregate marker. Per-record, aggregate, and marker overflow remains explicit incomplete evidence and cannot authorize a write or successful cleanup.
+- Preserve metadata/off, attachment, redaction, and call-ID behavior, and consume metadata-mode pending evidence after a successful capture using the same effective policy on pending and inbox sides.
+- Add provider-copy, capture-to-process, loss-defer, marker-capacity and lifecycle regressions. Deterministic tests do not claim real-model semantic quality, real-session replay, or customer acceptance.
+
 ## 0.2.30 — 2026-09-07
 
 - Add a physical evidence projection for the Gate: the complete local inventory remains available for provenance, replay and audit, while only user-origin units and complete external observations become bindable model evidence. Assistant synthesis, retrieved memory, incomplete observations and metadata-only records remain context or unresolved ledger state.

{memleaf-0.2.30/src/memleaf.egg-info → memleaf-0.2.32}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: memleaf
-Version: 0.2.30
+Version: 0.2.32
 Summary: A local-first Markdown memory core for AI agents
 Author: memleaf contributors
 License-Expression: MIT
@@ -23,8 +23,8 @@ Dynamic: license-file
 
 [English](README.en.md) · [PyPI](https://pypi.org/project/memleaf/) · [GitHub](https://github.com/miffyblueboo/memleaf)
 
-> **版本：0.2.30。**
-> 核心库、Vault、stdio MCP Server、初始化 CLI、模型路由、提炼流程、受控检索协议和宿主适配器已经实现。本版将完整证据清单与 Gate 可绑定的 physical evidence projection 分开，统一 `candidates`、`coverage`、`evidence_bindings` 协议，保留 legacy candidate-only 的 exact/bound compatibility，对未决证据安全收口并提供 allowlisted `evidence_check` 诊断，同时保持 source-neutral 语义、Markdown 唯一事实源和无 SQLite 运行时依赖。真实模型语义效果仍需结合本地模型和代表性样本验收。
+> **版本：0.2.32。**
+> 核心库、Vault、stdio MCP Server、初始化 CLI、模型路由、提炼流程、受控检索协议和宿主适配器已经实现。本版在既有 UTF-8 证据留存预算和 metadata pending 消费策略上，为 Gate 的 `unknown_unit` 提供字段路径、类型/长度/摘要和期望集合摘要等受限诊断，并在同一合法 ID 清单约束下最多三次尝试、最多两次纠正；不记录原始值、不做 ID 猜测，持续失败不推进水位且不触发清理。保持 source-neutral 语义、Markdown 唯一事实源和无 SQLite 运行时依赖。真实模型语义效果仍需结合本地模型和代表性样本验收。
 > **当前版本支持 Hermes 和 Codex。** Antigravity（反重力）不检测、不安装、不配置。
 
 ## 项目定位
@@ -490,7 +490,7 @@ MIT，见 [LICENSE](LICENSE)。
 *Your memories, in files you own.*
 
 
-## 通用处理与只读验收（0.2.30）
+## 通用处理与只读验收（0.2.32）
 
 邮件、日历、工单、文件、浏览器与普通对话共用证据准入、覆盖检查和写入路径。
 自动摘要只能使用获准引用的原文；助手复述和旧记忆回读不能单独授权新增写入。

{memleaf-0.2.30 → memleaf-0.2.32}/README.en.md

@@ -4,8 +4,8 @@
 
 [中文](README.md) · [PyPI](https://pypi.org/project/memleaf/) · [GitHub](https://github.com/miffyblueboo/memleaf)
 
-> **Version: 0.2.30.**
-> The core library, Vault, stdio MCP server, initialization CLI, model routing, memory extraction, controlled retrieval protocol, and host adapters are implemented. This release separates the complete evidence inventory from the Gate's bindable physical evidence projection, unifies the `candidates`, `coverage`, and `evidence_bindings` contract, preserves legacy candidate-only exact/bound compatibility, keeps unresolved evidence safe, and adds allowlisted `evidence_check` diagnostics while preserving source-neutral semantics, Markdown as the sole source of truth, and zero SQLite runtime dependencies. Real-model semantics still require local acceptance with the selected model and representative inputs.
+> **Version: 0.2.32.**
+> The core library, Vault, stdio MCP server, initialization CLI, model routing, memory extraction, controlled retrieval protocol, and host adapters are implemented. Building on the existing UTF-8 evidence budget and metadata-mode pending-evidence policy, this release adds bounded `unknown_unit` Gate diagnostics for field path, type/length/digest and expected-set summaries, with at most three attempts and two corrections constrained by the same legal-ID inventory; it never logs the raw value or guesses an ID, and a persistently failed Gate does not advance the watermark or trigger cleanup. This preserves source-neutral semantics, Markdown as the sole source of truth, and zero SQLite runtime dependencies. Real-model semantics still require local acceptance with the selected model and representative inputs.
 > **The current release supports Hermes and Codex.** Antigravity is not detected, installed, or configured.
 
 ## Project scope
@@ -474,7 +474,7 @@ MIT; see [LICENSE](LICENSE).
 *Your memories, in files you own.*
 
 
-## General processing and read-only inspection (0.2.30)
+## General processing and read-only inspection (0.2.32)
 
 Dialogue, calendars, tickets, files, web results and other tools share the evidence, coverage and write path.
 Models interpret semantics; Core validates physical provenance and exact original quotations.

{memleaf-0.2.30 → memleaf-0.2.32}/README.md

@@ -4,8 +4,8 @@
 
 [English](README.en.md) · [PyPI](https://pypi.org/project/memleaf/) · [GitHub](https://github.com/miffyblueboo/memleaf)
 
-> **版本：0.2.30。**
-> 核心库、Vault、stdio MCP Server、初始化 CLI、模型路由、提炼流程、受控检索协议和宿主适配器已经实现。本版将完整证据清单与 Gate 可绑定的 physical evidence projection 分开，统一 `candidates`、`coverage`、`evidence_bindings` 协议，保留 legacy candidate-only 的 exact/bound compatibility，对未决证据安全收口并提供 allowlisted `evidence_check` 诊断，同时保持 source-neutral 语义、Markdown 唯一事实源和无 SQLite 运行时依赖。真实模型语义效果仍需结合本地模型和代表性样本验收。
+> **版本：0.2.32。**
+> 核心库、Vault、stdio MCP Server、初始化 CLI、模型路由、提炼流程、受控检索协议和宿主适配器已经实现。本版在既有 UTF-8 证据留存预算和 metadata pending 消费策略上，为 Gate 的 `unknown_unit` 提供字段路径、类型/长度/摘要和期望集合摘要等受限诊断，并在同一合法 ID 清单约束下最多三次尝试、最多两次纠正；不记录原始值、不做 ID 猜测，持续失败不推进水位且不触发清理。保持 source-neutral 语义、Markdown 唯一事实源和无 SQLite 运行时依赖。真实模型语义效果仍需结合本地模型和代表性样本验收。
 > **当前版本支持 Hermes 和 Codex。** Antigravity（反重力）不检测、不安装、不配置。
 
 ## 项目定位
@@ -471,7 +471,7 @@ MIT，见 [LICENSE](LICENSE)。
 *Your memories, in files you own.*
 
 
-## 通用处理与只读验收（0.2.30）
+## 通用处理与只读验收（0.2.32）
 
 邮件、日历、工单、文件、浏览器与普通对话共用证据准入、覆盖检查和写入路径。
 自动摘要只能使用获准引用的原文；助手复述和旧记忆回读不能单独授权新增写入。

memleaf-0.2.32/docs/capture-budget-design.md

@@ -0,0 +1,74 @@
+# Tool evidence capture budget
+
+## Problem and ownership
+
+Capture permissions and capture capacity are separate contracts. `metadata`
+deliberately removes source bodies; `off` removes observations. Neither setting
+can produce new external facts for extraction. Enabling `bounded` authorizes
+bounded body retention, but does not reconstruct previously discarded data.
+
+The previous implementation also applied independent eight-record and
+2,000-character limits in the Hermes adapter, provenance normalization and
+pending-state processing. Early discovery and skill results could consume the
+entire record allowance before later source results arrived. Larger plain-text
+results were truncated before Core could account for their complete content.
+This is an ingestion-capacity problem; changing Gate semantics cannot recover
+those bytes.
+
+## Required implementation contract
+
+- Keep one shared, deterministic budget implementation for Core and the copied
+  Hermes provider. The provider must continue to work without importing Core
+  from the host's Python environment.
+- Bound record count, each body and aggregate body size. Publish the limits as
+  engineering capacity, not a guarantee of complete capture for arbitrary turns.
+  The implementation target is 64 source records, 32 KiB of UTF-8 text per body
+  and 128 KiB of total body text. Omission markers have a separate allowance of
+  64 call identities and one aggregate marker. They do not consume source-record
+  capacity or count themselves as newly lost observations on a later normalization
+  pass. Marker bodies are fixed diagnostics, never retained source excerpts.
+- Preserve complete, matched source results within those limits. Do not rank
+  business topics or tool names, infer facts from assistant text, split arbitrary
+  stdout into invented document records, or promote truncated prefixes to
+  complete observations.
+- Normalize retained evidence idempotently. Cache, capture, inbox read and
+  planning must not each discard another part of an already bounded inventory.
+- Keep explicit omission/incompleteness accounting when a limit is exceeded.
+  Policy-authorized but incomplete observations remain unresolved and cannot
+  authorize a write or successful source cleanup.
+- Apply `metadata`, `off`, attachment exclusions and redaction before persistent
+  writes. A capacity increase must not silently change capture permission.
+- Keep the existing Gate/Summarize semantic responsibility and Markdown storage
+  model. Larger source capacity is not a claim of successful model extraction.
+
+## Acceptance
+
+Use synthetic host messages shaped like an ordinary discovery/read turn: early
+tool discovery, skills and memory search followed by nine later results, with
+individual bodies ranging from hundreds to tens of thousands of characters.
+The old path loses all nine later results; the revised path must retain complete
+results when the full input is within its declared capacity.
+
+Verify the same evidence across adapter, pending cache, capture and inbox read;
+then verify a supported later-source candidate can pass the actual processing
+path in an isolated Vault. Check record/body/aggregate overflow separately,
+including repeated reads, accurate loss accounting, no write from incomplete
+evidence, and no cleanup of unresolved turns. Retain permission, redaction,
+cross-turn isolation and standalone installation coverage.
+
+Tests must use synthetic data and a deterministic backend. Live model semantics,
+changes to a user's capture permission, installation and release are separate
+operations with separately reported results.
+
+## Model capacity remains a separate limit
+
+The capture limits bound retained source data, not the final prompt or model
+token count. Evidence annotations, conversation context, retrieved context and
+the output allowance also consume model capacity. The current `llm.context_window`
+setting is not a pre-call tokenizer check. A source inventory within the capture
+budget can still exceed a selected model's usable context.
+
+Do not silently drop evidence to make that call succeed. Existing model-failure
+handling must preserve the failed turn and prevent cleanup. Automatic batching
+or model-specific token accounting needs its own design and is outside this
+capture-budget change.

{memleaf-0.2.30 → memleaf-0.2.32}/docs/evidence-retention.md

@@ -17,12 +17,20 @@ legacy boolean (false/absent -> metadata, true -> bounded). An explicit new mode
 wins over the legacy boolean. Invalid strings and non-boolean flags fail closed.
 Loading config does not rewrite it. Normal saves make the effective mode explicit.
 
-The inherited limits remain eight records, 2,000 characters per body, and 320 per
-metadata field. Oversized eligible evidence remains incomplete; its prefix is
-never promoted into a complete fact. Policy-excluded observations are marked
+The shared capture budget allows 64 source records, 32 KiB of UTF-8 text per body,
+128 KiB of total body text, and 320 characters per metadata field. Loss markers
+have a separate allowance of 64 call identities plus one aggregate marker, with
+fixed diagnostic bodies. Reapplying normalization does not reduce an
+already bounded inventory or count existing omissions again. Oversized eligible
+evidence remains incomplete; its prefix is never promoted into a complete fact.
+Policy-excluded observations are marked
 retention=metadata, have no content, and are not retried as missing evidence.
 No later relaxation recreates discarded original content.
 
+These are source-retention limits, not a guarantee that every configured model
+can process the resulting prompt. See [capture budget design](capture-budget-design.md)
+for the ingestion boundary, model-capacity limitation and acceptance contract.
+
 Document/attachment bodies require include_attachments=true and bounded mode.
 HostRuntime and the standalone Hermes adapter classify structural file arguments
 using the same tested contract (path/file_path/file_id/attachment_id/file URI,

{memleaf-0.2.30 → memleaf-0.2.32}/docs/gate-evidence-boundary.md

@@ -62,6 +62,18 @@ Record a separate, allowlisted evidence-check identifier for the failing
 constraint. Diagnostics must not contain raw model responses, message text,
 credentials or arbitrary exception strings.
 
+An unknown unit reference must identify the exact response field, such as
+`coverage[2].unit_id` or `evidence_bindings[0].claims[1].unit_id`. Persist only
+the field path, value type/length/digest and expected-set count/digest. The
+invalid value and legal ID list do not belong in normal logs or failed state.
+
+A bounded retry receives the failed constraint, exact field path and the legal
+IDs from the same immutable model-visible inventory used by the validator.
+These IDs are a reference constraint, not a suggested semantic decision. The
+model must regenerate a consistent response; the host must not substitute a
+nearby ID, guess a source, or relax evidence authority. Prompt examples must not
+provide a literal placeholder that looks like a usable evidence reference.
+
 A failed Gate cannot advance the turn watermark or create a cleanup deadline.
 A successful no-change decision can advance the watermark and start the normal
 retention period. Incomplete physical observations remain deferred even if other
@@ -97,3 +109,9 @@ That adapter/capture limitation must be reported independently of Gate success.
 - Deterministic tests establish the protocol. Live synthetic tests establish
   behavior only for those samples. A synthetic success does not identify the
   cause of an earlier real-model failure; do not claim otherwise.
+
+A successful replay is evidence about that new run. If an earlier raw response
+was not retained, its precise invalid value cannot be recovered from an
+`unknown_unit` category alone. Verify recovery with controlled invalid coverage
+and binding references as well as ordinary live samples, and distinguish those
+results from reproduction of the original incident.

{memleaf-0.2.30 → memleaf-0.2.32}/docs/general-processing.md

@@ -1,4 +1,4 @@
-# General processing reliability contract — 0.2.30
+# General processing reliability contract — 0.2.32
 
 This is source-neutral processing, not a mail extractor. Dialogue, documents,
 calendars, issue trackers and terminal/tool observations use the same admission
@@ -80,12 +80,15 @@ labels them NO_CHANGE. Incomplete turns retain their source instead of being
 cleaned after the usual grace period. Scope-filtered retries may revisit them;
 no endless automatic model retry or extra external tool call is introduced.
 
-Tool evidence is bounded to eight records with at most 2,000 content characters
-per captured result record and 320-character metadata fields, with redaction at
-Core capture. Large unambiguous top-level record collections retain complete
-records and enclosing context within that budget. Per-record provenance takes
-precedence over common source metadata. An overflow slot reports omitted
-records. Arbitrary large prose is not split into falsely complete facts;
+Tool evidence uses one shared budget: 64 source records, at most 32 KiB of UTF-8
+body text per record and 128 KiB of total body text, plus 320-character metadata
+fields, with redaction at Core capture. Cache and inbox normalization are
+idempotent under this budget. Large unambiguous top-level record collections
+retain complete records and enclosing context within that budget. Per-record provenance takes
+precedence over common source metadata. Separate loss markers report omitted
+records, with at most 64 call identities plus one aggregate marker. These markers
+have fixed diagnostic bodies and cannot supply external facts.
+Arbitrary large prose is not split into falsely complete facts;
 unsupported/incomplete content needs a supported complete source excerpt or a
 later source input. Switching from `metadata` to `bounded` does not reconstruct
 body content that was never captured; bounded retention can still produce an

{memleaf-0.2.30 → memleaf-0.2.32}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "memleaf"
-version = "0.2.30"
+version = "0.2.32"
 description = "A local-first Markdown memory core for AI agents"
 readme = "README.md"
 requires-python = ">=3.11"

{memleaf-0.2.30 → memleaf-0.2.32}/src/memleaf/__init__.py

@@ -1,6 +1,6 @@
 """Local-first Markdown memory core for AI agents."""
 
-__version__ = "0.2.30"
+__version__ = "0.2.32"
 
 from .config import DEFAULT_CONFIG, default_config, load_config, save_config
 from .frontmatter import FrontmatterError, dump_frontmatter, dump_yaml, load_yaml, parse_frontmatter

{memleaf-0.2.30 → memleaf-0.2.32}/src/memleaf/admission.py

@@ -242,7 +242,7 @@ def validate_bindings(value: Any, units: Iterable[EvidenceUnit],
                                evidence_check="binding_shape")
     result: dict[str, list[dict[str, Any]]] = {}
     allowed_roles = {"assertion", "source_excerpt", "user_confirmation"}
-    for row in value:
+    for binding_index, row in enumerate(value):
         if not isinstance(row, dict) or set(row) != {"candidate_id", "claims"}:
             raise ModelOutputError("invalid evidence binding", validation_detail="invalid_evidence",
                                    evidence_check="binding_shape")
@@ -255,7 +255,7 @@ def validate_bindings(value: Any, units: Iterable[EvidenceUnit],
             raise ModelOutputError("empty evidence claims", validation_detail="invalid_evidence",
                                    evidence_check="binding_shape")
         checked = []
-        for claim in claims:
+        for claim_index, claim in enumerate(claims):
             if not isinstance(claim, dict) or set(claim) not in (
                     {"unit_id", "start", "end", "quote", "role"}, {"unit_id", "quote", "role"}):
                 raise ModelOutputError("invalid evidence claim", validation_detail="invalid_evidence",
@@ -263,8 +263,13 @@ def validate_bindings(value: Any, units: Iterable[EvidenceUnit],
             claim = dict(claim)
             uid = claim["unit_id"]
             if not isinstance(uid, str) or uid not in by_unit:
-                raise ModelOutputError("unknown evidence unit", validation_detail="invalid_evidence",
-                                       evidence_check="unknown_unit")
+                error = ModelOutputError("unknown evidence unit", validation_detail="invalid_evidence",
+                                         evidence_check="unknown_unit")
+                raise error.with_evidence_context(
+                    path=f"evidence_bindings[{binding_index}].claims[{claim_index}].unit_id",
+                    actual=uid,
+                    expected_ids=tuple(by_unit),
+                )
             unit = by_unit[uid]
             quote = claim["quote"]
             if "start" not in claim:
@@ -356,20 +361,27 @@ _DEFERRED_COVERAGE_REASONS = frozenset({
 
 def parse_coverage(value: Any, units: Iterable[EvidenceUnit], candidates: Iterable[Mapping[str, Any]], *, require_complete: bool = True) -> dict[str, dict[str, Any]]:
     """Validate accounting without trusting the model's evidence identities."""
+    units = tuple(units)
+    expected_units = tuple(dict.fromkeys(u.unit_id for u in units))
     units = {u.unit_id: u for u in units}
     candidates = {c["candidate_id"]: c for c in candidates}
     if not isinstance(value, list):
         raise ModelOutputError("coverage must be a list", validation_detail="invalid_evidence",
                                evidence_check="coverage_shape")
     result = {}
-    for row in value:
+    for row_index, row in enumerate(value):
         if not isinstance(row, dict) or set(row) - {"unit_id", "decision", "candidate_ids", "reason"}:
             raise ModelOutputError("invalid coverage row", validation_detail="invalid_evidence",
                                    evidence_check="coverage_shape")
         uid = row.get("unit_id")
         if not isinstance(uid, str) or uid not in units:
-            raise ModelOutputError("invalid coverage unit", validation_detail="invalid_evidence",
-                                   evidence_check="unknown_unit")
+            error = ModelOutputError("invalid coverage unit", validation_detail="invalid_evidence",
+                                     evidence_check="unknown_unit")
+            raise error.with_evidence_context(
+                path=f"coverage[{row_index}].unit_id",
+                actual=uid,
+                expected_ids=expected_units,
+            )
         if uid in result:
             raise ModelOutputError("duplicate coverage unit", validation_detail="invalid_evidence",
                                    evidence_check="duplicate_coverage")
@@ -466,9 +478,10 @@ def evidence_prompt(units: Iterable[EvidenceUnit]) -> str:
         "candidates, coverage, and evidence_bindings. "
         "Coverage must contain exactly one row for EVERY supplied evidence unit. "
         "A response with coverage omitted or with coverage=[] is complete only when no units are supplied. "
-        "Each row is "
-        '{"unit_id":"supplied id","decision":"CANDIDATE","candidate_ids":["id"]} or '
-        '{"unit_id":"supplied id","decision":"NO_CHANGE or DEFERRED","reason":"reason"}. '
+        "For each row, copy unit_id character-for-character from the supplied evidence list. "
+        "Use decision=CANDIDATE with candidate_ids, or decision=NO_CHANGE/DEFERRED with reason. "
+        "The words in this schema description are labels only; never return a placeholder, event key, "
+        "call ID, or digest as unit_id. "
         'Allowed reasons: ' + ', '.join(sorted(COVERAGE_REASONS)) + '. '
         'Use NO_CHANGE only with reasons: ' + ', '.join(sorted(_NO_CHANGE_COVERAGE_REASONS)) + '. '
         'Use DEFERRED only with reasons: ' + ', '.join(sorted(_DEFERRED_COVERAGE_REASONS)) + '. '
@@ -489,9 +502,12 @@ def evidence_prompt(units: Iterable[EvidenceUnit]) -> str:
 
 
 SEMANTIC_BINDING_INSTRUCTIONS = """
-For every worth=true candidate, also return top-level evidence_bindings:
-[{"candidate_id":"existing candidate id","claims":[{"unit_id":"supplied id",
-"start":0,"end":10,"quote":"exact substring","role":"assertion"}]}].
+For every worth=true candidate, also return top-level evidence_bindings. Each
+binding must name a candidate_id copied from the candidates list and claims
+whose unit_id is copied character-for-character from the supplied evidence
+list. Do not return schema labels, placeholders, event keys, call IDs or
+digests as unit_id values. Each claim contains an exact quote and role, and
+may include start/end offsets.
 Offsets are relative to the supplied unit text. Roles: assertion (a current
 statement of fact or change), source_excerpt (actual quoted material, not a
 demonstration), user_confirmation (explicit adoption of a uniquely identified