memleaf 0.2.29__tar.gz → 0.2.31__tar.gz

{memleaf-0.2.29 → memleaf-0.2.31}/CHANGELOG.md

@@ -2,6 +2,20 @@
 
 All notable changes to memleaf are documented here.
 
+## 0.2.31 — 2026-09-07
+
+- Unify Core and the copied Hermes provider on one idempotent UTF-8 evidence budget: at most 64 source records, 32 KiB per body, and 128 KiB of aggregate body text. Complete matched sources above the former 2,000-character/eight-record limits remain available within those bounds.
+- Keep loss diagnostics bounded to 64 marker identities plus one aggregate marker. Per-record, aggregate, and marker overflow remains explicit incomplete evidence and cannot authorize a write or successful cleanup.
+- Preserve metadata/off, attachment, redaction, and call-ID behavior, and consume metadata-mode pending evidence after a successful capture using the same effective policy on pending and inbox sides.
+- Add provider-copy, capture-to-process, loss-defer, marker-capacity and lifecycle regressions. Deterministic tests do not claim real-model semantic quality, real-session replay, or customer acceptance.
+
+## 0.2.30 — 2026-09-07
+
+- Add a physical evidence projection for the Gate: the complete local inventory remains available for provenance, replay and audit, while only user-origin units and complete external observations become bindable model evidence. Assistant synthesis, retrieved memory, incomplete observations and metadata-only records remain context or unresolved ledger state.
+- Require one unified Gate response with `candidates`, `coverage` and `evidence_bindings`, including explicit per-unit coverage and bounded correction for missing physical units. Preserve legacy candidate-only compatibility when exact or validated bound support already accounts for a unit.
+- Preserve the public `invalid_evidence` failure category while adding safe, allowlisted `evidence_check` diagnostics for distinguishable coverage, binding and span failures. Diagnostics do not retain raw model output or error text.
+- Add regression coverage and documentation for the evidence boundary, metadata-only capture behavior and cleanup/watermark safety. This release does not claim reproduction or repair of any earlier real-model session.
+
 ## 0.2.29 — 2026-09-07
 
 - Clarify the source-neutral Gate contract for tool records retained as `metadata`: they are not evidence units, cannot be bound by metadata identifiers, and cannot authorize CREATE or UPDATE.

{memleaf-0.2.29/src/memleaf.egg-info → memleaf-0.2.31}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: memleaf
-Version: 0.2.29
+Version: 0.2.31
 Summary: A local-first Markdown memory core for AI agents
 Author: memleaf contributors
 License-Expression: MIT
@@ -23,8 +23,8 @@ Dynamic: license-file
 
 [English](README.en.md) · [PyPI](https://pypi.org/project/memleaf/) · [GitHub](https://github.com/miffyblueboo/memleaf)
 
-> **版本：0.2.29。**
-> 核心库、Vault、stdio MCP Server、初始化 CLI、模型路由、提炼流程、受控检索协议和宿主适配器已经实现。本版补齐 metadata 工具证据的 Gate 覆盖协议，修复查询和助手复述误标 DEFERRED 的收口问题，同时保持 source-neutral 语义、Markdown 唯一事实源和无 SQLite 运行时依赖。真实模型语义效果仍需结合本地模型和代表性样本验收。
+> **版本：0.2.31。**
+> 核心库、Vault、stdio MCP Server、初始化 CLI、模型路由、提炼流程、受控检索协议和宿主适配器已经实现。本版统一 Core 与 Hermes Provider 的 UTF-8 证据留存预算：最多 64 条 source records、每条正文 32 KiB、正文总量 128 KiB，并保留有界 loss markers 与幂等规范化；同时将 metadata 模式下成功捕获的超限 pending 证据按相同有效策略消费。Gate 继续使用 physical evidence projection、`candidates`、`coverage`、`evidence_bindings` 和 allowlisted `evidence_check` 诊断，保持 source-neutral 语义、Markdown 唯一事实源和无 SQLite 运行时依赖。真实模型语义效果仍需结合本地模型和代表性样本验收。
 > **当前版本支持 Hermes 和 Codex。** Antigravity（反重力）不检测、不安装、不配置。
 
 ## 项目定位
@@ -490,7 +490,7 @@ MIT，见 [LICENSE](LICENSE)。
 *Your memories, in files you own.*
 
 
-## 通用处理与只读验收（0.2.29）
+## 通用处理与只读验收（0.2.31）
 
 邮件、日历、工单、文件、浏览器与普通对话共用证据准入、覆盖检查和写入路径。
 自动摘要只能使用获准引用的原文；助手复述和旧记忆回读不能单独授权新增写入。

{memleaf-0.2.29 → memleaf-0.2.31}/README.en.md

@@ -4,8 +4,8 @@
 
 [中文](README.md) · [PyPI](https://pypi.org/project/memleaf/) · [GitHub](https://github.com/miffyblueboo/memleaf)
 
-> **Version: 0.2.29.**
-> The core library, Vault, stdio MCP server, initialization CLI, model routing, memory extraction, controlled retrieval protocol, and host adapters are implemented. This release completes the metadata-only tool evidence Gate contract and prevents query or assistant restatement coverage from remaining DEFERRED, while preserving source-neutral semantics, Markdown as the sole source of truth, and zero SQLite runtime dependencies. Real-model semantics still require local acceptance with the selected model and representative inputs.
+> **Version: 0.2.31.**
+> The core library, Vault, stdio MCP server, initialization CLI, model routing, memory extraction, controlled retrieval protocol, and host adapters are implemented. This release unifies Core and the Hermes provider on an idempotent UTF-8 evidence budget: at most 64 source records, 32 KiB per body, and 128 KiB of aggregate body text, with bounded loss markers; metadata-mode pending evidence from an oversized observation is consumed after successful capture under the same effective policy. The Gate still uses the physical evidence projection, the `candidates`, `coverage`, and `evidence_bindings` contract, and allowlisted `evidence_check` diagnostics while preserving source-neutral semantics, Markdown as the sole source of truth, and zero SQLite runtime dependencies. Real-model semantics still require local acceptance with the selected model and representative inputs.
 > **The current release supports Hermes and Codex.** Antigravity is not detected, installed, or configured.
 
 ## Project scope
@@ -474,7 +474,7 @@ MIT; see [LICENSE](LICENSE).
 *Your memories, in files you own.*
 
 
-## General processing and read-only inspection (0.2.29)
+## General processing and read-only inspection (0.2.31)
 
 Dialogue, calendars, tickets, files, web results and other tools share the evidence, coverage and write path.
 Models interpret semantics; Core validates physical provenance and exact original quotations.

{memleaf-0.2.29 → memleaf-0.2.31}/README.md

@@ -4,8 +4,8 @@
 
 [English](README.en.md) · [PyPI](https://pypi.org/project/memleaf/) · [GitHub](https://github.com/miffyblueboo/memleaf)
 
-> **版本：0.2.29。**
-> 核心库、Vault、stdio MCP Server、初始化 CLI、模型路由、提炼流程、受控检索协议和宿主适配器已经实现。本版补齐 metadata 工具证据的 Gate 覆盖协议，修复查询和助手复述误标 DEFERRED 的收口问题，同时保持 source-neutral 语义、Markdown 唯一事实源和无 SQLite 运行时依赖。真实模型语义效果仍需结合本地模型和代表性样本验收。
+> **版本：0.2.31。**
+> 核心库、Vault、stdio MCP Server、初始化 CLI、模型路由、提炼流程、受控检索协议和宿主适配器已经实现。本版统一 Core 与 Hermes Provider 的 UTF-8 证据留存预算：最多 64 条 source records、每条正文 32 KiB、正文总量 128 KiB，并保留有界 loss markers 与幂等规范化；同时将 metadata 模式下成功捕获的超限 pending 证据按相同有效策略消费。Gate 继续使用 physical evidence projection、`candidates`、`coverage`、`evidence_bindings` 和 allowlisted `evidence_check` 诊断，保持 source-neutral 语义、Markdown 唯一事实源和无 SQLite 运行时依赖。真实模型语义效果仍需结合本地模型和代表性样本验收。
 > **当前版本支持 Hermes 和 Codex。** Antigravity（反重力）不检测、不安装、不配置。
 
 ## 项目定位
@@ -471,7 +471,7 @@ MIT，见 [LICENSE](LICENSE)。
 *Your memories, in files you own.*
 
 
-## 通用处理与只读验收（0.2.29）
+## 通用处理与只读验收（0.2.31）
 
 邮件、日历、工单、文件、浏览器与普通对话共用证据准入、覆盖检查和写入路径。
 自动摘要只能使用获准引用的原文；助手复述和旧记忆回读不能单独授权新增写入。

memleaf-0.2.31/docs/capture-budget-design.md

@@ -0,0 +1,74 @@
+# Tool evidence capture budget
+
+## Problem and ownership
+
+Capture permissions and capture capacity are separate contracts. `metadata`
+deliberately removes source bodies; `off` removes observations. Neither setting
+can produce new external facts for extraction. Enabling `bounded` authorizes
+bounded body retention, but does not reconstruct previously discarded data.
+
+The previous implementation also applied independent eight-record and
+2,000-character limits in the Hermes adapter, provenance normalization and
+pending-state processing. Early discovery and skill results could consume the
+entire record allowance before later source results arrived. Larger plain-text
+results were truncated before Core could account for their complete content.
+This is an ingestion-capacity problem; changing Gate semantics cannot recover
+those bytes.
+
+## Required implementation contract
+
+- Keep one shared, deterministic budget implementation for Core and the copied
+  Hermes provider. The provider must continue to work without importing Core
+  from the host's Python environment.
+- Bound record count, each body and aggregate body size. Publish the limits as
+  engineering capacity, not a guarantee of complete capture for arbitrary turns.
+  The implementation target is 64 source records, 32 KiB of UTF-8 text per body
+  and 128 KiB of total body text. Omission markers have a separate allowance of
+  64 call identities and one aggregate marker. They do not consume source-record
+  capacity or count themselves as newly lost observations on a later normalization
+  pass. Marker bodies are fixed diagnostics, never retained source excerpts.
+- Preserve complete, matched source results within those limits. Do not rank
+  business topics or tool names, infer facts from assistant text, split arbitrary
+  stdout into invented document records, or promote truncated prefixes to
+  complete observations.
+- Normalize retained evidence idempotently. Cache, capture, inbox read and
+  planning must not each discard another part of an already bounded inventory.
+- Keep explicit omission/incompleteness accounting when a limit is exceeded.
+  Policy-authorized but incomplete observations remain unresolved and cannot
+  authorize a write or successful source cleanup.
+- Apply `metadata`, `off`, attachment exclusions and redaction before persistent
+  writes. A capacity increase must not silently change capture permission.
+- Keep the existing Gate/Summarize semantic responsibility and Markdown storage
+  model. Larger source capacity is not a claim of successful model extraction.
+
+## Acceptance
+
+Use synthetic host messages shaped like an ordinary discovery/read turn: early
+tool discovery, skills and memory search followed by nine later results, with
+individual bodies ranging from hundreds to tens of thousands of characters.
+The old path loses all nine later results; the revised path must retain complete
+results when the full input is within its declared capacity.
+
+Verify the same evidence across adapter, pending cache, capture and inbox read;
+then verify a supported later-source candidate can pass the actual processing
+path in an isolated Vault. Check record/body/aggregate overflow separately,
+including repeated reads, accurate loss accounting, no write from incomplete
+evidence, and no cleanup of unresolved turns. Retain permission, redaction,
+cross-turn isolation and standalone installation coverage.
+
+Tests must use synthetic data and a deterministic backend. Live model semantics,
+changes to a user's capture permission, installation and release are separate
+operations with separately reported results.
+
+## Model capacity remains a separate limit
+
+The capture limits bound retained source data, not the final prompt or model
+token count. Evidence annotations, conversation context, retrieved context and
+the output allowance also consume model capacity. The current `llm.context_window`
+setting is not a pre-call tokenizer check. A source inventory within the capture
+budget can still exceed a selected model's usable context.
+
+Do not silently drop evidence to make that call succeed. Existing model-failure
+handling must preserve the failed turn and prevent cleanup. Automatic batching
+or model-specific token accounting needs its own design and is outside this
+capture-budget change.

{memleaf-0.2.29 → memleaf-0.2.31}/docs/evidence-retention.md

@@ -17,12 +17,20 @@ legacy boolean (false/absent -> metadata, true -> bounded). An explicit new mode
 wins over the legacy boolean. Invalid strings and non-boolean flags fail closed.
 Loading config does not rewrite it. Normal saves make the effective mode explicit.
 
-The inherited limits remain eight records, 2,000 characters per body, and 320 per
-metadata field. Oversized eligible evidence remains incomplete; its prefix is
-never promoted into a complete fact. Policy-excluded observations are marked
+The shared capture budget allows 64 source records, 32 KiB of UTF-8 text per body,
+128 KiB of total body text, and 320 characters per metadata field. Loss markers
+have a separate allowance of 64 call identities plus one aggregate marker, with
+fixed diagnostic bodies. Reapplying normalization does not reduce an
+already bounded inventory or count existing omissions again. Oversized eligible
+evidence remains incomplete; its prefix is never promoted into a complete fact.
+Policy-excluded observations are marked
 retention=metadata, have no content, and are not retried as missing evidence.
 No later relaxation recreates discarded original content.
 
+These are source-retention limits, not a guarantee that every configured model
+can process the resulting prompt. See [capture budget design](capture-budget-design.md)
+for the ingestion boundary, model-capacity limitation and acceptance contract.
+
 Document/attachment bodies require include_attachments=true and bounded mode.
 HostRuntime and the standalone Hermes adapter classify structural file arguments
 using the same tested contract (path/file_path/file_id/attachment_id/file URI,

memleaf-0.2.31/docs/gate-evidence-boundary.md

@@ -0,0 +1,99 @@
+# Gate evidence boundary
+
+## Design decision
+
+Keep Markdown as the permanent-memory source of truth, the shared Vault,
+`_index/` / `_state/` separation, scope-directed retrieval, and the separate Gate
+and Summarize stages. Do not repair extraction failures by accepting unverified
+evidence or by adding business-specific classifiers.
+
+The Gate owns semantic decisions. The host owns physical evidence authority and
+accounting. A model must not be asked to re-decide a physical constraint the host
+already knows, such as whether text was written by the assistant or whether an
+external observation has retained source content.
+
+## Input and accounting responsibilities
+
+Keep the complete evidence inventory for provenance, input digests, replay and
+local audit. Project only physically admissible units to the model's evidence
+list:
+
+| Source | Model evidence | Local handling |
+| --- | --- | --- |
+| User text, including questions, examples and quotations | Yes | Model decides future value and coverage |
+| Complete, matched external observation | Yes | Model decides future value and coverage |
+| Assistant synthesis | No | Context only; no independent write authority |
+| Retrieved memleaf/native memory | No | Context only; no new evidence |
+| Missing, failed or incomplete external observation | No | Unresolved; retain the source turn |
+| Intentionally metadata-only tool record | No | Honor capture policy; never fabricate source text |
+
+Use the physical `can_support` boundary for this projection. `origin` and
+`eligible` contain syntax hints and must not become semantic filters for user
+text. A user can state a real fact inside a question or a code example; the Gate
+must still see and evaluate that text. Conversely, an exact quote from user text
+proves provenance, not the truth or future value of a proposed memory.
+
+The Gate can retain user/assistant conversation context for pronouns and explicit
+adoption of proposals. Context is not an additional source of bindable unit IDs.
+Metadata call IDs, digests and tool names are not substitute source text.
+
+## One Gate contract
+
+The documented response contains `candidates`, `coverage` and
+`evidence_bindings`. Coverage accounts for every model-visible evidence unit,
+including a decision that no candidate is warranted. An empty candidate list is
+not, by itself, proof of complete coverage.
+
+The three empty lists describe a complete response only when there are no
+model-visible evidence units. With units present, no-admission still needs
+explicit `NO_CHANGE` or `DEFERRED` accounting. Examples must obey the same rules
+as the validator and must not preclassify the first real input as a worthy fact.
+
+Legacy candidate-only responses use the same evidence checks. Exact support or
+validated bindings account for the supported units; remaining units take the
+bounded coverage-repair path. Missing accounting must not
+silently become permission to clean the source turn. Unknown unit IDs,
+contradictory coverage, invalid spans and unauthorized sources remain errors.
+
+## Diagnostics and recovery
+
+Preserve the existing failure category, retry bound and watermark behavior.
+Record a separate, allowlisted evidence-check identifier for the failing
+constraint. Diagnostics must not contain raw model responses, message text,
+credentials or arbitrary exception strings.
+
+A failed Gate cannot advance the turn watermark or create a cleanup deadline.
+A successful no-change decision can advance the watermark and start the normal
+retention period. Incomplete physical observations remain deferred even if other
+parts of the turn finish. Replaying an already completed operation must remain
+idempotent.
+
+## Capture policy is a separate capability boundary
+
+`metadata` intentionally excludes tool-result bodies from extraction. The host
+having read a document does not mean the extraction model has its source text.
+Do not change this policy implicitly, reconstruct removed bodies from assistant
+prose, or re-read external systems merely to make a failed Gate pass.
+
+`bounded` permits retained complete records; it does not promise complete
+extraction from arbitrary bulk stdout. Record-count and body-size limits can
+produce explicit omissions or partial observations. A large batch of results
+inside one text blob is not equivalent to separately matched complete records.
+That adapter/capture limitation must be reported independently of Gate success.
+
+## Verification
+
+- A long assistant answer does not expand the model's evidence coverage list;
+  the full local audit inventory remains available.
+- User questions, quotations and mixed question/assertion text remain in the
+  model projection. No local keyword rule decides their future value.
+- Complete external facts can be admitted; assistant, retrieved, metadata and
+  incomplete observations cannot authorize independent writes.
+- Missing coverage is corrected or retained as unresolved, not silently cleaned.
+- Invalid unit IDs, duplicate coverage, invalid reasons, invalid spans and
+  binding/coverage conflicts produce safe distinguishable diagnostics.
+- Failure, retry, watermark, cleanup eligibility and replay are verified together
+  in an isolated Vault without changing an existing user's Vault.
+- Deterministic tests establish the protocol. Live synthetic tests establish
+  behavior only for those samples. A synthetic success does not identify the
+  cause of an earlier real-model failure; do not claim otherwise.

{memleaf-0.2.29 → memleaf-0.2.31}/docs/general-processing.md

@@ -1,4 +1,4 @@
-# General processing reliability contract — 0.2.29
+# General processing reliability contract — 0.2.30
 
 This is source-neutral processing, not a mail extractor. Dialogue, documents,
 calendars, issue trackers and terminal/tool observations use the same admission
@@ -27,6 +27,14 @@ quotation, in which case Core locates it without model character counting.
 Malformed/ambiguous references fail the contract. Matching a quote proves
 provenance, not semantic truth. This mechanism is not a universal NLP proof.
 
+Core keeps the complete evidence inventory for audit, replay and input digests.
+The Gate receives a bounded physical-source projection: all user-origin units
+and complete current-turn external observations are included, while assistant
+prose, retrieved memory and incomplete observations remain host-side context or
+unresolved ledger entries. This is a provenance boundary, not a semantic
+classification; query, example and quoted-document hints remain visible to the
+Gate so it can judge them in context.
+
 Legacy candidate-only output has no n-gram or short-text authorization bypass.
 It must repeat a complete non-query source statement, or produce an explicit
 validated quotation via the bounded correction path. Automatic summarization
@@ -36,10 +44,24 @@ scope/type/target/date checks remain active.
 
 ## Coverage, limits and no-op behavior
 
-A Gate may map multiple facts in one evidence unit to several candidates, or
-one candidate to several units. Coverage is checked against real supplied IDs.
-One source-neutral correction can classify missing units; its candidates pass
-the same validator/deduplication path, never a post-Gate business-pattern writer.
+A Gate response always has the top-level fields `candidates`, `coverage` and
+`evidence_bindings`. When physical evidence units are supplied, `coverage` must
+contain exactly one row for each supplied unit, including units the model marks
+`NO_CHANGE` or `DEFERRED`; an omitted or empty coverage list is incomplete in
+that case. With zero physical units, the complete no-admission object is
+`{"candidates":[],"coverage":[],"evidence_bindings":[]}`. A Gate may map
+multiple facts in one evidence unit to several candidates, or one candidate to
+several units. Coverage is checked against real supplied IDs. One source-neutral
+correction can classify missing units; its candidates pass the same
+validator/deduplication path, never a post-Gate business-pattern writer.
+Legacy candidate-only output remains compatible when a candidate already has
+validated exact or bound physical support. Any remaining physical unit without
+coverage or validated candidate support is sent through bounded correction and
+remains unresolved; it cannot silently authorize inbox cleanup.
+Evidence failures retain the public `invalid_evidence` category and may expose
+an allowlisted `evidence_check` such as an unknown unit, duplicate row, invalid
+span or incomplete coverage. Diagnostic state never stores raw model output or
+error text.
 Tool records retained as `metadata` may remain visible in the event envelope for
 diagnostics, but they are not evidence units and cannot be bound by call ID,
 digest, tool name or other metadata. They therefore require no coverage row and
@@ -58,14 +80,20 @@ labels them NO_CHANGE. Incomplete turns retain their source instead of being
 cleaned after the usual grace period. Scope-filtered retries may revisit them;
 no endless automatic model retry or extra external tool call is introduced.
 
-Tool evidence is bounded to eight records with at most 2,000 content characters
-per captured result record and 320-character metadata fields, with redaction at
-Core capture. Large unambiguous top-level record collections retain complete
-records and enclosing context within that budget. Per-record provenance takes
-precedence over common source metadata. An overflow slot reports omitted
-records. Arbitrary large prose is not split into falsely complete facts;
+Tool evidence uses one shared budget: 64 source records, at most 32 KiB of UTF-8
+body text per record and 128 KiB of total body text, plus 320-character metadata
+fields, with redaction at Core capture. Cache and inbox normalization are
+idempotent under this budget. Large unambiguous top-level record collections
+retain complete records and enclosing context within that budget. Per-record provenance takes
+precedence over common source metadata. Separate loss markers report omitted
+records, with at most 64 call identities plus one aggregate marker. These markers
+have fixed diagnostic bodies and cannot supply external facts.
+Arbitrary large prose is not split into falsely complete facts;
 unsupported/incomplete content needs a supported complete source excerpt or a
-later source input. Execution outcome and completeness are distinct.
+later source input. Switching from `metadata` to `bounded` does not reconstruct
+body content that was never captured; bounded retention can still produce an
+unknown or overflow record when the adapter/result exceeds its capture limits.
+Execution outcome and completeness are distinct.
 
 Codex pending tool data retains sixteen turns; bounded tombstones make evicted
 uncaptured evidence visible as incomplete. Older loss beyond 256 tombstones

{memleaf-0.2.29 → memleaf-0.2.31}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "memleaf"
-version = "0.2.29"
+version = "0.2.31"
 description = "A local-first Markdown memory core for AI agents"
 readme = "README.md"
 requires-python = ">=3.11"

{memleaf-0.2.29 → memleaf-0.2.31}/src/memleaf/__init__.py

@@ -1,6 +1,6 @@
 """Local-first Markdown memory core for AI agents."""
 
-__version__ = "0.2.29"
+__version__ = "0.2.31"
 
 from .config import DEFAULT_CONFIG, default_config, load_config, save_config
 from .frontmatter import FrontmatterError, dump_frontmatter, dump_yaml, load_yaml, parse_frontmatter