memleaf 0.2.32__tar.gz → 0.2.33__tar.gz

{memleaf-0.2.32 → memleaf-0.2.33}/CHANGELOG.md

@@ -2,6 +2,15 @@
 
 All notable changes to memleaf are documented here.
 
+## 0.2.33 — 2026-09-08
+
+- Extend the Gate with bounded physical-evidence batches, exact unit/quote bindings, isolated cross-batch candidate IDs, same-target update coordination, and bounded model reconciliation for compatible CREATE proposals. Failed batches remain retryable and fail closed without advancing the watermark or cleanup.
+- Align Core, HostRuntime, and the copied Hermes provider on document/attachment classification and effective capture-policy reporting: ordinary structural files follow the selected retention mode, while explicitly identified attachments still require attachment opt-in and bounded retention.
+- Keep automatic UPDATEs as `NO_CHANGE` when current evidence only restates the target or adds provenance/source metadata; preserve the selected target for real semantic state changes. Add synthetic-document real-model lifecycle acceptance and regressions for coverage, deadlines, todo completion, capture policy, and lifecycle behavior.
+- Harden automatic processing around source/date grounding, target reconciliation, update review, duplicate/no-op collision handling, and partial-retry idempotency so unresolved ownership, target, evidence, or timing stays deferred without fabricated writes.
+- Isolate read-only/general queries from stale deferred automatic turns while preserving retries for assertions, explicit scopes, and external observations; extend Core/Hermes transport and evidence regressions for these boundaries.
+- Validation for this release: the full suite ran 911 tests successfully with 2 skips. A four-phase synthetic-input real-model acceptance also passed; this does not claim real-mail or customer-business acceptance.
+
 ## 0.2.32 — 2026-09-07
 
 - Add bounded `unknown_unit` Gate diagnostics that identify the exact response field and retain only allowlisted type/length/digest and expected-set summaries; raw invalid values, legal ID lists and model output remain excluded from normal logs and failed state.

{memleaf-0.2.32/src/memleaf.egg-info → memleaf-0.2.33}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: memleaf
-Version: 0.2.32
+Version: 0.2.33
 Summary: A local-first Markdown memory core for AI agents
 Author: memleaf contributors
 License-Expression: MIT
@@ -23,8 +23,8 @@ Dynamic: license-file
 
 [English](README.en.md) · [PyPI](https://pypi.org/project/memleaf/) · [GitHub](https://github.com/miffyblueboo/memleaf)
 
-> **版本：0.2.32。**
-> 核心库、Vault、stdio MCP Server、初始化 CLI、模型路由、提炼流程、受控检索协议和宿主适配器已经实现。本版在既有 UTF-8 证据留存预算和 metadata pending 消费策略上，为 Gate 的 `unknown_unit` 提供字段路径、类型/长度/摘要和期望集合摘要等受限诊断，并在同一合法 ID 清单约束下最多三次尝试、最多两次纠正；不记录原始值、不做 ID 猜测，持续失败不推进水位且不触发清理。保持 source-neutral 语义、Markdown 唯一事实源和无 SQLite 运行时依赖。真实模型语义效果仍需结合本地模型和代表性样本验收。
+> **版本：0.2.33。**
+> 本版扩展有界物理证据 Gate：支持精确 unit/quote 绑定、分批覆盖、跨批候选隔离与受限协调，并在失败时保留可重试的原始轮次；统一 Core、HostRuntime 与 Hermes Provider 的文件/附件留存分类，普通结构化文件遵循所选模式，显式附件仍需单独放行；自动 UPDATE 在仅措辞、复述或来源元数据变化时保持 `NO_CHANGE`，真实状态变化才更新原目标。新增合成文档真实模型生命周期验收示例与对应回归测试。Markdown 仍是唯一事实源，运行时不引入 SQLite。验收仅覆盖合成输入和真实模型路由，不代表真实邮件或客户业务验收。
 > **当前版本支持 Hermes 和 Codex。** Antigravity（反重力）不检测、不安装、不配置。
 
 ## 项目定位
@@ -443,7 +443,7 @@ history:
 
 ## 隐私与安全边界
 
-- 对话捕获只接收 user/assistant 可见文本，不捕获 system/developer 指令或隐藏推理。匹配到当前轮工具调用的证据另由 `capture.tool_evidence_mode` 控制；新建 Vault 默认 `bounded`（脱敏、有界正文），文件/附件正文默认不留存。旧配置显式关闭工具输出时不自动升级为保留正文；
+- 对话捕获只接收 user/assistant 可见文本，不捕获 system/developer 指令或隐藏推理。匹配到当前轮工具调用的证据另由 `capture.tool_evidence_mode` 控制；新建 Vault 默认 `bounded`（脱敏、有界正文），显式标记为附件的正文默认不留存，普通结构化文件正文按该模式保留。旧配置显式关闭工具输出时不自动升级为保留正文；
 - 捕获落盘前尽力脱敏常见 API key、Bearer token、Cookie、JWT 和私钥，但脱敏不是加密，也不能保证识别所有敏感信息；
 - 路径校验、符号链接检查、Vault 锁、同目录临时文件、fsync 和原子替换用于保护本地写入；
 - memleaf 不主动上传整个 Vault，也没有托管后台、遥测或账号系统；
@@ -490,7 +490,7 @@ MIT，见 [LICENSE](LICENSE)。
 *Your memories, in files you own.*
 
 
-## 通用处理与只读验收（0.2.32）
+## 通用处理与只读验收（0.2.33）
 
 邮件、日历、工单、文件、浏览器与普通对话共用证据准入、覆盖检查和写入路径。
 自动摘要只能使用获准引用的原文；助手复述和旧记忆回读不能单独授权新增写入。
@@ -507,6 +507,10 @@ memleaf process --vault /path/to/existing/vault --source hermes --session-id SES
 
 工具执行状态与证据完整性分别记录；超限、丢失或不完整内容不会被模型的 NO_CHANGE 升级为完整。
 执行成功但尚有未解决项时，结果显示 `coverage_status=partial`，并保留来源以供有界重试或补充证据。
+`external_evidence_status` 另外说明本批工具正文是否实际可供提炼：`metadata_only` 是仅留元数据，
+`disabled` 是关闭采集，`unavailable` 是没有可用完整正文，`partial` 是只有部分可用，
+`available` 是有可用正文，`not_provided` 是本批没有外部记录。可用正文不等于业务事项已提炼完整，
+也不证明工具读取了原始邮件或文档的全文。
 行为、限制、测试协议变更见 [通用处理说明](docs/general-processing.md)。
 
 ### 工具证据留存配置
@@ -524,9 +528,11 @@ capture:
 
 现有配置未提供新字段时，旧 `include_tool_output: false` 或未设置该开关均按
 `metadata` 处理，旧 `true` 按 `bounded` 处理。显式新字段优先；新建 Vault 只写
-新字段，不再同时写含义冲突的旧开关。`include_attachments: true` 仍受上述总模式限制。
-宿主通过结构化文件路径、文件/附件标识识别文档；不承诺识别任意 Shell 命令或不透明工具
-隐藏读取的文件。用户粘贴的可见文档和显式 `remember` 内容不属于自动附件抓取。
+新字段，不再同时写含义冲突的旧开关。`include_attachments: true` 只放行显式标记的
+attachment，且仍受上述总模式限制；普通结构化文件结果不需要该开关。宿主把路径、file、
+file_id 或 file URI 归为 document，只有明确的 `attachment_id` 才归为 attachment；无法仅凭
+路径判断某个文件是否为附件，也不承诺识别任意 Shell 命令或不透明工具隐藏读取的文件。用户
+粘贴的可见文档和显式 `remember` 内容不属于自动附件抓取。
 
 策略在待捕获缓存、inbox 写入和新模型提炼输入处共同执行。配置收紧不会自动删除已提交
 记忆或改写已捕获 inbox；失败前已经冻结的提交计划也不作为新的模型调用重新提炼。

{memleaf-0.2.32 → memleaf-0.2.33}/README.en.md

@@ -4,8 +4,8 @@
 
 [中文](README.md) · [PyPI](https://pypi.org/project/memleaf/) · [GitHub](https://github.com/miffyblueboo/memleaf)
 
-> **Version: 0.2.32.**
-> The core library, Vault, stdio MCP server, initialization CLI, model routing, memory extraction, controlled retrieval protocol, and host adapters are implemented. Building on the existing UTF-8 evidence budget and metadata-mode pending-evidence policy, this release adds bounded `unknown_unit` Gate diagnostics for field path, type/length/digest and expected-set summaries, with at most three attempts and two corrections constrained by the same legal-ID inventory; it never logs the raw value or guesses an ID, and a persistently failed Gate does not advance the watermark or trigger cleanup. This preserves source-neutral semantics, Markdown as the sole source of truth, and zero SQLite runtime dependencies. Real-model semantics still require local acceptance with the selected model and representative inputs.
+> **Version: 0.2.33.**
+> This release extends bounded physical-evidence Gate processing with exact unit/quote bindings, batch coverage, isolated cross-batch candidates and bounded model reconciliation, while failed batches remain retryable and fail closed. Core, HostRuntime and the Hermes provider now share document/attachment classification: ordinary structural files follow the selected retention mode, while explicitly marked attachments still require separate opt-in. Automatic UPDATEs return `NO_CHANGE` for wording, restatement or provenance-only changes and retain the selected target only for real semantic state changes. It adds a synthetic-document real-model lifecycle acceptance example and regression coverage. Markdown remains the sole source of truth with no SQLite runtime dependency. Acceptance covers synthetic inputs and the configured real-model route; it does not claim real-mail or customer-business acceptance.
 > **The current release supports Hermes and Codex.** Antigravity is not detected, installed, or configured.
 
 ## Project scope
@@ -426,7 +426,7 @@ Directories are normally created with mode `0700`, and files are stored as plain
 
 ## Privacy and security boundaries
 
-- Conversation capture accepts visible user/assistant text, never system/developer instructions or hidden reasoning. Matched current-turn tool evidence is controlled separately by `capture.tool_evidence_mode`: new Vaults use bounded/redacted observations; document/attachment bodies are excluded by default. Legacy configurations disabling tool output are not silently opted into body retention.
+- Conversation capture accepts visible user/assistant text, never system/developer instructions or hidden reasoning. Matched current-turn tool evidence is controlled separately by `capture.tool_evidence_mode`: new Vaults use bounded/redacted observations; explicitly marked attachment bodies are excluded by default, while ordinary structural file/document bodies follow the selected mode. Legacy configurations disabling tool output are not silently opted into body retention.
 - Common API keys, Bearer tokens, cookies, JWTs, and private keys are redacted on a best-effort basis before capture is written. Redaction is not encryption and cannot detect every secret.
 - Path validation, symlink checks, Vault locks, same-directory temporary files, fsync, and atomic replacement protect local writes.
 - memleaf does not upload the entire Vault and has no hosted backend, telemetry, or account system.
@@ -474,7 +474,7 @@ MIT; see [LICENSE](LICENSE).
 *Your memories, in files you own.*
 
 
-## General processing and read-only inspection (0.2.32)
+## General processing and read-only inspection (0.2.33)
 
 Dialogue, calendars, tickets, files, web results and other tools share the evidence, coverage and write path.
 Models interpret semantics; Core validates physical provenance and exact original quotations.
@@ -508,8 +508,10 @@ assistant synthesis or retrieved old memory independent evidence of new facts.
 For an existing file without the new mode, legacy `include_tool_output: false` or an
 absent boolean means `metadata`; true means `bounded`. An explicit new mode takes
 precedence. New Vaults write only the new mode. Attachment opt-in remains subject to the
-mode. Adapters classify structural file paths, file IDs and attachment handles, not
-arbitrary opaque shell commands. Pasted visible documents and explicit remember text
+mode and applies only to explicitly marked attachments. Adapters classify structural file
+paths, file IDs and file URIs as documents, and an explicit `attachment_id` as an attachment;
+a path alone cannot identify every file that happens to be an attachment. They do not
+classify arbitrary opaque shell commands. Pasted visible documents and explicit remember text
 are not automatic attachment capture.
 
 The policy applies to pending cache, inbox writes, and new model-planning inputs.

{memleaf-0.2.32 → memleaf-0.2.33}/README.md

@@ -4,8 +4,8 @@
 
 [English](README.en.md) · [PyPI](https://pypi.org/project/memleaf/) · [GitHub](https://github.com/miffyblueboo/memleaf)
 
-> **版本：0.2.32。**
-> 核心库、Vault、stdio MCP Server、初始化 CLI、模型路由、提炼流程、受控检索协议和宿主适配器已经实现。本版在既有 UTF-8 证据留存预算和 metadata pending 消费策略上，为 Gate 的 `unknown_unit` 提供字段路径、类型/长度/摘要和期望集合摘要等受限诊断，并在同一合法 ID 清单约束下最多三次尝试、最多两次纠正；不记录原始值、不做 ID 猜测，持续失败不推进水位且不触发清理。保持 source-neutral 语义、Markdown 唯一事实源和无 SQLite 运行时依赖。真实模型语义效果仍需结合本地模型和代表性样本验收。
+> **版本：0.2.33。**
+> 本版扩展有界物理证据 Gate：支持精确 unit/quote 绑定、分批覆盖、跨批候选隔离与受限协调，并在失败时保留可重试的原始轮次；统一 Core、HostRuntime 与 Hermes Provider 的文件/附件留存分类，普通结构化文件遵循所选模式，显式附件仍需单独放行；自动 UPDATE 在仅措辞、复述或来源元数据变化时保持 `NO_CHANGE`，真实状态变化才更新原目标。新增合成文档真实模型生命周期验收示例与对应回归测试。Markdown 仍是唯一事实源，运行时不引入 SQLite。验收仅覆盖合成输入和真实模型路由，不代表真实邮件或客户业务验收。
 > **当前版本支持 Hermes 和 Codex。** Antigravity（反重力）不检测、不安装、不配置。
 
 ## 项目定位
@@ -424,7 +424,7 @@ history:
 
 ## 隐私与安全边界
 
-- 对话捕获只接收 user/assistant 可见文本，不捕获 system/developer 指令或隐藏推理。匹配到当前轮工具调用的证据另由 `capture.tool_evidence_mode` 控制；新建 Vault 默认 `bounded`（脱敏、有界正文），文件/附件正文默认不留存。旧配置显式关闭工具输出时不自动升级为保留正文；
+- 对话捕获只接收 user/assistant 可见文本，不捕获 system/developer 指令或隐藏推理。匹配到当前轮工具调用的证据另由 `capture.tool_evidence_mode` 控制；新建 Vault 默认 `bounded`（脱敏、有界正文），显式标记为附件的正文默认不留存，普通结构化文件正文按该模式保留。旧配置显式关闭工具输出时不自动升级为保留正文；
 - 捕获落盘前尽力脱敏常见 API key、Bearer token、Cookie、JWT 和私钥，但脱敏不是加密，也不能保证识别所有敏感信息；
 - 路径校验、符号链接检查、Vault 锁、同目录临时文件、fsync 和原子替换用于保护本地写入；
 - memleaf 不主动上传整个 Vault，也没有托管后台、遥测或账号系统；
@@ -471,7 +471,7 @@ MIT，见 [LICENSE](LICENSE)。
 *Your memories, in files you own.*
 
 
-## 通用处理与只读验收（0.2.32）
+## 通用处理与只读验收（0.2.33）
 
 邮件、日历、工单、文件、浏览器与普通对话共用证据准入、覆盖检查和写入路径。
 自动摘要只能使用获准引用的原文；助手复述和旧记忆回读不能单独授权新增写入。
@@ -488,6 +488,10 @@ memleaf process --vault /path/to/existing/vault --source hermes --session-id SES
 
 工具执行状态与证据完整性分别记录；超限、丢失或不完整内容不会被模型的 NO_CHANGE 升级为完整。
 执行成功但尚有未解决项时，结果显示 `coverage_status=partial`，并保留来源以供有界重试或补充证据。
+`external_evidence_status` 另外说明本批工具正文是否实际可供提炼：`metadata_only` 是仅留元数据，
+`disabled` 是关闭采集，`unavailable` 是没有可用完整正文，`partial` 是只有部分可用，
+`available` 是有可用正文，`not_provided` 是本批没有外部记录。可用正文不等于业务事项已提炼完整，
+也不证明工具读取了原始邮件或文档的全文。
 行为、限制、测试协议变更见 [通用处理说明](docs/general-processing.md)。
 
 ### 工具证据留存配置
@@ -505,9 +509,11 @@ capture:
 
 现有配置未提供新字段时，旧 `include_tool_output: false` 或未设置该开关均按
 `metadata` 处理，旧 `true` 按 `bounded` 处理。显式新字段优先；新建 Vault 只写
-新字段，不再同时写含义冲突的旧开关。`include_attachments: true` 仍受上述总模式限制。
-宿主通过结构化文件路径、文件/附件标识识别文档；不承诺识别任意 Shell 命令或不透明工具
-隐藏读取的文件。用户粘贴的可见文档和显式 `remember` 内容不属于自动附件抓取。
+新字段，不再同时写含义冲突的旧开关。`include_attachments: true` 只放行显式标记的
+attachment，且仍受上述总模式限制；普通结构化文件结果不需要该开关。宿主把路径、file、
+file_id 或 file URI 归为 document，只有明确的 `attachment_id` 才归为 attachment；无法仅凭
+路径判断某个文件是否为附件，也不承诺识别任意 Shell 命令或不透明工具隐藏读取的文件。用户
+粘贴的可见文档和显式 `remember` 内容不属于自动附件抓取。
 
 策略在待捕获缓存、inbox 写入和新模型提炼输入处共同执行。配置收紧不会自动删除已提交
 记忆或改写已捕获 inbox；失败前已经冻结的提交计划也不作为新的模型调用重新提炼。

{memleaf-0.2.32 → memleaf-0.2.33}/docs/config-migrations.md

@@ -6,7 +6,7 @@ This document describes the configuration and Vault-layout compatibility boundar
 
 The current persisted top-level sections are `vault`, `agents`, `scopes`, `native_sources`, `process`, `history`, `capture`, and `llm`. Retrieval remains Scope Map -> search -> read; there is no configurable legacy injection mode.
 
-`capture.tool_evidence_mode` is the current tool-evidence retention setting and accepts `bounded`, `metadata`, or `off`. `capture.include_attachments` is independent and defaults to `false`.
+`capture.tool_evidence_mode` is the current tool-evidence retention setting and accepts `bounded`, `metadata`, or `off`. `capture.include_attachments` is independent, defaults to `false`, and gates only evidence explicitly identified as an attachment. Ordinary structural file/document results follow the selected mode.
 
 ## Deprecated fields
 

{memleaf-0.2.32 → memleaf-0.2.33}/docs/evidence-retention.md

@@ -31,12 +31,17 @@ These are source-retention limits, not a guarantee that every configured model
 can process the resulting prompt. See [capture budget design](capture-budget-design.md)
 for the ingestion boundary, model-capacity limitation and acceptance contract.
 
-Document/attachment bodies require include_attachments=true and bounded mode.
-HostRuntime and the standalone Hermes adapter classify structural file arguments
-using the same tested contract (path/file_path/file_id/attachment_id/file URI,
-including bounded nesting). This is not a claim to identify every file hidden
-behind arbitrary terminal commands or undocumented remote tools. Direct callers
-must truthfully identify document evidence with source_type=document.
+Document bodies follow the selected `tool_evidence_mode`; ordinary structural
+file arguments do not require `include_attachments=true`. Bodies explicitly
+identified as attachments (`source_type=attachment`, or an `attachment_id`
+argument in the host adapters) require `include_attachments=true` and bounded
+mode. A path that happens to point at an attachment cannot be classified as an
+attachment from the path alone. HostRuntime and the standalone Hermes adapter
+use the same tested contract: path/file/file_id/file URI means `document`, while
+an explicit `attachment_id` means `attachment`, including bounded nesting. This
+is not a claim to identify every file hidden behind arbitrary terminal commands
+or undocumented remote tools. Direct callers must truthfully identify document
+or attachment evidence with the matching `source_type`.
 
 ## Lifecycle
 

{memleaf-0.2.32 → memleaf-0.2.33}/docs/gate-evidence-boundary.md

@@ -44,6 +44,12 @@ The documented response contains `candidates`, `coverage` and
 including a decision that no candidate is warranted. An empty candidate list is
 not, by itself, proof of complete coverage.
 
+Candidates with exact `evidence_bindings` may omit `evidence_event_ids`. Core
+validates the bound unit, quote and role before deriving the source event IDs
+from those units. Explicit event IDs remain strict constraints and are never
+silently replaced. Likewise, unique exact quotes can omit numeric offsets;
+Core computes their source positions without asking the model to count them.
+
 The three empty lists describe a complete response only when there are no
 model-visible evidence units. With units present, no-admission still needs
 explicit `NO_CHANGE` or `DEFERRED` accounting. Examples must obey the same rules
@@ -55,6 +61,49 @@ bounded coverage-repair path. Missing accounting must not
 silently become permission to clean the source turn. Unknown unit IDs,
 contradictory coverage, invalid spans and unauthorized sources remain errors.
 
+## Bounded external records and Gate batches
+
+Retained JSON and unstructured external observations remain complete source
+records. Plain-text documents with explicit paragraphs, headings or list items
+use those structural boundaries and preserve the parent section path. Commas
+and sentence punctuation do not create independent units. Every unit retains
+its source record identity and exact text. If a legacy
+record exceeds the capture bound, the host may split it into contiguous
+UTF-8-safe blocks. Every block keeps the same tool/call/record identity and
+exact character offsets; the host never inserts a header or copies text from
+another record into a block.
+
+Physical units are sent to the Gate in ordered batches of at most eight units
+and 64 KiB of serialized evidence metadata/body. A complete unit is never
+truncated to fit a batch; a single oversized unit remains a singleton and the
+normal model-output limit is still a hard failure boundary. Every batch receives
+the full current-turn conversation and the same bounded related-memory and
+scope context. Coverage is complete per batch. Candidate IDs are namespaced by
+batch before they enter the turn audit, while unit IDs and source spans remain
+global and immutable.
+
+The retained physical text also supplies local related-memory search terms, so
+a short conversation accompanying a document can still retrieve the facts
+already stored from that document. Scope filtering and related-context limits
+still apply. Native memory readers receive the original conversation query;
+this local retrieval step does not send document bodies to another reader.
+
+The host waits for all Gate batches before running admission or summarization,
+so a hard failure in a later batch cannot commit an earlier batch's proposals.
+Each batch uses its own candidate IDs and does not receive earlier batches'
+proposals. Same-target updates are reconciled by the update coordinator.
+Cross-batch CREATE proposals with the same validated type and scopes go through
+a bounded model reconciliation step. It must account for every proposal and
+retain the contributing evidence; Core does not merge by keyword or text
+similarity. Failed reconciliation produces a deferred outcome.
+
+Partial coverage keeps the unresolved physical units deferred and the source
+turn available for retry without a cleanup deadline. A hard Gate/model failure
+keeps the existing failed processing marker and does not advance the turn
+watermark. These are separate outcomes: accepted partial coverage retains the
+existing journal behavior, while a failed batch never reaches the commit
+boundary.
+
 ## Diagnostics and recovery
 
 Preserve the existing failure category, retry bound and watermark behavior.

{memleaf-0.2.32 → memleaf-0.2.33}/docs/general-processing.md

@@ -1,4 +1,4 @@
-# General processing reliability contract — 0.2.32
+# General processing reliability contract — 0.2.33
 
 This is source-neutral processing, not a mail extractor. Dialogue, documents,
 calendars, issue trackers and terminal/tool observations use the same admission
@@ -22,8 +22,23 @@ be evidence; examples, suggestions and hypothetical content are not new facts.
 The Gate still decides future value, entailment, semantic role, ownership and
 CREATE/UPDATE/NO_CHANGE. Each writable candidate quotes actual evidence units.
 Core validates the unit/event identity, physical source, exact text and bounds.
+Summary title/body dates are also checked against the admitted source spans.
+Current user timestamps can anchor supported relative dates; external retrieval
+timestamps cannot. An explicit yearless source date can remain yearless, and
+an update may preserve dates already present in its selected target. Neither
+case authorizes inventing a new year or borrowing dates from another memory.
 Start/end are relative to unit.text; they may both be omitted for a unique exact
 quotation, in which case Core locates it without model character counting.
+The model may instead explicitly select a complete supplied unit with
+`unit_id`, `whole_unit: true`, and `role`, omitting quote/start/end. Core resolves
+the original text and canonical offsets from that same batch's inventory.
+This avoids copying multiline text back through the model; it does not repair
+an inaccurate quote, grant authority to metadata or assistant text, or prove
+semantic entailment. Both forms use the same downstream source checks.
+An unregistered model-generated project name must also be supported by that
+candidate's own bound source unit. A name introduced only in the model's
+proposal cannot establish a new Scope. Registered names/aliases and explicit
+user/session scope attribution keep their existing rules.
 Malformed/ambiguous references fail the contract. Matching a quote proves
 provenance, not semantic truth. This mechanism is not a universal NLP proof.
 
@@ -80,6 +95,20 @@ labels them NO_CHANGE. Incomplete turns retain their source instead of being
 cleaned after the usual grace period. Scope-filtered retries may revisit them;
 no endless automatic model retry or extra external tool call is introduced.
 
+`external_evidence_status` reports the effective capture-policy and physical
+source boundary separately from model coverage. Its detail object counts raw
+external records, usable retained body records/bytes, and metadata-only,
+incomplete or unusable records. `available` means complete source text is
+available to the planner; it is not a semantic extraction verdict or proof
+that the host read an entire underlying document. Error and partial native
+execution outputs do not gain authority merely because they contain text.
+
+Native Hermes terminal/code `output` envelopes are projected as observed text,
+with execution and host truncation kept as metadata. Explicit text record
+dividers retain each record's header and paragraphs in one exact source span;
+arbitrary application JSON remains JSON. Historical document dates must not
+be shifted to the time at which a tool retrieved the document.
+
 Tool evidence uses one shared budget: 64 source records, at most 32 KiB of UTF-8
 body text per record and 128 KiB of total body text, plus 320-character metadata
 fields, with redaction at Core capture. Cache and inbox normalization are
@@ -122,6 +151,33 @@ independent titled tasks are not merged solely because their bodies match.
 Model-assisted same-future-use matching still uses bounded existing candidates;
 it is not replaced with fuzzy string authorization or embeddings.
 
+When candidate-specific retrieval discovers an active local memory missing
+from the initial Gate context, a bounded target reconciliation stage compares
+the validated proposal and its evidence against the current records. The model
+chooses CREATE, UPDATE, NO_CHANGE or DEFERRED; an UPDATE must explicitly retain
+the target's type. Insufficient or oversized context defers the proposal.
+This contract is shared by conversation, document and arbitrary tool evidence.
+
+Final automatic UPDATE proposals receive a separate semantic review after
+same-target consolidation. The reviewer compares the selected current target,
+admitted source spans and proposed replacement, retaining still-valid old
+information unless current evidence supersedes it. It can accept, revise,
+return NO_CHANGE or defer. Revisions must pass the same source, date, type,
+Scope and target checks; review failure preserves the original memory. This
+adds a bounded model stage, not a local text-concatenation or keyword rule.
+Automatic duplicate observations remain NO_CHANGE ledger entries and do not
+enter the mutation batch as empty metadata operations.
+
+Partial semantic retries submit only unresolved evidence units. Settled outcomes
+are retained in the ledger; an identical external observation in a later turn
+is recognized by its source identity and exact content, without interpreting
+business keywords. New conversation assertions keep their distinct turn identity.
+When all newly pending turns in a session are read-only, automatic processing
+does not bundle retries of older deferred turns into that query. Those deferred
+records and their retry allowance remain available; an explicit scope retry
+keeps its existing behavior. Classification uses the current capture policy,
+including when an older inbox record still contains an excluded body.
+
 A retry resumes a matching persisted plan without asking the model for a new
 summary. CREATE/UPDATE outcomes survive interrupted final-ledger writes.
 Explicit cross-project correction and retirement preserve their original
@@ -189,8 +245,8 @@ not a passing semantic test. Never publish based only on deterministic mocks.
 
 ## Capture policy (shared-core refactor)
 
-Tool evidence is controlled by `capture.tool_evidence_mode` and document opt-in,
-not the presence of business words. The same policy runs before cache/inbox writes
+Tool evidence is controlled by `capture.tool_evidence_mode`; only explicitly
+identified attachment evidence also requires the attachment opt-in. The same policy runs before cache/inbox writes
 and new model-planning calls. Intentional exclusion is not missing evidence.
 See [retention contract](evidence-retention.md) for legacy settings, plaintext
 metadata, opaque-resource limitations and the distinction from explicit forget.

memleaf-0.2.33/examples/README.md

@@ -0,0 +1,35 @@
+# Examples
+
+`basic_usage.py` runs entirely offline. Without `--vault` it creates a
+temporary vault; pass a directory explicitly when you want to inspect the
+generated Markdown after the process exits.
+
+The example shows a lightweight `context()` directory followed by an explicit
+`read_page()` call for the selected entry. Directory results contain no body;
+long bodies can be read in subsequent pages using `next_offset` and `version`.
+
+The `mcp_stdio.ndjson` file contains one legacy initialization request and one
+modern discovery request. Pipe it to the stdio adapter with an explicit vault:
+
+```sh
+python -m memleaf.mcp_server --vault <your-vault> < examples/mcp_stdio.ndjson
+```
+
+The NDJSON file is request-only and intentionally contains no tool call,
+network endpoint, local path, or secret.
+
+`live_core_lifecycle_acceptance.py` is an opt-in real-model check. It sends only
+generated Cedar/Birch documents to the configured model and writes to a fresh
+temporary Vault. Only the supplied config's `llm` route is read; credentials
+are kept in memory, and existing inboxes, memories, and attachments are never read.
+
+```sh
+PYTHONPATH=src python examples/live_core_lifecycle_acceptance.py --model-config ~/.memleaf/config.yaml
+```
+
+It checks large tool inputs, cross-batch duplicate CREATEs, completion of an
+existing todo, structured deadlines, public todo readback, repeat processing,
+and read-only queries. It fails on incorrect business results even if processing
+reports success. Up to 60 real model calls are permitted; this script is not
+part of the offline test suite. The printed temporary directory contains only
+the synthetic Vault, model prompts/replies, and acceptance results.

memleaf-0.2.33/examples/live_core_lifecycle_acceptance.py

@@ -0,0 +1,156 @@
+"""Opt-in real-model acceptance with synthetic documents and a fresh Vault.
+
+Only the supplied config's llm section is read, in memory. No real inbox,
+knowledge, native source, or attachment is read. Credentials are never copied
+to the diagnostic Vault. This is excluded from automatic unittest discovery.
+"""
+from __future__ import annotations
+
+import argparse
+from datetime import datetime, timedelta, timezone
+import json
+import os
+from pathlib import Path
+import tempfile
+from typing import Any
+
+from memleaf import Memleaf
+from memleaf.config import save_config
+from memleaf.frontmatter import load_yaml
+from memleaf.llm import ModelRouter
+
+
+class RecordedRoute:
+    def __init__(self, route: ModelRouter, output: Path):
+        self.route, self.output, self.calls = route, output, 0
+
+    def complete(self, prompt: str, **kwargs: Any) -> str:
+        if self.calls >= 60:
+            raise RuntimeError("synthetic acceptance call budget exhausted")
+        self.calls += 1
+        number = self.calls
+        # Inputs and replies contain only the generated synthetic scenario.
+        (self.output / f"{number}-request.json").write_text(
+            json.dumps({"prompt": prompt, **kwargs}, ensure_ascii=False), encoding="utf-8")
+        print(f"call {number}: {kwargs.get('purpose')}", flush=True)
+        response = self.route.complete(prompt, **kwargs)
+        (self.output / f"{number}-response.json").write_text(response, encoding="utf-8")
+        return response
+
+
+def run(output: Path, backend: RecordedRoute) -> list[dict[str, Any]]:
+    core = Memleaf.initialize(output / "vault")
+    config = core.vault.config()
+    config["native_sources"] = {}
+    config["capture"].update(tool_evidence_mode="bounded", include_attachments=False)
+    config["scopes"] = {"project:Cedar": {}, "project:Birch": {}}
+    save_config(core.vault.config_path, config)
+    core.create_memory(memory_id="cedar-checklist", title="Cedar deployment checklist",
+        body="The Cedar deployment checklist is awaiting completion.", type="todo",
+        scopes=["project:Cedar"], scope_source="user", status="active")
+    core.create_memory(memory_id="birch-contact", title="Birch contact",
+        body="The Birch contact is Morgan.", type="fact", scopes=["project:Birch"], scope_source="user")
+
+    today = datetime.now(timezone.utc).date()
+    report_due, handover_due = (today + timedelta(days=7)).isoformat(), (today + timedelta(days=8)).isoformat()
+    actions = {
+        0: f"Project Cedar. Please prepare the Cedar acceptance report by {report_due}.",
+        1: f"Project Cedar. The Cedar deployment checklist is now complete, confirmed on {today}.",
+        2: "Project Birch. The current Birch contact is Morgan.",
+        13: f"For Project Cedar, you need to finish the acceptance report by {report_due}.",
+        22: f"Project Birch. Please arrange the Birch handover meeting by {handover_due}.",
+    }
+    records = []
+    for number in range(23):
+        log = "\n".join(f"run-{number:02d}-{row:03d}, check result=normal, observed value={row:03d};"
+                        for row in range(56))
+        records.append({"tool_name": "read_file", "call_id": f"synthetic-read-{number}",
+            "record_id": f"synthetic-record-{number}", "schema_version": "2",
+            "kind": "external_observation", "result_status": "success",
+            "execution_status": "success", "completeness": "complete", "source_type": "document",
+            "content": json.dumps({"record": number, "body": actions.get(number,
+                "Routine diagnostic output only; no new project decision, assignment, or state change."),
+                "diagnostic_log": log}, ensure_ascii=False)})
+
+    def capture(number: int, *, query_only: bool = False) -> None:
+        user = ("What are my current Project Cedar and Project Birch todos?" if query_only else
+            "I manage Project Cedar and Project Birch. Review these source records and retain confirmed "
+            "follow-up actions and state changes. Do not retain routine diagnostic logs.")
+        assistant = ("Your active follow-ups are the Cedar acceptance report and Birch handover meeting. "
+                     "The Cedar deployment checklist is completed." if query_only else
+                     "I reviewed the supplied source records.")
+        core.capture("hermes", "synthetic-lifecycle", f"turn-{number}", "user", user,
+                     event_id=f"synthetic-u-{number}")
+        core.capture("hermes", "synthetic-lifecycle", f"turn-{number}", "assistant", assistant,
+                     event_id=f"synthetic-a-{number}", tool_evidence=None if query_only else records)
+
+    def snapshot() -> dict[str, bytes]:
+        return {str(path.relative_to(core.vault.root)): path.read_bytes()
+                for area in ("knowledge", "history") for path in core.vault.list_markdown(area)}
+
+    rows = []
+    capture(1)
+    for phase in ("first", "same_turn", "new_turn", "query_only"):
+        if phase == "new_turn":
+            capture(2)
+        elif phase == "query_only":
+            capture(3, query_only=True)
+        calls_before, before = backend.calls, snapshot()
+        result = core.process(source="hermes", session_id="synthetic-lifecycle", model=backend)
+        row = {"phase": phase, "calls": backend.calls - calls_before, "result": result}
+        rows.append(row)
+        (output / "results.json").write_text(json.dumps(rows, ensure_ascii=False, indent=2), encoding="utf-8")
+        memories = [record.memory for record in core._read_memories_unlocked("knowledge")]
+        (output / f"{phase}-memories.json").write_text(
+            json.dumps([memory.to_dict() for memory in memories], ensure_ascii=False, indent=2), encoding="utf-8")
+        todos = core.list_todos(status="active")
+        (output / f"{phase}-todos.json").write_text(json.dumps(todos, ensure_ascii=False), encoding="utf-8")
+
+        assert result.get("coverage_status") == "complete" and result.get("deferred_candidates", 0) == 0, \
+            "confirmed synthetic evidence remains unresolved"
+        assert len(memories) == 4, "expected two seeded memories and two distinct new todos"
+        assert core.read("cedar-checklist").status == "completed", "existing checklist was not completed"
+        cedar = [memory for memory in memories if memory.type == "todo"
+                 and memory.scopes == ["project:Cedar"] and memory.memory_id != "cedar-checklist"]
+        birch = [memory for memory in memories if memory.type == "todo" and memory.scopes == ["project:Birch"]]
+        assert len(cedar) == 1 and cedar[0].status == "active", "Cedar report was lost, duplicated, or closed"
+        assert len(birch) == 1 and birch[0].status == "active", "final-source Birch handover was lost or duplicated"
+        assert cedar[0].due_date == report_due and birch[0].due_date == handover_due, "source deadlines were lost"
+        assert len(todos["results"]) == 2, "public todo readback did not return both active follow-ups"
+        if phase != "first":
+            assert result["memories_written"] == 0 and snapshot() == before, "repeated facts or a query changed memory"
+        if phase == "same_turn":
+            assert backend.calls == calls_before, "already-processed turn called the model again"
+        row["status"] = "pass"
+        print(json.dumps({"phase": phase, "status": "pass", "calls": row["calls"]}), flush=True)
+    return rows
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument("--model-config", type=Path, required=True,
+                        help="read only llm routing from this config; never read its Vault content")
+    args = parser.parse_args()
+    os.umask(0o077)
+    output = Path(tempfile.mkdtemp(prefix="memleaf-live-core-"))
+    print(f"Synthetic acceptance output: {output}", flush=True)
+    try:
+        # Parse this file without validating or probing unrelated native paths.
+        route_config = load_yaml(args.model_config.read_text(encoding="utf-8"))["llm"]
+        backend = RecordedRoute(ModelRouter.from_config({"llm": route_config}), output)
+        rows = run(output, backend)
+        result = {"status": "pass", "calls": backend.calls, "phases": rows}
+    except Exception as error:
+        result = {"status": "fail", "error_type": type(error).__name__,
+                  "validation_detail": getattr(error, "validation_detail", None),
+                  "evidence_check": getattr(error, "evidence_check", None)}
+        # Assertion messages are authored above; do not expose server exception text.
+        if isinstance(error, AssertionError):
+            result["assertion"] = str(error)
+    (output / "summary.json").write_text(json.dumps(result, ensure_ascii=False, indent=2), encoding="utf-8")
+    print(json.dumps({key: value for key, value in result.items() if key != "phases"}), flush=True)
+    return 0 if result["status"] == "pass" else 1
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

{memleaf-0.2.32 → memleaf-0.2.33}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "memleaf"
-version = "0.2.32"
+version = "0.2.33"
 description = "A local-first Markdown memory core for AI agents"
 readme = "README.md"
 requires-python = ">=3.11"

{memleaf-0.2.32 → memleaf-0.2.33}/src/memleaf/__init__.py

@@ -1,6 +1,6 @@
 """Local-first Markdown memory core for AI agents."""
 
-__version__ = "0.2.32"
+__version__ = "0.2.33"
 
 from .config import DEFAULT_CONFIG, default_config, load_config, save_config
 from .frontmatter import FrontmatterError, dump_frontmatter, dump_yaml, load_yaml, parse_frontmatter