memleaf 0.2.91__tar.gz → 0.2.93__tar.gz

{memleaf-0.2.91 → memleaf-0.2.93}/CHANGELOG.md

@@ -4,6 +4,53 @@ All notable changes to memleaf are documented here.
 
 ## Unreleased
 
+
+## 0.2.93 — 2026-10-05
+
+### Keep useful body compaction when another candidate has no net savings
+
+- Skip each valid compaction proposal that does not reduce both the content
+  estimate and stored record bytes, including maintenance metadata. Other safe
+  proposals in the same response can still commit; all-no-gain batches return
+  noop without writing history or a compaction journal.
+- Keep strict whole-response validation, protected metadata, selected-source
+  snapshot checks and transactional recovery. Invalid identities, duplicate rows,
+  multi-ID merges and extra business fields still reject the batch.
+- Add no model request or retry. Automatic maintenance keeps its existing
+  durable budget and input deduplication, including across restarts.
+- Validation: 58 focused regressions and 12 isolated replays of saved real
+  DeepSeek responses passed. The original mixed-gain case now shortens its useful
+  record while keeping the other file byte-for-byte unchanged. Replays made no
+  external model calls; they are not a new live-model run.
+
+## 0.2.92 — 2026-10-05
+
+### Preserve unresolved review outcomes and schedule bounded lifecycle maintenance
+
+- Retain source-bound DEFERRED when review rejects UPDATE, MERGE, or explicitly
+  requested CREATE, even if other NO_CHANGE candidates survive. Only a validated
+  same-target correction covering the evidence and fields can resolve a rejected
+  UPDATE. Unrelated candidates cannot consume an unresolved host retention intent.
+- Run deterministic history/provenance maintenance after normal process and
+  remember operations. Apply bounded version/age retention or keep_all, and
+  protect pending mutation and recovery dependencies without replaying old work.
+- Schedule at most one optional body-compaction request per processing batch
+  after the configured threshold is reached. Use a freshly completed turn's
+  remaining durable allowance, with extraction and maintenance sharing its
+  five-request limit; never reopen completed budgets or allocate a new allowance.
+- Persist recent input fingerprints and reservations before dispatch, including
+  no-op, failed and interrupted requests. Preserve compaction identity and all
+  metadata, serialize model ownership, recheck dependencies at commit, and report
+  maintenance status and observed token usage independently of committed facts.
+- Ordinary turns below the threshold and cached replays add no model calls.
+  No new user-facing tool or model-output protocol field is introduced.
+- Validation: 175 focused checks and seven isolated replay scenarios passed,
+  including the frozen real draft under approval and veto, host-intent retention,
+  process exit/restart, budget exhaustion, concurrent mutation protection, and
+  41 public-process updates retaining one identity, 32 historical versions and
+  cumulative source accounting. External model calls were zero; new live-model
+  acceptance remains separate.
+
 ## 0.2.91 — 2026-10-05
 
 ### Preserve historical reads in incremental comparison

{memleaf-0.2.91/src/memleaf.egg-info → memleaf-0.2.93}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: memleaf
-Version: 0.2.91
+Version: 0.2.93
 Summary: A local-first Markdown memory core for AI agents
 Author: memleaf contributors
 License-Expression: MIT
@@ -23,7 +23,7 @@ Dynamic: license-file
 
 [English](README.en.md) · [PyPI](https://pypi.org/project/memleaf/) · [GitHub](https://github.com/miffyblueboo/memleaf)
 
-> **版本：0.2.91。**
+> **版本：0.2.93。**
 > 自动提炼和复核共用同一条“未来记忆价值”标准：模型综合未来复用、信息增量、再次读取时的直接可用性和忘记成本，只保留对未来理解、判断或行动有实质影响的最小核心；没有明确价值的信息不提炼，不按具体业务场景硬编码排除。
 > 提炼先比较旧事项中的编号、目标、进展、责任和期限；补全未知信息也应维护原事项，另建独立待办不能替代项目事实更新。确认重复的记忆在现有处理请求中合并，保留历史与来源；项目和独立任务保持各自生命周期，Core 不按标题强行合并。
 > 新请求兼容等价的字段表示；未知执行人保持 `null`，新增或变更负责人和期限需要真实来源；等待条件在正文维护，不再提炼 `waiting_on`。行动及相关字段经过有界语义复核，提炼与复核共用每份工作的最多五次模型请求额度，成功即停止；复核失败不写入。来源时间用于解析相对日期，不补造事实日期。
@@ -462,8 +462,9 @@ history:
 - active Markdown 每条最多保留 16 个来源明细，同时记录累计 `source_count`、`source_digest` 和省略数量，避免 UPDATE 让 `sources` 无限膨胀；
 - 已完成/已取消 todo 保留在 `knowledge/` 中作为稳定当前身份，默认 active 清单不展示，但 `list_todos(status=completed|cancelled|all)` 和后续维护仍可按原 ID 定位；`closed_todo_retention_days` 仅作为旧配置兼容项保留；
 - `history.policy: bounded` 默认每个稳定记忆身份最多保留 32 个完整历史版本，且超过 3650 天的版本可被维护流程淘汰；需要永久审计时可显式设置 `history.policy: keep_all`；
-- compaction 会保留一个既有 canonical `memory_id`，不会再为整理结果创建新的 `mem-compact-*` 身份；单条精简保持原 ID，多条合并选择一个稳定 survivor。
-- 这些维护动作在正常 `process()` / `remember()` / `compact()` 生命周期中执行，不需要 daemon；普通只读检索本身不触发维护写入。
+- 正文 compaction 逐条精简，保持原 `memory_id` 和来源、责任、状态、期限，不合并独立身份；同一事项的语义合并由提炼协议的 `MERGE` 处理。
+- 正常 `process()` / `remember()` 结束后执行确定性历史维护；有未完成写入或恢复依赖时推迟维护，保留恢复输入。普通只读检索不触发维护写入。
+- 新完成的轮次达到正文压缩阈值时，每批最多一次压缩调用，与该轮提炼共用不超过 5 次的持久额度。低于阈值、缓存重放、近期相同输入已尝试时不增加调用；持久保留最近 64 个输入指纹，压缩无收益、失败或进程中断也计入去重。维护结果独立报告，不将已提交的事实回滚。显式 `compact()` 仍只执行正文压缩，不隐式清理历史；无需 daemon。
 
 默认目录尽量使用 `0700`，文件默认明文保存。memleaf 没有内置加密层，请自行保护 Vault、备份和模型调用凭证。
 

{memleaf-0.2.91 → memleaf-0.2.93}/README.en.md

@@ -4,7 +4,7 @@
 
 [中文](README.md) · [PyPI](https://pypi.org/project/memleaf/) · [GitHub](https://github.com/miffyblueboo/memleaf)
 
-> **Version: 0.2.91.**
+> **Version: 0.2.93.**
 > Automatic extraction and review share one future-value standard: the model weighs likely reuse, information gain, direct usability when read again, and the cost of forgetting. It keeps only the smallest core that can materially help future understanding, decisions, or actions; information without clear value is not extracted, and no business-specific exclusion rule is hard-coded.
 > Extraction compares existing identifiers, objectives, progress, responsibility and deadlines first. Filling an unknown also maintains the original item; a separate task does not replace updating shared project facts. Confirmed duplicates can be merged in the existing processing requests with their sources and history preserved. Independent project and task lifecycles remain separate; Core does not merge matching titles.
 > New requests accept equivalent field representations. Unknown executors remain `null`; new or changed responsibility and deadlines require actual source evidence. Dependencies belong in the body; `waiting_on` is no longer extracted. Proposed actions and related fields undergo bounded semantic verification within a shared five-request limit per work item, stopping on success; failed verification blocks writes. Source time resolves relative dates without inventing factual dates.
@@ -442,8 +442,9 @@ history:
 - Each active Markdown memory retains at most 16 detailed provenance rows while tracking cumulative `source_count`, `source_digest`, and omitted rows, so repeated UPDATEs cannot grow `sources` without bound.
 - Completed/cancelled todos remain in `knowledge/` as stable current identities. The default active view hides them, while `list_todos(status=completed|cancelled|all)` and later maintenance can still resolve the original ID. `closed_todo_retention_days` remains only as a legacy configuration key.
 - With `history.policy: bounded`, each stable memory identity keeps at most 32 full historical versions, and versions older than 3650 days are eligible for pruning. Set `history.policy: keep_all` explicitly when permanent audit retention is required.
-- Compaction preserves an existing canonical `memory_id` instead of creating a new `mem-compact-*` identity; a single-memory rewrite keeps its ID and a multi-memory merge chooses one stable survivor.
-- Maintenance runs through normal `process()` / `remember()` / `compact()` lifecycle calls and requires no daemon. Ordinary read-only retrieval does not trigger maintenance writes.
+- Body compaction shortens each memory independently, preserving its ID, provenance, responsibility, status and deadline. Semantic consolidation of the same matter uses the extraction protocol's `MERGE` operation.
+- Normal `process()` / `remember()` calls perform deterministic history maintenance after processing. Pending mutations and recovery dependencies postpone maintenance and preserve recovery inputs. Read-only retrieval does not trigger maintenance writes.
+- Freshly completed turns can schedule at most one compaction request per batch when the configured threshold is reached, sharing that turn's durable five-request allowance with extraction. Below-threshold turns, cached replays and recently attempted identical inputs add no calls. The latest 64 input fingerprints are persisted, including no-op, failed and interrupted attempts. Maintenance results are reported separately without undoing committed facts. Explicit `compact()` remains body-only and does not implicitly prune history. No daemon is required.
 
 Directories are normally created with mode `0700`, and files are stored as plaintext by default. memleaf has no built-in encryption layer; protect the Vault, backups, and model credentials yourself.
 

{memleaf-0.2.91 → memleaf-0.2.93}/README.md

@@ -4,7 +4,7 @@
 
 [English](README.en.md) · [PyPI](https://pypi.org/project/memleaf/) · [GitHub](https://github.com/miffyblueboo/memleaf)
 
-> **版本：0.2.91。**
+> **版本：0.2.93。**
 > 自动提炼和复核共用同一条“未来记忆价值”标准：模型综合未来复用、信息增量、再次读取时的直接可用性和忘记成本，只保留对未来理解、判断或行动有实质影响的最小核心；没有明确价值的信息不提炼，不按具体业务场景硬编码排除。
 > 提炼先比较旧事项中的编号、目标、进展、责任和期限；补全未知信息也应维护原事项，另建独立待办不能替代项目事实更新。确认重复的记忆在现有处理请求中合并，保留历史与来源；项目和独立任务保持各自生命周期，Core 不按标题强行合并。
 > 新请求兼容等价的字段表示；未知执行人保持 `null`，新增或变更负责人和期限需要真实来源；等待条件在正文维护，不再提炼 `waiting_on`。行动及相关字段经过有界语义复核，提炼与复核共用每份工作的最多五次模型请求额度，成功即停止；复核失败不写入。来源时间用于解析相对日期，不补造事实日期。
@@ -443,8 +443,9 @@ history:
 - active Markdown 每条最多保留 16 个来源明细，同时记录累计 `source_count`、`source_digest` 和省略数量，避免 UPDATE 让 `sources` 无限膨胀；
 - 已完成/已取消 todo 保留在 `knowledge/` 中作为稳定当前身份，默认 active 清单不展示，但 `list_todos(status=completed|cancelled|all)` 和后续维护仍可按原 ID 定位；`closed_todo_retention_days` 仅作为旧配置兼容项保留；
 - `history.policy: bounded` 默认每个稳定记忆身份最多保留 32 个完整历史版本，且超过 3650 天的版本可被维护流程淘汰；需要永久审计时可显式设置 `history.policy: keep_all`；
-- compaction 会保留一个既有 canonical `memory_id`，不会再为整理结果创建新的 `mem-compact-*` 身份；单条精简保持原 ID，多条合并选择一个稳定 survivor。
-- 这些维护动作在正常 `process()` / `remember()` / `compact()` 生命周期中执行，不需要 daemon；普通只读检索本身不触发维护写入。
+- 正文 compaction 逐条精简，保持原 `memory_id` 和来源、责任、状态、期限，不合并独立身份；同一事项的语义合并由提炼协议的 `MERGE` 处理。
+- 正常 `process()` / `remember()` 结束后执行确定性历史维护；有未完成写入或恢复依赖时推迟维护，保留恢复输入。普通只读检索不触发维护写入。
+- 新完成的轮次达到正文压缩阈值时，每批最多一次压缩调用，与该轮提炼共用不超过 5 次的持久额度。低于阈值、缓存重放、近期相同输入已尝试时不增加调用；持久保留最近 64 个输入指纹，压缩无收益、失败或进程中断也计入去重。维护结果独立报告，不将已提交的事实回滚。显式 `compact()` 仍只执行正文压缩，不隐式清理历史；无需 daemon。
 
 默认目录尽量使用 `0700`，文件默认明文保存。memleaf 没有内置加密层，请自行保护 Vault、备份和模型调用凭证。
 

{memleaf-0.2.91 → memleaf-0.2.93}/docs/body-compaction.md

@@ -7,8 +7,9 @@ extraction, a task-state update, or a multi-ID deduplication service.
 ## One existing entry, one narrower model job
 
 `service.compact(model=..., router=...)` keeps the existing explicit maintenance
-entry and threshold/low-priority candidate selection. Normal extraction still
-never invokes it on its return path. The configured threshold is a local storage
+entry and threshold/low-priority candidate selection. Completed `process` and
+`remember` calls can also schedule threshold-driven body maintenance within the
+source turn's remaining request budget. The configured threshold is a local storage
 estimate, not a claim about a model's context capacity. No default route,
 package version, incremental prompt or automatic request allowance changes.
 
@@ -58,7 +59,13 @@ counts this invocation boundary, not independently observed provider billing or
 hidden transport retries within an externally supplied backend.
 
 The output must reduce both the existing content estimate and the actual stored
-record bytes. Results expose `record_bytes_before` and `record_bytes_after` for
+record bytes, including new maintenance metadata. Each valid proposal with no
+net savings is skipped independently; other safe proposals in the same response
+can still be committed. If none have savings, the result is `noop` with no
+history or journal writes. All rows are validated before applying this filter;
+invalid structure, identities or protected fields still reject the batch.
+Skipping a row does not trigger another model call. Results expose
+`record_bytes_before` and `record_bytes_after` for
 changed records; these exclude the retained historical copy and are not total
 Vault-disk savings. `active_tokens_*` remain local estimates, not tokenizer or
 latency measurements. A more concise body is not proof of semantic equivalence:

{memleaf-0.2.91 → memleaf-0.2.93}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "memleaf"
-version = "0.2.91"
+version = "0.2.93"
 description = "A local-first Markdown memory core for AI agents"
 readme = "README.md"
 requires-python = ">=3.11"

{memleaf-0.2.91 → memleaf-0.2.93}/src/memleaf/__init__.py

@@ -1,6 +1,6 @@
 """Local-first Markdown memory core for AI agents."""
 
-__version__ = "0.2.91"
+__version__ = "0.2.93"
 
 from .config import DEFAULT_CONFIG, default_config, load_config, save_config
 from .frontmatter import FrontmatterError, dump_frontmatter, dump_yaml, load_yaml, parse_frontmatter

{memleaf-0.2.91 → memleaf-0.2.93}/src/memleaf/compaction.py

@@ -546,10 +546,13 @@ class Compactor:
             if summary["body"] == source_candidates[0].memory.body:
                 continue
             replacement = self._build_replacement(summary, source_candidates, now=now)
+            # A valid proposal without net savings is a per-record no-op.
+            # Include maintenance metadata in byte cost; other safe rows may
+            # still be useful. Protocol and snapshot errors remain batch errors.
             if len(replacement.raw.encode("utf-8")) >= len(source_candidates[0].raw.encode("utf-8")):
-                raise CompactionError("compaction replacement does not reduce stored bytes")
+                continue
             if replacement.replacement_tokens >= replacement.source_tokens:
-                raise CompactionError("compaction replacement is not smaller than its sources")
+                continue
             replacements.append(replacement)
         return replacements
 
@@ -605,10 +608,13 @@ class Compactor:
         replacements: list[_Replacement],
         *,
         now: str,
+        before_commit: Any = None,
     ) -> tuple[list[str], list[str]]:
         replacement_ids = [replacement.memory.memory_id for replacement in replacements]
         selected_by_id = {candidate.memory.memory_id: candidate for candidate in selected}
         with self.service._mutation_boundary():
+            if before_commit is not None:
+                before_commit()
             current_scan = scan_memories(self.service.vault)
             if current_scan.issues:
                 raise CompactionError("current-memory scan changed during compaction")
@@ -737,7 +743,8 @@ class Compactor:
             "history_written": [],
         }
 
-    def _run(self, *, model: Any = None, router: Any = None, explicit: bool) -> dict[str, Any]:
+    def _run(self, *, model: Any = None, router: Any = None, explicit: bool,
+             admit: Any = None, before_commit: Any = None) -> dict[str, Any]:
         threshold, ratio = self._config()
         now = _clock_now(getattr(self.service, "clock", None))
         selected, all_active, active_tokens = self._snapshot(threshold, ratio)
@@ -758,6 +765,11 @@ class Compactor:
                 raise CompactionError("compaction input exceeds size limit")
             result["request_bytes"] = request_bytes
             backend = self._resolve_backend(model=model, router=router)
+            if admit is not None:
+                reason = admit(selected, threshold, ratio)
+                if reason is not None:
+                    result.update(status="deferred", code=reason)
+                    return result
             try:
                 result["backend_calls"] += 1
                 raw = backend.complete(
@@ -788,6 +800,7 @@ class Compactor:
                 all_active,
                 replacements,
                 now=now,
+                before_commit=before_commit,
             )
             with self.service.vault.lock():
                 active_after = [record.memory for record in self.service._read_memories_unlocked("knowledge")]
@@ -826,9 +839,11 @@ class Compactor:
     def compact(self, *, model: Any = None, router: Any = None) -> dict[str, Any]:
         return self._run(model=model, router=router, explicit=True)
 
-    def auto(self, *, model: Any = None, router: Any = None) -> dict[str, Any]:
+    def auto(self, *, model: Any = None, router: Any = None,
+             admit: Any = None, before_commit: Any = None) -> dict[str, Any]:
         try:
-            return self._run(model=model, router=router, explicit=False)
+            return self._run(model=model, router=router, explicit=False,
+                             admit=admit, before_commit=before_commit)
         except Exception as error:
             status = _error_kind(error)
             return {

{memleaf-0.2.91 → memleaf-0.2.93}/src/memleaf/extraction_work_state.py

@@ -28,6 +28,7 @@ _MAX_BUDGET_BYTES = 8 * 1024 * 1024
 _PROCESS_JOB_VERSION = 1
 _MAX_WORKS = 128
 _MAX_TURNS_PER_WORK = 64
+_MAX_MAINTENANCE_SNAPSHOTS = 64
 
 
 class ExtractionWorkStateError(RuntimeError):
@@ -95,11 +96,15 @@ def _normalize_turn_state(value: Any) -> dict[str, Any]:
     completed = value.get("completed", False)
     if not isinstance(completed, bool):
         raise ExtractionWorkStateError("invalid extraction request budget completion")
+    maintenance_reserved = value.get("maintenance_reserved", False)
+    if type(maintenance_reserved) is not bool or (maintenance_reserved and not completed):
+        raise ExtractionWorkStateError("invalid maintenance request reservation")
     return {
         "requests": requests,
         "started_at_epoch": float(started) if started is not None else None,
         "request_limit_at_creation": stored_limit,
         "completed": completed,
+        **({"maintenance_reserved": True} if maintenance_reserved else {}),
     }
 
 
@@ -164,7 +169,13 @@ def _read_budget_state_unlocked(vault: Any) -> dict[str, Any]:
     retired_count = sum(w.get("retired", False) for w in normalized_works.values())
     if len(normalized_works) - retired_count > _MAX_WORKS or retired_count > _MAX_RETIRED_WORKS:
         raise ExtractionWorkStateError("extraction request budget state exceeds bound")
-    return {"version": value["version"], "works": normalized_works, "order": normalized_order}
+    snapshots = value.get("maintenance_snapshots", [])
+    if (not isinstance(snapshots, list) or len(snapshots) > _MAX_MAINTENANCE_SNAPSHOTS
+            or any(not isinstance(s, str) or re.fullmatch(r"[0-9a-f]{64}", s) is None for s in snapshots)
+            or len(set(snapshots)) != len(snapshots)):
+        raise ExtractionWorkStateError("invalid maintenance snapshots")
+    return {"version": value["version"], "works": normalized_works, "order": normalized_order,
+            **({"maintenance_snapshots": snapshots} if snapshots else {})}
 
 
 def _save_budget_state_unlocked(vault: Any, state: dict[str, Any]) -> None:
@@ -401,6 +412,42 @@ def complete_turn_budget(vault: Any, *, work_id: str, turn_id: str) -> bool:
         return False
 
 
+def _remember_maintenance_snapshot_unlocked(vault: Any, state: dict[str, Any], snapshot: str) -> None:
+    if not isinstance(snapshot, str) or re.fullmatch(r"[0-9a-f]{64}", snapshot) is None:
+        raise ExtractionWorkStateError("invalid maintenance snapshot")
+    snapshots = [s for s in state.get("maintenance_snapshots", []) if s != snapshot]
+    state["maintenance_snapshots"] = (snapshots + [snapshot])[-_MAX_MAINTENANCE_SNAPSHOTS:]
+    _save_budget_state_unlocked(vault, state)
+
+
+def _reserve_maintenance_unlocked(vault: Any, *, work_id: str, turn_id: str,
+                                  snapshot: str, minimum_requests: int = 0) -> tuple[int | None, str | None]:
+    """Caller holds the lock and proves the linked run fully committed.
+
+    Only one optional maintenance call can use the remaining allowance of a
+    completed turn. Completion stays sealed; extraction can never resume from
+    this reservation. Count and input deduplication precede outbound IO in the
+    same atomic file write, including uncertain process exits.
+    """
+    state = _read_budget_state_unlocked(vault)
+    work = state["works"].get(work_id)
+    turn = work.get("turns", {}).get(turn_id) if work else None
+    if not turn or not turn["completed"] or work.get("retired"):
+        return None, "maintenance_budget_unavailable"
+    if turn["requests"] < minimum_requests:
+        raise ExtractionWorkStateError("maintenance budget evidence incomplete")
+    if snapshot in state.get("maintenance_snapshots", []):
+        return None, "maintenance_snapshot_already_attempted"
+    if turn.get("maintenance_reserved"):
+        return None, "maintenance_already_reserved"
+    if turn["requests"] >= min(turn["request_limit_at_creation"], MAX_INCREMENTAL_REQUESTS):
+        return None, "request_budget_exhausted"
+    turn["requests"] += 1
+    turn["maintenance_reserved"] = True
+    _remember_maintenance_snapshot_unlocked(vault, state, snapshot)
+    return turn["requests"], None
+
+
 __all__ = [
     "ExtractionWorkStateError",
     "active_background_work_id",

{memleaf-0.2.91 → memleaf-0.2.93}/src/memleaf/hermes_provider/plugin.yaml

@@ -1,5 +1,5 @@
 name: memleaf
-version: 0.2.91
+version: 0.2.93
 description: "Hermes-native external MemoryProvider for local-first Markdown memory shared with the memleaf MCP server."
 hooks:
   - prefetch

{memleaf-0.2.91 → memleaf-0.2.93}/src/memleaf/incremental_semantics.py

@@ -153,6 +153,15 @@ def apply_review(request, response):
         if set(review) - {"decisions", "updates"} or not isinstance(decisions, list) or len(decisions) != len(rows):
             raise ValueError
         positions = set()
+        rejected = []
+        def discard(row):
+            # A rejected maintenance proposal is not proof that the source
+            # needs no change. Keep its unresolved disposition even when other
+            # candidates survive the review. Ordinary rejected automatic
+            # CREATEs can still mean there was no durable fact to retain.
+            if (row.get("action") in {"UPDATE", "MERGE"}
+                    or data["original_input"].get("request_kind") == "explicit_remember"):
+                rejected.append(deepcopy(row))
         for decision in decisions:
             if (not isinstance(decision, dict) or not {"item", "keep", "field_support"} <= set(decision)
                     or set(decision) - {"item", "keep", "field_support", "replace"}):
@@ -206,6 +215,7 @@ def apply_review(request, response):
             if isinstance(row, dict) and row.get("action") in {"NO_CHANGE", "NO_MEMORY", "DEFERRED"}:
                 continue  # Review cannot erase an unresolved control disposition.
             if not decision["keep"] or any(not support.get(k, True) for k in _REQUIRED.get(row["action"].upper(), set())):
+                discard(row)
                 rows[i] = None
                 continue
             payload = row.get("memory" if row["action"].upper() == "CREATE" else "patch", {})
@@ -241,6 +251,7 @@ def apply_review(request, response):
                     if name == "deadline":
                         row.pop("deadline_decision", None)
             if row["action"].upper() == "UPDATE" and not payload:
+                discard(row)
                 rows[i] = None
         rows = [r for r in rows if r is not None]
         additions = review.get("updates", [])
@@ -256,6 +267,21 @@ def apply_review(request, response):
                 raise ValueError
             rows.append(row)
             used.add(row["target"])
+        for rejected_row in rejected:
+            # The same review can supply a complete, independently validated
+            # correction for this target. An unrelated NO_CHANGE cannot cover
+            # a rejected proposal, nor can UPDATE stand in for a rejected MERGE.
+            if rejected_row.get("action") == "UPDATE" and any(
+                    row.get("action") == "UPDATE" and row.get("target") == rejected_row.get("target")
+                    and set(rejected_row.get("evidence", [])) <= set(row.get("evidence", []))
+                    and fields(rejected_row) <= fields(row)
+                    for row in rows):
+                continue
+            rows.append({"action": "DEFERRED", "evidence": rejected_row.get("evidence", []),
+                         "reason": "missing_context",
+                         "need": "The proposed retention or maintenance was rejected by source review."})
+        if len(rows) > MAX_ITEMS:
+            raise ValueError
         if not rows:
             original = data["original_input"]
             if original.get("request_kind") == "explicit_remember":

memleaf-0.2.93/src/memleaf/lifecycle_maintenance.py

@@ -0,0 +1,195 @@
+"""Optional maintenance after extraction, sharing its sealed request allowance.
+
+No model call below the configured threshold. At most one batch request can
+use a freshly completed turn's remaining allowance; neither polling nor a
+process restart grants another budget. Only hashes and counters are persisted.
+"""
+from __future__ import annotations
+
+from dataclasses import replace
+import hashlib
+import json
+import os
+import uuid
+
+from .compaction import Compactor, CompactionError, _clock_now
+from .extraction_work_state import (_read_budget_state_unlocked, _reserve_maintenance_unlocked,
+                                    _remember_maintenance_snapshot_unlocked)
+from .incremental_run_state import (OWNER, load_run, owner_live, register_owner,
+                                    unregister_owner, safe_call_metrics)
+from .locking import atomic_write_json
+from .models import Memory
+from .process_common import _read_processed
+from .prompts import COMPACT_SYSTEM
+from .retention import RetentionManager
+
+
+def _signature(selected):
+    values = sorted((Compactor._prompt_memory(c) for c in selected), key=lambda m: m["memory_id"])
+    raw = json.dumps([COMPACT_SYSTEM, values], ensure_ascii=False,
+                     sort_keys=True, separators=(",", ":"))
+    return hashlib.sha256(raw.encode("utf-8")).hexdigest()
+
+
+class _MaintenanceBackend:
+    def __init__(self):
+        self.backend = None
+        self.calls = 0
+        self.failed = False
+        self.usage = {}
+
+    def complete(self, prompt, **kwargs):
+        consume = getattr(self.backend, "consume_call_metrics", None)
+        if callable(consume):
+            try:
+                consume()
+            except Exception:
+                pass
+        try:
+            self.calls += 1
+            return self.backend.complete(prompt, **kwargs)
+        except Exception:
+            self.failed = True
+            raise
+        finally:
+            if callable(consume):
+                try:
+                    self.usage = safe_call_metrics(consume())
+                except Exception:
+                    pass
+
+    def metrics(self, status):
+        if not self.calls:
+            return {}
+        bucket = {"call_count": self.calls, "failed_calls": int(self.failed),
+                  "invalid_output_count": int(status == "invalid_output"), **self.usage}
+        return {"total": bucket, "stages": {"compact": bucket},
+                "calls": [{"stage": "compact", "operation": "compact_primary",
+                           "call_index": 1, "failed": self.failed,
+                           "invalid_output": status == "invalid_output", **self.usage}]}
+
+
+def maintain_lifecycle(service, result, *, run_ids=(), model=None, router=None):
+    """Attach independent maintenance outcomes without undoing committed facts."""
+    result["maintenance_model_calls"] = 0
+    manager = RetentionManager(service)
+    try:
+        result["retention"] = manager.maintain(_clock_now(getattr(service, "clock", None)))
+    except (OSError, ValueError, RuntimeError):
+        result["retention"] = {"maintenance_status": "deferred", "code": "maintenance_state_unavailable"}
+    retention = result["retention"]
+    retention.setdefault("maintenance_status", "completed")
+    if retention.get("maintenance_status") in {"deferred", "partial"}:
+        result["compaction"] = {"status": "deferred", "code": retention.get("code"), "backend_calls": 0}
+        return result
+    if not run_ids or result.get("execution_status") != "completed":
+        result["compaction"] = {"status": "not_run", "reason": "no_fresh_completed_turn", "backend_calls": 0}
+        return result
+
+    backend = _MaintenanceBackend()
+    token = uuid.uuid4().hex
+    claimed = False
+    selected_input = []
+    ordinal = None
+
+    def safe_dependencies(processed):
+        _, protected, unresolved = manager._dependencies_unlocked(processed)
+        if protected or unresolved:
+            raise CompactionError("pending mutation dependencies")
+
+    def admit(selected, threshold, ratio):
+        nonlocal claimed, selected_input, ordinal
+        fingerprint = _signature(selected)
+        with service._mutation_boundary():
+            processed = _read_processed(service.vault.processed_state_path)
+            if owner_live(processed):
+                return "processing_busy"
+            safe_dependencies(processed)
+            # One successful source allowance, never a newly allocated budget.
+            run = next((r for rid in reversed(run_ids) if (r := load_run(processed, rid)) is not None
+                        and r["status"] == "completed" and r.get("budget_finalized")), None)
+            if run is None:
+                return "maintenance_budget_unavailable"
+            from .incremental_runtime import _resolve
+            backend.backend = _resolve(service, model, router)
+            ordinal, reason = _reserve_maintenance_unlocked(service.vault,
+                work_id=run["budget_id"], turn_id=run["turn_budget_id"], snapshot=fingerprint,
+                minimum_requests=run["reserved_requests"])
+            if reason:
+                return reason
+            register_owner(token)
+            claimed = True
+            processed[OWNER] = {"run_id": run["run_id"], "pid": os.getpid(), "token": token}
+            atomic_write_json(service.vault.processed_state_path, processed)
+            selected_input = selected
+        return None
+
+    def before_commit():
+        # Called inside the compactor's mutation boundary. New explicit work,
+        # forget/cancel, or an ownership change during IO postpones the write.
+        processed = _read_processed(service.vault.processed_state_path)
+        owner = processed.get(OWNER, {})
+        run = load_run(processed, owner.get("run_id")) if owner.get("token") == token else None
+        if run is None or run["status"] != "completed":
+            raise CompactionError("maintenance ownership changed")
+        safe_dependencies(processed)
+
+    try:
+        compaction = Compactor(service).auto(model=backend, admit=admit, before_commit=before_commit)
+        if compaction.get("status") == "compacted":
+            # Remember the resulting input too: another turn must not pay to
+            # compress these same heads again merely because total size is high.
+            try:
+                with service.vault.lock():
+                    from .query_scan import MAX_FILE_BYTES
+                    current = []
+                    for candidate in selected_input:
+                        if candidate.path.is_symlink():
+                            raise CompactionError("unsafe maintenance target")
+                        with candidate.path.open("rb") as stream:
+                            raw = stream.read(MAX_FILE_BYTES + 1)
+                        if len(raw) > MAX_FILE_BYTES:
+                            raise CompactionError("unsafe maintenance target")
+                        current.append(replace(candidate, memory=Memory.from_markdown(raw.decode("utf-8"), candidate.path)))
+                    state = _read_budget_state_unlocked(service.vault)
+                    _remember_maintenance_snapshot_unlocked(service.vault, state, _signature(current))
+            except (OSError, ValueError, RuntimeError):
+                compaction["deduplication_status"] = "deferred"
+                compaction["code"] = "maintenance_state_unavailable"
+    except (OSError, ValueError, RuntimeError):
+        compaction = {"status": "deferred", "code": "maintenance_state_unavailable"}
+    finally:
+        if claimed:
+            try:
+                with service.vault.lock():
+                    processed = _read_processed(service.vault.processed_state_path)
+                    if processed.get(OWNER, {}).get("token") == token:
+                        processed.pop(OWNER)
+                        atomic_write_json(service.vault.processed_state_path, processed)
+            except (OSError, ValueError, RuntimeError):
+                pass  # A stale local/dead owner cannot grant another request.
+            finally:
+                unregister_owner(token)
+
+    compaction["backend_calls"] = backend.calls
+    if ordinal is not None:
+        compaction["request_ordinal"] = ordinal
+    metrics = backend.metrics(compaction.get("status"))
+    if metrics:
+        compaction["model_metrics"] = metrics
+        from .process_jobs import _aggregate_model_metrics
+        result["model_metrics"] = _aggregate_model_metrics([result.get("model_metrics", {}), metrics])
+    result["compaction"] = compaction
+    result["maintenance_model_calls"] = backend.calls
+    count_key = "model_calls" if "model_calls" in result else "model_calls_this_invocation"
+    result[count_key] = result.get(count_key, 0) + backend.calls
+    if compaction.get("status") == "compacted":
+        try:
+            after = manager.maintain(_clock_now(getattr(service, "clock", None)))
+            for key in ("history_pruned", "closed_todos_retired", "provenance_rewritten"):
+                after[key] = after.get(key, 0) + retention.get(key, 0)
+            result["retention"] = after
+        except (OSError, ValueError, RuntimeError):
+            result["retention"]["maintenance_status"] = "deferred"
+            result["retention"]["code"] = "maintenance_state_unavailable"
+    return result

{memleaf-0.2.91 → memleaf-0.2.93}/src/memleaf/process_jobs.py

@@ -71,6 +71,7 @@ _MODEL_METRIC_STAGES = frozenset({
     "coordination",
     "target_reconciliation",
     "single_pass",
+    "compact",
     "other",
 })
 _MODEL_METRIC_OPERATIONS = frozenset(
@@ -415,11 +416,22 @@ def _safe_result(value: Any) -> dict[str, Any]:
         result["memory_ids"] = [item[:200] for item in ids if isinstance(item, str)][:100]
     if isinstance(value.get("pipeline"), str) and value["pipeline"] in {"legacy", "incremental"}:
         result["pipeline"] = value["pipeline"]
-    for key in ("model_calls", "attempted_turns", "unlocated_issue_count"):
+    for key in ("model_calls", "maintenance_model_calls", "attempted_turns", "unlocated_issue_count"):
         if type(value.get(key)) is int and value[key] >= 0:
             result[key] = value[key]
     if value.get("cleanup_status") == "recovery_required":
         result["cleanup_status"] = "recovery_required"
+    for key in ("retention", "compaction"):
+        detail = value.get(key)
+        if isinstance(detail, Mapping):
+            safe = {name: detail[name][:100] for name in ("status", "maintenance_status", "code", "reason", "history_policy", "deduplication_status")
+                    if isinstance(detail.get(name), str)}
+            for name in ("history_pruned", "provenance_rewritten", "closed_todos_retired", "protected_history_groups",
+                         "backend_calls", "request_ordinal", "compacted", "active_tokens_before", "active_tokens_after"):
+                if type(detail.get(name)) is int and detail[name] >= 0:
+                    safe[name] = detail[name]
+            if safe:
+                result[key] = safe
     if isinstance(value.get("execution_status"), str):
         result["execution_status"] = value["execution_status"][:80]
     rows = value.get("results")
@@ -530,7 +542,7 @@ def _aggregate_attempt_results(attempts: list[Any]) -> dict[str, Any]:
         value = attempt.get("result", {}) if isinstance(attempt, Mapping) else {}
         if not isinstance(value, Mapping):
             continue
-        for key in ("pipeline", "execution_status", "results", "results_truncated", "cleanup_status"):
+        for key in ("pipeline", "execution_status", "results", "results_truncated", "cleanup_status", "retention", "compaction"):
             if key in value:
                 aggregate[key] = value[key]
         break
@@ -542,7 +554,7 @@ def _aggregate_attempt_results(attempts: list[Any]) -> dict[str, Any]:
                     "deferred_inbox_turns", "unresolved_evidence_count", "unlocated_issue_count", "retryable_deferred_turns"):
             if key in last:
                 aggregate[key] = last[key]
-        for key in ("model_calls", "attempted_turns", "unlocated_issue_count"):
+        for key in ("model_calls", "maintenance_model_calls", "attempted_turns", "unlocated_issue_count"):
             aggregate[key] = sum(v.get(key, 0) for v in incremental if type(v.get(key, 0)) is int)
         if "cleanup_status" in last:
             aggregate["cleanup_status"] = last["cleanup_status"]