memleaf 0.2.80__tar.gz → 0.2.82__tar.gz

{memleaf-0.2.80 → memleaf-0.2.82}/CHANGELOG.md

@@ -4,6 +4,34 @@ All notable changes to memleaf are documented here.
 
 ## Unreleased
 
+## 0.2.82 — 2026-09-30
+
+### Hermes installation tool discovery
+
+- Derive the Hermes MCP installation check from the installed server tool
+  declarations instead of a separately maintained tool-count constant. This
+  fixes the 0.2.81 installation rollback after adding `recover_failed_run`.
+- Verify actual stdio tool discovery and Hermes count acceptance/rejection in
+  both installed distributions on every supported CI platform.
+
+## 0.2.81 — 2026-09-30
+
+### HTTP failure classification and controlled run recovery
+
+- Preserve safe numeric HTTP status in model errors, request attempts and
+  processing diagnostics. Allow 408/500/502/503/504 to use the remaining original
+  transport allowance; known permanent errors stay closed.
+- Add revision-bound failed automatic run preview/apply through the Python API,
+  `recover-run` CLI and `recover_failed_run` MCP tool. Historical unclassified
+  HTTP errors require explicit acknowledgement; ordinary recovery stays unchanged.
+- Rebuild current comparison context for unchanged sources while retaining run,
+  commit and budget identities, prior attempts and recovery audit. Reject consumed,
+  changed or revoked sources, conflicting ownership, exhausted budgets and full
+  active receipt capacity. Repeated execution and interrupted writes remain bounded.
+- Validation: 121 focused checks passed and 10 historical failed runs exercised
+  isolated recovery with an injected HTTP503, without network/model calls or
+  formal Vault changes. Real-model semantic quality remains unverified.
+
 ## 0.2.80 — 2026-09-29
 
 ### Explicit deadline annotations

{memleaf-0.2.80/src/memleaf.egg-info → memleaf-0.2.82}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: memleaf
-Version: 0.2.80
+Version: 0.2.82
 Summary: A local-first Markdown memory core for AI agents
 Author: memleaf contributors
 License-Expression: MIT
@@ -23,7 +23,7 @@ Dynamic: license-file
 
 [English](README.en.md) · [PyPI](https://pypi.org/project/memleaf/) · [GitHub](https://github.com/miffyblueboo/memleaf)
 
-> **版本：0.2.80。**
+> **版本：0.2.82。**
 > 自动提炼和复核共用同一条“未来记忆价值”标准：模型综合未来复用、信息增量、再次读取时的直接可用性和忘记成本，只保留对未来理解、判断或行动有实质影响的最小核心；没有明确价值的信息不提炼，不按具体业务场景硬编码排除。
 > 提炼契约现在只有一份中文语义契约（旧版英文 B3 提示词已删除），首轮、主题选择、候选修复和 `semantic_maintenance` 使用同一套保留价值判断；同一件事只写一次。
 > Core 不再因为归属、日期、任务依据或跨项目而丢弃整条候选：归属缺失会归一到证据里唯一被点名的项目、否则落 `global`；日期只接受能在本候选证据里找到出处、且能锚定成 ISO 的写法（`明天`、`周五`、`月底` 都会被解析），无法解析时只丢掉日期字段，记忆照常写入；未获授权的改归属沿用目标原归属。
@@ -167,12 +167,12 @@ python -m pip install -U memleaf && python -m memleaf install
 4. 激活 `memory.provider=memleaf`；
 5. 通过 Hermes 官方 CLI 配置 memleaf MCP；
 6. 配置 MCP lazy/idle 生命周期；
-7. 验证 MCP Server 能发现 13 个工具；
+7. 验证 MCP Server 能发现当前版本声明的全部工具；
 8. 写入本地 Agent 状态索引。
 
 完成后重启 Hermes。
 
-如果没有检测到 Hermes、无法取得完整模型路由、Provider 激活失败或 MCP 的 13 工具验证失败，安装会明确返回失败，不会把未完成的接入报告为成功。
+如果没有检测到 Hermes、无法取得完整模型路由、Provider 激活失败或 MCP 工具发现验证失败，安装会明确返回失败，不会把未完成的接入报告为成功。
 
 仓库中的 `install.sh` 保留给源码开发、离线源码安装和故障排查；普通 PyPI 用户不需要执行它。
 
@@ -269,7 +269,7 @@ python -m memleaf.mcp_server --vault "$HOME/.memleaf"
 
 不传 `--vault` 时默认使用 `~/.memleaf`；也可以设置 `MEMLEAF_VAULT`。正常情况下不需要手工常驻，Hermes 或 Codex 会按需启动它。stdout 只输出 JSON-RPC，日志不会污染协议通道。
 
-当前提供 13 个工具：
+当前提供以下工具：
 
 | 工具 | 用途 |
 | --- | --- |
@@ -277,9 +277,11 @@ python -m memleaf.mcp_server --vault "$HOME/.memleaf"
 | `context` | 兼容保留的有界轻量目录，不用于自动检索路径 |
 | `scope_catalog` | 返回 Scope、父级和别名，不返回具体记忆正文 |
 | `search` | 返回有界候选目录和 `found`/`no_match` 状态 |
+| `list_todos` | 跨 Scope 按状态和日期筛选并分页列出当前待办 |
 | `read` | 使用当前 `retrieval_id` 分页读取选中的记忆正文 |
 | `process` | 处理完整 inbox 轮次并按准入规则提炼；`background=true` 时异步入队 |
 | `process_status` | 只读查询异步处理 job 状态 |
+| `recover_failed_run` | 预览并显式恢复终态传输失败，保留原请求预算 |
 | `remember` | 用户明确要求时创建或更新记忆 |
 | `forget_memory` | 按精确 ID 删除一条记忆 |
 | `forget_about` | 忘记明确主题；有歧义时只返回候选 |
@@ -529,3 +531,5 @@ memleaf process --vault /path/to/existing/vault --source hermes --session-id SES
 边界在宿主采集、Core capture 和历史 inbox 的新处理处执行。
 已提交记忆不会自动删除。旧版本已冻结但未完成的计划不会被静默丢弃或重新规划，
 不含新来源边界标记的计划会停止恢复并保留现场，避免越过新的输入约定。
+
+历史 HTTP 失败的受控恢复及 CLI 用法见 [失败任务恢复](docs/failed-run-recovery.md)。

{memleaf-0.2.80 → memleaf-0.2.82}/README.en.md

@@ -4,7 +4,7 @@
 
 [中文](README.md) · [PyPI](https://pypi.org/project/memleaf/) · [GitHub](https://github.com/miffyblueboo/memleaf)
 
-> **Version: 0.2.80.**
+> **Version: 0.2.82.**
 > Automatic extraction and review share one future-value standard: the model weighs likely reuse, information gain, direct usability when read again, and the cost of forgetting. It keeps only the smallest core that can materially help future understanding, decisions, or actions; information without clear value is not extracted, and no business-specific exclusion rule is hard-coded.
 > A single Chinese semantic contract is sent for extraction, topic selection, candidate repair, and `semantic_maintenance`; the older English B3 prompt has been deleted and B3 remains only as a compatibility read format. One topic is written once.
 > Core no longer discards a whole candidate over ownership, dates, task basis or cross-project wording. A missing owner normalizes to the single project named by that candidate's own evidence, otherwise `global`. A deadline is accepted only when it appears in the candidate's own evidence and can be anchored to ISO (`明天`, `周五`, `月底` all resolve); when it cannot, only the date field is dropped and the memory is still written. An unauthorized ownership change keeps the target's existing scope.
@@ -148,12 +148,12 @@ Both installation paths automatically:
 4. Activate `memory.provider=memleaf`.
 5. Configure the memleaf MCP entry through Hermes' official CLI.
 6. Configure MCP lazy/idle lifecycle settings.
-7. Verify that the MCP server exposes all 13 tools.
+7. Verify that the MCP server exposes all tools declared by the installed version.
 8. Record the local Agent integration status.
 
 Restart Hermes after installation.
 
-If Hermes cannot be detected, no complete model route can be configured, Provider activation fails, or the 13-tool MCP verification fails, the installer returns an explicit failure rather than reporting an incomplete integration as successful.
+If Hermes cannot be detected, no complete model route can be configured, Provider activation fails, or MCP tool-discovery verification fails, the installer returns an explicit failure rather than reporting an incomplete integration as successful.
 
 The repository `install.sh` remains for source development, offline source installation, and troubleshooting. Normal PyPI users do not need to run it.
 
@@ -250,7 +250,7 @@ python -m memleaf.mcp_server --vault "$HOME/.memleaf"
 
 Without `--vault`, the server uses `~/.memleaf`; `MEMLEAF_VAULT` can also specify the Vault. In normal use the server does not need to be kept running manually: Hermes or Codex starts it on demand. stdout contains only JSON-RPC messages so logs do not corrupt the protocol stream.
 
-The server currently exposes 13 tools:
+The server exposes the following tools:
 
 | Tool | Purpose |
 | --- | --- |
@@ -262,6 +262,7 @@ The server currently exposes 13 tools:
 | `read` | Read a selected memory body in pages using the current `retrieval_id` |
 | `process` | Process complete inbox turns under the admission rules; `background=true` enqueues a job |
 | `process_status` | Read the status of a background processing job |
+| `recover_failed_run` | Preview and explicitly recover terminal transport failures within the original request budget |
 | `remember` | Create or update memory after an explicit request |
 | `forget_memory` | Delete one memory by exact ID |
 | `forget_about` | Forget an unambiguous topic; return candidates when ambiguous |
@@ -516,3 +517,5 @@ The boundary is enforced by host capture, Core capture and new processing of his
 inbox records. Committed memories are not deleted automatically. Frozen operations without
 the conversation-only policy marker are not silently replayed or replanned; they remain
 available for inspection so the new input contract is not bypassed.
+
+Controlled historical HTTP recovery and CLI usage: [failed-run recovery](docs/failed-run-recovery.md).

{memleaf-0.2.80 → memleaf-0.2.82}/README.md

@@ -4,7 +4,7 @@
 
 [English](README.en.md) · [PyPI](https://pypi.org/project/memleaf/) · [GitHub](https://github.com/miffyblueboo/memleaf)
 
-> **版本：0.2.80。**
+> **版本：0.2.82。**
 > 自动提炼和复核共用同一条“未来记忆价值”标准：模型综合未来复用、信息增量、再次读取时的直接可用性和忘记成本，只保留对未来理解、判断或行动有实质影响的最小核心；没有明确价值的信息不提炼，不按具体业务场景硬编码排除。
 > 提炼契约现在只有一份中文语义契约（旧版英文 B3 提示词已删除），首轮、主题选择、候选修复和 `semantic_maintenance` 使用同一套保留价值判断；同一件事只写一次。
 > Core 不再因为归属、日期、任务依据或跨项目而丢弃整条候选：归属缺失会归一到证据里唯一被点名的项目、否则落 `global`；日期只接受能在本候选证据里找到出处、且能锚定成 ISO 的写法（`明天`、`周五`、`月底` 都会被解析），无法解析时只丢掉日期字段，记忆照常写入；未获授权的改归属沿用目标原归属。
@@ -148,12 +148,12 @@ python -m pip install -U memleaf && python -m memleaf install
 4. 激活 `memory.provider=memleaf`；
 5. 通过 Hermes 官方 CLI 配置 memleaf MCP；
 6. 配置 MCP lazy/idle 生命周期；
-7. 验证 MCP Server 能发现 13 个工具；
+7. 验证 MCP Server 能发现当前版本声明的全部工具；
 8. 写入本地 Agent 状态索引。
 
 完成后重启 Hermes。
 
-如果没有检测到 Hermes、无法取得完整模型路由、Provider 激活失败或 MCP 的 13 工具验证失败，安装会明确返回失败，不会把未完成的接入报告为成功。
+如果没有检测到 Hermes、无法取得完整模型路由、Provider 激活失败或 MCP 工具发现验证失败，安装会明确返回失败，不会把未完成的接入报告为成功。
 
 仓库中的 `install.sh` 保留给源码开发、离线源码安装和故障排查；普通 PyPI 用户不需要执行它。
 
@@ -250,7 +250,7 @@ python -m memleaf.mcp_server --vault "$HOME/.memleaf"
 
 不传 `--vault` 时默认使用 `~/.memleaf`；也可以设置 `MEMLEAF_VAULT`。正常情况下不需要手工常驻，Hermes 或 Codex 会按需启动它。stdout 只输出 JSON-RPC，日志不会污染协议通道。
 
-当前提供 13 个工具：
+当前提供以下工具：
 
 | 工具 | 用途 |
 | --- | --- |
@@ -258,9 +258,11 @@ python -m memleaf.mcp_server --vault "$HOME/.memleaf"
 | `context` | 兼容保留的有界轻量目录，不用于自动检索路径 |
 | `scope_catalog` | 返回 Scope、父级和别名，不返回具体记忆正文 |
 | `search` | 返回有界候选目录和 `found`/`no_match` 状态 |
+| `list_todos` | 跨 Scope 按状态和日期筛选并分页列出当前待办 |
 | `read` | 使用当前 `retrieval_id` 分页读取选中的记忆正文 |
 | `process` | 处理完整 inbox 轮次并按准入规则提炼；`background=true` 时异步入队 |
 | `process_status` | 只读查询异步处理 job 状态 |
+| `recover_failed_run` | 预览并显式恢复终态传输失败，保留原请求预算 |
 | `remember` | 用户明确要求时创建或更新记忆 |
 | `forget_memory` | 按精确 ID 删除一条记忆 |
 | `forget_about` | 忘记明确主题；有歧义时只返回候选 |
@@ -510,3 +512,5 @@ memleaf process --vault /path/to/existing/vault --source hermes --session-id SES
 边界在宿主采集、Core capture 和历史 inbox 的新处理处执行。
 已提交记忆不会自动删除。旧版本已冻结但未完成的计划不会被静默丢弃或重新规划，
 不含新来源边界标记的计划会停止恢复并保留现场，避免越过新的输入约定。
+
+历史 HTTP 失败的受控恢复及 CLI 用法见 [失败任务恢复](docs/failed-run-recovery.md)。

memleaf-0.2.82/docs/failed-run-recovery.md

@@ -0,0 +1,56 @@
+# HTTP 失败与历史任务恢复
+
+HTTP 传输层保留经过校验的数字 `http_status`，不保存响应正文、URL、请求头或密钥。
+408、500、502、503、504 可在原预算仍有余额时进入 `retryable`；429 使用现有
+限流处理。400、401、403、404、501 等不因新的 HTTP 分类而获得重试。
+缺少状态码的历史 `model_http_error` 保持终态，普通 `process --recover` 不会重开它。
+
+## 按 run_id 预览及执行
+
+```sh
+# 预览：只读，无模型调用。旧 HTTP 回执必须明确接受其状态码未知。
+memleaf recover-run --vault /path/to/vault --run-id inc-run-... \
+  --allow-legacy-http --json
+
+# 仅当 recoverable=true 时，使用此次预览返回的完整 expected_revision。
+memleaf recover-run --vault /path/to/vault --run-id inc-run-... \
+  --allow-legacy-http --apply --expected-revision PREVIEW_REVISION --json
+```
+
+`--allow-legacy-http` 不放行已知的永久 HTTP 错误，也不增加预算。执行使用当前
+Vault 配置的固定 API 模型路由；预览不解析模型路由、不初始化 Vault、不创建锁文件。
+MCP 对应工具为 `recover_failed_run`，参数为 `run_id`、`allow_legacy_http`、
+`apply`（默认 false）及 `expected_revision`。
+
+Python API：
+
+```python
+preview = service.recover_failed_run(run_id, allow_legacy_http=True)
+if preview["recoverable"]:
+    result = service.recover_failed_run(
+        run_id, dry_run=False, allow_legacy_http=True,
+        expected_revision=preview["expected_revision"],
+    )
+```
+
+## 恢复边界
+
+- 只处理自动捕获任务的已知第一次传输失败，原请求预算为已用 1、尚有余额。
+  显式 remember 的授权正文已清除时，不能借此重新取得保留授权。
+- 原轮必须完整且可处理，来源摘要及原预算身份必须一致。来源被删除、修改、禁录，
+  已有消费记录、自动提交、旧流程待提交计划或其他活动所有者时拒绝恢复。
+- 原 run_id、budget_id、turn_budget_id、commit identity 和第一次尝试记录全部保留。
+  执行在锁内绑定此次预览版本，并记录旧失败、旧来源窗口、旧快照及恢复时间。
+  不修改原额度、不新建 run、不删除旧回执。
+- 原请求正文已清除，因此执行根据当前记忆重新生成快照和请求。完整后续对话仅以
+  `use=context` 参与比较，不独立创建新事项，也不随本轮结算。已有日期、来源及
+  `stale_observation` 校验继续生效；不能证明安全的修改留下未解决项。
+- 预览后状态、预算、来源或比较目标变化，需要重新预览。绑定后的调用前/提交前
+  校验不允许再次悄悄更新快照。
+- 第二次请求仍先预留再派发。崩溃后的未知调用保守计入预算；重复 apply 使用同一
+  恢复绑定，完成后只返回旧回执。保存的回复/提交可在没有模型配置时恢复。
+- compact 回执重开需要一个可用 full 槽位；容量不足时预览明确拒绝，不能删除
+  回执来换取执行机会。
+
+真实模型可能输出 NO_MEMORY、NO_CHANGE、UPDATE、CREATE 或未解决项；入口可执行
+不等于模型语义正确，也不承诺所有历史内容都应保存。

{memleaf-0.2.80 → memleaf-0.2.82}/docs/hermes-mcp-runtime.md

@@ -152,7 +152,7 @@ Vault. Once changes begin, it snapshots:
 - Hermes `memleaf.json`; and
 - the installed `plugins/memleaf` Provider directory.
 
-If MCP persistence, MCP lifecycle configuration, the 13-tool test, Provider
+If MCP persistence, MCP lifecycle configuration, the declared-tool discovery test, Provider
 copy/version validation, Provider activation, or native-source registration
 fails, those Hermes paths are restored to their pre-install state. Each
 snapshot is restored independently: a failure on one path does not prevent the

{memleaf-0.2.80 → memleaf-0.2.82}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "memleaf"
-version = "0.2.80"
+version = "0.2.82"
 description = "A local-first Markdown memory core for AI agents"
 readme = "README.md"
 requires-python = ">=3.11"

{memleaf-0.2.80 → memleaf-0.2.82}/src/memleaf/__init__.py

@@ -1,6 +1,6 @@
 """Local-first Markdown memory core for AI agents."""
 
-__version__ = "0.2.80"
+__version__ = "0.2.82"
 
 from .config import DEFAULT_CONFIG, default_config, load_config, save_config
 from .frontmatter import FrontmatterError, dump_frontmatter, dump_yaml, load_yaml, parse_frontmatter

{memleaf-0.2.80 → memleaf-0.2.82}/src/memleaf/adapters/hermes.py

@@ -24,8 +24,14 @@ from .base import (
 
 
 _MCP_IDLE_TIMEOUT_SECONDS = 60
-MCP_EXPECTED_TOOL_COUNT = 13
-_MCP_EXPECTED_TOOL_COUNT = MCP_EXPECTED_TOOL_COUNT
+
+
+def expected_mcp_tool_count() -> int:
+    """Read the installed server's declarations instead of a second tool inventory."""
+
+    from ..mcp_server import _TOOLS
+
+    return len(_TOOLS)
 
 
 def hermes_home_for_platform(
@@ -418,7 +424,7 @@ class HermesAdapter:
         self,
         detection: Detection | None = None,
         *,
-        expected_tools: int = MCP_EXPECTED_TOOL_COUNT,
+        expected_tools: int | None = None,
     ) -> bool:
         """Run the official MCP test and require the expected tool count."""
 
@@ -426,7 +432,9 @@ class HermesAdapter:
         executable = detection.executable if detection.detected else None
         if detection.confidence != "high" or not executable:
             return False
-        if isinstance(expected_tools, bool) or expected_tools < 0:
+        if expected_tools is None:
+            expected_tools = expected_mcp_tool_count()
+        if not isinstance(expected_tools, int) or isinstance(expected_tools, bool) or expected_tools < 0:
             return False
         try:
             result = run_argv(

{memleaf-0.2.80 → memleaf-0.2.82}/src/memleaf/cli.py

@@ -100,6 +100,13 @@ def build_parser() -> argparse.ArgumentParser:
                          help="allow remaining incremental transport recovery, not partial replan")
     process.add_argument("--dry-run", action="store_true", help="no source Vault writes; may call the configured model")
     process.add_argument("--json", action="store_true")
+    recovery = commands.add_parser("recover-run", help="preview or explicitly replan a failed automatic transport run")
+    recovery.add_argument("--vault", type=Path, default=None)
+    recovery.add_argument("--run-id", required=True)
+    recovery.add_argument("--apply", action="store_true", help="use the original remaining budget; requires preview revision")
+    recovery.add_argument("--expected-revision", default=None)
+    recovery.add_argument("--allow-legacy-http", action="store_true", help="explicitly retry one old HTTP failure whose status was not recorded")
+    recovery.add_argument("--json", action="store_true")
     maintenance = commands.add_parser("maintain-state", help="preview lossless terminal control-state compaction")
     maintenance.add_argument("--vault", type=Path, default=None)
     maintenance.add_argument("--apply", action="store_true", help="apply only to --expected-revision from preview")
@@ -274,6 +281,17 @@ def main(argv: Sequence[str] | None = None) -> int:
                 from .service import Memleaf
                 output = Memleaf(existing_root(args.vault)).process(
                     source=args.source, session_id=args.session_id, scope=args.scope, pipeline=args.pipeline, recover=args.recover)
+        elif args.command == "recover-run":
+            from .inspection import existing_root
+            from .service import Memleaf
+            from .incremental_failed_recovery import FailedRecoveryError
+            service = Memleaf(Vault(existing_root(args.vault), create=False))
+            try:
+                output = service.recover_failed_run(args.run_id, dry_run=not args.apply,
+                    expected_revision=args.expected_revision, allow_legacy_http=args.allow_legacy_http)
+            except FailedRecoveryError as error:
+                print(json.dumps({"run_id": args.run_id, "code": error.code, "status": "rejected"}))
+                return 1
         elif args.command == "maintain-state":
             from .inspection import existing_root
             from .service import Memleaf
@@ -318,15 +336,17 @@ def main(argv: Sequence[str] | None = None) -> int:
                 print("{}")
             return 0
         if getattr(args, "json", False):
-            print(json.dumps({"error": "operation failed" if args.command in {"audit", "process", "maintain-state", "migration-check", "migration-backup"}
+            print(json.dumps({"error": "operation failed" if args.command in {"audit", "process", "recover-run", "maintain-state", "migration-check", "migration-backup"}
                               else "initialization failed", "stage": args.command}, ensure_ascii=False))
         else:
             action = getattr(args, "command", "init")
             print(f"memleaf {action} failed unexpectedly", file=sys.stderr)
         return 1
 
-    if args.command in {"audit", "process", "maintain-state", "migration-check", "migration-backup"}:
+    if args.command in {"audit", "process", "recover-run", "maintain-state", "migration-check", "migration-backup"}:
         print(json.dumps(output, ensure_ascii=False, sort_keys=True, indent=None if args.json else 2))
+        if args.command == "recover-run":
+            return 2 if output.get("recoverable") is False or output.get("execution_status") in {"failed", "blocked", "cancelled"} else 0
         return 2 if args.command == "migration-check" and output["local_status"] != "clear" else 0
     if args.command == "host-event":
         print(json.dumps(output, ensure_ascii=False, separators=(",", ":")))

{memleaf-0.2.80 → memleaf-0.2.82}/src/memleaf/hermes_provider/README.md

@@ -61,7 +61,8 @@ mcp_servers:
 
 Use `python -m memleaf install` to configure both entries. The installer writes
 the MCP entry through `hermes config set`, reads `config.yaml` back, and tests
-that all 13 tools are discoverable before it reports success. When two memleaf
+that all tools declared by the installed MCP server are discoverable before it
+reports success. When two memleaf
 virtual environments are present, use `--mcp-runtime current` to migrate to the
 runtime executing the installer or `--mcp-runtime existing` to retain an
 already configured executable after an exact version check. See the

{memleaf-0.2.80 → memleaf-0.2.82}/src/memleaf/hermes_provider/plugin.yaml

@@ -1,5 +1,5 @@
 name: memleaf
-version: 0.2.80
+version: 0.2.82
 description: "Hermes-native external MemoryProvider for local-first Markdown memory shared with the memleaf MCP server."
 hooks:
   - prefetch

{memleaf-0.2.80 → memleaf-0.2.82}/src/memleaf/incremental_execution.py

@@ -28,7 +28,7 @@ from .incremental_run_state import (
 )
 from .index import turn_key
 from .inbox import captured_turn_selector
-from .llm.base import ModelError
+from .llm.base import ModelError, HTTP_RETRYABLE_STATUSES
 from .locking import atomic_write_json
 from .models import utc_now
 from .process_common import _read_processed
@@ -295,12 +295,14 @@ def _drive(service, run_id, token, backend, calls):
             if ordinal > 1 and not run.get("partial_used"):
                 request["system"] += RETRY_SYSTEM
         error_code = None
+        http_status = None
         response = None
         try:
             calls[0] += 1
             response = backend.complete(request["user"], system=request["system"], purpose="single_pass")
         except ModelError as error:
             error_code = error.code
+            http_status = error.http_status
         except Exception:
             error_code = "model_failed"  # Never persist provider exception text.
         with service.vault.lock():
@@ -309,7 +311,10 @@ def _drive(service, run_id, token, backend, calls):
                 return public_result(run, calls=calls[0])
             if error_code is not None:
                 run["attempts"][-1]["outcome"] = error_code
-                retryable = error_code in TRANSIENT and run["reserved_requests"] < 2
+                if http_status is not None:
+                    run["attempts"][-1]["http_status"] = http_status
+                retryable = (error_code in TRANSIENT or
+                             (error_code == "model_http_error" and http_status in HTTP_RETRYABLE_STATUSES)) and run["reserved_requests"] < 2
                 _finish(service, processed, run, "retryable" if retryable else "failed", error_code)
                 return public_result(run, calls=calls[0])
             if not isinstance(response, str) or not response.strip() or len(response.encode("utf-8")) > MAX_BYTES:

memleaf-0.2.82/src/memleaf/incremental_failed_recovery.py

@@ -0,0 +1,184 @@
+"""Explicit, revision-bound recovery of failed automatic transport runs.
+
+Preview never opens a Vault lock, calls a model or reconstructs persisted
+requests. Apply rebuilds a current request for unchanged source under the same
+run and budget, retaining the original failed attempt and a bounded audit.
+"""
+from __future__ import annotations
+
+import json
+import re
+from typing import Any
+
+from .extraction_work_state import _read_budget_state_unlocked, ExtractionWorkStateError
+from .incremental_commit import _window
+from .incremental_execution import (TRANSIENT, RETRY_SYSTEM, _guard_legacy, _protocol_digest,
+                                    resume_incremental_run)
+from .incremental_journal import digest, load_work
+from .incremental_preview import _prepare_incremental_unlocked
+from .incremental_prompts import INCREMENTAL_SYSTEM
+from .incremental_protocol import MAX_BYTES
+from .incremental_run_state import TERMINAL, MAX_RUNS, COMPACT_VERSION, load_run, owner_live, public_result, save_run
+from .llm.base import HTTP_RETRYABLE_STATUSES
+from .models import utc_now
+from .process_common import _read_processed
+from .recording_policy import recording_allowed
+from .turn_plan import input_digest, turn_identity_key
+from .receipt_codec import is_compact, ledger_usage
+
+
+class FailedRecoveryError(ValueError):
+    """Bounded public rejection, without private provider/source text."""
+
+    def __init__(self, code: str):
+        super().__init__(code)
+        self.code = code
+
+
+def _reject(code: str):
+    raise FailedRecoveryError(code)
+
+
+def _plan(service: Any, run_id: str, *, allow_legacy_http: bool):
+    processed = _read_processed(service.vault.processed_state_path)
+    run = load_run(processed, run_id)
+    if run is None:
+        _reject('incremental_run_not_found')
+    if (is_compact(processed['incremental_runs'][run_id], COMPACT_VERSION)
+            and ledger_usage(processed['incremental_runs'], compact_version=COMPACT_VERSION)['full'] >= MAX_RUNS):
+        _reject('active_run_capacity_unavailable')
+    if turn_identity_key(run['source'], run['session_id'], run['turn_key']) in processed.get('pending_turn_plans', {}):
+        _reject('legacy_pending_plan')
+    if run['status'] != 'failed' or 'terminal_recovery' in run:
+        _reject('terminal_recovery_not_available')
+    if run.get('request_kind', 'automatic') != 'automatic' or run['arguments'].get('selection'):
+        _reject('explicit_retention_requires_new_authorization')
+    if owner_live(processed):
+        _reject('incremental_model_busy')
+    _guard_legacy(service, processed)
+    attempts = run['attempts']
+    # Only a known failed first transport dispatch. Unknown reservations,
+    # responses, partial commits and exhausted attempts cannot be reopened.
+    if (len(attempts) != 1 or attempts[0]['ordinal'] != 1
+            or attempts[0]['outcome'] != run.get('code')
+            or run['reserved_requests'] != 1 or run.get('partial_used')):
+        _reject('terminal_recovery_attempt_not_eligible')
+    code = run.get('code')
+    status = attempts[0].get('http_status')
+    legacy = code == 'model_http_error' and status is None
+    if code == 'model_http_error':
+        if legacy and not allow_legacy_http:
+            _reject('legacy_http_status_unknown')
+        if not legacy and status not in HTTP_RETRYABLE_STATUSES:
+            _reject('http_error_not_retryable')
+    elif code not in TRANSIENT:
+        _reject('failure_not_retryable')
+    if not recording_allowed(processed, run['source'], run['session_id'], run['turn_key']):
+        _reject('source_recording_revoked')
+    if run['commit_work_id'] in processed.get('incremental_commits', {}):
+        _reject('source_already_owned')
+    turn, window = _window(service, run['source'], run['session_id'], run['turn_key'])
+    if not turn.processable or input_digest(turn) != run['source_digest']:
+        _reject('source_changed')
+    from .extraction_work_state import extraction_work_id
+    if extraction_work_id(turn, request_kind='automatic', intent_id='automatic') != run['budget_id']:
+        _reject('source_budget_identity_changed')
+    session = processed.get('sessions', {}).get(f"{run['source']}/{run['session_id']}", {})
+    if any(isinstance(entry, dict) and entry.get('turn_key') == run['turn_key']
+           for entry in session.get('processed_turns', [])):
+        _reject('source_already_processed')
+    for key in processed.get('incremental_commits', {}):
+        work = load_work(processed, key)
+        if (work['source'] == run['source'] and work['session_id'] == run['session_id']
+                and work['turn_key'] == run['turn_key']
+                and work.get('request_kind', 'automatic') == 'automatic'):
+            _reject('source_already_owned')
+    for key in processed.get('incremental_runs', {}):
+        other = load_run(processed, key)
+        if (key != run_id and other['source'] == run['source']
+                and other['session_id'] == run['session_id'] and other['turn_key'] == run['turn_key']
+                and other['status'] not in TERMINAL):
+            _reject('source_already_owned')
+    try:
+        budget = _read_budget_state_unlocked(service.vault)
+    except ExtractionWorkStateError:
+        _reject('remaining_request_budget_unavailable')
+    work_budget = budget['works'].get(run['budget_id'], {})
+    row = work_budget.get('turns', {}).get(run['turn_budget_id'])
+    if (not row or work_budget.get('retired') or row['completed'] or run.get('budget_finalized')
+            or row['requests'] != 1 or row['request_limit_at_creation'] <= row['requests']):
+        _reject('remaining_request_budget_unavailable')
+    snapshot = _prepare_incremental_unlocked(service, **run['arguments'])
+    request = {'system': INCREMENTAL_SYSTEM,
+               'user': json.dumps(snapshot.model_input(), ensure_ascii=False, separators=(',', ':'))}
+    if sum(len(s.encode('utf-8')) for s in request.values()) + len(RETRY_SYSTEM.encode('utf-8')) > MAX_BYTES:
+        _reject('blocked_context')
+    # Preview has no advisory lock. Confirm the independently read planning
+    # state/budget stayed stable; apply repeats this inside the existing lock.
+    if (processed != _read_processed(service.vault.processed_state_path)
+            or budget != _read_budget_state_unlocked(service.vault)):
+        _reject('recovery_preview_changed')
+    revision = digest({'processed': processed, 'budget': budget, 'source_window': window,
+                       'snapshot': snapshot.snapshot_id, 'protocol': _protocol_digest(),
+                       'allow_legacy_http': allow_legacy_http, 'run_id': run_id})
+    result = {'run_id': run_id, 'recoverable': True, 'read_only': True, 'model_calls': 0,
+              'expected_revision': revision, 'remaining_requests': 1,
+              'http_status': status, 'legacy_http_status_unknown': legacy,
+              'source_unchanged': True, 'source_window_changed': window != run['source_window'],
+              'snapshot_changed': snapshot.snapshot_id != run['snapshot_id'],
+              'new_events': sum(e['use'] == 'new' for e in snapshot.state()['evidence']),
+              'context_events': sum(e['use'] == 'context' for e in snapshot.state()['evidence'])}
+    return processed, run, snapshot, window, request, result
+
+
+def recover_failed_run(service: Any, run_id: str, *, dry_run: bool = True,
+                       expected_revision: str | None = None, allow_legacy_http: bool = False,
+                       model: Any = None, router: Any = None) -> dict[str, Any]:
+    if type(dry_run) is not bool or type(allow_legacy_http) is not bool:
+        _reject('invalid_recovery_options')
+    if dry_run:
+        try:
+            return _plan(service, run_id, allow_legacy_http=allow_legacy_http)[-1]
+        except (ValueError, OSError, ExtractionWorkStateError) as error:
+            # Known local validators emit reason codes, never source bodies.
+            code = error.code if isinstance(error, FailedRecoveryError) else 'recovery_preflight_failed'
+            return {'run_id': run_id, 'recoverable': False, 'read_only': True,
+                    'model_calls': 0, 'code': code}
+    if not isinstance(expected_revision, str) or re.fullmatch(r'[0-9a-f]{64}', expected_revision) is None:
+        _reject('expected_recovery_revision_required')
+    from .incremental_runtime import _resolve
+    # A repeated apply is the same original run, not another authorization.
+    with service.vault.lock():
+        stored = load_run(_read_processed(service.vault.processed_state_path), run_id)
+        audit = stored.get('terminal_recovery') if stored else None
+        if audit is not None:
+            if audit['expected_revision'] != expected_revision or audit['allow_legacy_http'] != allow_legacy_http:
+                _reject('recovery_already_bound')
+            if stored['status'] in TERMINAL:
+                return public_result(stored)
+            # Active/resumable same-intent work uses the existing owner guard.
+            already_bound = True
+        else:
+            already_bound = False
+        if not already_bound:
+            processed, run, snapshot, window, request, preview = _plan(
+                service, run_id, allow_legacy_http=allow_legacy_http)
+            if preview['expected_revision'] != expected_revision:
+                _reject('stale_recovery_preview')
+            backend = _resolve(service, model, router)
+            audit = {'version': 1, 'expected_revision': expected_revision,
+                     'allow_legacy_http': allow_legacy_http, 'previous_code': run['code'],
+                     'previous_snapshot_id': run['snapshot_id'], 'previous_source_window': run['source_window'],
+                     'requested_at': utc_now()}
+            run.update(status='ready', code=None, snapshot_id=snapshot.snapshot_id, source_window=window,
+                       request=request, request_digest=digest(request), protocol_digest=_protocol_digest(),
+                       source_keys=[e['event_key'] for e in snapshot.state()['evidence']],
+                       target_ids=[t['memory']['memory_id'] for t in snapshot.state()['targets'].values()],
+                       terminal_recovery=audit)
+            save_run(service, processed, run)
+    if already_bound:
+        # Resolve only if a dispatch is still needed. Saved responses/commits
+        # remain recoverable with zero model calls and no API configuration.
+        backend = (_resolve(service, model, router) if stored['status'] in {'ready', 'retryable', 'dispatching'}
+                   else None)
+    return resume_incremental_run(service, run_id, backend=backend)