stratagate-dsh 0.3.5 → 0.3.7

package/docs/DSH.md

@@ -197,11 +197,13 @@ config:
   # model: deepseek-chat
 ```
 
-Topic Projection has an independent output bound of 32,768 tokens. Advertised model context capacity can lower it after subtracting estimated input and a 1,024-token margin. Other memory tasks retain their existing budget policies. The Host's `defaultMaxTokens` is a default, not a hard capacity declaration; unadvertised provider limits can still fail a request.
+Topic Projection has an independent output bound of 32,768 tokens. Exact-route advertised model context capacity can lower it after subtracting estimated input and a 1,024-token margin; a context that cannot accommodate the input reports `provider-failed`, rather than a content validation error. The Host's `defaultMaxTokens` is an ordinary default and does not cap Topic requests, even when set to 512 or 1,024. No verified model output hard limit is exposed by the current resolved-model metadata; unadvertised provider limits can still fail a request. Other memory tasks retain their existing budget policies.
 
 Topic technical details and task details in `/api/stratagate/overview` distinguish `max-tokens`, `schema-invalid`, `validation-failed`, `timeout`, `provider-failed`, `worker-failed`, and `source-changed`. Diagnostics retain only safe validation rules and numeric/runtime metadata, never raw chat, Event/Topic prose, full model responses, or reasoning. `requested-unverified` means off was requested, not that the provider honored it; observed reasoning is marked separately.
 
-Only explicit `finish=max-tokens` splits a failed batch in half, with at most four levels and three attempts per singleton. Each historical child still consumes the shared Bootstrap allowance, without trimming Event evidence. Other failures keep bounded retry and backoff. After automatic attempts are exhausted, the directory retry action (`POST /api/stratagate/topics/retry`, with namespace, jobId and expectedRevision) or `POST /api/stratagate/jobs/retry?namespace=...&kind=topic-projection&jobId=...` queues a fresh job ID. Both paths enforce source versions, reject old results, and persist across restart. Upgrades never automatically revive historical terminal failures.
+Topic Projection prioritizes complete chapter and section assignments for every batch Event. New overview paragraphs are optional, usually 0–2, with a suggested length of 300–600 Unicode code points and a hard limit of 1,000 per paragraph; text is never truncated to pass validation. An unsaved invalid draft may be omitted on any retry, including returning `overview: []`; valid persisted overviews are inherited while their sections remain. Legacy `overviewRepairRequirements` are read for snapshot compatibility and targeted recovery, never treated as authoritative content or sent to the model. Event coverage, legal sources, versions, leases and final commit validation remain mandatory. Settled failures retain their original historical status but leave current fault counts. The directory, Dashboard and task details include only valid unfinished failures, and manual retry processes only remaining Events.
+
+Only explicit `finish=max-tokens` splits a failed batch in half, with at most four levels and three attempts per singleton. Each historical child still consumes the shared Bootstrap allowance, without trimming Event evidence. Other failures keep bounded retry and backoff. After automatic attempts are exhausted, the directory retry action (`POST /api/stratagate/topics/retry`, with namespace, jobId and expectedRevision) or `POST /api/stratagate/jobs/retry?namespace=...&kind=topic-projection&jobId=...` queues a fresh job ID. Both paths enforce source versions, reject old results, and persist across restart. Recovery Version 2 reuses the existing persistent writer-initialization mechanism to queue one new cycle of at most three task attempts for current terminal Topic failures with unfinished work that match overview validation, legacy draft requirements, Provider failure or the original `invalid-output` signature. Unrelated worker, timeout and content validation failures retain their attempt budgets. This cycle consumes the existing historical allowance. Successful work, stale sources and retired history are excluded. The recovery marker and queued jobs commit together; restarts, refreshes and repeated initialization never reset attempts again. If the new cycle fails, partial progress, error details and manual retry remain available. Only fully validated current work reports `completed`; exhausted outstanding work reports `failed`.
 
 `blockTurnSize` and `blockDecayLambda` are initial fallbacks. Once changed in **Advanced Settings**, persisted UI values take precedence. λ defaults to `0.3`; smaller values forget more slowly and consume more tokens, and values above `0.4` are not recommended.
 

package/docs/DSH.zh-CN.md

@@ -100,6 +100,8 @@ DSH_HOME/stratagate/memory.db
 
 其余行为与普通 Event 一致：agent 记录会投影进 Knowledge Graph，跨会话持久保存，走同样的 turn 衰减与引用强化生命周期（`memory_assess` → `memory_record_use`），也可以通过该生命周期遗忘。检索为两个池各开一条独立的 top-k 通道 —— 被动池与 agent 池分别独立排序，再按加权 RRF 融合，任何一方都无法把另一方挤出结果窗口。agent 通道的占比由 `agentMemoryRetrievalWeight` 配置（默认 `1` = 平权；`0` = 记录保留但不再浮现；更大值提升 agent 记录的排序权重），卡片带 `source: 'agent-recorded'` 标记。记忆面板通过 `/api/stratagate/agent-memories`（查询参数 `session` 与 `includeArchived=true`）展示这些记录。可用 `agentMemoryEnabled: false` 整体关闭该功能，同时注销对应工具。
 
+主动 Event 可直接引用真实开放尾部消息，因此可能没有 `sourceBlockId`；搜索卡片仅在该字段存在时返回它。检索返回的原始结构必须通过 DSH 的无损 JSON 规则后，才登记可评估的证据批次；无效返回不会留下未解决批次。此处理不删除 Event 或改写来源，排序、证据引用和后续采用强化沿用原有流程。
+
 插件注册以下工具：
 
 记忆目录默认提供约 400 词元的导航。`memory_list_topics` 可分类分页或查询主题，`memory_expand_topic` 展开有来源的概览；这两个工具不生成证据批次，也不强化记忆。概览中的事实需用 `memory_search_events` 的 `topic_id` 参数查证，再走证据评估和采用流程。后台从已有事件小批量整理，失败或尚未整理时仍保留事件入口。详见[记忆目录与主题整合](MEMORY_TOPICS.zh-CN.md)。
@@ -172,11 +174,13 @@ config:
   # model: deepseek-chat
 ```
 
-Topic Projection 使用独立的 32,768 token 输出上限；若宿主公布了模型 context window，会扣除估算输入和 1,024 token 余量后进一步降低上限。其他记忆任务继续使用原有预算策略。宿主的 `defaultMaxTokens` 是默认值，不代表模型硬上限；未公布的 provider 限制仍可能导致调用失败。
+Topic Projection 使用独立的 32,768 token 输出上限；若实际模型路由公布了 context window，会扣除估算输入和 1,024 token 余量后进一步降低上限。上下文无法容纳输入时报告 `provider-failed`，不归类成内容校验错误。宿主的 `defaultMaxTokens` 是普通默认值，不限制 Topic 请求，即使它设为 512 或 1,024 也不压低主题投影预算。当前 resolved-model 元数据没有公布经过验证的模型输出硬上限，未知 provider 限制仍可能导致调用失败。其他记忆任务继续使用原有预算策略。
 
 Topic 技术详情及 `/api/stratagate/overview` 的任务详情会区分 `max-tokens`、`schema-invalid`、`validation-failed`、`timeout`、`provider-failed`、`worker-failed` 和 `source-changed`。诊断仅记录安全校验规则、数量、估算 token、请求预算、finish reason、reasoning off 请求/回退状态和尝试次数，不记录原始聊天、Event/Topic 正文、完整模型响应或 reasoning。`requested-unverified` 表示发送了 off 请求，不能证明 provider 实际关闭了 reasoning；若仍观察到 reasoning，会明确标记。
 
-只有明确的 `finish=max-tokens` 才会二分失败批次，最多四层，单条批次最多尝试三次；每个历史子批次仍受共用 Bootstrap 额度约束，不裁剪 Event 证据。其他错误维持原有有限重试与退避。自动尝试耗尽后，可通过目录“重新整理这 N 条”（`POST /api/stratagate/topics/retry`，携带 namespace、jobId 和 expectedRevision），或 `POST /api/stratagate/jobs/retry?namespace=...&kind=topic-projection&jobId=...` 重新排队。两条路径均开启新 jobId 周期，检查来源版本并拒绝旧结果，状态在重启后保留。升级不会自动复活历史终态失败。
+Topic Projection 优先将全部本批 Event 完整分配到章节和小节。新增概要按需生成，通常 0～2 段，建议每段 300～600 个 Unicode 码点，硬上限为 1,000；不通过截断文字满足校验。尚未保存的不合格草稿可在任意重试中舍弃，包括返回 `overview: []`；有效的已保存概要在所属小节仍存在时由存储层继承。旧 `overviewRepairRequirements` 仅兼容读取并用于识别定向恢复对象，不作为权威内容，也不发送给模型。完整 Event 覆盖、合法来源、版本、租约及最终提交校验仍然必须通过。已成功完成的失败记录保留原失败历史并退出当前故障统计；目录、Dashboard 和任务详情都只展示仍有效且存在未完成工作的失败，人工重试只处理剩余 Event。
+
+只有明确的 `finish=max-tokens` 才会二分失败批次，最多四层，单条批次最多尝试三次；每个历史子批次仍受共用 Bootstrap 额度约束，不裁剪 Event 证据。其他错误维持原有有限重试与退避。自动尝试耗尽后，可通过目录“重新整理这 N 条”（`POST /api/stratagate/topics/retry`，携带 namespace、jobId 和 expectedRevision），或 `POST /api/stratagate/jobs/retry?namespace=...&kind=topic-projection&jobId=...` 重新排队。两条路径均开启新 jobId 周期，检查来源版本并拒绝旧结果，状态在重启后保留。Recovery Version 2 复用现有持久化 writer 初始化机制，仅为命中概要校验、旧草稿要求、Provider 失败或原始 `invalid-output` 签名，且当前有效、仍有未完成工作的 Topic 终态失败任务自动开启一次新的最多三次任务尝试，仍消耗已有历史额度。无关 worker、超时及内容校验失败保留原尝试额度。已成功、来源过期及失效的历史记录不重跑；恢复标记与新任务一起提交，重启、刷新和重复初始化不再次清零。本轮再次失败后保留部分完成进度、错误详情和人工重试入口。只有当前应完成工作全部通过校验才返回 `completed`，仅剩终态失败时返回 `failed`。
 
 配置文件中的 `blockTurnSize` 和 `blockDecayLambda` 是初始后备值；一旦在“高级设置”中修改，持久化的界面值优先生效。λ 默认值为 `0.3`；数字越小，记忆遗忘越慢、消耗 token 越多，不建议大于 `0.4`。
 

package/docs/validation/issue119-after.json

@@ -0,0 +1,30 @@
+{
+  "verification": "deterministic mocked streams, not live model generation",
+  "base": "e954eaf2f2cbbef4bb6226a2ebb35191424868c2",
+  "rows": [
+    {
+      "initialCharacters": 656,
+      "retryOverview": "same",
+      "success": true,
+      "calls": 1
+    },
+    {
+      "initialCharacters": 778,
+      "retryOverview": "same",
+      "success": true,
+      "calls": 1
+    },
+    {
+      "initialCharacters": 1000,
+      "retryOverview": "same",
+      "success": true,
+      "calls": 1
+    },
+    {
+      "initialCharacters": 1001,
+      "retryOverview": "empty",
+      "success": true,
+      "calls": 2
+    }
+  ]
+}

package/docs/validation/issue119-before.json

@@ -0,0 +1,94 @@
+{
+  "verification": "deterministic mocked streams, not live model generation",
+  "base": "e954eaf2f2cbbef4bb6226a2ebb35191424868c2",
+  "rows": [
+    {
+      "initialCharacters": 656,
+      "retryOverview": "same",
+      "success": false,
+      "calls": 2,
+      "diagnostics": {
+        "category": "validation-failed",
+        "reason": "overview.text exceeds the character limit",
+        "eventCount": 1,
+        "candidateTopicCount": 1,
+        "estimatedInputTokens": 3769,
+        "requestedOutputTokens": 32768,
+        "maxOutputTokens": 32768,
+        "modelCalls": 2,
+        "actualCharacters": 656,
+        "minCharacters": 1,
+        "maxCharacters": 600,
+        "fieldPath": "topics[0].overview[1].text",
+        "finishReason": "tool-calls",
+        "reasoningOff": "unavailable",
+        "reasoningObserved": false
+      }
+    },
+    {
+      "initialCharacters": 778,
+      "retryOverview": "same",
+      "success": false,
+      "calls": 2,
+      "diagnostics": {
+        "category": "validation-failed",
+        "reason": "overview.text exceeds the character limit",
+        "eventCount": 1,
+        "candidateTopicCount": 1,
+        "estimatedInputTokens": 3769,
+        "requestedOutputTokens": 32768,
+        "maxOutputTokens": 32768,
+        "modelCalls": 2,
+        "actualCharacters": 778,
+        "minCharacters": 1,
+        "maxCharacters": 600,
+        "fieldPath": "topics[0].overview[1].text",
+        "finishReason": "tool-calls",
+        "reasoningOff": "unavailable",
+        "reasoningObserved": false
+      }
+    },
+    {
+      "initialCharacters": 1000,
+      "retryOverview": "same",
+      "success": false,
+      "calls": 2,
+      "diagnostics": {
+        "category": "validation-failed",
+        "reason": "overview.text exceeds the character limit",
+        "eventCount": 1,
+        "candidateTopicCount": 1,
+        "estimatedInputTokens": 3769,
+        "requestedOutputTokens": 32768,
+        "maxOutputTokens": 32768,
+        "modelCalls": 2,
+        "actualCharacters": 1000,
+        "minCharacters": 1,
+        "maxCharacters": 600,
+        "fieldPath": "topics[0].overview[1].text",
+        "finishReason": "tool-calls",
+        "reasoningOff": "unavailable",
+        "reasoningObserved": false
+      }
+    },
+    {
+      "initialCharacters": 1001,
+      "retryOverview": "empty",
+      "success": false,
+      "calls": 2,
+      "diagnostics": {
+        "category": "validation-failed",
+        "reason": "Topic retry must replace invalid overview without removing entries.",
+        "eventCount": 1,
+        "candidateTopicCount": 1,
+        "estimatedInputTokens": 3769,
+        "requestedOutputTokens": 32768,
+        "maxOutputTokens": 32768,
+        "modelCalls": 2,
+        "finishReason": "tool-calls",
+        "reasoningOff": "unavailable",
+        "reasoningObserved": false
+      }
+    }
+  ]
+}

package/docs/validation/issue119.md

@@ -0,0 +1,41 @@
+# Issue #119 修复与验证
+
+基线：`e954eaf2f2cbbef4bb6226a2ebb35191424868c2`（main，stratagate-dsh 0.3.6）。
+
+## 根因与修改
+
+- 日志中的 656、778 字符概要均正常结束于 `tool-calls`，被 600 字符业务上限拒绝，不能归因于 token 截断。现在使用 `TOPIC_OVERVIEW_MAX_CHARACTERS = 1000`，按 Unicode code point 计数，保留原文，不截断。
+- DSH `StringValueSchemaSpec` 只支持类型、枚举、常量和注解，不支持 `maxLength`。工具字段说明与提示词引用同一常量，解析器和存储提交门调用同一校验函数。
+- 旧 `TopicOverviewRepairGuard` 把未保存的违规草稿计数和来源持久化成补写义务，还从历史字段错误推导“至少一段”的要求。删除这套草稿守卫、错误携带和跨批分配逻辑，保留语义去重。旧 `overviewRepairRequirements` 字段仅供快照兼容与识别恢复对象，不发送给模型、不约束提交，重排、失败或成功时清理。
+- 有效旧概要仍由原有存储逻辑继承；章节成员与小节归属、来源权限、批次覆盖、源版本、租约、陈旧响应和原子提交检查继续生效。原有合法重分类、来源失效和遗忘语义不变，旧小节消失时仍按现有规则移除孤立概要。
+- 提示词先要求完整事件归类；新增概要通常 0～2 段，无必要时允许空数组，建议 300～600 字符，不为凑长度省略事实，不复述已有内容。
+- 沿用 `recoveryVersion`，升级为 2。仅恢复仍有效、有未完成工作且命中概要字段、旧草稿要求、Provider 失败或原始 `invalid-output` 签名的终态任务。无关内容校验、超时和 worker 失败不获得新周期。已完成来源、失效来源、已有排队工作、重启和并发写入沿用原有排除与持久化机制；单周期仍最多三次任务尝试，没有新增恢复状态机。
+
+## Provider 预算与剩余证据
+
+DSH `LlmResolvedModelInfo` 公开 `context.contextWindow` 和 `defaultMaxTokens`；后者仅是未显式指定输出预算时使用的普通默认值，不证明模型能力，也不限制 Topic 的独立请求。Topic 保留 32,768 上限，仅按实际 provider/model 路由已知的上下文剩余空间收缩。当前元数据没有可用的已验证模型输出硬上限，不据默认值猜测 Provider 限制。提示中的预算与每次实际请求保持一致，包括内容重试和 reasoning-off 回退。审查后的实现删除了普通默认值限流和对应缓存。
+
+无法容纳输入的模型配置报告 `provider-failed`，不归类成内容校验错误。输出实际截断继续用 `max-tokens` 和现有有界拆批；Provider 异常继续走现有任务退避，无新增专属补丁或重试次数。普通默认值为 512、1024 的路由测试使用 12 张 Event 和两段 1000 字符概要，完整响应的估算 token 超过这些默认值，断言实际请求仍使用独立预算并只调用一次；这是模拟流的请求策略验证，不是低实际预算下的真实生成能力测试。人工重试路径同时保留携带敏感响应的普通 `worker-failed` 脱敏测试。
+
+Issue 中那次 `finishReason=error` 的根因尚未确认。32,768 可能超过部分模型的输出能力，但也可能是限流、路由或其他请求错误。确认仍需要实际 Provider/model 标识、路由配置的 defaultMaxTokens 与 contextWindow、Provider 错误码/HTTP 状态/请求编号，以及 token usage（含 reasoning）和对应失败请求日志。不得凭当前安全摘要断言是哪一项。
+
+## 固定模拟响应对比
+
+在固定 main 与首轮修复提交 `6edcb34445e58e9df1ede0189e118672f90fcb42` 上运行同一临时重放测试，使用 `tests/llm.test.ts` 的 `modelBridge`、`context`、`proposal` 辅助函数；只把新概要 text 替换成 `𠮷.repeat(length)`。第一次与第二次流均正常结束于 `tool-calls`；656、778、1000 的第二次响应重复原响应；1001 的第二次响应返回空概要且保留全部章节/小节归类。其余输入与输出一致。保留这次原始测量；审查修复不改变这些未配置路由默认值的概要校验场景，新的定向测试继续覆盖其行为。
+
+| 首次字符数 | 修复前结果 / 调用数 | 修复后结果 / 调用数 |
+| --- | --- | --- |
+| 656 | 长度校验失败 / 2 | 首次通过 / 1 |
+| 778 | 长度校验失败 / 2 | 首次通过 / 1 |
+| 1000 | 长度校验失败 / 2 | 首次通过 / 1 |
+| 1001 → 空概要 | 草稿守卫拒绝 / 2 | 合法重试通过 / 2 |
+
+这组样例首次通过率 0/4 → 3/4，总调用数 8 → 5，最终完成 0/4 → 4/4。1001 字符始终拒绝，成功来自第二次合法结果。原始输出保存在 [修复前](issue119-before.json) 与 [修复后](issue119-after.json)。这是模拟流经过真实实现的测量，不是线上成功率、真实模型遵从度或提示词效果的测量；未调用真实 Provider，未读取用户数据库。
+
+## 回归范围
+
+定向测试覆盖 656、778、1000 / 1001 与补充平面 Unicode 字符，字段级诊断、Schema 与提示词、草稿舍弃、完整 Event 分类、旧概要和来源继承、伪造来源和遗漏 Event 拒绝、SQLite 重启、旧恢复要求、部分完成与重叠任务、重复提交、并发初始化、失败后有界拆批，以及 Provider 预算配置与分类。
+
+首轮修复提交在 Node 24.19.0 下：`npm run check` 全部通过；`npm test` 的 core 286、DSH 580、WorkBuddy 10 项通过，共 876 项通过、1 项既有跳过；`npm run build` 的 core、DSH 和 WorkBuddy 全部通过。隔离安装最初跳过安装脚本，导致旧数据库迁移测试缺少 better-sqlite3 绑定；用相同 Node 24 执行 `npm rebuild better-sqlite3` 后，完整测试通过。未修改依赖或锁文件。
+
+审查修复后的本地验证：`tests/llm.test.ts` 与 `tests/memory-topics.test.ts` 共 194 项通过，`packages/core/tests/topic-recovery.test.ts` 25 项通过，共 219 项定向测试通过；`npm run check` 和 `npm run build` 全部通过。已自查实际预算反馈、内容重试与 reasoning-off 回退、上下文预检分类、持久化来源/归属/旧概要保护和有界恢复路径。新提交仍需独立复审及其对应的 CI 结果；未执行用户真实 Windows + DSH 数据库的升级验收。

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "stratagate-dsh",
-  "version": "0.3.5",
+  "version": "0.3.7",
   "description": "Recent conversations stay vivid. Older ones fade into summaries, not oblivion. StrataGate gives DeepSeek Harness six-layer, time-decaying memory, while lasting events and relationships settle into a knowledge graph. Bring your memories from other AIs with you—no need to start over.",
   "type": "module",
   "main": "./dist/index.js",