dsh-continual-evolve 0.7.0 → 0.7.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +22 -8
- package/README.zh.md +17 -8
- package/lib/apply.js +17 -9
- package/lib/auto.d.ts +24 -5
- package/lib/auto.js +159 -71
- package/lib/command.d.ts +5 -0
- package/lib/command.js +141 -3
- package/lib/evolve-event.d.ts +4 -2
- package/lib/evolve-event.js +12 -2
- package/lib/fate.d.ts +1 -1
- package/lib/fate.js +11 -5
- package/lib/index.d.ts +60 -4
- package/lib/index.js +52 -22
- package/lib/llm-text.d.ts +23 -0
- package/lib/llm-text.js +50 -33
- package/lib/planner.d.ts +17 -0
- package/lib/planner.js +23 -5
- package/lib/prefix-cache.d.ts +78 -0
- package/lib/prefix-cache.js +168 -0
- package/lib/review-scheduler.d.ts +37 -0
- package/lib/review-scheduler.js +110 -0
- package/lib/review.d.ts +14 -1
- package/lib/review.js +23 -2
- package/lib/runtime.d.ts +24 -0
- package/lib/runtime.js +87 -0
- package/lib/service.d.ts +11 -1
- package/lib/service.js +19 -3
- package/lib/store.d.ts +38 -0
- package/lib/store.js +88 -1
- package/lib/token-usage.d.ts +68 -0
- package/lib/token-usage.js +211 -0
- package/lib/tool.d.ts +9 -0
- package/lib/tool.js +37 -6
- package/lib/turn-snapshot.d.ts +58 -0
- package/lib/turn-snapshot.js +188 -0
- package/lib/wrapup-command.js +10 -1
- package/lib/wrapup.d.ts +5 -0
- package/lib/wrapup.js +6 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -7,7 +7,7 @@
|
|
|
7
7
|
[](https://github.com/ZK-Andy/dsh-continual-evolve/actions/workflows/ci.yml)
|
|
8
8
|
[](LICENSE)
|
|
9
9
|
[](package.json)
|
|
10
|
-
[]()
|
|
11
11
|
|
|
12
12
|
Continual self-evolution for [DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness): a versioned, auditable, rollback-safe harness state layer — prompt notes, memories, skills, subagent specs — refined from session trajectories.
|
|
13
13
|
|
|
@@ -25,7 +25,7 @@ Agents accumulate reusable experience (repeated failures, durable facts, reusabl
|
|
|
25
25
|
|
|
26
26
|
## How it works
|
|
27
27
|
|
|
28
|
-
1. **Sediment** — the model creates entries via `evolve_add`, or the automatic review gate proposes them from
|
|
28
|
+
1. **Sediment** — the model creates entries via `evolve_add`, or the automatic review gate proposes them from incremental snapshots captured after successful turns (plus compaction checkpoints).
|
|
29
29
|
2. **Guard** — code-enforced validation: edit schema, blast-radius/scope coherence, and the promotion policy (project-scoped markers, thin content, near-duplicate detection, credential screening keep the global store clean — secrets are rejected at every write sink, including mount materialization). Global creates that near-duplicate an existing entry are rejected at write time (≥0.8 similarity); moderate overlaps carry a `conflictHint` for later consolidation.
|
|
30
30
|
3. **Approve** — global and project writes require explicit human approval; local-fate proposals are consulted before they land.
|
|
31
31
|
4. **Apply & inject** — atomic apply with snapshot + audit event. Prompt notes and delegation specs inject into the system prompt (capped, relevance-ranked, contradicted entries demoted, zero tokens when empty); memories/skills appear as a relevance-ordered capped directory index (`- [memory:type:id] title` hooks, full text one `evolve_list` away).
|
|
@@ -61,11 +61,15 @@ Commands (in-session):
|
|
|
61
61
|
| `/evolve mount · unmount <skillId>` | hot-mount an executable skill as a live plugin |
|
|
62
62
|
| `/evolve goal [objective · done · block]` | round-driven auto-review goal |
|
|
63
63
|
| `/evolve benchmark …` | case lifecycle, runs, acceptance |
|
|
64
|
+
| `/evolve pause · resume · status` | pause/resume the auto-review gate (manual tools keep working), gate state |
|
|
65
|
+
| `/evolve usage` | per-entry injection counts + exact provider-reported tokens for direct review/planner/wrapup/fate calls (benchmark host subagents excluded) |
|
|
64
66
|
|
|
65
|
-
Model tools: `evolve_list / add / update / delete / rollback
|
|
67
|
+
Model tools: `evolve_list / add / update / delete / rollback` (`evolve_delete` takes `id` or a batch `ids` array — one refinement, one approval).
|
|
66
68
|
|
|
67
69
|
For third-party consumers: every applied evolution (gate or manual) appends a structured `evolve_complete` event to `reviews.jsonl` (`src/evolve-event.ts` defines the shape) alongside the human-readable audit records.
|
|
68
70
|
|
|
71
|
+
`/evolve usage` also reads `evolve/token-usage.jsonl`: exact provider-reported input/cache/output/total tokens for the plugin's direct review, planner, manual-wrapup, and automatic-fate calls. The report covers a retained tail rather than lifetime usage, distinguishes missing provider samples, and explicitly excludes host benchmark subagents, their agent-loop calls, and per-entry injection attribution.
|
|
72
|
+
|
|
69
73
|
Injection shape: prompt notes and delegation specs inject with content (≤6/kind × 180 chars, relevance-ranked). Memories and skills appear as a relevance-ordered directory index (`[memory:type:id] title` hooks, capped at 15 lines with a fold counter) — full text via `evolve_list`. Empty store = zero injected tokens.
|
|
70
74
|
|
|
71
75
|
## Configuration
|
|
@@ -73,8 +77,8 @@ Injection shape: prompt notes and delegation specs inject with content (≤6/kin
|
|
|
73
77
|
| Key | Default | Meaning |
|
|
74
78
|
|---|---|---|
|
|
75
79
|
| `baseDir` | resolved DSH home | root for the `evolve/` stores |
|
|
76
|
-
| `autoReview` | `false` |
|
|
77
|
-
| `reviewIntervalTurns` | `6` |
|
|
80
|
+
| `autoReview` | `false` | initial automatic-review default when no runtime switch exists; the listener is always registered |
|
|
81
|
+
| `reviewIntervalTurns` | `6` | legacy local-fate cadence fallback; successful-turn review no longer waits for this interval |
|
|
78
82
|
| `maxReviewInputChars` | `40000` | trajectory slice handed to the gate |
|
|
79
83
|
| `reviewBudgetTokens` | `4096` | output budget for the gate call |
|
|
80
84
|
| `notifyOnAutoReview` | `true` | visible follow-up notice after an applied gate run |
|
|
@@ -92,6 +96,9 @@ Injection shape: prompt notes and delegation specs inject with content (≤6/kin
|
|
|
92
96
|
| `autoRollbackOnReject` | `true` | deterministic rollback after a benchmark rejection |
|
|
93
97
|
| `autoCase` | `true` | failed evolution attempts are captured as draft regression cases (`auto_regression` benchmark) |
|
|
94
98
|
| `reviewModel` | agent's own | optional cheaper model for the gate (`"provider/model"`) |
|
|
99
|
+
| `plannerPrefixCache` | `auto` | Route A session-prefix input when cache evidence exists (`session` always, `off` legacy flat text) |
|
|
100
|
+
| `plannerPrefixMaxChars` | `12000` | session-prefix budget for Route A planning inputs (chars) |
|
|
101
|
+
| `historyRetain` | `{snapshots: 20, refinements: 500, reviews: 500, tokenUsage: 500}` | storage hygiene: snapshots per store, tail lines per store history, shared `reviews.jsonl` tail, and direct-call `token-usage.jsonl` tail |
|
|
95
102
|
|
|
96
103
|
Example profile patch:
|
|
97
104
|
|
|
@@ -102,11 +109,18 @@ Example profile patch:
|
|
|
102
109
|
reviewIntervalTurns: 6
|
|
103
110
|
```
|
|
104
111
|
|
|
112
|
+
The listener is registered even when `autoReview` is `false`. Use `/evolve resume`
|
|
113
|
+
to enable successful-turn snapshots immediately, `/evolve pause` to suppress new
|
|
114
|
+
snapshots and fate/model work, and `/evolve status` to inspect the configured
|
|
115
|
+
default plus the current runtime state. The runtime switch is stored in
|
|
116
|
+
`evolve/runtime.json`; manual `evolve_*` tools and `/evolve` commands are not
|
|
117
|
+
paused.
|
|
118
|
+
|
|
105
119
|
## Development
|
|
106
120
|
|
|
107
121
|
```bash
|
|
108
122
|
pnpm install && pnpm build # deps + tsc -> lib/
|
|
109
|
-
pnpm test # vitest (
|
|
123
|
+
pnpm test # vitest (675 tests)
|
|
110
124
|
pnpm test:coverage # v8 coverage, thresholds enforced in CI
|
|
111
125
|
pnpm lint # oxlint src test
|
|
112
126
|
```
|
|
@@ -114,8 +128,8 @@ pnpm lint # oxlint src test
|
|
|
114
128
|
Project layout:
|
|
115
129
|
|
|
116
130
|
```
|
|
117
|
-
├── src/ # engine, tools, commands, gate, fate, benchmark, usage…
|
|
118
|
-
├── test/ # vitest suites (
|
|
131
|
+
├── src/ # engine, tools, commands, gate, fate, benchmark, injection + token usage…
|
|
132
|
+
├── test/ # vitest suites (43 files)
|
|
119
133
|
├── lib/ # build output (tsc)
|
|
120
134
|
├── docs/
|
|
121
135
|
│ ├── design.md # full design doc (hardening matrix)
|
package/README.zh.md
CHANGED
|
@@ -7,7 +7,7 @@
|
|
|
7
7
|
[](https://github.com/ZK-Andy/dsh-continual-evolve/actions/workflows/ci.yml)
|
|
8
8
|
[](LICENSE)
|
|
9
9
|
[](package.json)
|
|
10
|
-
[]()
|
|
11
11
|
|
|
12
12
|
[DeepSeek Harness](https://github.com/deepseek-ai/deepseek-harness)(`dsh`)的持续自进化插件:一套**版本化、可审计、可回滚**的 harness 状态层——提示词补充、记忆、技能、子代理规格——从会话轨迹中沉淀而来。
|
|
13
13
|
|
|
@@ -25,7 +25,7 @@ Agent 在每个会话里积累可复用经验(重复失败、持久事实、
|
|
|
25
25
|
|
|
26
26
|
## 工作原理
|
|
27
27
|
|
|
28
|
-
1. **沉淀**——模型经 `evolve_add` 创建条目,或自动 review
|
|
28
|
+
1. **沉淀**——模型经 `evolve_add` 创建条目,或自动 review 门禁从成功回合后的增量 snapshot(以及压缩检查点)提议。
|
|
29
29
|
2. **守卫**——代码强制校验:编辑 schema、blast-radius 与作用域一致性、晋升政策(项目专属标记 / 过薄内容 / 近似重复检测 / 凭据筛查保持全局库干净——密钥类内容在所有写入出口被拒,含 mount 物化)。全局 create 与既有条目高度相似(≥0.8)时写入即拒;中等重叠带 `conflictHint` 供后续合并。
|
|
30
30
|
3. **审批**——全局与项目写入需明确人工批准;local 归宿提议先征询后落地。
|
|
31
31
|
4. **应用与注入**——原子应用带快照与审计事件。prompt 补充与委派规格注入系统提示词(封顶、按相关性排序、被证伪条目降权、空 store 零 token);memory/skill 以按相关性排序的目录索引出现(`[memory:type:id] 标题`钩子)。
|
|
@@ -61,11 +61,15 @@ dsh plugin add ZK-Andy/dsh-continual-evolve
|
|
|
61
61
|
| `/evolve mount · unmount <skillId>` | 把可执行技能热挂载为 live 插件 |
|
|
62
62
|
| `/evolve goal [objective · done · block]` | 回合驱动的自进化目标 |
|
|
63
63
|
| `/evolve benchmark …` | 用例生命周期、运行、接受决策 |
|
|
64
|
+
| `/evolve pause · resume · status` | 暂停/恢复自动门禁(手动工具不受影响)、门禁状态 |
|
|
65
|
+
| `/evolve usage` | 每条目注入次数 + review/planner/wrapup/fate 直属调用的 provider 精确 token(不含 benchmark 宿主子代理) |
|
|
64
66
|
|
|
65
|
-
模型工具:`evolve_list / add / update / delete / rollback
|
|
67
|
+
模型工具:`evolve_list / add / update / delete / rollback`(`evolve_delete` 支持 `id` 或批量 `ids` 数组——一次 refinement、一次审批)。
|
|
66
68
|
|
|
67
69
|
第三方消费:每次进化落地(门禁或手动)都会向 `reviews.jsonl` 追加结构化 `evolve_complete` 事件(shape 见 `src/evolve-event.ts`),与人类可读的审计记录并存。
|
|
68
70
|
|
|
71
|
+
`/evolve usage` 还会读取 `evolve/token-usage.jsonl`:插件直属 review、planner、手动 wrapup 与自动 fate 调用的 provider 精确 input/cache/output/total token。报告只覆盖保留尾部而非终身累计,单独显示 provider 未返回 usage 的调用,并明确排除宿主 benchmark 子代理、其 agent-loop 调用与逐条 memory 注入归因。
|
|
72
|
+
|
|
69
73
|
注入形态:prompt 补充与委派规格带内容注入(每 kind ≤6 条 × 180 字符,按相关性排序)。memory/skill 以按相关性排序的目录索引出现(`[memory:type:id] 标题`钩子,15 行封顶 + 折叠计数行)——全文经 `evolve_list` 获取。空 store = 零注入 token。
|
|
70
74
|
|
|
71
75
|
## 配置
|
|
@@ -73,8 +77,8 @@ dsh plugin add ZK-Andy/dsh-continual-evolve
|
|
|
73
77
|
| 键 | 默认 | 含义 |
|
|
74
78
|
|---|---|---|
|
|
75
79
|
| `baseDir` | 解析后的 DSH home | `evolve/` 存储根目录 |
|
|
76
|
-
| `autoReview` | `false` |
|
|
77
|
-
| `reviewIntervalTurns` | `6` |
|
|
80
|
+
| `autoReview` | `false` | 没有 runtime 开关时的自动 review 初始默认;监听器始终注册 |
|
|
81
|
+
| `reviewIntervalTurns` | `6` | local-fate 的兼容节奏;成功回合 review 不再等待这个间隔 |
|
|
78
82
|
| `maxReviewInputChars` | `40000` | 交给门禁的轨迹切片 |
|
|
79
83
|
| `reviewBudgetTokens` | `4096` | 门禁调用输出预算 |
|
|
80
84
|
| `notifyOnAutoReview` | `true` | 门禁应用后发可见跟进通知 |
|
|
@@ -92,6 +96,9 @@ dsh plugin add ZK-Andy/dsh-continual-evolve
|
|
|
92
96
|
| `autoRollbackOnReject` | `true` | benchmark 拒绝后自动确定性回滚 |
|
|
93
97
|
| `autoCase` | `true` | 失败的进化尝试自动沉淀为 draft 回归用例(`auto_regression` 基准) |
|
|
94
98
|
| `reviewModel` | agent 自身 | 门禁可选更便宜的模型(`"provider/model"`) |
|
|
99
|
+
| `plannerPrefixCache` | `auto` | 有缓存证据时用会话前缀输入(`session` 总是前缀,`off` 保持旧扁平文本) |
|
|
100
|
+
| `plannerPrefixMaxChars` | `12000` | Route A 会话前缀预算(字符) |
|
|
101
|
+
| `historyRetain` | `{snapshots: 20, refinements: 500, reviews: 500, tokenUsage: 500}` | 存储卫生:每 store 快照数、每 store 历史尾行、共享 `reviews.jsonl` 尾行、直属调用 `token-usage.jsonl` 尾行 |
|
|
95
102
|
|
|
96
103
|
profile patch 示例:
|
|
97
104
|
|
|
@@ -102,11 +109,13 @@ profile patch 示例:
|
|
|
102
109
|
reviewIntervalTurns: 6
|
|
103
110
|
```
|
|
104
111
|
|
|
112
|
+
即使 `autoReview` 为 `false`,监听器也会注册。使用 `/evolve resume` 立即开启成功回合 snapshot,使用 `/evolve pause` 抑制新 snapshot、fate 和模型调用,使用 `/evolve status` 查看配置默认值与运行时状态。开关保存在 `evolve/runtime.json`;手动 `evolve_*` 工具和 `/evolve` 命令不受暂停影响。
|
|
113
|
+
|
|
105
114
|
## 开发
|
|
106
115
|
|
|
107
116
|
```bash
|
|
108
117
|
pnpm install && pnpm build # 依赖 + tsc -> lib/
|
|
109
|
-
pnpm test # vitest(
|
|
118
|
+
pnpm test # vitest(675 例)
|
|
110
119
|
pnpm test:coverage # v8 覆盖率,CI 强制阈值
|
|
111
120
|
pnpm lint # oxlint src test
|
|
112
121
|
```
|
|
@@ -114,8 +123,8 @@ pnpm lint # oxlint src test
|
|
|
114
123
|
目录结构:
|
|
115
124
|
|
|
116
125
|
```
|
|
117
|
-
├── src/ # 引擎、工具、命令、门禁、fate、benchmark
|
|
118
|
-
├── test/ # vitest 测试套件(
|
|
126
|
+
├── src/ # 引擎、工具、命令、门禁、fate、benchmark、注入 + token 用量…
|
|
127
|
+
├── test/ # vitest 测试套件(43 个文件)
|
|
119
128
|
├── lib/ # 构建产物(tsc)
|
|
120
129
|
├── docs/
|
|
121
130
|
│ ├── design.md # 完整设计文档(硬化矩阵)
|
package/lib/apply.js
CHANGED
|
@@ -7,15 +7,23 @@ export function applyRefinementProposal(state, proposal, options) {
|
|
|
7
7
|
const now = new Date().toISOString();
|
|
8
8
|
for (const edit of proposal.edits) {
|
|
9
9
|
// Store-prefix hygiene: planner edits are written against the MERGED
|
|
10
|
-
// view, where colliding local ids carry a `
|
|
11
|
-
// A CREATE must never bake that view
|
|
12
|
-
// (observed: global entries literally
|
|
13
|
-
//
|
|
14
|
-
//
|
|
15
|
-
//
|
|
16
|
-
//
|
|
17
|
-
|
|
18
|
-
|
|
10
|
+
// view, where colliding project/local ids carry a `project:` /
|
|
11
|
+
// `local:` / `global:` prefix. A CREATE must never bake that view
|
|
12
|
+
// prefix into a permanent id (observed: global entries literally
|
|
13
|
+
// named "local:handoff_todo_…").
|
|
14
|
+
// Non-create edits address existing entries: the literal id wins —
|
|
15
|
+
// legacy stores contain entries whose literal id carries a colon
|
|
16
|
+
// prefix, and stripping unconditionally would resolve them to a
|
|
17
|
+
// sibling that does not exist ("entry not found", forcing file-level
|
|
18
|
+
// surgery). Only when the literal id is absent does the stripped
|
|
19
|
+
// form serve as the merged-view fallback.
|
|
20
|
+
// (Review audit 2026-08-28 S7 origin; literal-first fix 2026-09-24.)
|
|
21
|
+
const strippedId = edit.id?.replace(/^(?:local|project|global):/, "");
|
|
22
|
+
const requestedId = edit.action === "create"
|
|
23
|
+
? strippedId
|
|
24
|
+
: edit.id !== undefined && state.entries[edit.kind]?.[edit.id] !== undefined
|
|
25
|
+
? edit.id
|
|
26
|
+
: edit.id && strippedId;
|
|
19
27
|
const computedId = requestedId ?? (edit.action === "create" ? slug(edit.title ?? edit.kind, edit.kind) : undefined);
|
|
20
28
|
const id = computedId ?? "";
|
|
21
29
|
// Unknown kinds fail per-edit — a malformed proposal must never crash
|
package/lib/auto.d.ts
CHANGED
|
@@ -4,8 +4,12 @@ import type { HarnessState, RefinementEdit, RefinementProposal } from "./types.j
|
|
|
4
4
|
import type { EvolutionEngine } from "./service.js";
|
|
5
5
|
import { type AutoRefineReason } from "./review.js";
|
|
6
6
|
import type { PromotionPolicy } from "./promotion.js";
|
|
7
|
+
import type { PlannerPrefixCacheMode } from "./prefix-cache.js";
|
|
7
8
|
export interface AutoReviewConfig {
|
|
9
|
+
/** Legacy fate cadence; successful turns are no longer gated by this value. */
|
|
8
10
|
intervalTurns: number;
|
|
11
|
+
/** Initial runtime state when no runtime.json exists. */
|
|
12
|
+
enabledByDefault?: boolean;
|
|
9
13
|
maxInputChars: number;
|
|
10
14
|
budgetTokens: number;
|
|
11
15
|
/** Queue a visible follow-up notice after an approved, applied gate run. */
|
|
@@ -17,10 +21,9 @@ export interface AutoReviewConfig {
|
|
|
17
21
|
*/
|
|
18
22
|
localFate: boolean;
|
|
19
23
|
/**
|
|
20
|
-
* Minimum turns between local-fate assessments on the turn
|
|
21
|
-
* (compaction is unconditional). Independent of
|
|
22
|
-
*
|
|
23
|
-
* round.
|
|
24
|
+
* Minimum turns between local-fate assessments on the successful-turn path
|
|
25
|
+
* (compaction is unconditional). Independent of snapshot review cadence so
|
|
26
|
+
* eligible turns do not pay an assessment every round.
|
|
24
27
|
*/
|
|
25
28
|
fateIntervalTurns: number;
|
|
26
29
|
/**
|
|
@@ -29,6 +32,12 @@ export interface AutoReviewConfig {
|
|
|
29
32
|
* When absent, the review gate uses the agent's own provider/model.
|
|
30
33
|
*/
|
|
31
34
|
reviewModel?: string;
|
|
35
|
+
/**
|
|
36
|
+
* Prefix-cache routing for the gate input (see reviewAutoRefine).
|
|
37
|
+
* Absent mode → auto-detect; absent budget → the default prefix budget.
|
|
38
|
+
*/
|
|
39
|
+
prefixCacheMode?: PlannerPrefixCacheMode;
|
|
40
|
+
prefixMaxChars?: number;
|
|
32
41
|
/**
|
|
33
42
|
* Goal-blocked trigger (D3): after this many CONSECUTIVE gate runs that
|
|
34
43
|
* observe the session goal in phase "blocked", run one local-fate
|
|
@@ -52,9 +61,19 @@ export interface AutoReviewConfig {
|
|
|
52
61
|
autoCase: boolean;
|
|
53
62
|
/** Resolved rubric key for the capture's encrypted scaffold rubric. */
|
|
54
63
|
rubricKey?: Buffer;
|
|
64
|
+
/**
|
|
65
|
+
* Storage hygiene (#20): tail lines kept in the shared reviews.jsonl
|
|
66
|
+
* audit trail. Absent → the store default (readers need a recent
|
|
67
|
+
* window, not the full past).
|
|
68
|
+
*/
|
|
69
|
+
reviewsRetain?: number;
|
|
55
70
|
}
|
|
56
71
|
export interface GateState {
|
|
57
72
|
turns: number;
|
|
73
|
+
/** Latest successful turn boundary seen by the listener. */
|
|
74
|
+
completedTurn: number;
|
|
75
|
+
/** Latest turn whose snapshot acquisition was reserved. */
|
|
76
|
+
lastSnapshotTurn: number;
|
|
58
77
|
lastReviewAt: number;
|
|
59
78
|
running: boolean;
|
|
60
79
|
/**
|
|
@@ -85,7 +104,7 @@ export interface ReviewRecord {
|
|
|
85
104
|
sessionId: string;
|
|
86
105
|
reason: AutoRefineReason;
|
|
87
106
|
turnsSinceLastReview: number;
|
|
88
|
-
outcome: "approved" | "declined" | "failed" | "assessed" | "deferred";
|
|
107
|
+
outcome: "approved" | "declined" | "failed" | "assessed" | "deferred" | "skipped" | "armed";
|
|
89
108
|
rationale?: string;
|
|
90
109
|
refinementId?: string;
|
|
91
110
|
}
|
package/lib/auto.js
CHANGED
|
@@ -5,36 +5,42 @@
|
|
|
5
5
|
* fire-and-forget with error containment: an auto-review failure never
|
|
6
6
|
* disturbs the agent loop.
|
|
7
7
|
*
|
|
8
|
-
* Every gate decision (approved / declined / failed) is appended to
|
|
8
|
+
* Every gate decision (approved / declined / failed / skipped) is appended to
|
|
9
9
|
* `<dshHome>/evolve/reviews.jsonl` so auto-review activity is durably
|
|
10
10
|
* auditable — the server console is not a reliable place to look.
|
|
11
11
|
*
|
|
12
12
|
* Hook wiring:
|
|
13
|
-
* - `agent/turn-stopping`
|
|
14
|
-
* - `agent/status` (idle)
|
|
15
|
-
*
|
|
16
|
-
*
|
|
13
|
+
* - `agent/turn-stopping` records the successful turn boundary.
|
|
14
|
+
* - `agent/status` (idle) captures an incremental snapshot and feeds the
|
|
15
|
+
* per-session latest-pending serial scheduler.
|
|
16
|
+
* - `session/event` (compaction/start) forces a snapshot flush before data is
|
|
17
|
+
* summarized away.
|
|
17
18
|
*/
|
|
18
19
|
import { appendFileSync, mkdirSync } from "node:fs";
|
|
19
20
|
import { join } from "node:path";
|
|
20
21
|
import { slug } from "./types.js";
|
|
21
22
|
import { planWithLlm } from "./planner.js";
|
|
22
|
-
import { reviewAutoRefine
|
|
23
|
-
import {
|
|
23
|
+
import { reviewAutoRefine } from "./review.js";
|
|
24
|
+
import { goalServiceOf } from "./goal.js";
|
|
24
25
|
import { notifyAutoReview } from "./notify.js";
|
|
25
26
|
import { runLocalFatePhase } from "./fate.js";
|
|
26
|
-
import { entrySourceOf } from "./source.js";
|
|
27
27
|
import { mergeHarnessStates } from "./state.js";
|
|
28
28
|
import { projectKeyOf } from "./project.js";
|
|
29
29
|
import { questionServiceOf } from "./approval.js";
|
|
30
30
|
import { buildEvolveCompleteEvent, emitEvolveComplete } from "./evolve-event.js";
|
|
31
|
+
import { DEFAULT_REVIEWS_RETAIN, pruneJsonlFile } from "./store.js";
|
|
32
|
+
import { isGateEnabled } from "./runtime.js";
|
|
33
|
+
import { createReviewScheduler } from "./review-scheduler.js";
|
|
34
|
+
import { captureTurnSnapshot } from "./turn-snapshot.js";
|
|
31
35
|
import { captureAutoCase } from "./autocase.js";
|
|
32
36
|
/** Turns a rejected skill candidate stays silent before being offered again. */
|
|
33
37
|
export const SKILL_CONSULT_COOLDOWN_TURNS = 10;
|
|
34
38
|
export function registerAutoReview(ctx, engine, config) {
|
|
35
39
|
const perSession = new Map();
|
|
40
|
+
const schedulers = new Map();
|
|
36
41
|
const logger = ctx.logger("continual-evolve");
|
|
37
42
|
const reviewsPath = join(engine.baseDir, "evolve", "reviews.jsonl");
|
|
43
|
+
const defaultEnabled = config.enabledByDefault ?? true;
|
|
38
44
|
const record = (entry) => {
|
|
39
45
|
try {
|
|
40
46
|
mkdirSync(join(engine.baseDir, "evolve"), { recursive: true });
|
|
@@ -42,11 +48,87 @@ export function registerAutoReview(ctx, engine, config) {
|
|
|
42
48
|
}
|
|
43
49
|
catch (cause) {
|
|
44
50
|
logger.warn(`failed to record auto-review: ${cause instanceof Error ? cause.message : String(cause)}`);
|
|
51
|
+
return;
|
|
52
|
+
}
|
|
53
|
+
// Storage hygiene (#20): bound the audit trail at write time.
|
|
54
|
+
try {
|
|
55
|
+
pruneJsonlFile(reviewsPath, config.reviewsRetain ?? DEFAULT_REVIEWS_RETAIN);
|
|
56
|
+
}
|
|
57
|
+
catch {
|
|
58
|
+
// ignored — the next record retries
|
|
45
59
|
}
|
|
46
60
|
};
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
61
|
+
const schedulerFor = (agent, state) => {
|
|
62
|
+
const existing = schedulers.get(agent.id);
|
|
63
|
+
if (existing)
|
|
64
|
+
return existing;
|
|
65
|
+
const scheduler = createReviewScheduler(async ({ snapshot, signal }) => {
|
|
66
|
+
if (!isGateEnabled(engine.baseDir, defaultEnabled))
|
|
67
|
+
return "aborted";
|
|
68
|
+
if (!snapshot.eligible) {
|
|
69
|
+
record({
|
|
70
|
+
sessionId: snapshot.sessionId,
|
|
71
|
+
reason: snapshot.reason,
|
|
72
|
+
turnsSinceLastReview: state.turns - state.lastReviewAt,
|
|
73
|
+
outcome: "skipped",
|
|
74
|
+
rationale: `snapshot ${snapshot.cursor} skipped: ${snapshot.skipReason ?? "not eligible"}`,
|
|
75
|
+
});
|
|
76
|
+
return "no-op";
|
|
77
|
+
}
|
|
78
|
+
try {
|
|
79
|
+
await runGate(ctx, engine, agent, config, state, snapshot, record, signal);
|
|
80
|
+
return "success";
|
|
81
|
+
}
|
|
82
|
+
catch (cause) {
|
|
83
|
+
const message = cause instanceof Error ? cause.message : String(cause);
|
|
84
|
+
logger.warn(`auto-review failed for ${agent.id}: ${message}`);
|
|
85
|
+
record({
|
|
86
|
+
sessionId: agent.id,
|
|
87
|
+
reason: snapshot.reason,
|
|
88
|
+
turnsSinceLastReview: state.turns - state.lastReviewAt,
|
|
89
|
+
outcome: "failed",
|
|
90
|
+
rationale: `gate error: ${message}`,
|
|
91
|
+
});
|
|
92
|
+
return "error";
|
|
93
|
+
}
|
|
94
|
+
}, (snapshot) => snapshot.cursor);
|
|
95
|
+
schedulers.set(agent.id, scheduler);
|
|
96
|
+
return scheduler;
|
|
97
|
+
};
|
|
98
|
+
const captureAndSchedule = (agent, state, reason) => {
|
|
99
|
+
if (!isGateEnabled(engine.baseDir, defaultEnabled))
|
|
100
|
+
return;
|
|
101
|
+
const turn = state.completedTurn;
|
|
102
|
+
const previousSnapshotTurn = state.lastSnapshotTurn;
|
|
103
|
+
if (reason === "turn_snapshot" && turn <= previousSnapshotTurn)
|
|
104
|
+
return;
|
|
105
|
+
if (reason === "turn_snapshot")
|
|
106
|
+
state.lastSnapshotTurn = turn;
|
|
107
|
+
const cursor = schedulerFor(agent, state).getCursor();
|
|
108
|
+
void captureTurnSnapshot(ctx, agent, {
|
|
109
|
+
turn,
|
|
110
|
+
reason,
|
|
111
|
+
...(cursor !== undefined ? { cursor } : {}),
|
|
112
|
+
maxChars: config.maxInputChars,
|
|
113
|
+
})
|
|
114
|
+
.then((snapshot) => schedulerFor(agent, state).schedule(snapshot))
|
|
115
|
+
.catch((cause) => {
|
|
116
|
+
const message = cause instanceof Error ? cause.message : String(cause);
|
|
117
|
+
logger.warn(`auto-review snapshot failed for ${agent.id}: ${message}`);
|
|
118
|
+
record({
|
|
119
|
+
sessionId: agent.id,
|
|
120
|
+
reason,
|
|
121
|
+
turnsSinceLastReview: state.turns - state.lastReviewAt,
|
|
122
|
+
outcome: "failed",
|
|
123
|
+
rationale: `snapshot error: ${message}`,
|
|
124
|
+
});
|
|
125
|
+
if (reason === "turn_snapshot" && state.lastSnapshotTurn === turn) {
|
|
126
|
+
state.lastSnapshotTurn = previousSnapshotTurn;
|
|
127
|
+
}
|
|
128
|
+
});
|
|
129
|
+
};
|
|
130
|
+
// DSH emits this awaited boundary when a turn is about to close. The idle
|
|
131
|
+
// transition is the safe point at which to read the durable surface.
|
|
50
132
|
ctx.on("agent/turn-stopping", (payload) => {
|
|
51
133
|
const agent = payload.agent;
|
|
52
134
|
if (!agent) {
|
|
@@ -55,36 +137,20 @@ export function registerAutoReview(ctx, engine, config) {
|
|
|
55
137
|
}
|
|
56
138
|
const state = stateFor(perSession, agent.id);
|
|
57
139
|
state.turns += 1;
|
|
140
|
+
if (typeof payload.turn === "number")
|
|
141
|
+
state.completedTurn = Math.max(state.completedTurn, payload.turn);
|
|
58
142
|
});
|
|
59
143
|
ctx.on("agent/status", (payload) => {
|
|
60
144
|
const agent = payload.agent;
|
|
61
145
|
if (!agent || payload.status !== "idle")
|
|
62
146
|
return;
|
|
63
147
|
const state = stateFor(perSession, agent.id);
|
|
64
|
-
|
|
65
|
-
// (the goal's round machine keeps the session continuing); without a
|
|
66
|
-
// goal the plain turn interval applies.
|
|
67
|
-
const goalDriven = goalDrivesRounds(goalServiceOf(ctx)?.get(agent));
|
|
68
|
-
if (!goalDriven && state.turns - state.lastReviewAt < config.intervalTurns)
|
|
148
|
+
if (state.completedTurn <= state.lastSnapshotTurn)
|
|
69
149
|
return;
|
|
70
|
-
|
|
71
|
-
// Every failure is durably recorded — nothing fails silently.
|
|
72
|
-
void runGate(ctx, engine, agent, config, state, "turn_interval", record).catch((cause) => {
|
|
73
|
-
const message = cause instanceof Error ? cause.message : String(cause);
|
|
74
|
-
logger.warn(`auto-review failed for ${agent.id}: ${message}`);
|
|
75
|
-
record({
|
|
76
|
-
sessionId: agent.id,
|
|
77
|
-
reason: "turn_interval",
|
|
78
|
-
turnsSinceLastReview: state.turns - state.lastReviewAt,
|
|
79
|
-
outcome: "failed",
|
|
80
|
-
rationale: `gate error: ${message}`,
|
|
81
|
-
});
|
|
82
|
-
state.lastReviewAt = state.turns; // back off until the interval elapses again
|
|
83
|
-
});
|
|
150
|
+
captureAndSchedule(agent, state, "turn_snapshot");
|
|
84
151
|
});
|
|
85
|
-
// Diagnostic: the armed marker proves
|
|
86
|
-
//
|
|
87
|
-
// trigger events are not reaching this listener.
|
|
152
|
+
// Diagnostic: the armed marker proves the listener was registered even when
|
|
153
|
+
// its runtime default is off, distinguishing "not registered" from "off".
|
|
88
154
|
try {
|
|
89
155
|
mkdirSync(join(engine.baseDir, "evolve"), { recursive: true });
|
|
90
156
|
appendFileSync(reviewsPath, `${JSON.stringify({
|
|
@@ -93,12 +159,26 @@ export function registerAutoReview(ctx, engine, config) {
|
|
|
93
159
|
reason: "boot",
|
|
94
160
|
turnsSinceLastReview: 0,
|
|
95
161
|
outcome: "armed",
|
|
96
|
-
rationale: `auto-review gate registered (
|
|
162
|
+
rationale: `auto-review gate registered (default=${defaultEnabled ? "on" : "off"}; per-success-turn snapshots; local-fate every ${config.fateIntervalTurns} turns)`,
|
|
97
163
|
})}\n`, "utf8");
|
|
98
164
|
}
|
|
99
165
|
catch (cause) {
|
|
100
166
|
logger.warn(`failed to write armed marker: ${cause instanceof Error ? cause.message : String(cause)}`);
|
|
101
167
|
}
|
|
168
|
+
try {
|
|
169
|
+
pruneJsonlFile(reviewsPath, config.reviewsRetain ?? DEFAULT_REVIEWS_RETAIN);
|
|
170
|
+
}
|
|
171
|
+
catch {
|
|
172
|
+
// ignored — the next record retries
|
|
173
|
+
}
|
|
174
|
+
ctx.on("agent/disposed", (payload) => {
|
|
175
|
+
const agent = payload.agent;
|
|
176
|
+
if (!agent)
|
|
177
|
+
return;
|
|
178
|
+
schedulers.get(agent.id)?.shutdown();
|
|
179
|
+
schedulers.delete(agent.id);
|
|
180
|
+
perSession.delete(agent.id);
|
|
181
|
+
});
|
|
102
182
|
ctx.on("session/event", (session, event) => {
|
|
103
183
|
if (event.type !== "compaction/start")
|
|
104
184
|
return;
|
|
@@ -107,17 +187,7 @@ export function registerAutoReview(ctx, engine, config) {
|
|
|
107
187
|
if (!agent)
|
|
108
188
|
return; // no live agent for that session (e.g. cold read)
|
|
109
189
|
const state = stateFor(perSession, agent.id);
|
|
110
|
-
|
|
111
|
-
void runGate(ctx, engine, agent, config, state, "compact", record).catch((cause) => {
|
|
112
|
-
logger.warn(`auto-review failed at compaction for ${agent.id}: ${cause instanceof Error ? cause.message : String(cause)}`);
|
|
113
|
-
record({
|
|
114
|
-
sessionId: agent.id,
|
|
115
|
-
reason: "compact",
|
|
116
|
-
turnsSinceLastReview: state.turns - state.lastReviewAt,
|
|
117
|
-
outcome: "failed",
|
|
118
|
-
rationale: `gate error at compaction: ${cause instanceof Error ? cause.message : String(cause)}`,
|
|
119
|
-
});
|
|
120
|
-
});
|
|
190
|
+
captureAndSchedule(agent, state, "compact");
|
|
121
191
|
});
|
|
122
192
|
}
|
|
123
193
|
/**
|
|
@@ -141,6 +211,8 @@ function stateFor(map, sessionId) {
|
|
|
141
211
|
if (!state) {
|
|
142
212
|
state = {
|
|
143
213
|
turns: 0,
|
|
214
|
+
completedTurn: 0,
|
|
215
|
+
lastSnapshotTurn: 0,
|
|
144
216
|
lastReviewAt: 0,
|
|
145
217
|
running: false,
|
|
146
218
|
skillRejects: new Map(),
|
|
@@ -173,23 +245,22 @@ export function loadGateHarnessView(engine, sessionId, opts) {
|
|
|
173
245
|
* review's baseline fresh — fate re-loads the store and never races the
|
|
174
246
|
* review's optimistic-concurrency checks.
|
|
175
247
|
*/
|
|
176
|
-
async function runGate(ctx, engine, agent, config, state,
|
|
248
|
+
async function runGate(ctx, engine, agent, config, state, snapshot, record, signal) {
|
|
177
249
|
// Reentry guard (review audit 2026-08-28 S3): a gate run holds LLM calls
|
|
178
250
|
// and possibly a user question for a long time; an idle/compaction
|
|
179
251
|
// trigger overlapping the run would start a second concurrent pipeline
|
|
180
252
|
// whose stale whole-file saves clobber the first run's writes.
|
|
181
253
|
if (state.running) {
|
|
182
|
-
|
|
183
|
-
return;
|
|
254
|
+
throw new Error(`previous gate run still in flight for ${agent.id}`);
|
|
184
255
|
}
|
|
185
256
|
state.running = true;
|
|
186
257
|
try {
|
|
187
|
-
await runReviewPhase(ctx, engine, agent, config, state,
|
|
258
|
+
await runReviewPhase(ctx, engine, agent, config, state, snapshot, record, signal);
|
|
188
259
|
// D3: a goal stuck in "blocked" for consecutive gate runs gets one
|
|
189
260
|
// local-fate assessment (the pipeline below), so whatever led the
|
|
190
261
|
// goal astray is distilled before the session moves on.
|
|
191
|
-
await runGoalBlockedFate(ctx, engine, agent, config, state, reason, record);
|
|
192
|
-
await runLocalFatePhase(ctx, engine, agent, config, state, reason, record);
|
|
262
|
+
await runGoalBlockedFate(ctx, engine, agent, config, state, snapshot.reason, record);
|
|
263
|
+
await runLocalFatePhase(ctx, engine, agent, config, state, snapshot.reason, record);
|
|
193
264
|
}
|
|
194
265
|
finally {
|
|
195
266
|
state.running = false;
|
|
@@ -224,18 +295,20 @@ export async function runGoalBlockedFate(ctx, engine, agent, config, state, _rea
|
|
|
224
295
|
logger.info(`auto-review goal-blocked trigger [${agent.id}]: ${config.goalBlockedWrapupTurns} consecutive blocked gate runs → local-fate assessment`);
|
|
225
296
|
await runLocalFatePhase(ctx, engine, agent, config, state, "goal_blocked", record);
|
|
226
297
|
}
|
|
227
|
-
async function runReviewPhase(ctx, engine, agent, config, state,
|
|
298
|
+
async function runReviewPhase(ctx, engine, agent, config, state, snapshot, record, signal) {
|
|
228
299
|
const sessionId = agent.id;
|
|
300
|
+
const reason = snapshot.reason;
|
|
229
301
|
const turnsSinceLastReview = state.turns - state.lastReviewAt;
|
|
230
302
|
const logger = ctx.logger("continual-evolve");
|
|
231
|
-
const
|
|
232
|
-
|
|
233
|
-
|
|
234
|
-
|
|
235
|
-
|
|
303
|
+
const tokenUsage = {
|
|
304
|
+
baseDir: engine.baseDir,
|
|
305
|
+
sessionId,
|
|
306
|
+
retain: engine.retention.tokenUsage,
|
|
307
|
+
onError: (cause) => logger.warn(`token-usage ledger failed for ${sessionId}: ${cause instanceof Error ? cause.message : String(cause)}`),
|
|
308
|
+
};
|
|
309
|
+
const trajectory = snapshot.trajectory;
|
|
236
310
|
if (!trajectory) {
|
|
237
|
-
|
|
238
|
-
return;
|
|
311
|
+
throw new Error("eligible snapshot has no trajectory");
|
|
239
312
|
}
|
|
240
313
|
// The gate judges the merged view (global + local, scopes labeled) so it
|
|
241
314
|
// can recognize topics already covered globally and decline duplicates;
|
|
@@ -251,9 +324,20 @@ async function runReviewPhase(ctx, engine, agent, config, state, reason, record)
|
|
|
251
324
|
state: harnessState,
|
|
252
325
|
history,
|
|
253
326
|
trajectory,
|
|
327
|
+
trajectoryEvents: snapshot.events,
|
|
328
|
+
...(signal ? { signal } : {}),
|
|
254
329
|
context: { reason, turnsSinceLastReview },
|
|
255
330
|
budgetTokens: config.budgetTokens,
|
|
331
|
+
tokenUsage,
|
|
256
332
|
...(reviewRoute ? { overrideProvider: reviewRoute.provider, overrideModel: reviewRoute.model } : {}),
|
|
333
|
+
...((config.prefixCacheMode !== undefined || config.prefixMaxChars !== undefined
|
|
334
|
+
? {
|
|
335
|
+
prefixCache: {
|
|
336
|
+
...(config.prefixCacheMode !== undefined ? { mode: config.prefixCacheMode } : {}),
|
|
337
|
+
...(config.prefixMaxChars !== undefined ? { maxChars: config.prefixMaxChars } : {}),
|
|
338
|
+
},
|
|
339
|
+
}
|
|
340
|
+
: {})),
|
|
257
341
|
});
|
|
258
342
|
state.lastReviewAt = state.turns;
|
|
259
343
|
if (!review.shouldRefine) {
|
|
@@ -265,11 +349,23 @@ async function runReviewPhase(ctx, engine, agent, config, state, reason, record)
|
|
|
265
349
|
agent,
|
|
266
350
|
state: harnessState,
|
|
267
351
|
history,
|
|
352
|
+
trajectory: snapshot.trajectory,
|
|
353
|
+
trajectoryEvents: snapshot.events,
|
|
354
|
+
...(signal ? { signal } : {}),
|
|
268
355
|
...(review.instructions ? { instructions: review.instructions } : {}),
|
|
269
356
|
global: false,
|
|
270
357
|
// Read the skill-creator template facts (fallback: builtin distilled
|
|
271
358
|
// guide) so skill proposals follow the standard.
|
|
272
359
|
skillsRoot: join(engine.baseDir, "skills"),
|
|
360
|
+
tokenUsage,
|
|
361
|
+
...((config.prefixCacheMode !== undefined || config.prefixMaxChars !== undefined
|
|
362
|
+
? {
|
|
363
|
+
prefixCache: {
|
|
364
|
+
...(config.prefixCacheMode !== undefined ? { mode: config.prefixCacheMode } : {}),
|
|
365
|
+
...(config.prefixMaxChars !== undefined ? { maxChars: config.prefixMaxChars } : {}),
|
|
366
|
+
},
|
|
367
|
+
}
|
|
368
|
+
: {})),
|
|
273
369
|
});
|
|
274
370
|
// Skills are governed resources: an auto-created skill is OFFERED to the
|
|
275
371
|
// user for a decision (固化/不固化) before it lands — the gate never
|
|
@@ -308,7 +404,7 @@ async function runReviewPhase(ctx, engine, agent, config, state, reason, record)
|
|
|
308
404
|
record({ sessionId, reason, turnsSinceLastReview, outcome: "declined", rationale: `${review.rationale}${withheld}` });
|
|
309
405
|
return;
|
|
310
406
|
}
|
|
311
|
-
const source =
|
|
407
|
+
const source = { sessionId, ...(snapshot.sourceSeqs.length > 0 ? { seqs: [...snapshot.sourceSeqs] } : {}) };
|
|
312
408
|
const result = engine.apply("local", sessionId, finalProposal, {
|
|
313
409
|
scope: "local",
|
|
314
410
|
baselineState: localState,
|
|
@@ -317,23 +413,15 @@ async function runReviewPhase(ctx, engine, agent, config, state, reason, record)
|
|
|
317
413
|
logger.info(`auto-review approved (${reason}) [${sessionId}] after ${turnsSinceLastReview} turns; auto-refine ${result.id}: ${result.appliedEdits.filter((e) => e.applied).length} applied, ${result.appliedEdits.filter((e) => !e.applied).length} failed — ${review.rationale}`);
|
|
318
414
|
record({ sessionId, reason, turnsSinceLastReview, outcome: "approved", rationale: review.rationale, refinementId: result.id });
|
|
319
415
|
// Gap C4: emit structured evolve_complete event for third-party consumers.
|
|
320
|
-
emitEvolveComplete(engine.baseDir, buildEvolveCompleteEvent(result, `auto_review:${reason}`, sessionId));
|
|
416
|
+
emitEvolveComplete(engine.baseDir, buildEvolveCompleteEvent(result, `auto_review:${reason}`, sessionId), config.reviewsRetain ?? DEFAULT_REVIEWS_RETAIN);
|
|
321
417
|
// Visibility: tell the user what the gate just persisted. Only the
|
|
322
|
-
// turn
|
|
418
|
+
// turn snapshot path notifies — a compaction-triggered gate must not wake
|
|
323
419
|
// the agent mid-compaction — and only when something was actually applied
|
|
324
420
|
// (a notice for zero edits is noise). Failure is contained in notifyAutoReview.
|
|
325
|
-
if (config.notifyOnAutoReview && reason === "
|
|
421
|
+
if (config.notifyOnAutoReview && reason === "turn_snapshot" && result.appliedEdits.some((e) => e.applied)) {
|
|
326
422
|
notifyAutoReview(ctx, agent, result, turnsSinceLastReview);
|
|
327
423
|
}
|
|
328
424
|
}
|
|
329
|
-
async function readTrajectory(ctx, agent, maxChars) {
|
|
330
|
-
const sessionQuery = ctx.sessionQuery;
|
|
331
|
-
if (!sessionQuery) {
|
|
332
|
-
throw new Error("sessionQuery unavailable");
|
|
333
|
-
}
|
|
334
|
-
const snapshot = await sessionQuery.readSurface(agent.id);
|
|
335
|
-
return serializeSurface(snapshot.events, maxChars);
|
|
336
|
-
}
|
|
337
425
|
/**
|
|
338
426
|
* Split a proposal into skill edits and everything else. Skill edits are the
|
|
339
427
|
* governed part: they need explicit user consent before the gate applies
|