@deepseek-ai/dsh-llm-replay 0.1.3-alpha.2 → 0.1.5-alpha.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.i18n.yaml +2 -2
- package/README.md +5 -5
- package/README.zh.md +6 -6
- package/lib/index.js +20 -147
- package/lib/types/index.d.ts +5 -10
- package/package.json +11 -11
package/README.i18n.yaml
CHANGED
|
@@ -2,5 +2,5 @@
|
|
|
2
2
|
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
|
3
3
|
# after editing either side, bring the other along and re-record with:
|
|
4
4
|
# pnpm run verify-translation-pairing --write packages/test-support/llm-replay/README.md
|
|
5
|
-
README.md:
|
|
6
|
-
README.zh.md:
|
|
5
|
+
README.md: 5877db20babb07523e870d6f3041e1c35f076130
|
|
6
|
+
README.zh.md: 17a0815591051ea9dc6a4d1aa847d970bbb87c16
|
package/README.md
CHANGED
|
@@ -9,7 +9,7 @@ English | [中文](README.zh.md)
|
|
|
9
9
|
|
|
10
10
|
## Summary
|
|
11
11
|
|
|
12
|
-
`dsh-llm-replay`
|
|
12
|
+
`dsh-llm-replay` lets snapshot tests run the real agent without an API key by replaying model streams from recorded Session JSONL fixtures. Each parent and subagent session receives its recorded script in first-call order, while calls within a session advance independently. A `replay.override.json` sidecar represents pre-chunk failures, cancellation, hangs, and injected retries that durable settlements cannot reconstruct. Use it for deterministic ACP, headless, and Web browser scenarios that need real loop behavior with fixed model output.
|
|
13
13
|
|
|
14
14
|
## Table of Contents
|
|
15
15
|
|
|
@@ -58,14 +58,14 @@ With `providers` configured, the plugin registers a replay-only adapter whose ca
|
|
|
58
58
|
| `file` | `$DSH_SNAPSHOT_FILE` | Path to the selected primary fixture: `session.jsonl` for v0 or `session.vN.jsonl` for a positive generation; required (config or env) |
|
|
59
59
|
| `overrideFile` | `$DSH_SNAPSHOT_OVERRIDE` | Optional `ReplayOverrideDoc` sidecar for the primary session |
|
|
60
60
|
| `childFiles` | `$DSH_SNAPSHOT_CHILD_FILES` | Recorded subagent child-session logs for a nested scenario |
|
|
61
|
-
| `providers` | — | Optional replay-only provider and model catalog; a model may declare `contextWindow`, text/image modalities,
|
|
61
|
+
| `providers` | — | Optional replay-only provider and model catalog; a model may declare `contextWindow`, text/image modalities, positive `imageRequestTokens` when image-capable, and `systemPromptUpdate: in-history` so a keyless scenario exercises in-history system prompt replacement; invalid values fail at load (`llm-replay: provider "…" model "…" systemPromptUpdate must be "in-history" when present`) and routes never perform provider I/O |
|
|
62
62
|
| `paceMs` | — (burst) | Optional per-chunk delay in ms for genuinely incremental delivery |
|
|
63
63
|
|
|
64
64
|
The generated [configuration catalog](../../../docs/config-catalog.md#deepseek-aidsh-llm-replay) is the exhaustive source for every accepted field and its JSDoc.
|
|
65
65
|
|
|
66
66
|
### How the fixture works
|
|
67
67
|
|
|
68
|
-
The fixture is a projection of one selected persisted Session generation produced by running the real agent once — this plugin does not record. The snapshot harness supplies the numerically highest canonical parent path (`<scenario>/session.jsonl` for v0 or `<scenario>/session.vN.jsonl` for a positive generation), and validates filename/header agreement before replay. The fixture keeps the header and every event payload but omits body `seq`/`time` envelopes (`seq0`/`time0` for historical packed rows). Replay supplies contiguous sequences and deterministic timestamps, restores typed values replaced by snapshot tokens, rejects partial or mixed envelopes, decodes the complete physical artifact through the build-static Session format catalog, and migrates historical input in memory before it exposes events or the inherited cut; current input takes direct restoration. For a projected v0 header only, an absent `delegationDepth` denotes `0`. The parser never rewrites or renames the fixture. Runtime persistence continues to write complete logs. Replay expands the compact stream on each current-view `assistant/message` or `assistant/attempt`, so a recorded fixture replays the same logical stream the live model produced. A fixture may carry its `request/header` content tokenized to `{{system}}`/`{{tools}}`; replay materializes validation-only values, while derivation reads only Assistant settlements, marked summary events, and Session metadata. Every replay and comparison fixture must pass the same content-only catalog validation; replay never repairs a refused artifact.
|
|
68
|
+
The fixture is a projection of one selected persisted Session generation produced by running the real agent once — this plugin does not record. The snapshot harness supplies the numerically highest canonical parent path (`<scenario>/session.jsonl` for v0 or `<scenario>/session.vN.jsonl` for a positive generation), and validates filename/header agreement before replay. The fixture keeps the header and every event payload but omits body `seq`/`time` envelopes (`seq0`/`time0` for historical packed rows). Replay supplies contiguous sequences and deterministic timestamps, restores typed values replaced by snapshot tokens, rejects partial or mixed envelopes, decodes the complete physical artifact through the build-static Session format catalog, and migrates historical input in memory before it exposes events or the inherited cut; current input takes direct restoration. For a projected v0 header only, an absent `delegationDepth` denotes `0`. The parser never rewrites or renames the fixture. Runtime persistence continues to write complete logs. Replay expands the compact stream on each current-view `assistant/message` or `assistant/attempt`, so a recorded fixture replays the same logical stream the live model produced. A fixture may carry its `request/header` content tokenized to `{{system}}`/`{{tools}}`; replay materializes validation-only values, while derivation reads only Assistant settlements, marked summary events, and Session metadata. Every replay and comparison fixture must pass the same content-only catalog validation; replay never repairs a refused artifact. Comparison encoding preserves accepted catalog output, including extension request-header fields; current `header.system` is rejected. Wire-notification expected outputs compare directly with current-writer output, retaining event order, inserted system messages, wrapper fields, and opaque delivery and captured-generation values; only complete Session artifacts use the format migration catalog.
|
|
69
69
|
|
|
70
70
|
### Nested agents
|
|
71
71
|
|
|
@@ -97,14 +97,14 @@ This section explains the design of the replay plugin; the observable behavior i
|
|
|
97
97
|
|
|
98
98
|
Replay treats the selected projected Session generation as the fixture. One parser completes projected envelopes, validates and migrates the whole artifact through `sessionFormatCatalog`, and returns the current header, inherited cut, and event list as one result. `deriveReplayScript` expands each `assistant/message` or `assistant/attempt` stream in log order, so each durable settlement becomes one `chunks` entry; a non-empty stream without a `finish` chunk is the fingerprint of a thrown `stream()` and must be expressed through an override sidecar. A `compaction/summary` carrying `llmStreamCall: true` and a complete `rawOutput` replays as one canonical successful stream at that event's position. Scripted strings may embed `{{fromRequest:<regex>}}`; at stream time each placeholder resolves against the live request's string leaves, taking the pattern's last match and its first capture group (or the whole match) in place.
|
|
99
99
|
|
|
100
|
-
The committed-corpus test
|
|
100
|
+
The [committed-corpus test](tests/session-format-corpus.spec.ts) restores each versioned `session*.jsonl` under `snapshots/`, `packages/`, and `scripts/snapshots/python-sdk-single-exe/` through the real catalog without changing source bytes. Its [inventory](tests/session-format-corpus-inventory.ts) pins deliberate historical refusals by path, source generation, error type, and exact reason; a refusal that disappears or changes fails. Current-generation artifacts cannot receive an exception. Headerless snapshot-harness protocol examples have a separate explicit exemption. All other restoration errors fail with the artifact path; historical files remain unchanged, and native current fixtures require owner correction.
|
|
101
101
|
|
|
102
102
|
### Source map
|
|
103
103
|
|
|
104
104
|
| File | Role |
|
|
105
105
|
|---|---|
|
|
106
106
|
| [`src/index.ts`](src/index.ts) | Types, fixture derivation, override validation, placeholder resolution, session binding, `installLlmReplay`, and the plugin export |
|
|
107
|
-
| [`tests/session-format-corpus.spec.ts`](tests/session-format-corpus.spec.ts) |
|
|
107
|
+
| [`tests/session-format-corpus.spec.ts`](tests/session-format-corpus.spec.ts) | Committed-generation restoration and exact historical refusal checks |
|
|
108
108
|
| — | No runtime invariant companion is published; this test-only adapter consumes a fixed replay script; its stream grammar is checked by the LLM companion and fixture derivation tests. |
|
|
109
109
|
|
|
110
110
|
### Binding and stream flow
|
package/README.zh.md
CHANGED
|
@@ -9,7 +9,7 @@ kind: "package-reference"
|
|
|
9
9
|
|
|
10
10
|
## 概述
|
|
11
11
|
|
|
12
|
-
`dsh-llm-replay`
|
|
12
|
+
`dsh-llm-replay` 从已记录的 Session JSONL fixture(测试前置数据)回放模型流,让快照测试无需 API 密钥即可运行真实 agent(智能体)。每个 parent 与 subagent 会话按首次调用顺序取得各自的已记录脚本,而同一会话内的调用会独立推进。`replay.override.json` 伴随文件表示持久 settlement 无法重建的分片前失败、取消、挂起与注入重试。需要以固定模型输出确定性测试真实 loop 行为时,可在 ACP、headless 与 Web 浏览器场景中使用本包。
|
|
13
13
|
|
|
14
14
|
## 目录
|
|
15
15
|
|
|
@@ -58,14 +58,14 @@ kind: "package-reference"
|
|
|
58
58
|
| `file` | `$DSH_SNAPSHOT_FILE` | 选定 primary fixture 路径:v0 为 `session.jsonl`,正 generation 为 `session.vN.jsonl`;必需(config 或 env) |
|
|
59
59
|
| `overrideFile` | `$DSH_SNAPSHOT_OVERRIDE` | 主会话的可选 `ReplayOverrideDoc` 伴随文件 |
|
|
60
60
|
| `childFiles` | `$DSH_SNAPSHOT_CHILD_FILES` | 嵌套场景中已记录的 subagent 子会话日志 |
|
|
61
|
-
| `providers` | 无 | 可选的仅回放提供方与模型目录;模型可声明 `contextWindow
|
|
61
|
+
| `providers` | 无 | 可选的仅回放提供方与模型目录;模型可声明 `contextWindow`、文本/图片模态、图片模型使用的正整数 `imageRequestTokens`,以及让无密钥场景演练历史内系统提示词替换的 `systemPromptUpdate: in-history`;非法值会在加载时失败(`llm-replay: provider "…" model "…" systemPromptUpdate must be "in-history" when present`),路由绝不执行提供方 I/O |
|
|
62
62
|
| `paceMs` | 无(突发) | 可选的每分片延迟(毫秒),用于真正的增量投递 |
|
|
63
63
|
|
|
64
64
|
生成的[配置目录](../../../docs/config-catalog.zh.md#deepseek-aidsh-llm-replay)是每个受支持字段及其 JSDoc 的穷尽式真源。
|
|
65
65
|
|
|
66
66
|
### fixture 的工作方式
|
|
67
67
|
|
|
68
|
-
fixture 是运行一次真实 agent 所产生的一份选定持久化 Session generation 投影,本插件不录制。snapshot harness 会提供数值最高的规范 parent 路径(v0 为 `<scenario>/session.jsonl`,正 generation 为 `<scenario>/session.vN.jsonl`),并在 replay 前校验文件名与 header 一致。fixture 保留 header 与每个事件 payload,但省略正文的 `seq`/`time` envelope(历史 packed row 使用 `seq0`/`time0`)。replay 补充连续序号与确定性 timestamp,恢复被 snapshot token 替换的类型化值,拒绝不完整或混合 envelope,通过构建期静态 Session 格式 catalog 解码完整物理产物,并在公开事件或继承 cut 前于内存中迁移历史输入;当前输入直接 restore。仅对投影 v0 header,缺失的 `delegationDepth` 表示 `0`。parser 从不重写或重命名 fixture。runtime persistence 继续写入完整日志。replay 会展开当前视图中每个 `assistant/message` 或 `assistant/attempt` 的紧凑 stream,因此已记录 fixture 会 replay 与在线模型产生的相同逻辑流。fixture 的 `request/header` 内容可能 token 化为 `{{system}}`/`{{tools}}`;replay 会物化仅用于校验的值,而派生只读取 Assistant settlement、带标记的 summary 事件与 Session metadata。每个 replay 与 comparison fixture 都必须通过同一个只基于内容的 catalog 校验;replay
|
|
68
|
+
fixture 是运行一次真实 agent 所产生的一份选定持久化 Session generation 投影,本插件不录制。snapshot harness 会提供数值最高的规范 parent 路径(v0 为 `<scenario>/session.jsonl`,正 generation 为 `<scenario>/session.vN.jsonl`),并在 replay 前校验文件名与 header 一致。fixture 保留 header 与每个事件 payload,但省略正文的 `seq`/`time` envelope(历史 packed row 使用 `seq0`/`time0`)。replay 补充连续序号与确定性 timestamp,恢复被 snapshot token 替换的类型化值,拒绝不完整或混合 envelope,通过构建期静态 Session 格式 catalog 解码完整物理产物,并在公开事件或继承 cut 前于内存中迁移历史输入;当前输入直接 restore。仅对投影 v0 header,缺失的 `delegationDepth` 表示 `0`。parser 从不重写或重命名 fixture。runtime persistence 继续写入完整日志。replay 会展开当前视图中每个 `assistant/message` 或 `assistant/attempt` 的紧凑 stream,因此已记录 fixture 会 replay 与在线模型产生的相同逻辑流。fixture 的 `request/header` 内容可能 token 化为 `{{system}}`/`{{tools}}`;replay 会物化仅用于校验的值,而派生只读取 Assistant settlement、带标记的 summary 事件与 Session metadata。每个 replay 与 comparison fixture 都必须通过同一个只基于内容的 catalog 校验;replay 绝不修复被拒绝的产物。比较编码保留已接受的 catalog 输出,包括扩展 request-header 字段;当前版本的 `header.system` 会被拒绝。协议通知的预期输出直接与当前写入器输出比较,保留事件顺序、插入的 system 消息、封装字段,以及不透明的交付和捕获代际值;只有完整 Session 产物使用格式迁移 catalog。
|
|
69
69
|
|
|
70
70
|
### 嵌套 agent
|
|
71
71
|
|
|
@@ -97,15 +97,15 @@ parent agent 委托给进程内 subagent 的场景会为每个 Session 记录一
|
|
|
97
97
|
|
|
98
98
|
replay 把选定的投影 Session generation 视为 fixture。一个 parser 补全投影 envelope,通过 `sessionFormatCatalog` 校验并迁移完整产物,再以一个结果返回当前 header、继承 cut 与事件列表。`deriveReplayScript` 按日志顺序展开每个 `assistant/message` 或 `assistant/attempt` stream,因此每个持久 settlement 都成为一条 `chunks` entry;非空 stream 缺少 `finish` chunk 是 `stream()` 抛出异常的 fingerprint,必须通过 override sidecar 表达。携带 `llmStreamCall: true` 与完整 `rawOutput` 的 `compaction/summary` 会在该事件位置 replay 为一条规范成功 stream。脚本字符串可以内嵌 `{{fromRequest:<regex>}}`;stream 输出时每个 placeholder 针对 live request 的 string leaf 解析,取该 pattern 的最后一次 match,用其第一个 capture group(无 capture group 时用整个 match)原位替换。
|
|
99
99
|
|
|
100
|
-
|
|
100
|
+
[已提交语料测试](tests/session-format-corpus.spec.ts) 通过真实 catalog 还原 `snapshots/`、`packages/` 与 `scripts/snapshots/python-sdk-single-exe/` 下每个带版本的 `session*.jsonl`,且不改变源字节。其[清单](tests/session-format-corpus-inventory.ts) 按路径、源代际、错误类型与精确原因固定有意拒绝的历史转换;拒绝消失或变化都会使测试失败。当前代际产物不能获得例外。没有版本 header 的快照框架协议示例具有独立的显式豁免。其他所有还原错误都携带产物路径并使测试失败;历史文件保持不变,原生当前 fixture 则由 owner 修正。
|
|
101
101
|
|
|
102
102
|
### 源码地图
|
|
103
103
|
|
|
104
104
|
| 文件 | 职责 |
|
|
105
105
|
|---|---|
|
|
106
106
|
| [`src/index.ts`](src/index.ts) | 类型、fixture 派生、override 校验、占位符解析、会话绑定、`installLlmReplay` 与插件导出 |
|
|
107
|
-
| [`tests/session-format-corpus.spec.ts`](tests/session-format-corpus.spec.ts) |
|
|
108
|
-
| — |
|
|
107
|
+
| [`tests/session-format-corpus.spec.ts`](tests/session-format-corpus.spec.ts) | 已提交代际还原与精确历史拒绝检查 |
|
|
108
|
+
| — | 不发布运行时不变式伴生入口;该仅测试适配器消费固定的回放脚本;其流语法由 LLM 伴生插件与 fixture 派生测试检验。 |
|
|
109
109
|
|
|
110
110
|
### 绑定与流式流程
|
|
111
111
|
|
package/lib/index.js
CHANGED
|
@@ -7,7 +7,7 @@ import { assertNever } from "@deepseek-ai/dsh-util-values";
|
|
|
7
7
|
//#region lib/types/index.js
|
|
8
8
|
/**
|
|
9
9
|
* Keyless snapshot-test LLM replay. It derives one model-call script per
|
|
10
|
-
* recorded session from
|
|
10
|
+
* recorded session from v3 embedded Assistant streams and explicitly marked local
|
|
11
11
|
* compaction calls, then binds fresh live sessions to parent/child scripts by
|
|
12
12
|
* first-call order. Throw and hang cases require an explicit override because
|
|
13
13
|
* a session log cannot reconstruct them alone.
|
|
@@ -107,46 +107,15 @@ function parsedSessionFixture(artifact, sourceHeader) {
|
|
|
107
107
|
}
|
|
108
108
|
/**
|
|
109
109
|
* Convert one persisted or projected snapshot fixture to the current physical format in memory for expected-output comparison.
|
|
110
|
-
* Projected cwd tokens remain tokens
|
|
110
|
+
* Projected cwd and request-tool tokens remain tokens for comparison with a fresh run.
|
|
111
111
|
* @param text - one complete Session fixture.
|
|
112
112
|
* @returns current-format JSONL with complete event envelopes; the input string and source file remain unchanged.
|
|
113
113
|
*/
|
|
114
114
|
function prepareSessionSnapshotFixtureForComparison(text) {
|
|
115
115
|
return encodeCurrentSessionSnapshotFixture(text, parseSessionFixture(text));
|
|
116
116
|
}
|
|
117
|
-
/** Read one headless or SDK event-notification wrapper. */
|
|
118
|
-
function wrappedSessionEvent(row) {
|
|
119
|
-
if (row["type"] === "session_event" && typeof row["sessionId"] === "string" && row["event"] !== null && typeof row["event"] === "object" && !Array.isArray(row["event"])) return {
|
|
120
|
-
sessionId: row["sessionId"],
|
|
121
|
-
event: row["event"],
|
|
122
|
-
replace: (event) => ({
|
|
123
|
-
...row,
|
|
124
|
-
event
|
|
125
|
-
})
|
|
126
|
-
};
|
|
127
|
-
const params = row["params"];
|
|
128
|
-
if (row["method"] !== "session.event" || params === null || typeof params !== "object" || Array.isArray(params)) return;
|
|
129
|
-
const record = params;
|
|
130
|
-
if (typeof record["sessionId"] !== "string" || record["event"] === null || typeof record["event"] !== "object" || Array.isArray(record["event"])) return;
|
|
131
|
-
return {
|
|
132
|
-
sessionId: record["sessionId"],
|
|
133
|
-
event: record["event"],
|
|
134
|
-
replace: (event) => ({
|
|
135
|
-
...row,
|
|
136
|
-
params: {
|
|
137
|
-
...record,
|
|
138
|
-
event
|
|
139
|
-
}
|
|
140
|
-
})
|
|
141
|
-
};
|
|
142
|
-
}
|
|
143
|
-
/** Append one migrated event to the output assigned to its v1 source row. */
|
|
144
|
-
function assignMigratedEvent(assigned, rowIndex, event) {
|
|
145
|
-
assigned.set(rowIndex, [...assigned.get(rowIndex) ?? [], event]);
|
|
146
|
-
}
|
|
147
117
|
/** Restore fixture tokens materialized only to satisfy released-format validation. */
|
|
148
118
|
function restoreProjectedRequestHeader(target, source) {
|
|
149
|
-
if (target["type"] !== "request/header" || source["type"] !== "request/header") return target;
|
|
150
119
|
const targetData = target["data"];
|
|
151
120
|
const sourceData = source["data"];
|
|
152
121
|
const targetHeader = targetData["header"];
|
|
@@ -163,118 +132,22 @@ function restoreProjectedRequestHeader(target, source) {
|
|
|
163
132
|
}
|
|
164
133
|
};
|
|
165
134
|
}
|
|
166
|
-
/**
|
|
167
|
-
function normalizeWrappedEventProvenance(event) {
|
|
168
|
-
if (event["type"] !== "session-log-deepseek/delivery-accepted") return event;
|
|
169
|
-
const data = event["data"];
|
|
170
|
-
if (data === null || typeof data !== "object" || Array.isArray(data)) return event;
|
|
171
|
-
const normalized = { ...data };
|
|
172
|
-
delete normalized["sessionFormatVersion"];
|
|
173
|
-
delete normalized["throughSeq"];
|
|
174
|
-
return {
|
|
175
|
-
...event,
|
|
176
|
-
data: normalized
|
|
177
|
-
};
|
|
178
|
-
}
|
|
179
|
-
/** Migrate one session's contiguous v1 notification tail and align its surviving rows. */
|
|
180
|
-
function migrateWrappedEventGroup(entries) {
|
|
181
|
-
const first = entries[0];
|
|
182
|
-
if (!entries.some((entry) => entry.event["type"] === "assistant/chunk")) return /* @__PURE__ */ new Map();
|
|
183
|
-
const firstSeq = first.event["seq"];
|
|
184
|
-
if (!Number.isSafeInteger(firstSeq) || firstSeq < 0) throw new Error("session event comparison requires a non-negative first seq");
|
|
185
|
-
for (const [index, entry] of entries.entries()) if (entry.event["seq"] !== firstSeq + index || typeof entry.event["type"] !== "string") throw new Error(`session event comparison requires a contiguous event tail for ${first.sessionId}`);
|
|
186
|
-
const prefix = Array.from({ length: firstSeq }, (_, seq) => ({
|
|
187
|
-
type: "feedback/record",
|
|
188
|
-
seq,
|
|
189
|
-
time: 0,
|
|
190
|
-
data: { text: `comparison prefix ${String(seq)}` }
|
|
191
|
-
}));
|
|
192
|
-
const restore = sessionFormatCatalog.createRestore({
|
|
193
|
-
type: "session",
|
|
194
|
-
version: 1,
|
|
195
|
-
id: first.sessionId,
|
|
196
|
-
createdAt: 0,
|
|
197
|
-
delegationDepth: 0
|
|
198
|
-
}, {
|
|
199
|
-
recovery: "strict",
|
|
200
|
-
validation: "current"
|
|
201
|
-
});
|
|
202
|
-
for (const event of prefix) restore.decodeRow(event);
|
|
203
|
-
for (const entry of entries) restore.decodeRow(normalizeProjectedRow(entry.event));
|
|
204
|
-
const migrated = restore.finish().events.slice(prefix.length);
|
|
205
|
-
const assigned = /* @__PURE__ */ new Map();
|
|
206
|
-
let migratedIndex = 0;
|
|
207
|
-
let lastChunkRow;
|
|
208
|
-
for (const entry of entries) {
|
|
209
|
-
if (entry.event["type"] === "assistant/chunk") {
|
|
210
|
-
lastChunkRow = entry.rowIndex;
|
|
211
|
-
continue;
|
|
212
|
-
}
|
|
213
|
-
while (migrated[migratedIndex]?.["type"] === "assistant/attempt") {
|
|
214
|
-
assignMigratedEvent(assigned, lastChunkRow, migrated[migratedIndex]);
|
|
215
|
-
migratedIndex += 1;
|
|
216
|
-
}
|
|
217
|
-
const next = migrated[migratedIndex];
|
|
218
|
-
assignMigratedEvent(assigned, entry.rowIndex, restoreProjectedRequestHeader(next, entry.event));
|
|
219
|
-
migratedIndex += 1;
|
|
220
|
-
}
|
|
221
|
-
while (migrated[migratedIndex]?.["type"] === "assistant/attempt") {
|
|
222
|
-
assignMigratedEvent(assigned, lastChunkRow, migrated[migratedIndex]);
|
|
223
|
-
migratedIndex += 1;
|
|
224
|
-
}
|
|
225
|
-
return assigned;
|
|
226
|
-
}
|
|
227
|
-
/**
|
|
228
|
-
* Project v1 session-event notifications to current settlement cardinality in memory.
|
|
229
|
-
* Non-event protocol rows retain their exact positions, and current v2 input passes through.
|
|
230
|
-
* @param text - headless `session_event` or SDK `session.event` JSONL.
|
|
231
|
-
* @returns comparison JSONL using current Session events without modifying its source file.
|
|
232
|
-
*/
|
|
233
|
-
function prepareSessionEventNotificationsForComparison(text) {
|
|
234
|
-
const trailingNewline = text.endsWith("\n");
|
|
235
|
-
const rows = text.split("\n").filter((line) => line.trim().length > 0).map((line, index) => {
|
|
236
|
-
const value = JSON.parse(line);
|
|
237
|
-
if (value === null || typeof value !== "object" || Array.isArray(value)) throw new Error(`session event comparison line ${String(index + 1)} must be an object`);
|
|
238
|
-
return value;
|
|
239
|
-
});
|
|
240
|
-
const entries = rows.flatMap((row, rowIndex) => {
|
|
241
|
-
const wrapped = wrappedSessionEvent(row);
|
|
242
|
-
return wrapped === void 0 ? [] : [{
|
|
243
|
-
...wrapped,
|
|
244
|
-
rowIndex
|
|
245
|
-
}];
|
|
246
|
-
});
|
|
247
|
-
const groups = [];
|
|
248
|
-
for (const entry of entries) {
|
|
249
|
-
const nextSeq = entry.event["seq"];
|
|
250
|
-
const group = groups.findLast((candidate) => {
|
|
251
|
-
const previous = candidate.at(-1);
|
|
252
|
-
const previousSeq = previous?.event["seq"];
|
|
253
|
-
return previous?.sessionId === entry.sessionId && Number.isSafeInteger(previousSeq) && Number.isSafeInteger(nextSeq) && nextSeq === previousSeq + 1;
|
|
254
|
-
});
|
|
255
|
-
if (group === void 0) groups.push([entry]);
|
|
256
|
-
else group.push(entry);
|
|
257
|
-
}
|
|
258
|
-
const assigned = /* @__PURE__ */ new Map();
|
|
259
|
-
for (const group of groups) for (const [rowIndex, events] of migrateWrappedEventGroup(group)) assigned.set(rowIndex, events);
|
|
260
|
-
const output = rows.flatMap((row, rowIndex) => {
|
|
261
|
-
const wrapped = wrappedSessionEvent(row);
|
|
262
|
-
if (wrapped === void 0) return [row];
|
|
263
|
-
const migrated = assigned.get(rowIndex);
|
|
264
|
-
if (migrated === void 0) return wrapped.event["type"] === "assistant/chunk" ? [] : [wrapped.replace(normalizeWrappedEventProvenance(wrapped.event))];
|
|
265
|
-
return migrated.map((event) => wrapped.replace(normalizeWrappedEventProvenance(event)));
|
|
266
|
-
}).map((row) => JSON.stringify(row)).join("\n");
|
|
267
|
-
return trailingNewline ? `${output}\n` : output;
|
|
268
|
-
}
|
|
269
|
-
/** Encode one migrated fixture while retaining a projected cwd token. */
|
|
135
|
+
/** Encode one migrated fixture while retaining projected cwd and request-tool tokens. */
|
|
270
136
|
function encodeCurrentSessionSnapshotFixture(text, parsed) {
|
|
271
137
|
const header = { ...sessionFormatCatalog.encodeCurrentHeader(parsed.artifact.header, parsed.artifact.inheritedEventCount) };
|
|
272
138
|
const sourceCwd = parsed.sourceHeader["cwd"];
|
|
273
139
|
if (typeof sourceCwd === "string" && /^\{\{cwd\}\}(?:\/|$)/.test(sourceCwd)) header["cwd"] = sourceCwd;
|
|
274
|
-
const
|
|
140
|
+
const sourceRequests = text.split(/\r?\n/).filter((line) => line.trim().length > 0).slice(1).map((line) => JSON.parse(line)).filter((row) => row["type"] === "request/header");
|
|
141
|
+
let requestIndex = 0;
|
|
142
|
+
const output = [JSON.stringify(header), ...parsed.artifact.events.map((event) => {
|
|
143
|
+
const encoded = sessionFormatCatalog.encodeCurrentEvent(event);
|
|
144
|
+
if (event.type !== "request/header") return JSON.stringify(encoded);
|
|
145
|
+
const source = sourceRequests[requestIndex++];
|
|
146
|
+
return JSON.stringify(restoreProjectedRequestHeader(encoded, source));
|
|
147
|
+
})].join("\n");
|
|
275
148
|
return text.endsWith("\n") ? `${output}\n` : output;
|
|
276
149
|
}
|
|
277
|
-
/**
|
|
150
|
+
/** Omit exact request-tool sidecar tokens and materialize projected tool names for validation. */
|
|
278
151
|
function normalizeProjectedRow(source) {
|
|
279
152
|
const record = { ...source };
|
|
280
153
|
if (record["type"] !== "request/header") return record;
|
|
@@ -283,9 +156,9 @@ function normalizeProjectedRow(source) {
|
|
|
283
156
|
const header = data["header"];
|
|
284
157
|
if (header === null || typeof header !== "object" || Array.isArray(header)) return record;
|
|
285
158
|
const tools = header["tools"];
|
|
286
|
-
|
|
287
|
-
if (tools === "{{tools}}")
|
|
288
|
-
else if (Array.isArray(tools) && tools.every((tool) => typeof tool === "string" && tool.length > 0))
|
|
159
|
+
const normalizedHeader = { ...header };
|
|
160
|
+
if (tools === "{{tools}}") delete normalizedHeader["tools"];
|
|
161
|
+
else if (Array.isArray(tools) && tools.length > 0 && tools.every((tool) => typeof tool === "string" && tool.length > 0)) normalizedHeader["tools"] = tools.map((name) => ({
|
|
289
162
|
name,
|
|
290
163
|
description: "",
|
|
291
164
|
parameters: {}
|
|
@@ -293,10 +166,7 @@ function normalizeProjectedRow(source) {
|
|
|
293
166
|
else return record;
|
|
294
167
|
record["data"] = {
|
|
295
168
|
...data,
|
|
296
|
-
header:
|
|
297
|
-
...header,
|
|
298
|
-
tools: materializedTools
|
|
299
|
-
}
|
|
169
|
+
header: normalizedHeader
|
|
300
170
|
};
|
|
301
171
|
return record;
|
|
302
172
|
}
|
|
@@ -725,6 +595,7 @@ var ReplayAdapter = class extends LlmAdapter {
|
|
|
725
595
|
...configuredModel?.inputModalities === void 0 ? {} : { inputModalities: [...configuredModel.inputModalities] },
|
|
726
596
|
...configuredModel?.contextWindow === void 0 ? {} : { context: { contextWindow: configuredModel.contextWindow } },
|
|
727
597
|
...configuredModel?.defaultMaxTokens === void 0 ? {} : { defaultMaxTokens: configuredModel.defaultMaxTokens },
|
|
598
|
+
...configuredModel?.systemPromptUpdate === void 0 ? {} : { systemPromptUpdate: configuredModel.systemPromptUpdate },
|
|
728
599
|
...configuredModel?.reasoningEfforts === void 0 ? {} : { reasoning: {
|
|
729
600
|
efforts: configuredModel.reasoningEfforts.map((id) => ({
|
|
730
601
|
id: ReasoningEffortId(id),
|
|
@@ -900,6 +771,8 @@ function validateConfiguredModels(providers) {
|
|
|
900
771
|
const imageRequestTokens = model.imageRequestTokens;
|
|
901
772
|
if (imageRequestTokens !== void 0 && (!Number.isSafeInteger(imageRequestTokens) || imageRequestTokens <= 0)) throw new Error(`llm-replay: provider "${provider.id}" model "${model.id}" imageRequestTokens must be a positive safe integer`);
|
|
902
773
|
if (imageRequestTokens !== void 0 && model.inputModalities?.includes("image") !== true) throw new Error(`llm-replay: provider "${provider.id}" model "${model.id}" imageRequestTokens requires inputModalities to include "image"`);
|
|
774
|
+
const systemPromptUpdate = model.systemPromptUpdate;
|
|
775
|
+
if (systemPromptUpdate !== void 0 && systemPromptUpdate !== "in-history") throw new Error(`llm-replay: provider "${provider.id}" model "${model.id}" systemPromptUpdate must be "in-history" when present`);
|
|
903
776
|
}
|
|
904
777
|
}
|
|
905
778
|
function apply(ctx, config = {}) {
|
|
@@ -918,4 +791,4 @@ function apply(ctx, config = {}) {
|
|
|
918
791
|
});
|
|
919
792
|
}
|
|
920
793
|
//#endregion
|
|
921
|
-
export { apply, deriveReplayScript, inject, installLlmReplay, loadReplayScript, loadSessionScripts, name, parseSessionHeader, parseSessionLog,
|
|
794
|
+
export { apply, deriveReplayScript, inject, installLlmReplay, loadReplayScript, loadSessionScripts, name, parseSessionHeader, parseSessionLog, prepareSessionSnapshotFixtureForComparison, resolveScriptedEntry };
|
package/lib/types/index.d.ts
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* Keyless snapshot-test LLM replay. It derives one model-call script per
|
|
3
|
-
* recorded session from
|
|
3
|
+
* recorded session from v3 embedded Assistant streams and explicitly marked local
|
|
4
4
|
* compaction calls, then binds fresh live sessions to parent/child scripts by
|
|
5
5
|
* first-call order. Throw and hang cases require an explicit override because
|
|
6
6
|
* a session log cannot reconstruct them alone.
|
|
@@ -9,7 +9,7 @@
|
|
|
9
9
|
import type { Context } from '@deepseek-ai/cordis';
|
|
10
10
|
import { type SessionEvent } from '@deepseek-ai/dsh-session';
|
|
11
11
|
import type { SessionLogOffset as SessionLogOffsetType } from '@deepseek-ai/dsh-session';
|
|
12
|
-
import type { GenerateOptions, ModelModality, RetryPolicyConfig, StreamChunk } from '@deepseek-ai/dsh-llm';
|
|
12
|
+
import type { GenerateOptions, ModelModality, RetryPolicyConfig, StreamChunk, SystemPromptUpdate } from '@deepseek-ai/dsh-llm';
|
|
13
13
|
/**
|
|
14
14
|
* One recorded model call. `throw` may replay prefix chunks before failing;
|
|
15
15
|
* `hang` models cancellation. Derived chunk entries come from ordinary model
|
|
@@ -63,6 +63,8 @@ export interface ReplayModelConfig {
|
|
|
63
63
|
* {@link reasoningEfforts} or call resolution rejects the route.
|
|
64
64
|
*/
|
|
65
65
|
defaultReasoningEffort?: string;
|
|
66
|
+
/** Optional in-history system prompt replacement for a keyless replay route. */
|
|
67
|
+
systemPromptUpdate?: SystemPromptUpdate;
|
|
66
68
|
}
|
|
67
69
|
/** One provider route exposed by the replay adapter. */
|
|
68
70
|
export interface ReplayProviderConfig {
|
|
@@ -160,18 +162,11 @@ export interface SessionScript {
|
|
|
160
162
|
export declare function parseSessionLog(text: string): SessionEvent[];
|
|
161
163
|
/**
|
|
162
164
|
* Convert one persisted or projected snapshot fixture to the current physical format in memory for expected-output comparison.
|
|
163
|
-
* Projected cwd tokens remain tokens
|
|
165
|
+
* Projected cwd and request-tool tokens remain tokens for comparison with a fresh run.
|
|
164
166
|
* @param text - one complete Session fixture.
|
|
165
167
|
* @returns current-format JSONL with complete event envelopes; the input string and source file remain unchanged.
|
|
166
168
|
*/
|
|
167
169
|
export declare function prepareSessionSnapshotFixtureForComparison(text: string): string;
|
|
168
|
-
/**
|
|
169
|
-
* Project v1 session-event notifications to current settlement cardinality in memory.
|
|
170
|
-
* Non-event protocol rows retain their exact positions, and current v2 input passes through.
|
|
171
|
-
* @param text - headless `session_event` or SDK `session.event` JSONL.
|
|
172
|
-
* @returns comparison JSONL using current Session events without modifying its source file.
|
|
173
|
-
*/
|
|
174
|
-
export declare function prepareSessionEventNotificationsForComparison(text: string): string;
|
|
175
170
|
/**
|
|
176
171
|
* Read replay identity, ordering, and fork-seed facts from the JSONL header.
|
|
177
172
|
*
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@deepseek-ai/dsh-llm-replay",
|
|
3
3
|
"description": "Replay LLM plugin: short-circuits llm/stream with model chunks reconstructed from a recorded session JSONL (keyless snapshot tests)",
|
|
4
|
-
"version": "0.1.
|
|
4
|
+
"version": "0.1.5-alpha.2",
|
|
5
5
|
"publishConfig": {
|
|
6
6
|
"access": "public"
|
|
7
7
|
},
|
|
@@ -28,10 +28,10 @@
|
|
|
28
28
|
"license": "MIT",
|
|
29
29
|
"peerDependencies": {
|
|
30
30
|
"@deepseek-ai/cordis": "^4.0.2",
|
|
31
|
-
"@deepseek-ai/dsh-compaction": "^0.1.
|
|
32
|
-
"@deepseek-ai/dsh-deepseek-llm-api-extensions": "^0.1.
|
|
33
|
-
"@deepseek-ai/dsh-llm": "^0.1.
|
|
34
|
-
"@deepseek-ai/dsh-session": "^0.1.
|
|
31
|
+
"@deepseek-ai/dsh-compaction": "^0.1.5-alpha.2",
|
|
32
|
+
"@deepseek-ai/dsh-deepseek-llm-api-extensions": "^0.1.5-alpha.2",
|
|
33
|
+
"@deepseek-ai/dsh-llm": "^0.1.5-alpha.2",
|
|
34
|
+
"@deepseek-ai/dsh-session": "^0.1.5-alpha.2"
|
|
35
35
|
},
|
|
36
36
|
"peerDependenciesMeta": {
|
|
37
37
|
"@deepseek-ai/dsh-deepseek-llm-api-extensions": {
|
|
@@ -39,14 +39,14 @@
|
|
|
39
39
|
}
|
|
40
40
|
},
|
|
41
41
|
"devDependencies": {
|
|
42
|
-
"@deepseek-ai/dsh-compaction": "^0.1.3-alpha.2",
|
|
43
|
-
"@deepseek-ai/dsh-deepseek-llm-api-extensions": "^0.1.3-alpha.2",
|
|
44
|
-
"@deepseek-ai/dsh-llm": "^0.1.3-alpha.2",
|
|
45
42
|
"@deepseek-ai/cordis": "^4.0.2",
|
|
46
|
-
"@deepseek-ai/dsh-
|
|
43
|
+
"@deepseek-ai/dsh-compaction": "^0.1.5-alpha.2",
|
|
44
|
+
"@deepseek-ai/dsh-deepseek-llm-api-extensions": "^0.1.5-alpha.2",
|
|
45
|
+
"@deepseek-ai/dsh-llm": "^0.1.5-alpha.2",
|
|
46
|
+
"@deepseek-ai/dsh-session": "^0.1.5-alpha.2"
|
|
47
47
|
},
|
|
48
48
|
"dependencies": {
|
|
49
|
-
"@deepseek-ai/dsh-session-format-catalog": "^0.1.
|
|
50
|
-
"@deepseek-ai/dsh-util-values": "^0.1.
|
|
49
|
+
"@deepseek-ai/dsh-session-format-catalog": "^0.1.5-alpha.2",
|
|
50
|
+
"@deepseek-ai/dsh-util-values": "^0.1.5-alpha.2"
|
|
51
51
|
}
|
|
52
52
|
}
|