@deepseek-ai/dsh-session-persistence-jsonl 0.0.1-rc.1 → 0.0.1-rc.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.i18n.yaml +2 -2
- package/README.md +2 -2
- package/README.zh.md +11 -11
- package/lib/index.js +100 -28
- package/lib/types/index.d.ts +27 -3
- package/package.json +8 -8
package/README.i18n.yaml
CHANGED
|
@@ -2,5 +2,5 @@
|
|
|
2
2
|
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
|
3
3
|
# after editing either side, bring the other along and re-record with:
|
|
4
4
|
# pnpm run verify-translation-pairing --write packages/session/session-persistence-jsonl/README.md
|
|
5
|
-
README.md:
|
|
6
|
-
README.zh.md:
|
|
5
|
+
README.md: 4cff3215cdb083d2fdb7c4a8f1b60e8c4028ba84
|
|
6
|
+
README.zh.md: 38b41cb9f7fa0f144923feced06152a185254c0a
|
package/README.md
CHANGED
|
@@ -42,7 +42,7 @@ A root belongs to one encoding. Startup discovery and targeted lookup reject the
|
|
|
42
42
|
- **Bound storage identity.** Lookup requires one matching session directory across the readable project directories, then verifies that the header id equals the requested id and that the header's id/cwd derive the selected transcript path. Listing applies the same path check and rejects duplicate ids. Identity failures occur before repair or append.
|
|
43
43
|
- **Lazy materialization.** `create(meta)` writes nothing; on the first `append`, the backend writes and `fsync`s the encoded header and first batch in a temporary file. POSIX publishes it without overwrite via a hard link and `fsync`s the parent directory. Windows publishes it without overwrite via `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)` and creates missing directories through the same write-through pattern. A created-but-never-appended session leaves nothing on disk and is absent from `list`.
|
|
44
44
|
- **Append-only.** Flushed events are never rewritten. Subsequent raw batches append lines; compressed batches append one frame. Both paths `fsync`, and a caught write or sync failure rolls the file back to its prior byte length.
|
|
45
|
-
- **Crash recovery — preserve valid tail work.** `load` validates every complete compressed frame and scans their decompressed JSONL. If the last frame is structurally incomplete, the reader keeps its complete decoded records, truncates from that frame's start, and re-encodes those records with the synthetic tool, step, and turn closers required by the shared [persistence contract](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md). Raw mode truncates from its first incomplete line.
|
|
45
|
+
- **Crash recovery — preserve valid tail work.** `load` validates every complete compressed frame and scans their decompressed JSONL. If the last frame is structurally incomplete, the reader keeps its complete decoded records, truncates from that frame's start, and re-encodes those records with the synthetic tool, step, and turn closers required by the shared [persistence contract](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md). Raw mode truncates from its first incomplete line. An existing compressed artifact with no complete header frame, a checksum/decompression failure in a complete frame, or a defect at or before the last committed `turn/end` is corruption and rejects.
|
|
46
46
|
- **Non-mutating inspection.** `inspect()` returns an immutable balanced logical view and may synthesize recovery closers in memory, without truncating an incomplete tail or changing the lightweight revision.
|
|
47
47
|
- **Contiguous-seq.** `append` rejects a batch whose first `seq` does not continue the stored log, and rejects non-JSON-serializable `event.data` naming the offending event type.
|
|
48
48
|
- **Lightweight revisions.** `listSnapshots(signal?)` identifies a log by its device, inode, size, and nanosecond timestamps, avoiding a full-log parse while changing after append, repair, replacement, or store changes. A full-prefix read requires the same identity before and after reading the bytes, and `readStoredRevision()` uses that identity to validate retained preparations without loading the log. Snapshot listing forwards the exact signal through artifact discovery and checks cancellation around every `stat`; because filesystem `stat` is not interruptible, cancellation waits for the active call to settle, then rejects without starting another.
|
|
@@ -72,6 +72,6 @@ JSONL storage does not mutate live request prefixes. A resumed loop can reuse pr
|
|
|
72
72
|
- **Only the configured encoding and current `SESSION_FORMAT_VERSION` (v0) load** — changing compression requires a separate/fresh root or selecting the legacy raw mode; the pre-release format has no migration.
|
|
73
73
|
- **The flat-file storage layout does not load** — use a separate root or move pre-release artifacts into the project/session directory layout before loading.
|
|
74
74
|
- **Compressed files are not directly line-readable** — use the backend to load them, or select `compression: 'none'` before writing a fresh root when external line readers are required.
|
|
75
|
-
- **Nothing deletes session files** — logs accumulate under `root` until removed externally (the seam has no deletion
|
|
75
|
+
- **Nothing deletes session files** — logs accumulate under `root` until removed externally (the seam has no deletion API).
|
|
76
76
|
- **One live writer per session** — append and repair are coordinated only inside the owning backend instance. Another backend instance or process must not write the same session until that owner reaches quiescent disposal; initial same-id publication remains collision-safe through the POSIX no-overwrite hard link or Windows write-through rename without replacement.
|
|
77
77
|
- **POSIX materialization requires hard-link support** — first append uses `link()` so same-id races fail instead of overwriting a committed log; Windows uses write-through rename without replacement.
|
package/README.zh.md
CHANGED
|
@@ -14,9 +14,9 @@ JSONL 持久会话存储后端:`SessionPersistence` 的一个具体实现(`d
|
|
|
14
14
|
session.jsonl # only with compression: 'none'
|
|
15
15
|
```
|
|
16
16
|
|
|
17
|
-
- 第一个逻辑行是不可变的 `SessionHeader`,标记为 `{ type: 'session', version, id, cwd?, createdAt, parentSession?, seedLength?, origin?, delegationDepth, agentPreset? }`。`delegationDepth` 在磁盘上必需,顶层会话为 `0`;缺失或无效值会拒绝日志。`agentPreset`
|
|
18
|
-
- 存储记录是原样 `SessionEvent` JSON,或在 `packChunks` 已启用且连续段符合条件时写入的**打包分片行**(`text-chunks` / `reasoning-chunks` / `tool-call-chunks`;像 header 的 `session` 一样不带斜杠,因此行 tag 不会与事件类型混淆):一行保存至少 3 个连续同 block `assistant/chunk` delta 事件,`seq0`/`time0`
|
|
19
|
-
- 项目目录保留规范化 cwd
|
|
17
|
+
- 第一个逻辑行是不可变的 `SessionHeader`,标记为 `{ type: 'session', version, id, cwd?, createdAt, parentSession?, seedLength?, origin?, delegationDepth, agentPreset? }`。`delegationDepth` 在磁盘上必需,顶层会话为 `0`;缺失或无效值会拒绝日志。`agentPreset` 必须持久化,因为它决定了被恢复会话的工具与提示词——恢复成另一套组装,就会回放模型已无法据以行动的历史。后续每个逻辑行是一条存储记录;`assistant/chunk` 事件绝不丢弃,且 `seq` 在解码日志中保持连续(`events[i].seq === i`)。
|
|
18
|
+
- 存储记录是原样 `SessionEvent` JSON,或在 `packChunks` 已启用且连续段符合条件时写入的**打包分片行**(`text-chunks` / `reasoning-chunks` / `tool-call-chunks`;像 header 的 `session` 一样不带斜杠,因此行 tag 不会与事件类型混淆):一行保存至少 3 个连续同 block `assistant/chunk` delta 事件,`seq0`/`time0` 和各成员的 `dt` 间隔精确重建每个成员的 `seq`/`time`。无损 codec 位于 `@deepseek-ai/dsh-session`(`packChunkRuns`/`decodeStorageRecord`),并使用精确形态 allowlist:任何未识别内容原样存储。读取与布局无关:`load` 始终解码行,因此打包、非打包和混合文件加载结果一致。
|
|
19
|
+
- 项目目录保留规范化 cwd 的可读形式,便于导航,并限制在文件系统组件上限内。分隔符替换和截断刻意有损,因此规范化相同的 cwd 字符串共享项目目录;会话 id 仍选择不同会话目录。在不区分大小写的文件系统上,只有文件系统规范化将两种写法解析到同一 transcript(文本记录)时,身份验证才接受备选路径写法。配置根仍由部署控制:可以是项目本地、共享、临时或集中式。[项目会话目录决策](../../../.agents/notes/implemented/architecture/2026-07-24-project-session-directories.md) 记录这项取舍。
|
|
20
20
|
- 会话 id 是未验证的带品牌类型的字符串,因此在使用前单射转义为一个安全路径段(无遍历、无冲突)。结果目录保留给其他会话自有产物;发现只读取固定 transcript 文件名。
|
|
21
21
|
|
|
22
22
|
## 配置
|
|
@@ -24,7 +24,7 @@ JSONL 持久会话存储后端:`SessionPersistence` 的一个具体实现(`d
|
|
|
24
24
|
| 键 | 类型 | 说明 |
|
|
25
25
|
|---|---|---|
|
|
26
26
|
| `root` | `string`(必需) | 所有会话文件的根目录。**无默认值**:`process.cwd()` 默认值会随进程 cwd 变更(bash 调用、子进程)而分散文件。现有根必须是可读目录;缺失根在第一次实体化时创建。 |
|
|
27
|
-
| `packChunks` | `boolean`(默认 `true`) | 将符合条件的 delta
|
|
27
|
+
| `packChunks` | `boolean`(默认 `true`) | 将符合条件的 delta 分片连续段写为打包行(在真实编程会话上测得逻辑日志约小 60%)。设为 `false` 可用于每事件一行诊断;无论该写入侧开关如何,都能读取打包行。 |
|
|
28
28
|
| `compression` | `'zstd' \| 'none'` | 默认 `'zstd'`;`'none'` 保留换行分隔 UTF-8 文本。 |
|
|
29
29
|
| `preparedSessionCacheSize` | 正整数(默认 `5`) | 冷历史检查后保留、供恢复复用的未发布会话数量上限。 |
|
|
30
30
|
| `writeBatchMaxDelayMs` | 正整数(默认 `200`) | 空闲的活动事件队列收到待写入事件后开启的固定合并窗口。后续事件不会重置窗口;flush 与 teardown 会绕过它。该值不限制事件循环、串行化操作或后端延迟。最大值为 Node 计时器上限 `2_147_483_647` ms。 |
|
|
@@ -42,10 +42,10 @@ JSONL 持久会话存储后端:`SessionPersistence` 的一个具体实现(`d
|
|
|
42
42
|
- **绑定存储身份。** 查找要求可读项目目录中只有一个匹配会话目录,然后验证 header id 等于请求 id,且 header id/cwd 派生所选 transcript 路径。列表应用同一路径检查,并拒绝重复 id。身份失败发生在修复或 append 前。
|
|
43
43
|
- **延迟实体化。**`create(meta)` 不写入;第一次 `append` 将编码 header 和第一批写入临时文件并执行 `fsync`。POSIX 通过硬链接无覆盖发布,并对父目录 `fsync`。Windows 通过 `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)` 无覆盖发布,并通过同一 write-through pattern 创建缺失目录。已创建但从未 append 的会话不留下磁盘内容,不在 `list` 中。
|
|
44
44
|
- **仅追加。** 已 flush 事件绝不重写。后续原始批次 append 行;压缩批次 append 一个 frame。两条路径都执行 `fsync`,并在捕获到写入或同步失败时回滚到之前字节长度。
|
|
45
|
-
- **崩溃恢复:保留有效尾部工作。**`load` 验证每个完整压缩 frame,并扫描解压 JSONL。最后 frame 结构不完整时,读取器保留其完整解码记录,从 frame 开头截断,并使用共享[持久化约定](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md) 需要的合成工具、步骤和轮次 closer 重新编码这些记录。原始 mode
|
|
46
|
-
-
|
|
47
|
-
- **连续 seq。**`append` 拒绝第一个 `seq`
|
|
48
|
-
- **轻量修订。**`listSnapshots(signal?)` 使用 device、inode、size 和纳秒时间戳标识日志,避免解析完整日志;该标识会在 append、修复、替换或存储变更后改变。完整前缀读取要求读取字节前后的身份一致,`readStoredRevision()` 使用同一身份校验保留的 preparation
|
|
45
|
+
- **崩溃恢复:保留有效尾部工作。**`load` 验证每个完整压缩 frame,并扫描解压 JSONL。最后 frame 结构不完整时,读取器保留其完整解码记录,从 frame 开头截断,并使用共享[持久化约定](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md) 需要的合成工具、步骤和轮次 closer 重新编码这些记录。原始 mode 从第一个不完整行截断。已经存在却没有完整 header frame 的压缩工件、完整 frame 中的 checksum/解压失败,或位于最后已提交的 `turn/end` 处或之前的缺陷都属于损坏,会被拒绝。
|
|
46
|
+
- **非修改式检查。**`inspect()` 返回不可变、平衡的逻辑视图,并可在内存中合成恢复 closer,但不会截断不完整尾部或更改轻量修订。
|
|
47
|
+
- **连续 seq。**`append` 拒绝第一个 `seq` 不继续已存储日志的批次,并拒绝无法 JSON 序列化的 `event.data`,同时命名违规事件类型。
|
|
48
|
+
- **轻量修订。**`listSnapshots(signal?)` 使用 device、inode、size 和纳秒时间戳标识日志,避免解析完整日志;该标识会在 append、修复、替换或存储变更后改变。完整前缀读取要求读取字节前后的身份一致,`readStoredRevision()` 使用同一身份校验保留的 preparation,而不加载日志。快照列表通过产物发现原样转发该信号,并在每个 `stat` 前后检查取消;由于文件系统 `stat` 不可中断,取消会等待活动调用完成,然后在不启动另一次调用的情况下拒绝。
|
|
49
49
|
|
|
50
50
|
## 写入路径
|
|
51
51
|
|
|
@@ -57,7 +57,7 @@ JSONL 持久会话存储后端:`SessionPersistence` 的一个具体实现(`d
|
|
|
57
57
|
|
|
58
58
|
#### 模型看到的内容
|
|
59
59
|
|
|
60
|
-
JSONL
|
|
60
|
+
JSONL 存储不会向当前请求提供提示词或 schema。加载会恢复已存储的表层历史,并保留之前的请求 header 用于重建;新 loop 组合当前 envelope。恢复会用 `TOOL_NOT_STARTED` 平衡没有已持久化调用的 assistant 请求;已持久化调用无结果时则变为 `TOOL_OUTCOME_UNKNOWN`,它要求模型只重试只读或幂等工作,并验证可能的副作用或询问用户。原始 `assistant/chunk` 记录不会重复生成消息。
|
|
61
61
|
|
|
62
62
|
#### Token 影响
|
|
63
63
|
|
|
@@ -69,9 +69,9 @@ JSONL 存储不修改实时请求前缀。只有重建历史、当前 envelope
|
|
|
69
69
|
|
|
70
70
|
## 已知限制与暂缓事项
|
|
71
71
|
|
|
72
|
-
- **只加载已配置编码和当前 `SESSION_FORMAT_VERSION
|
|
72
|
+
- **只加载已配置编码和当前 `SESSION_FORMAT_VERSION`(v0)**:更改压缩需要独立/全新根,或选择遗留原始 mode;预发布格式没有迁移。
|
|
73
73
|
- **平铺文件存储布局不加载**:加载前使用独立根,或将预发布产物移入项目/会话目录布局。
|
|
74
74
|
- **压缩文件不能直接按行读取**:使用后端加载;或在写入新根前选择 `compression: 'none'`,以便外部行 reader 使用。
|
|
75
75
|
- **不删除会话文件**:日志在 `root` 下累积,直到外部移除(seam 无删除接口)。
|
|
76
|
-
-
|
|
76
|
+
- **每会话一个活动 writer**:append 和修复只在所属后端实例内协调。在所有者完成完全停稳的 dispose 前,其他后端实例或进程不得写入同一会话;初始同 id 发布仍通过 POSIX 无覆盖硬链接或 Windows 无替换 write-through rename 保持冲突安全。
|
|
77
77
|
- **POSIX 实体化需要硬链接支持**:第一次 append 使用 `link()`,使同 id 竞态失败,而不覆盖已提交日志;Windows 使用无替换 write-through rename。
|
package/lib/index.js
CHANGED
|
@@ -5,8 +5,8 @@ import { dirname, join, parse, resolve, toNamespacedPath } from "node:path";
|
|
|
5
5
|
import { performance } from "node:perf_hooks";
|
|
6
6
|
import { scheduler } from "node:timers/promises";
|
|
7
7
|
import { randomBytes } from "node:crypto";
|
|
8
|
-
import { DEFAULT_PREPARED_SESSION_CACHE_SIZE, DEFAULT_WRITE_BATCH_MAX_DELAY_MS, MAX_WRITE_BATCH_DELAY_MS, PersistenceCoordinator, SessionPersistence, SessionPersistenceRevision } from "@deepseek-ai/dsh-session-persistence";
|
|
9
|
-
import { decodeStorageRecord, packChunkRuns } from "@deepseek-ai/dsh-session";
|
|
8
|
+
import { DEFAULT_PREPARED_SESSION_CACHE_SIZE, DEFAULT_WRITE_BATCH_MAX_DELAY_MS, MAX_WRITE_BATCH_DELAY_MS, PersistenceCoordinator, SessionFormatUnsupportedError, SessionPersistence, SessionPersistenceRevision, sessionFormatVersionRefusal } from "@deepseek-ai/dsh-session-persistence";
|
|
9
|
+
import { SESSION_FORMAT_VERSION, decodeStorageRecord, packChunkRuns } from "@deepseek-ai/dsh-session";
|
|
10
10
|
import { constants, createZstdDecompress, zstdCompress, zstdDecompress, zstdDecompressSync } from "node:zlib";
|
|
11
11
|
import { promisify } from "node:util";
|
|
12
12
|
import { constants as constants$1 } from "node:buffer";
|
|
@@ -171,6 +171,19 @@ function eventLines(events, packChunks) {
|
|
|
171
171
|
return (packChunks ? packChunkRuns(events) : events).map((record) => JSON.stringify(record)).join("\n");
|
|
172
172
|
}
|
|
173
173
|
/** Parse one complete header record supplied independently from event rows. */
|
|
174
|
+
/**
|
|
175
|
+
* Refuse a header carrying a format version this build does not read BEFORE
|
|
176
|
+
* validating the current header shape or decoding any event row: a future
|
|
177
|
+
* format need not satisfy today's structural checks at all, and its user must
|
|
178
|
+
* see "upgrade the harness", never "corrupt session log".
|
|
179
|
+
* @param parsed - the JSON-parsed first line of a session artifact.
|
|
180
|
+
*/
|
|
181
|
+
function refuseForeignFormatVersion(parsed) {
|
|
182
|
+
if (typeof parsed !== "object" || parsed === null) return;
|
|
183
|
+
const { version, id } = parsed;
|
|
184
|
+
if (typeof version !== "number" || version === SESSION_FORMAT_VERSION) return;
|
|
185
|
+
throw new SessionFormatUnsupportedError(sessionFormatVersionRefusal(typeof id === "string" ? id : String(id), version));
|
|
186
|
+
}
|
|
174
187
|
function parseHeaderRecord(record) {
|
|
175
188
|
if (record.length === 0 || record.at(-1) !== 10 || record.indexOf(10) !== record.length - 1) throw new Error("empty or header-less session log");
|
|
176
189
|
let parsed;
|
|
@@ -179,6 +192,7 @@ function parseHeaderRecord(record) {
|
|
|
179
192
|
} catch {
|
|
180
193
|
throw new Error("corrupt session log: header line is not valid JSON");
|
|
181
194
|
}
|
|
195
|
+
refuseForeignFormatVersion(parsed);
|
|
182
196
|
if (!isHeaderLine(parsed)) throw new Error("corrupt session log: first line is not a session header");
|
|
183
197
|
return fromHeaderLine(parsed);
|
|
184
198
|
}
|
|
@@ -607,7 +621,7 @@ const ERROR_FILE_EXISTS = 80;
|
|
|
607
621
|
const ERROR_INVALID_NAME = 123;
|
|
608
622
|
const ERROR_ALREADY_EXISTS = 183;
|
|
609
623
|
let bindings;
|
|
610
|
-
/** Load the small Win32
|
|
624
|
+
/** Load the small Win32 API lazily so non-Windows processes never load Koffi. */
|
|
611
625
|
async function win32() {
|
|
612
626
|
if (bindings !== void 0) return bindings;
|
|
613
627
|
const kernel32 = (await import("koffi")).default.load("kernel32.dll");
|
|
@@ -749,8 +763,9 @@ function isENOENT(error) {
|
|
|
749
763
|
* listeners. Its torn-tail marker carries the byte offset and any events
|
|
750
764
|
* recovered from an incomplete final Zstandard frame.
|
|
751
765
|
*/
|
|
752
|
-
var
|
|
766
|
+
var JsonlSessionPersistence = class extends SessionPersistence {
|
|
753
767
|
config;
|
|
768
|
+
supportsRawArtifacts = true;
|
|
754
769
|
static inject = ["sessions"];
|
|
755
770
|
static Config = z.object({
|
|
756
771
|
root: z.string().required(),
|
|
@@ -839,37 +854,94 @@ var SessionPersistenceJsonl = class extends SessionPersistence {
|
|
|
839
854
|
}
|
|
840
855
|
}
|
|
841
856
|
/**
|
|
842
|
-
* Read a stored
|
|
843
|
-
*
|
|
857
|
+
* Read a session's stored artifact text verbatim: the durable file bytes
|
|
858
|
+
* decoded from this backend's physical encoding (complete zstd frames
|
|
859
|
+
* concatenated, or UTF-8 plaintext). The content is the exact JSONL text the
|
|
860
|
+
* backend wrote — never a reconstruction from parsed events — so packed-
|
|
861
|
+
* chunk rows, key order, and line breaks survive byte-for-byte. A torn
|
|
862
|
+
* final frame is omitted, matching the committed-prefix semantics of every
|
|
863
|
+
* other read.
|
|
864
|
+
* @param id - the persisted session to read.
|
|
865
|
+
* @param signal - optional cancellation for the stat/read/decode work.
|
|
866
|
+
* @returns the raw artifact text plus the header parsed from its own first
|
|
867
|
+
* line, or `undefined` when the session has no stored artifact.
|
|
844
868
|
*/
|
|
845
|
-
async
|
|
846
|
-
|
|
847
|
-
|
|
869
|
+
async readRaw(id, signal) {
|
|
870
|
+
signal?.throwIfAborted();
|
|
871
|
+
await this.ensureRootEncoding();
|
|
872
|
+
signal?.throwIfAborted();
|
|
873
|
+
const path = await this.findLog(id, signal);
|
|
874
|
+
if (path === void 0) return void 0;
|
|
875
|
+
const { buffer } = await this.readStableFile(path, signal);
|
|
876
|
+
let content;
|
|
877
|
+
if (this.compression === "zstd") {
|
|
878
|
+
const { frames } = scanZstdFrames(buffer);
|
|
879
|
+
if (frames.length === 0) throw new Error("empty or header-less Zstandard session log");
|
|
880
|
+
const decoder = createZstdFrameDecoder();
|
|
881
|
+
const plaintexts = [];
|
|
882
|
+
for (const plaintext of decoder.decode(buffer, frames)) {
|
|
883
|
+
signal?.throwIfAborted();
|
|
884
|
+
plaintexts.push(Buffer.from(plaintext));
|
|
885
|
+
}
|
|
886
|
+
content = Buffer.concat(plaintexts).toString("utf8");
|
|
887
|
+
} else content = buffer.toString("utf8");
|
|
888
|
+
const meta = parseHeaderMeta(content.split("\n", 1)[0]);
|
|
889
|
+
if (meta === void 0 || meta.id !== id) throw new Error(`corrupt session log: invalid header line in "${path}"`);
|
|
890
|
+
return {
|
|
891
|
+
meta,
|
|
892
|
+
filename: "session.jsonl",
|
|
893
|
+
content
|
|
894
|
+
};
|
|
895
|
+
}
|
|
896
|
+
/**
|
|
897
|
+
* Read a file's bytes under a revision-stable loop: a writer appending
|
|
898
|
+
* between stat and readFile would yield a torn physical file, so retry
|
|
899
|
+
* while the stat revision changes.
|
|
900
|
+
* @param path - the artifact file to read.
|
|
901
|
+
* @param signal - optional cancellation for the stat/read work.
|
|
902
|
+
* @returns the stable bytes and the revision that matched both stats.
|
|
903
|
+
*/
|
|
904
|
+
async readStableFile(path, signal) {
|
|
848
905
|
for (;;) {
|
|
849
906
|
signal?.throwIfAborted();
|
|
850
907
|
const before = fileRevision(await stat(path, { bigint: true }));
|
|
851
|
-
buffer = await readFile(path, { signal });
|
|
908
|
+
const buffer = await readFile(path, { signal });
|
|
852
909
|
signal?.throwIfAborted();
|
|
853
910
|
const after = fileRevision(await stat(path, { bigint: true }));
|
|
854
|
-
if (before === after) {
|
|
855
|
-
|
|
856
|
-
|
|
857
|
-
}
|
|
911
|
+
if (before === after) return {
|
|
912
|
+
buffer,
|
|
913
|
+
revision: after
|
|
914
|
+
};
|
|
858
915
|
}
|
|
916
|
+
}
|
|
917
|
+
/**
|
|
918
|
+
* Read a stored prefix and convert torn-tail state to the opaque marker the
|
|
919
|
+
* coordinator can round-trip without knowing the physical encoding.
|
|
920
|
+
*/
|
|
921
|
+
async readPrefix(path, expectedId, signal) {
|
|
922
|
+
const { buffer, revision } = await this.readStableFile(path, signal);
|
|
859
923
|
let prefix;
|
|
860
|
-
|
|
861
|
-
|
|
862
|
-
|
|
863
|
-
|
|
864
|
-
|
|
865
|
-
|
|
866
|
-
|
|
867
|
-
|
|
868
|
-
|
|
869
|
-
|
|
870
|
-
|
|
871
|
-
|
|
872
|
-
|
|
924
|
+
try {
|
|
925
|
+
if (this.compression === "zstd") prefix = await this.readZstdPrefix(buffer, signal);
|
|
926
|
+
else {
|
|
927
|
+
signal?.throwIfAborted();
|
|
928
|
+
const { meta, events, committedBytes } = scanLog(buffer);
|
|
929
|
+
signal?.throwIfAborted();
|
|
930
|
+
prefix = {
|
|
931
|
+
meta,
|
|
932
|
+
events,
|
|
933
|
+
...committedBytes < buffer.byteLength ? { tornMarker: {
|
|
934
|
+
truncateTo: committedBytes,
|
|
935
|
+
recoveredEvents: []
|
|
936
|
+
} } : {}
|
|
937
|
+
};
|
|
938
|
+
}
|
|
939
|
+
} catch (error) {
|
|
940
|
+
if (error instanceof SessionFormatUnsupportedError && error.location === void 0) throw new SessionFormatUnsupportedError(`${error.message} (raw log: ${path})`, {
|
|
941
|
+
kind: "jsonl",
|
|
942
|
+
path
|
|
943
|
+
});
|
|
944
|
+
throw error;
|
|
873
945
|
}
|
|
874
946
|
signal?.throwIfAborted();
|
|
875
947
|
await this.assertStoredIdentity(path, prefix.meta, expectedId, signal);
|
|
@@ -1386,4 +1458,4 @@ var SessionPersistenceJsonl = class extends SessionPersistence {
|
|
|
1386
1458
|
}
|
|
1387
1459
|
};
|
|
1388
1460
|
//#endregion
|
|
1389
|
-
export { JsonlCompressionSchema,
|
|
1461
|
+
export { JsonlCompressionSchema, JsonlSessionPersistence, JsonlSessionPersistence as default };
|
package/lib/types/index.d.ts
CHANGED
|
@@ -7,7 +7,7 @@
|
|
|
7
7
|
*/
|
|
8
8
|
import { Context } from '@deepseek-ai/cordis';
|
|
9
9
|
import z from '@deepseek-ai/schemastery';
|
|
10
|
-
import { SessionPersistence, type PersistenceBackend, type SessionLocation, type SessionPersistenceSnapshot, type SessionInspection, type SessionPersistenceRevision as PersistenceRevision, type StoredPrefix } from '@deepseek-ai/dsh-session-persistence';
|
|
10
|
+
import { SessionPersistence, type PersistenceBackend, type SessionLocation, type SessionPersistenceSnapshot, type SessionInspection, type SessionPersistenceRevision as PersistenceRevision, type SessionRawArtifact, type StoredPrefix } from '@deepseek-ai/dsh-session-persistence';
|
|
11
11
|
import type { SessionEvent, SessionId, SessionHeader, SessionPreparation } from '@deepseek-ai/dsh-session';
|
|
12
12
|
import { type JsonlCompression } from './format.ts';
|
|
13
13
|
export type { JsonlCompression } from './format.ts';
|
|
@@ -49,8 +49,9 @@ interface JsonlTornMarker {
|
|
|
49
49
|
* listeners. Its torn-tail marker carries the byte offset and any events
|
|
50
50
|
* recovered from an incomplete final Zstandard frame.
|
|
51
51
|
*/
|
|
52
|
-
export declare class
|
|
52
|
+
export declare class JsonlSessionPersistence extends SessionPersistence implements PersistenceBackend<JsonlTornMarker> {
|
|
53
53
|
config: Config;
|
|
54
|
+
readonly supportsRawArtifacts = true;
|
|
54
55
|
static inject: string[];
|
|
55
56
|
static Config: z<Config>;
|
|
56
57
|
/**
|
|
@@ -83,6 +84,29 @@ export declare class SessionPersistenceJsonl extends SessionPersistence implemen
|
|
|
83
84
|
* Resolving an id with unknown cwd still scans the project directories.
|
|
84
85
|
*/
|
|
85
86
|
readStoredRevision(id: SessionId, signal?: AbortSignal): Promise<PersistenceRevision | undefined>;
|
|
87
|
+
/**
|
|
88
|
+
* Read a session's stored artifact text verbatim: the durable file bytes
|
|
89
|
+
* decoded from this backend's physical encoding (complete zstd frames
|
|
90
|
+
* concatenated, or UTF-8 plaintext). The content is the exact JSONL text the
|
|
91
|
+
* backend wrote — never a reconstruction from parsed events — so packed-
|
|
92
|
+
* chunk rows, key order, and line breaks survive byte-for-byte. A torn
|
|
93
|
+
* final frame is omitted, matching the committed-prefix semantics of every
|
|
94
|
+
* other read.
|
|
95
|
+
* @param id - the persisted session to read.
|
|
96
|
+
* @param signal - optional cancellation for the stat/read/decode work.
|
|
97
|
+
* @returns the raw artifact text plus the header parsed from its own first
|
|
98
|
+
* line, or `undefined` when the session has no stored artifact.
|
|
99
|
+
*/
|
|
100
|
+
readRaw(id: SessionId, signal?: AbortSignal): Promise<SessionRawArtifact | undefined>;
|
|
101
|
+
/**
|
|
102
|
+
* Read a file's bytes under a revision-stable loop: a writer appending
|
|
103
|
+
* between stat and readFile would yield a torn physical file, so retry
|
|
104
|
+
* while the stat revision changes.
|
|
105
|
+
* @param path - the artifact file to read.
|
|
106
|
+
* @param signal - optional cancellation for the stat/read work.
|
|
107
|
+
* @returns the stable bytes and the revision that matched both stats.
|
|
108
|
+
*/
|
|
109
|
+
private readStableFile;
|
|
86
110
|
/**
|
|
87
111
|
* Read a stored prefix and convert torn-tail state to the opaque marker the
|
|
88
112
|
* coordinator can round-trip without knowing the physical encoding.
|
|
@@ -159,5 +183,5 @@ export declare class SessionPersistenceJsonl extends SessionPersistence implemen
|
|
|
159
183
|
private exists;
|
|
160
184
|
private assertLogParentAllowsAbsence;
|
|
161
185
|
}
|
|
162
|
-
export default
|
|
186
|
+
export default JsonlSessionPersistence;
|
|
163
187
|
//# sourceMappingURL=index.d.ts.map
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@deepseek-ai/dsh-session-persistence-jsonl",
|
|
3
3
|
"description": "JSONL durable session persistence backend for the DeepSeek Harness",
|
|
4
|
-
"version": "0.0.1-rc.
|
|
4
|
+
"version": "0.0.1-rc.3",
|
|
5
5
|
"publishConfig": {
|
|
6
6
|
"access": "restricted"
|
|
7
7
|
},
|
|
@@ -32,19 +32,19 @@
|
|
|
32
32
|
],
|
|
33
33
|
"license": "BSD-3-Clause",
|
|
34
34
|
"peerDependencies": {
|
|
35
|
-
"@deepseek-ai/dsh-
|
|
36
|
-
"@deepseek-ai/dsh-session-persistence": "^0.0.1-rc.
|
|
37
|
-
"@deepseek-ai/
|
|
38
|
-
"@deepseek-ai/
|
|
35
|
+
"@deepseek-ai/dsh-session": "^0.0.1-rc.3",
|
|
36
|
+
"@deepseek-ai/dsh-session-persistence": "^0.0.1-rc.3",
|
|
37
|
+
"@deepseek-ai/cordis": "^4.0.1-rc.1",
|
|
38
|
+
"@deepseek-ai/dsh-invariants": "^0.0.1-rc.3"
|
|
39
39
|
},
|
|
40
40
|
"dependencies": {
|
|
41
41
|
"koffi": "^3.1.0",
|
|
42
42
|
"@deepseek-ai/schemastery": "^3.18.1-rc.1"
|
|
43
43
|
},
|
|
44
44
|
"devDependencies": {
|
|
45
|
-
"@deepseek-ai/dsh-invariants": "^0.0.1-rc.
|
|
46
|
-
"@deepseek-ai/dsh-session": "^0.0.1-rc.
|
|
47
|
-
"@deepseek-ai/dsh-session-persistence": "^0.0.1-rc.
|
|
45
|
+
"@deepseek-ai/dsh-invariants": "^0.0.1-rc.3",
|
|
46
|
+
"@deepseek-ai/dsh-session": "^0.0.1-rc.3",
|
|
47
|
+
"@deepseek-ai/dsh-session-persistence": "^0.0.1-rc.3",
|
|
48
48
|
"@deepseek-ai/cordis": "^4.0.1-rc.1"
|
|
49
49
|
}
|
|
50
50
|
}
|