@deepseek-ai/dsh-session-persistence-jsonl 0.1.1-rc.1 → 0.1.2-alpha.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.i18n.yaml +2 -2
- package/README.md +115 -31
- package/README.zh.md +123 -39
- package/lib/index.js +53 -9
- package/lib/types/format.d.ts +3 -3
- package/lib/types/index.d.ts +6 -2
- package/package.json +10 -10
package/README.i18n.yaml
CHANGED
|
@@ -2,5 +2,5 @@
|
|
|
2
2
|
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
|
3
3
|
# after editing either side, bring the other along and re-record with:
|
|
4
4
|
# pnpm run verify-translation-pairing --write packages/session/session-persistence-jsonl/README.md
|
|
5
|
-
README.md:
|
|
6
|
-
README.zh.md:
|
|
5
|
+
README.md: 69fb2901d783c327878cd37570aec730a3ca0841
|
|
6
|
+
README.zh.md: 188982028374d4ca11a5a658acabce5fe5df9930
|
package/README.md
CHANGED
|
@@ -1,12 +1,62 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: "The shipped JSONL session-persistence backend for deployments and maintainers choosing, configuring, or debugging per-session durable logs with optional Zstandard compression."
|
|
3
|
+
kind: "package-reference"
|
|
4
|
+
---
|
|
5
|
+
|
|
1
6
|
# @deepseek-ai/dsh-session-persistence-jsonl
|
|
2
7
|
|
|
3
8
|
English | [中文](README.zh.md)
|
|
4
9
|
|
|
5
|
-
|
|
10
|
+
## Summary
|
|
11
|
+
|
|
12
|
+
`dsh-session-persistence-jsonl` stores each session in its own append-only JSONL log — checksummed Zstandard frames by default, raw newline-delimited lines when compression is disabled. It serves the same logical `SessionEvent` stream as any persistence backend, so choosing it changes nothing for the agent loop, the model, or replay; compression, packing, and crash recovery are storage-internal details. Choose it when consumers need a per-session artifact on disk: `locate(meta)` returns the transcript path, and the logs are readable as plain lines when `compression: 'none'` is selected. A root directory is the one required configuration; durability, lazy materialization, and interrupted-turn recovery come with the backend.
|
|
13
|
+
|
|
14
|
+
## Table of Contents
|
|
15
|
+
|
|
16
|
+
- [Use this package](#use-this-package)
|
|
17
|
+
- [Understand the implementation](#understand-the-implementation)
|
|
18
|
+
- [Further Exploration](#further-exploration)
|
|
19
|
+
- [Model Experience](#model-experience)
|
|
20
|
+
- [Known Limitations and Deferred Work](#known-limitations-and-deferred-work)
|
|
21
|
+
- [Dev Note](#dev-note)
|
|
22
|
+
|
|
23
|
+
-----
|
|
24
|
+
|
|
25
|
+
<a id="use-this-package"></a>
|
|
26
|
+
## Use this package
|
|
27
|
+
|
|
28
|
+
Mount this backend when a composition needs durable sessions backed by per-session files. The common path is explicit: load the session service, mount the backend, and give it a root directory.
|
|
29
|
+
|
|
30
|
+
### When to choose it
|
|
31
|
+
|
|
32
|
+
Choose this backend when consumers benefit from one artifact per session — navigation, external tooling, or a raw line-readable log. Choose [SQLite](../session-persistence-sqlite/README.md) when a single queryable database fits the deployment instead. The backend keeps sessions under a deployment-controlled root: project-local, shared, temporary, or centralized.
|
|
6
33
|
|
|
7
|
-
|
|
34
|
+
### Minimal configuration
|
|
8
35
|
|
|
36
|
+
```yaml
|
|
37
|
+
- name: '@deepseek-ai/dsh-session'
|
|
38
|
+
- name: '@deepseek-ai/dsh-session-persistence-jsonl'
|
|
39
|
+
config:
|
|
40
|
+
root: /absolute/path/to/session-logs
|
|
9
41
|
```
|
|
42
|
+
|
|
43
|
+
`root` is required and has no default: a `process.cwd()` default would scatter session files as the process's cwd changes. An existing root must be a readable directory; an absent root is created on first materialization.
|
|
44
|
+
|
|
45
|
+
| Field | Default | Meaning |
|
|
46
|
+
|---|---|---|
|
|
47
|
+
| `root` | required | Root directory for all session files |
|
|
48
|
+
| `packChunks` | `true` | Write eligible `assistant/chunk` runs as packed rows; `false` keeps one event per line for diagnostics |
|
|
49
|
+
| `compression` | `'zstd'` | Physical encoding: `'zstd'` checksummed frames, or `'none'` newline-delimited UTF-8 text |
|
|
50
|
+
| `preparedSessionCacheSize` | `5` | Cold session preparations retained for resume reuse |
|
|
51
|
+
| `writeBatchMaxDelayMs` | `200` | Fixed live-event coalescing window, in milliseconds |
|
|
52
|
+
|
|
53
|
+
The generated [configuration catalog](../../../docs/config-catalog.md#deepseek-aidsh-session-persistence-jsonl) is the exhaustive source for every accepted field and its JSDoc.
|
|
54
|
+
|
|
55
|
+
### On-disk layout
|
|
56
|
+
|
|
57
|
+
Each session gets a session-owned directory under a readable project directory; the first logical line of the log is the immutable `SessionHeader`, followed by one storage record per logical event (or one packed chunk row per eligible run). Storage records use the lossless provenance representation described below:
|
|
58
|
+
|
|
59
|
+
```text
|
|
10
60
|
<root>/
|
|
11
61
|
--<normalized-cwd>--/ # readable project directory (or _no-cwd/)
|
|
12
62
|
<encoded-id>/ # session-owned directory
|
|
@@ -14,43 +64,62 @@ The JSONL durable session-persistence backend — a concrete `SessionPersistence
|
|
|
14
64
|
session.jsonl # only with compression: 'none'
|
|
15
65
|
```
|
|
16
66
|
|
|
17
|
-
|
|
18
|
-
- A storage record is a `SessionEvent` JSON verbatim, or — for an eligible run when `packChunks` is enabled — a **packed chunk row** (`text-chunks` / `reasoning-chunks` / `tool-call-chunks`; bare slash-less tags like the header's `session`, so row tags cannot be confused with event types): one line holding a run of ≥3 consecutive same-block `assistant/chunk` delta events, `seq0`/`time0` plus per-member `dt` gaps reconstructing every member's `seq`/`time` exactly. The lossless codec lives in `@deepseek-ai/dsh-session` (`packChunkRuns`/`decodeStorageRecord`) and whitelists exact shapes — anything unrecognized stores verbatim. Reading is layout-blind: `load` always decodes rows, so packed, unpacked, and mixed files load identically.
|
|
19
|
-
- The project directory keeps the normalized cwd readable for navigation and is bounded for filesystem component limits. Separator replacement and truncation are intentionally lossy, so cwd strings that normalize alike share a project directory; session ids still select distinct session directories. On a case-insensitive filesystem, identity validation accepts an alternate path spelling only when filesystem canonicalization resolves both spellings to the same transcript. The configured root remains deployment-controlled: it may be project-local, shared, temporary, or centralized. The [project-session directory decision](../../../.agents/notes/implemented/architecture/2026-07-24-project-session-directories.md) records this tradeoff.
|
|
20
|
-
- Session ids are unvalidated branded strings, so they are injectively escaped to a single safe path segment before use (no traversal, no collision). The resulting directory is reserved for additional session-owned artifacts; discovery reads only the fixed transcript filename.
|
|
67
|
+
Session ids are injectively escaped to one safe path segment before use (no traversal, no collision). The normalized cwd keeps the project directory readable for navigation; cwd strings that normalize alike share a project directory while session ids still select distinct session directories. `locate(meta)` returns `{ kind: 'jsonl', path }` for the fixed transcript inside the resolved directories, performing no filesystem I/O.
|
|
21
68
|
|
|
22
|
-
|
|
69
|
+
### Durability and crash semantics
|
|
23
70
|
|
|
24
|
-
|
|
25
|
-
|---|---|---|
|
|
26
|
-
| `root` | `string` (required) | Root directory for all session files. **No default** — a `process.cwd()` default would scatter files as the process's cwd changes (bash calls, subprocesses). An existing root must be a readable directory; an absent root is created on first materialization. |
|
|
27
|
-
| `packChunks` | `boolean` (default `true`) | Write eligible delta-chunk runs as packed rows (~60% smaller logical logs measured on a real coding session). Set `false` for one-event-per-line diagnostics; reading packed rows works regardless of this write-side switch. |
|
|
28
|
-
| `compression` | `'zstd' \| 'none'` | Defaults to `'zstd'`; `'none'` retains newline-delimited UTF-8 text. |
|
|
29
|
-
| `preparedSessionCacheSize` | positive integer (default `5`) | Maximum unpublished Sessions retained after cold history inspection for reuse by resume. |
|
|
30
|
-
| `writeBatchMaxDelayMs` | positive integer (default `200`) | Fixed coalescing window after an idle live-event queue receives work. Later events do not reset it; flush and teardown bypass it. It does not bound event-loop, serialized-operation, or backend latency. At most Node's `2_147_483_647` ms timer limit. |
|
|
71
|
+
A session is materialized lazily: `create(meta)` writes nothing, and the first `append` writes and `fsync`s the encoded header and first batch through a no-overwrite publish — so a created-but-never-appended session leaves nothing on disk unless a lifecycle consumer calls `ensureMaterialized`, which publishes one header frame without an event. Flushed events are never rewritten; each subsequent batch appends lines or one compressed frame, and a caught write or sync failure rolls the file back to its prior length. After a crash, `load` preserves an interrupted final turn: it keeps the complete decoded records of an incomplete last frame, truncates from that frame's start, and re-encodes the records with the synthetic tool, step, and turn closers required by the shared persistence contract. Only a never-fully-written torn tail is discarded; checksum, decompression, or structural failure in the committed prefix rejects as corruption.
|
|
31
72
|
|
|
32
|
-
|
|
73
|
+
### Reading the logs
|
|
33
74
|
|
|
34
|
-
|
|
75
|
+
`inspect(id)` returns an immutable balanced view without committing recovery. `readFrom(id, fromSeq)` returns stored events at or past a sequence number for watermark consumers; sequential media like JSONL parse the whole artifact and skip forward. With `compression: 'none'`, the log is newline-delimited text an external reader can consume directly; the compressed default must be read through the backend.
|
|
35
76
|
|
|
36
|
-
|
|
77
|
+
-----
|
|
37
78
|
|
|
38
|
-
|
|
79
|
+
<a id="understand-the-implementation"></a>
|
|
80
|
+
## Understand the implementation
|
|
39
81
|
|
|
40
|
-
|
|
82
|
+
<details>
|
|
83
|
+
<summary>Implementation internals — click to expand</summary>
|
|
41
84
|
|
|
42
|
-
|
|
43
|
-
- **Lazy materialization.** `create(meta)` writes nothing; on the first `append`, the backend writes and `fsync`s the encoded header and first batch in a temporary file. POSIX publishes it without overwrite via a hard link and `fsync`s the parent directory. Windows publishes it without overwrite via `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)` and creates missing directories through the same write-through pattern. A created-but-never-appended session leaves nothing on disk and is absent from `list`.
|
|
44
|
-
- **Append-only.** Flushed events are never rewritten. Subsequent raw batches append lines; compressed batches append one frame. Both paths `fsync`, and a caught write or sync failure rolls the file back to its prior byte length.
|
|
45
|
-
- **Crash recovery — preserve valid tail work.** `load` validates every complete compressed frame and scans their decompressed JSONL. If the last frame is structurally incomplete, the reader keeps its complete decoded records, truncates from that frame's start, and re-encodes those records with the synthetic tool, step, and turn closers required by the shared [persistence contract](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.md). Raw mode truncates from its first incomplete line. An existing compressed artifact with no complete header frame, a checksum/decompression failure in a complete frame, or a defect at or before the last committed `turn/end` is corruption and rejects.
|
|
46
|
-
- **Non-mutating inspection.** `inspect()` returns an immutable balanced logical view and may synthesize recovery closers in memory, without truncating an incomplete tail or changing the lightweight revision.
|
|
47
|
-
- **Contiguous-seq.** `append` rejects a batch whose first `seq` does not continue the stored log, and rejects non-JSON-serializable `event.data` naming the offending event type.
|
|
48
|
-
- **Lightweight revisions.** `listSnapshots(signal?)` identifies a log by its device, inode, size, and nanosecond timestamps, avoiding a full-log parse while changing after append, repair, replacement, or store changes. A full-prefix read requires the same identity before and after reading the bytes, and `readStoredRevision()` uses that identity to validate retained preparations without loading the log. Snapshot listing forwards the exact signal through artifact discovery and checks cancellation around every `stat`; because filesystem `stat` is not interruptible, cancellation waits for the active call to settle, then rejects without starting another.
|
|
85
|
+
This section explains the physical encoding and write path; the observable contract is covered in [Use this package](#use-this-package).
|
|
49
86
|
|
|
50
|
-
|
|
87
|
+
### Design concept
|
|
51
88
|
|
|
52
|
-
The
|
|
89
|
+
The backend is a thin storage layer over the shared [PersistenceCoordinator](../session-persistence/README.md#understand-the-implementation): it loads stored records, appends batches, commits repairs, and delegates lifecycle orchestration to the coordinator. Its physical identity is a file revision: device, inode, size, and nanosecond timestamps identify one log and change after append or repair, which is what `listSnapshots` and retained-preparation validation use.
|
|
53
90
|
|
|
91
|
+
### Physical encoding
|
|
92
|
+
|
|
93
|
+
The default artifact is a standard concatenation of independent [Zstandard frames](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.md): one checksummed frame containing only the header line, then one checksummed frame per durable append batch, using Node's built-in Zstandard API at its default compression level (no level knob). `sourceEventSeqs` uses a lossless storage representation: consecutive runs of at least three sequence numbers become `[start, end]` pairs, any other list stays verbatim, and reading expands the exact in-memory array. Listing reads and validates only the header frame. `compression: 'none'` keeps the same storage-form logical lines without frame compression. A root belongs to one encoding: startup discovery and targeted lookup reject the opposite suffix, and there is no format or compression migration, mixed-root fallback, or dual write. When `packChunks` is enabled, an eligible run of ≥3 consecutive same-block `assistant/chunk` delta events becomes one packed row (`text-chunks`/`reasoning-chunks`/`tool-call-chunks`) whose `seq0`/`time0` and per-member `dt` gaps reconstruct every member exactly; the lossless codec lives in `dsh-session` and reading is layout-blind, so packed, unpacked, and mixed files load identically.
|
|
94
|
+
|
|
95
|
+
### Source map
|
|
96
|
+
|
|
97
|
+
| File | Role |
|
|
98
|
+
|---|---|
|
|
99
|
+
| [`src/index.ts`](src/index.ts) | Plugin entry: `Config` schema, backend class, coordinator wiring |
|
|
100
|
+
| [`src/format.ts`](src/format.ts) | Log path derivation, header encoding, record scanning, packed-row layout |
|
|
101
|
+
| [`src/zstd.ts`](src/zstd.ts) | Zstandard frame compression, decoding, and frame scanning |
|
|
102
|
+
| [`src/win32.ts`](src/win32.ts) | Windows write-through publish and directory creation |
|
|
103
|
+
| [`src/invariant.ts`](src/invariant.ts) | Invariant companion (no runtime invariant; identity is enforced at the storage layer) |
|
|
104
|
+
|
|
105
|
+
</details>
|
|
106
|
+
|
|
107
|
+
-----
|
|
108
|
+
|
|
109
|
+
<a id="further-exploration"></a>
|
|
110
|
+
## Further Exploration
|
|
111
|
+
|
|
112
|
+
Read these pages when the package-level contract is not enough. They move from the shared persistence model to the sibling backend and the physical-format decisions.
|
|
113
|
+
|
|
114
|
+
- [Session persistence subsystem](../../../docs/subsystems/persistence.md) — backend-neutral service semantics and provider relationships.
|
|
115
|
+
- [Session persistence seam](../session-persistence/README.md) — the service contract this backend implements.
|
|
116
|
+
- [SQLite persistence backend](../session-persistence-sqlite/README.md) — the opt-in single-database alternative.
|
|
117
|
+
- [Project-session directory decision](../../../.agents/notes/implemented/architecture/2026-07-24-project-session-directories.md) — the layout tradeoff behind project and session directories.
|
|
118
|
+
- [Zstandard JSONL session logs](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.md) — the checksummed-frame encoding rationale.
|
|
119
|
+
|
|
120
|
+
-----
|
|
121
|
+
|
|
122
|
+
<a id="model-experience"></a>
|
|
54
123
|
## Model Experience
|
|
55
124
|
|
|
56
125
|
### Resumed conversation history
|
|
@@ -69,9 +138,24 @@ JSONL storage does not mutate live request prefixes. A resumed loop can reuse pr
|
|
|
69
138
|
|
|
70
139
|
## Known Limitations and Deferred Work
|
|
71
140
|
|
|
72
|
-
-
|
|
141
|
+
<a id="known-limitations-and-deferred-work"></a>
|
|
142
|
+
|
|
143
|
+
|
|
144
|
+
These limits define when this backend is a poor fit or needs special operational care. They are current package constraints, not a task backlog.
|
|
145
|
+
|
|
146
|
+
- **Only the configured encoding and current `SESSION_FORMAT_VERSION` (v0) load** — changing compression requires a separate or fresh root, or selecting raw mode; the pre-release format has no migration.
|
|
73
147
|
- **The flat-file storage layout does not load** — use a separate root or move pre-release artifacts into the project/session directory layout before loading.
|
|
74
148
|
- **Compressed files are not directly line-readable** — use the backend to load them, or select `compression: 'none'` before writing a fresh root when external line readers are required.
|
|
75
|
-
- **Nothing deletes session files** — logs accumulate under `root` until removed externally
|
|
76
|
-
- **One live writer per session** — append and repair are coordinated only inside the owning backend instance
|
|
149
|
+
- **Nothing deletes session files** — logs accumulate under `root` until removed externally; the seam has no deletion API.
|
|
150
|
+
- **One live writer per session** — append and repair are coordinated only inside the owning backend instance; another instance or process must not write the same session until that owner reaches quiescent disposal.
|
|
77
151
|
- **POSIX materialization requires hard-link support** — first append uses `link()` so same-id races fail instead of overwriting a committed log; Windows uses write-through rename without replacement.
|
|
152
|
+
|
|
153
|
+
<a id="dev-note"></a>
|
|
154
|
+
### Dev Note
|
|
155
|
+
|
|
156
|
+
<details>
|
|
157
|
+
<summary>Working context for maintainers — click to expand</summary>
|
|
158
|
+
|
|
159
|
+
None.
|
|
160
|
+
|
|
161
|
+
</details>
|
package/README.zh.md
CHANGED
|
@@ -1,12 +1,62 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: "面向部署方与维护者的随产品交付 JSONL 会话持久化后端说明,用于选择、配置或排查带可选 Zstandard 压缩的逐会话持久日志。"
|
|
3
|
+
kind: "package-reference"
|
|
4
|
+
---
|
|
5
|
+
|
|
1
6
|
# @deepseek-ai/dsh-session-persistence-jsonl
|
|
2
7
|
|
|
3
8
|
[English](README.md) | 中文
|
|
4
9
|
|
|
5
|
-
|
|
10
|
+
## 概述
|
|
11
|
+
|
|
12
|
+
`dsh-session-persistence-jsonl` 把每个会话存为一份仅追加 JSONL 日志——默认以带校验和的 Zstandard 帧存储,禁用压缩时以换行分隔的原始文本行存储。它提供与任何持久化后端相同的逻辑 `SessionEvent` 流,因此选择它不会改变 agent loop、模型或回放的任何行为;压缩、打包与崩溃恢复都是存储内部细节。当消费方需要按会话的磁盘产物时选择它:`locate(meta)` 返回 transcript 路径,选择 `compression: 'none'` 后日志可作为纯文本按行读取。根目录是唯一必填配置;持久性、延迟实体化与中断轮次恢复都随后端提供。
|
|
13
|
+
|
|
14
|
+
## 目录
|
|
15
|
+
|
|
16
|
+
- [使用本包](#use-this-package)
|
|
17
|
+
- [理解实现](#understand-the-implementation)
|
|
18
|
+
- [进一步探索](#further-exploration)
|
|
19
|
+
- [模型体验](#model-experience)
|
|
20
|
+
- [已知限制与延期工作](#known-limitations-and-deferred-work)
|
|
21
|
+
- [开发备注](#dev-note)
|
|
22
|
+
|
|
23
|
+
-----
|
|
24
|
+
|
|
25
|
+
<a id="use-this-package"></a>
|
|
26
|
+
## 使用本包
|
|
27
|
+
|
|
28
|
+
当组合需要由按会话文件支撑的持久会话时挂载此后端。常用路径是显式的:加载会话服务、挂载后端,然后给出根目录。
|
|
29
|
+
|
|
30
|
+
### 何时选择
|
|
31
|
+
|
|
32
|
+
当消费方受益于每会话一份产物——导航、外部工具或可逐行读取的原始日志——时选择此后端。当单一可查询数据库更适合部署时,选择 [SQLite](../session-persistence-sqlite/README.zh.md)。后端把会话保存在部署控制的根下:项目本地、共享、临时或集中式。
|
|
6
33
|
|
|
7
|
-
|
|
34
|
+
### 最小配置
|
|
8
35
|
|
|
36
|
+
```yaml
|
|
37
|
+
- name: '@deepseek-ai/dsh-session'
|
|
38
|
+
- name: '@deepseek-ai/dsh-session-persistence-jsonl'
|
|
39
|
+
config:
|
|
40
|
+
root: /absolute/path/to/session-logs
|
|
9
41
|
```
|
|
42
|
+
|
|
43
|
+
`root` 必填且无默认值:`process.cwd()` 默认值会随进程 cwd 变更而分散会话文件。现有根必须是可读目录;缺失根在第一次实体化时创建。
|
|
44
|
+
|
|
45
|
+
| 字段 | 默认值 | 含义 |
|
|
46
|
+
|---|---|---|
|
|
47
|
+
| `root` | 必填 | 所有会话文件的根目录 |
|
|
48
|
+
| `packChunks` | `true` | 把符合条件的 `assistant/chunk` 连续段写为打包行;`false` 为诊断保留每事件一行 |
|
|
49
|
+
| `compression` | `'zstd'` | 物理编码:`'zstd'` 带校验和帧,或 `'none'` 换行分隔 UTF-8 文本 |
|
|
50
|
+
| `preparedSessionCacheSize` | `5` | 为恢复复用而保留的冷会话准备结果数量 |
|
|
51
|
+
| `writeBatchMaxDelayMs` | `200` | 实时事件的固定聚合窗口,单位为毫秒 |
|
|
52
|
+
|
|
53
|
+
生成的[配置目录](../../../docs/config-catalog.zh.md#deepseek-aidsh-session-persistence-jsonl)是每个受支持字段及其 JSDoc 的穷尽式真源。
|
|
54
|
+
|
|
55
|
+
### 磁盘布局
|
|
56
|
+
|
|
57
|
+
每个会话在可读项目目录下获得一个会话自有目录;日志第一个逻辑行是不可变 `SessionHeader`,之后每个逻辑事件一条存储记录(或每个符合条件的连续段一条打包分片行)。存储记录使用下文所述的无损来源序列表示:
|
|
58
|
+
|
|
59
|
+
```text
|
|
10
60
|
<root>/
|
|
11
61
|
--<normalized-cwd>--/ # readable project directory (or _no-cwd/)
|
|
12
62
|
<encoded-id>/ # session-owned directory
|
|
@@ -14,64 +64,98 @@ JSONL 持久会话存储后端:`SessionPersistence` 的一个具体实现(`d
|
|
|
14
64
|
session.jsonl # only with compression: 'none'
|
|
15
65
|
```
|
|
16
66
|
|
|
17
|
-
|
|
18
|
-
- 存储记录是原样 `SessionEvent` JSON,或在 `packChunks` 已启用且连续段符合条件时写入的**打包分片行**(`text-chunks` / `reasoning-chunks` / `tool-call-chunks`;像 header 的 `session` 一样不带斜杠,因此行 tag 不会与事件类型混淆):一行保存至少 3 个连续同 block `assistant/chunk` delta 事件,`seq0`/`time0` 和各成员的 `dt` 间隔精确重建每个成员的 `seq`/`time`。无损 codec 位于 `@deepseek-ai/dsh-session`(`packChunkRuns`/`decodeStorageRecord`),并使用精确形态 allowlist:任何未识别内容原样存储。读取与布局无关:`load` 始终解码行,因此打包、非打包和混合文件加载结果一致。
|
|
19
|
-
- 项目目录保留规范化 cwd 的可读形式,便于导航,并限制在文件系统组件上限内。分隔符替换和截断刻意有损,因此规范化相同的 cwd 字符串共享项目目录;会话 id 仍选择不同会话目录。在不区分大小写的文件系统上,只有文件系统规范化将两种写法解析到同一 transcript(文本记录)时,身份验证才接受备选路径写法。配置根仍由部署控制:可以是项目本地、共享、临时或集中式。[项目会话目录决策](../../../.agents/notes/implemented/architecture/2026-07-24-project-session-directories.zh.md) 记录这项取舍。
|
|
20
|
-
- 会话 id 是未验证的带品牌类型的字符串,因此在使用前单射转义为一个安全路径段(无遍历、无冲突)。结果目录保留给其他会话自有产物;发现只读取固定 transcript 文件名。
|
|
67
|
+
会话 id 在使用前被单射转义为一个安全路径段(无遍历、无冲突)。规范化 cwd 让项目目录保持可读、便于导航;规范化相同的 cwd 字符串共享项目目录,而会话 id 仍选择不同会话目录。`locate(meta)` 返回已解析目录内固定 transcript 的 `{ kind: 'jsonl', path }`,不执行任何文件系统 I/O。
|
|
21
68
|
|
|
22
|
-
|
|
69
|
+
### 持久性与崩溃语义
|
|
23
70
|
|
|
24
|
-
|
|
25
|
-
|---|---|---|
|
|
26
|
-
| `root` | `string`(必需) | 所有会话文件的根目录。**无默认值**:`process.cwd()` 默认值会随进程 cwd 变更(bash 调用、子进程)而分散文件。现有根必须是可读目录;缺失根在第一次实体化时创建。 |
|
|
27
|
-
| `packChunks` | `boolean`(默认 `true`) | 将符合条件的 delta 分片连续段写为打包行(在真实编程会话上测得逻辑日志约小 60%)。设为 `false` 可用于每事件一行诊断;无论该写入侧开关如何,都能读取打包行。 |
|
|
28
|
-
| `compression` | `'zstd' \| 'none'` | 默认 `'zstd'`;`'none'` 保留换行分隔 UTF-8 文本。 |
|
|
29
|
-
| `preparedSessionCacheSize` | 正整数(默认 `5`) | 冷历史检查后保留、供恢复复用的未发布会话数量上限。 |
|
|
30
|
-
| `writeBatchMaxDelayMs` | 正整数(默认 `200`) | 空闲的活动事件队列收到待写入事件后开启的固定合并窗口。后续事件不会重置窗口;flush 与 teardown 会绕过它。该值不限制事件循环、串行化操作或后端延迟。最大值为 Node 计时器上限 `2_147_483_647` ms。 |
|
|
71
|
+
会话延迟实体化:`create(meta)` 不写入任何内容,第一次 `append` 通过无覆盖发布写入并 `fsync` 编码后的 header 与第一批——因此已创建但从未 append 的会话不留下任何磁盘内容,除非生命周期消费方调用 `ensureMaterialized`,以无事件的单个 header 帧发布它。已 flush 事件绝不重写;后续每个批次追加行或一个压缩帧,捕获到写入或同步失败时把文件回滚到之前的字节长度。崩溃后,`load` 保留被中断的最终轮次:保留不完整最后帧中完整解码的记录,从该帧开头截断,并按共享持久化约定的要求,用合成工具、步骤与轮次 closer 重新编码这些记录。只有从未完整写入的撕裂尾部被丢弃;已提交前缀中的校验和、解压或结构失败以损坏拒绝。
|
|
31
72
|
|
|
32
|
-
|
|
73
|
+
### 读取日志
|
|
33
74
|
|
|
34
|
-
|
|
75
|
+
`inspect(id)` 返回不可变的平衡视图,不提交恢复。`readFrom(id, fromSeq)` 为水位消费方返回该序列号及之后的已存储事件;JSONL 这类顺序介质解析整个产物并向前跳过。选择 `compression: 'none'` 后,日志是外部读取方可直接消费的换行分隔文本;压缩默认值必须经后端读取。
|
|
35
76
|
|
|
36
|
-
|
|
77
|
+
-----
|
|
37
78
|
|
|
38
|
-
|
|
79
|
+
<a id="understand-the-implementation"></a>
|
|
80
|
+
## 理解实现
|
|
39
81
|
|
|
40
|
-
|
|
82
|
+
<details>
|
|
83
|
+
<summary>实现细节——点击展开</summary>
|
|
41
84
|
|
|
42
|
-
-
|
|
43
|
-
- **延迟实体化。**`create(meta)` 不写入;第一次 `append` 将编码 header 和第一批写入临时文件并执行 `fsync`。POSIX 通过硬链接无覆盖发布,并对父目录 `fsync`。Windows 通过 `MoveFileExW(..., MOVEFILE_WRITE_THROUGH)` 无覆盖发布,并通过同一 write-through pattern 创建缺失目录。已创建但从未 append 的会话不留下磁盘内容,不在 `list` 中。
|
|
44
|
-
- **仅追加。** 已 flush 事件绝不重写。后续原始批次 append 行;压缩批次 append 一个 frame。两条路径都执行 `fsync`,并在捕获到写入或同步失败时回滚到之前字节长度。
|
|
45
|
-
- **崩溃恢复:保留有效尾部工作。**`load` 验证每个完整压缩 frame,并扫描解压 JSONL。最后 frame 结构不完整时,读取器保留其完整解码记录,从 frame 开头截断,并使用共享[持久化约定](../../../.agents/notes/implemented/architecture/2026-06-14-session-persistence.zh.md) 需要的合成工具、步骤和轮次 closer 重新编码这些记录。原始 mode 从第一个不完整行截断。已经存在却没有完整 header frame 的压缩工件、完整 frame 中的 checksum/解压失败,或位于最后已提交的 `turn/end` 处或之前的缺陷都属于损坏,会被拒绝。
|
|
46
|
-
- **非修改式检查。**`inspect()` 返回不可变、平衡的逻辑视图,并可在内存中合成恢复 closer,但不会截断不完整尾部或更改轻量修订。
|
|
47
|
-
- **连续 seq。**`append` 拒绝第一个 `seq` 不继续已存储日志的批次,并拒绝无法 JSON 序列化的 `event.data`,同时命名违规事件类型。
|
|
48
|
-
- **轻量修订。**`listSnapshots(signal?)` 使用 device、inode、size 和纳秒时间戳标识日志,避免解析完整日志;该标识会在 append、修复、替换或存储变更后改变。完整前缀读取要求读取字节前后的身份一致,`readStoredRevision()` 使用同一身份校验保留的 preparation,而不加载日志。快照列表通过产物发现原样转发该信号,并在每个 `stat` 前后检查取消;由于文件系统 `stat` 不可中断,取消会等待活动调用完成,然后在不启动另一次调用的情况下拒绝。
|
|
85
|
+
本节说明物理编码与写入路径;可观察约定已在[使用本包](#use-this-package)中说明。
|
|
49
86
|
|
|
50
|
-
|
|
87
|
+
### 设计理念
|
|
51
88
|
|
|
52
|
-
|
|
89
|
+
该后端是共享 [PersistenceCoordinator](../session-persistence/README.zh.md#understand-the-implementation) 之上的一层薄存储:它加载已存储记录、追加批次、提交修复,并把生命周期编排委托给协调器。其物理身份是文件修订值:device、inode、size 与纳秒时间戳标识一份日志,并在追加或修复后改变,这正是 `listSnapshots` 与保留准备结果校验所使用的身份。
|
|
53
90
|
|
|
91
|
+
### 物理编码
|
|
92
|
+
|
|
93
|
+
默认产物是独立 [Zstandard 帧](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.zh.md) 的标准拼接:一个仅包含 header 行的带校验和帧,后跟每个持久 append 批次一个带校验和帧,使用 Node 内置 Zstandard API 的默认压缩级别(无级别开关)。`sourceEventSeqs` 使用无损存储形式:至少包含三个序列号的连续段会变成 `[start, end]` 区间对,其他列表原样保留;读取时会展开回精确的内存数组。列表只读取并验证 header 帧。`compression: 'none'` 保留相同的存储形式逻辑行,但不使用帧压缩。一个根只属于一种编码:启动发现与定向查找会拒绝相反后缀,且不提供格式或压缩迁移、混合根回退或双写。启用 `packChunks` 时,符合条件的 ≥3 个连续同 block `assistant/chunk` delta 事件连续段会变成一行打包行(`text-chunks`/`reasoning-chunks`/`tool-call-chunks`),其 `seq0`/`time0` 与各成员的 `dt` 间隔精确重建每个成员;无损 codec 位于 `dsh-session`,读取与布局无关,因此打包、非打包与混合文件加载结果一致。
|
|
94
|
+
|
|
95
|
+
### 源码地图
|
|
96
|
+
|
|
97
|
+
| 文件 | 职责 |
|
|
98
|
+
|---|---|
|
|
99
|
+
| [`src/index.ts`](src/index.ts) | 插件入口:`Config` schema、后端类、协调器接线 |
|
|
100
|
+
| [`src/format.ts`](src/format.ts) | 日志路径派生、header 编码、记录扫描、打包行布局 |
|
|
101
|
+
| [`src/zstd.ts`](src/zstd.ts) | Zstandard 帧压缩、解码与帧扫描 |
|
|
102
|
+
| [`src/win32.ts`](src/win32.ts) | Windows write-through 发布与目录创建 |
|
|
103
|
+
| [`src/invariant.ts`](src/invariant.ts) | 不变式伴生插件(无运行时不变式;身份在存储层强制) |
|
|
104
|
+
|
|
105
|
+
</details>
|
|
106
|
+
|
|
107
|
+
-----
|
|
108
|
+
|
|
109
|
+
<a id="further-exploration"></a>
|
|
110
|
+
## 进一步探索
|
|
111
|
+
|
|
112
|
+
当包级约定不够用时阅读以下页面。它们从共享持久化模型逐步进入同级后端与物理格式决策。
|
|
113
|
+
|
|
114
|
+
- [会话持久化子系统](../../../docs/subsystems/persistence.zh.md)——后端无关的服务语义与提供方关系。
|
|
115
|
+
- [会话持久化 seam](../session-persistence/README.zh.md)——本后端实现的服务约定。
|
|
116
|
+
- [SQLite 持久化后端](../session-persistence-sqlite/README.zh.md)——可选启用的单数据库替代方案。
|
|
117
|
+
- [项目会话目录决策](../../../.agents/notes/implemented/architecture/2026-07-24-project-session-directories.zh.md)——项目与会话目录布局背后的取舍。
|
|
118
|
+
- [Zstandard JSONL 会话日志](../../../.agents/notes/implemented/architecture/2026-07-19-zstandard-jsonl-session-logs.zh.md)——带校验和帧编码的理由。
|
|
119
|
+
|
|
120
|
+
-----
|
|
121
|
+
|
|
122
|
+
<a id="model-experience"></a>
|
|
54
123
|
## 模型体验
|
|
55
124
|
|
|
56
125
|
### 恢复的对话历史
|
|
57
126
|
|
|
58
|
-
####
|
|
127
|
+
#### 模型看到什么
|
|
59
128
|
|
|
60
|
-
JSONL
|
|
129
|
+
JSONL 存储不会向实时请求提供提示词或 schema。加载会恢复已存储的表层历史,并保留之前的请求 header 用于重建;新 loop 组合当前 envelope。恢复会用 `TOOL_NOT_STARTED` 平衡没有持久调用的 assistant 请求;持久调用无结果时则变为 `TOOL_OUTCOME_UNKNOWN`,它要求模型只重试只读或幂等工作,并验证可能的副作用或询问用户。原始 `assistant/chunk` 记录不会重复生成消息。
|
|
61
130
|
|
|
62
131
|
#### Token 影响
|
|
63
132
|
|
|
64
|
-
|
|
133
|
+
实时请求不新增 token。恢复后的 agent(智能体)会因保留的历史、当前 envelope,以及每个中断调用中以引用形式加入的修复结果文本而消耗 token。
|
|
65
134
|
|
|
66
135
|
#### KV Cache 影响
|
|
67
136
|
|
|
68
|
-
JSONL 存储不修改实时请求前缀。只有重建历史、当前 envelope
|
|
137
|
+
JSONL 存储不修改实时请求前缀。只有重建历史、当前 envelope 与模型路由匹配时,恢复 loop 才能重用提供方缓存;崩溃修复结果仅追加。
|
|
138
|
+
|
|
139
|
+
## 已知限制与延期工作
|
|
140
|
+
|
|
141
|
+
<a id="known-limitations-and-deferred-work"></a>
|
|
142
|
+
|
|
143
|
+
|
|
144
|
+
这些限制说明本后端何时不合适,或何时需要特别的运维注意。它们是当前包约束,不是任务积压。
|
|
145
|
+
|
|
146
|
+
- **只加载已配置编码和当前 `SESSION_FORMAT_VERSION`(v0)**——更改压缩需要独立或全新根,或选择原始文本模式;预发布格式没有迁移。
|
|
147
|
+
- **平铺文件存储布局不加载**——加载前使用独立根,或将预发布产物移入项目/会话目录布局。
|
|
148
|
+
- **压缩文件不能直接按行读取**——使用后端加载;或在写入新根前选择 `compression: 'none'`,供外部行读取方使用。
|
|
149
|
+
- **不删除会话文件**——日志在 `root` 下累积,直到外部移除;seam 无删除接口。
|
|
150
|
+
- **每会话一个活动写入方**——append 与修复只在所属后端实例内协调;在该所有者达到完全停稳的 dispose 前,另一实例或进程不得写入同一会话。
|
|
151
|
+
- **POSIX 实体化需要硬链接支持**——第一次 append 使用 `link()`,使同 id 竞态失败而不覆盖已提交日志;Windows 使用无替换 write-through rename。
|
|
152
|
+
|
|
153
|
+
<a id="dev-note"></a>
|
|
154
|
+
### 开发备注
|
|
155
|
+
|
|
156
|
+
<details>
|
|
157
|
+
<summary>维护者的工作上下文——点击展开</summary>
|
|
69
158
|
|
|
70
|
-
|
|
159
|
+
无。
|
|
71
160
|
|
|
72
|
-
|
|
73
|
-
- **平铺文件存储布局不加载**:加载前使用独立根,或将预发布产物移入项目/会话目录布局。
|
|
74
|
-
- **压缩文件不能直接按行读取**:使用后端加载;或在写入新根前选择 `compression: 'none'`,以便外部行 reader 使用。
|
|
75
|
-
- **不删除会话文件**:日志在 `root` 下累积,直到外部移除(seam 无删除接口)。
|
|
76
|
-
- **每会话一个活动 writer**:append 和修复只在所属后端实例内协调。在所有者完成完全停稳的 dispose 前,其他后端实例或进程不得写入同一会话;初始同 id 发布仍通过 POSIX 无覆盖硬链接或 Windows 无替换 write-through rename 保持冲突安全。
|
|
77
|
-
- **POSIX 实体化需要硬链接支持**:第一次 append 使用 `link()`,使同 id 竞态失败,而不覆盖已提交日志;Windows 使用无替换 write-through rename。
|
|
161
|
+
</details>
|
package/lib/index.js
CHANGED
|
@@ -6,7 +6,7 @@ import { performance } from "node:perf_hooks";
|
|
|
6
6
|
import { scheduler } from "node:timers/promises";
|
|
7
7
|
import { randomBytes } from "node:crypto";
|
|
8
8
|
import { DEFAULT_PREPARED_SESSION_CACHE_SIZE, DEFAULT_WRITE_BATCH_MAX_DELAY_MS, MAX_WRITE_BATCH_DELAY_MS, PersistenceCoordinator, SessionFormatUnsupportedError, SessionPersistence, SessionPersistenceRevision, sessionFormatVersionRefusal } from "@deepseek-ai/dsh-session-persistence";
|
|
9
|
-
import { SESSION_FORMAT_VERSION, decodeStorageRecord, packChunkRuns } from "@deepseek-ai/dsh-session";
|
|
9
|
+
import { SESSION_FORMAT_VERSION, decodeSeqRanges, decodeStorageRecord, encodeSeqRanges, packChunkRuns } from "@deepseek-ai/dsh-session";
|
|
10
10
|
import { constants, createZstdDecompress, zstdCompress, zstdDecompress, zstdDecompressSync } from "node:zlib";
|
|
11
11
|
import { promisify } from "node:util";
|
|
12
12
|
import { constants as constants$1 } from "node:buffer";
|
|
@@ -160,22 +160,53 @@ function logPath(root, cwd, id, compression) {
|
|
|
160
160
|
* Serialize an event batch as JSONL lines (no trailing newline). With
|
|
161
161
|
* `packChunks` on, delta-chunk runs pack into `text-chunks` /
|
|
162
162
|
* `reasoning-chunks` / `tool-call-chunks` storage rows; off writes one event
|
|
163
|
-
* per line
|
|
164
|
-
* either way ({@link scanLog} always decodes rows),
|
|
165
|
-
* newly written bytes.
|
|
163
|
+
* per line. Both modes range-encode provenance at the storage boundary.
|
|
164
|
+
* Reading is layout-blind either way ({@link scanLog} always decodes rows),
|
|
165
|
+
* so the switch changes only newly written bytes.
|
|
166
166
|
* @param events - the batch to serialize, in log order.
|
|
167
167
|
* @param packChunks - whether to pack delta runs into storage rows.
|
|
168
168
|
* @returns the batch's JSONL text; the writer adds the final newline.
|
|
169
169
|
*/
|
|
170
170
|
function eventLines(events, packChunks) {
|
|
171
|
-
return (packChunks ? packChunkRuns(events) : events).map((record) => JSON.stringify(record)).join("\n");
|
|
171
|
+
return (packChunks ? packChunkRuns(events) : events).map((record) => JSON.stringify(encodeProvenanceForStorage(record))).join("\n");
|
|
172
|
+
}
|
|
173
|
+
/**
|
|
174
|
+
* Losslessly shrink a record's `sourceEventSeqs` for the log: consecutive
|
|
175
|
+
* runs of at least three seqs become `[start, end]` pairs, and any other list
|
|
176
|
+
* stays verbatim.
|
|
177
|
+
* @param record - one stored record (event or packed row).
|
|
178
|
+
* @returns the record with its provenance in storage form (widened from the
|
|
179
|
+
* in-memory `number[]`; {@link expandProvenanceFromStorage} restores it).
|
|
180
|
+
*/
|
|
181
|
+
function encodeProvenanceForStorage(record) {
|
|
182
|
+
if (!("sourceEventSeqs" in record)) return record;
|
|
183
|
+
return {
|
|
184
|
+
...record,
|
|
185
|
+
sourceEventSeqs: encodeSeqRanges(record.sourceEventSeqs)
|
|
186
|
+
};
|
|
187
|
+
}
|
|
188
|
+
/**
|
|
189
|
+
* Expand a parsed line's storage-form provenance back to `number[]`.
|
|
190
|
+
* @param parsed - the JSON-parsed value of one stored line.
|
|
191
|
+
* @returns the value with provenance expanded.
|
|
192
|
+
* @throws when the record or its storage-form provenance is malformed.
|
|
193
|
+
*/
|
|
194
|
+
function expandProvenanceFromStorage(parsed) {
|
|
195
|
+
if (typeof parsed !== "object" || parsed === null || Array.isArray(parsed)) throw new TypeError("stored session records must be objects");
|
|
196
|
+
const record = parsed;
|
|
197
|
+
if (record.sourceEventSeqs === void 0) return parsed;
|
|
198
|
+
if (!Number.isSafeInteger(record.seq) || record.seq < 0) throw new TypeError("stored session event seq must be a non-negative safe integer");
|
|
199
|
+
return {
|
|
200
|
+
...record,
|
|
201
|
+
sourceEventSeqs: decodeSeqRanges(record.sourceEventSeqs, record.seq)
|
|
202
|
+
};
|
|
172
203
|
}
|
|
173
204
|
/** Parse one complete header record supplied independently from event rows. */
|
|
174
205
|
/**
|
|
175
206
|
* Refuse a header carrying a format version this build does not read BEFORE
|
|
176
207
|
* validating the current header shape or decoding any event row: a future
|
|
177
|
-
* format need not satisfy
|
|
178
|
-
* see "upgrade the harness", never "corrupt session log".
|
|
208
|
+
* format need not satisfy this build's structural checks at all, and its user
|
|
209
|
+
* must see "upgrade the harness", never "corrupt session log".
|
|
179
210
|
* @param parsed - the JSON-parsed first line of a session artifact.
|
|
180
211
|
*/
|
|
181
212
|
function refuseForeignFormatVersion(parsed) {
|
|
@@ -276,7 +307,7 @@ var SessionLogScanner = class {
|
|
|
276
307
|
this.eventLine += 1;
|
|
277
308
|
let decoded;
|
|
278
309
|
try {
|
|
279
|
-
decoded = decodeStorageRecord(JSON.parse(line.toString("utf8")));
|
|
310
|
+
decoded = decodeStorageRecord(expandProvenanceFromStorage(JSON.parse(line.toString("utf8"))));
|
|
280
311
|
} catch {
|
|
281
312
|
this.issue ??= /* @__PURE__ */ new Error(`corrupt session log: unparsable committed event at line ${this.eventLine}`);
|
|
282
313
|
return;
|
|
@@ -809,6 +840,9 @@ var JsonlSessionPersistence = class extends SessionPersistence {
|
|
|
809
840
|
create(meta) {
|
|
810
841
|
return this.coordinator.create(meta);
|
|
811
842
|
}
|
|
843
|
+
ensureMaterialized(session) {
|
|
844
|
+
return this.coordinator.ensureMaterialized(session);
|
|
845
|
+
}
|
|
812
846
|
append(id, events) {
|
|
813
847
|
return this.coordinator.append(id, events);
|
|
814
848
|
}
|
|
@@ -821,6 +855,9 @@ var JsonlSessionPersistence = class extends SessionPersistence {
|
|
|
821
855
|
inspect(id, signal) {
|
|
822
856
|
return this.coordinator.inspect(id, signal);
|
|
823
857
|
}
|
|
858
|
+
borrowSession(id, signal) {
|
|
859
|
+
return this.coordinator.borrowSession(id, signal);
|
|
860
|
+
}
|
|
824
861
|
readFrom(id, fromSeq, signal) {
|
|
825
862
|
return this.coordinator.readFrom(id, fromSeq, signal);
|
|
826
863
|
}
|
|
@@ -1023,6 +1060,10 @@ var JsonlSessionPersistence = class extends SessionPersistence {
|
|
|
1023
1060
|
if (isMaterialized) await this.appendLines(meta, events);
|
|
1024
1061
|
else await this.materialize(meta, events);
|
|
1025
1062
|
}
|
|
1063
|
+
/** Materialize a header-only JSONL artifact for an explicitly durable empty session. */
|
|
1064
|
+
async materializeHeader(meta) {
|
|
1065
|
+
await this.materialize(meta, []);
|
|
1066
|
+
}
|
|
1026
1067
|
/**
|
|
1027
1068
|
* Make a crash repair durable: truncate a torn tail, restore complete events
|
|
1028
1069
|
* decoded from it, then append synthetic closers. Two fsync'd steps — the seam
|
|
@@ -1032,6 +1073,7 @@ var JsonlSessionPersistence = class extends SessionPersistence {
|
|
|
1032
1073
|
if (tornMarker !== void 0) await this.repair(meta, tornMarker.truncateTo);
|
|
1033
1074
|
const repairedEvents = [...tornMarker?.recoveredEvents ?? [], ...closers];
|
|
1034
1075
|
if (repairedEvents.length > 0) await this.appendLines(meta, repairedEvents);
|
|
1076
|
+
if (tornMarker !== void 0) this.ctx.logger.warn(`${this.name}: session "${meta.id}" recovered from a torn tail; incomplete tail bytes were discarded`);
|
|
1035
1077
|
}
|
|
1036
1078
|
/** List valid unique stored sessions' metadata (header line only — no full-log parse). */
|
|
1037
1079
|
async list(signal) {
|
|
@@ -1170,6 +1212,7 @@ var JsonlSessionPersistence = class extends SessionPersistence {
|
|
|
1170
1212
|
/** Encode the header and first batch without combining their frame boundaries. */
|
|
1171
1213
|
async encodeMaterialization(meta, events) {
|
|
1172
1214
|
const header = JSON.stringify(toHeaderLine(meta)) + "\n";
|
|
1215
|
+
if (events.length === 0) return this.compression === "none" ? header : compressZstdFrame(header);
|
|
1173
1216
|
const body = eventLines(events, this.packChunks) + "\n";
|
|
1174
1217
|
if (this.compression === "none") return header + body;
|
|
1175
1218
|
const headerFrame = await compressZstdFrame(header);
|
|
@@ -1435,7 +1478,8 @@ var JsonlSessionPersistence = class extends SessionPersistence {
|
|
|
1435
1478
|
} catch (error) {
|
|
1436
1479
|
/* v8 ignore else -- Windows reports file-valued parents as ENOENT; POSIX covers direct ENOTDIR. */
|
|
1437
1480
|
if (isENOENT(error)) {
|
|
1438
|
-
|
|
1481
|
+
/* v8 ignore next -- native Windows coverage exercises this platform dispatch; POSIX reports ENOTDIR from open */
|
|
1482
|
+
if (process.platform === "win32") await this.assertLogParentAllowsAbsence(path);
|
|
1439
1483
|
return false;
|
|
1440
1484
|
}
|
|
1441
1485
|
/* v8 ignore next -- Windows repairs ENOTDIR from ENOENT above; POSIX covers direct ENOTDIR. */
|
package/lib/types/format.d.ts
CHANGED
|
@@ -97,9 +97,9 @@ export declare function logPath(root: string, cwd: string | undefined, id: Sessi
|
|
|
97
97
|
* Serialize an event batch as JSONL lines (no trailing newline). With
|
|
98
98
|
* `packChunks` on, delta-chunk runs pack into `text-chunks` /
|
|
99
99
|
* `reasoning-chunks` / `tool-call-chunks` storage rows; off writes one event
|
|
100
|
-
* per line
|
|
101
|
-
* either way ({@link scanLog} always decodes rows),
|
|
102
|
-
* newly written bytes.
|
|
100
|
+
* per line. Both modes range-encode provenance at the storage boundary.
|
|
101
|
+
* Reading is layout-blind either way ({@link scanLog} always decodes rows),
|
|
102
|
+
* so the switch changes only newly written bytes.
|
|
103
103
|
* @param events - the batch to serialize, in log order.
|
|
104
104
|
* @param packChunks - whether to pack delta runs into storage rows.
|
|
105
105
|
* @returns the batch's JSONL text; the writer adds the final newline.
|
package/lib/types/index.d.ts
CHANGED
|
@@ -7,8 +7,8 @@
|
|
|
7
7
|
*/
|
|
8
8
|
import { Context } from '@deepseek-ai/cordis';
|
|
9
9
|
import z from '@deepseek-ai/schemastery';
|
|
10
|
-
import { SessionPersistence, type PersistenceBackend, type SessionLocation, type SessionPersistenceSnapshot, type SessionInspection, type SessionPersistenceRevision as PersistenceRevision, type SessionRawArtifact, type StoredPrefix } from '@deepseek-ai/dsh-session-persistence';
|
|
11
|
-
import type { SessionEvent, SessionId, SessionHeader, SessionPreparation } from '@deepseek-ai/dsh-session';
|
|
10
|
+
import { SessionPersistence, type BorrowedSessionSource, type PersistenceBackend, type SessionLocation, type SessionPersistenceSnapshot, type SessionInspection, type SessionPersistenceRevision as PersistenceRevision, type SessionRawArtifact, type StoredPrefix } from '@deepseek-ai/dsh-session-persistence';
|
|
11
|
+
import type { Session, SessionEvent, SessionId, SessionHeader, SessionPreparation } from '@deepseek-ai/dsh-session';
|
|
12
12
|
import { type JsonlCompression } from './format.ts';
|
|
13
13
|
export type { JsonlCompression } from './format.ts';
|
|
14
14
|
/** Loader schema for the JSONL artifact's physical encoding. */
|
|
@@ -69,10 +69,12 @@ export declare class JsonlSessionPersistence extends SessionPersistence implemen
|
|
|
69
69
|
/** Resolve the absolute target path without touching the filesystem. */
|
|
70
70
|
locate(meta: SessionHeader): SessionLocation;
|
|
71
71
|
create(meta: SessionHeader): Promise<void>;
|
|
72
|
+
ensureMaterialized(session: Session): Promise<void>;
|
|
72
73
|
append(id: SessionId, events: readonly SessionEvent[]): Promise<void>;
|
|
73
74
|
prepare(id: SessionId, signal?: AbortSignal): Promise<SessionPreparation>;
|
|
74
75
|
load(id: SessionId): Promise<SessionInspection>;
|
|
75
76
|
inspect(id: SessionId, signal?: AbortSignal): Promise<SessionInspection>;
|
|
77
|
+
borrowSession(id: SessionId, signal?: AbortSignal): Promise<BorrowedSessionSource>;
|
|
76
78
|
readFrom(id: SessionId, fromSeq: number, signal?: AbortSignal): Promise<{
|
|
77
79
|
meta: SessionHeader;
|
|
78
80
|
events: SessionEvent[];
|
|
@@ -116,6 +118,8 @@ export declare class JsonlSessionPersistence extends SessionPersistence implemen
|
|
|
116
118
|
private readZstdPrefix;
|
|
117
119
|
/** Durably append a batch, lazily materializing the file when not yet present. */
|
|
118
120
|
appendBatch(meta: SessionHeader, events: readonly SessionEvent[], isMaterialized: boolean): Promise<void>;
|
|
121
|
+
/** Materialize a header-only JSONL artifact for an explicitly durable empty session. */
|
|
122
|
+
materializeHeader(meta: SessionHeader): Promise<void>;
|
|
119
123
|
/**
|
|
120
124
|
* Make a crash repair durable: truncate a torn tail, restore complete events
|
|
121
125
|
* decoded from it, then append synthetic closers. Two fsync'd steps — the seam
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@deepseek-ai/dsh-session-persistence-jsonl",
|
|
3
3
|
"description": "JSONL durable session persistence backend for the DeepSeek Harness",
|
|
4
|
-
"version": "0.1.
|
|
4
|
+
"version": "0.1.2-alpha.2",
|
|
5
5
|
"publishConfig": {
|
|
6
6
|
"access": "public"
|
|
7
7
|
},
|
|
@@ -32,19 +32,19 @@
|
|
|
32
32
|
],
|
|
33
33
|
"license": "MIT",
|
|
34
34
|
"peerDependencies": {
|
|
35
|
-
"@deepseek-ai/dsh-invariants": "^0.1.
|
|
36
|
-
"@deepseek-ai/dsh-session": "^0.1.
|
|
37
|
-
"@deepseek-ai/
|
|
38
|
-
"@deepseek-ai/
|
|
35
|
+
"@deepseek-ai/dsh-invariants": "^0.1.2-alpha.2",
|
|
36
|
+
"@deepseek-ai/dsh-session": "^0.1.2-alpha.2",
|
|
37
|
+
"@deepseek-ai/dsh-session-persistence": "^0.1.2-alpha.2",
|
|
38
|
+
"@deepseek-ai/cordis": "^4.0.2"
|
|
39
39
|
},
|
|
40
40
|
"dependencies": {
|
|
41
41
|
"koffi": "^3.1.0",
|
|
42
|
-
"@deepseek-ai/schemastery": "^3.18.
|
|
42
|
+
"@deepseek-ai/schemastery": "^3.18.2"
|
|
43
43
|
},
|
|
44
44
|
"devDependencies": {
|
|
45
|
-
"@deepseek-ai/dsh-invariants": "^0.1.
|
|
46
|
-
"@deepseek-ai/dsh-session": "^0.1.
|
|
47
|
-
"@deepseek-ai/
|
|
48
|
-
"@deepseek-ai/
|
|
45
|
+
"@deepseek-ai/dsh-invariants": "^0.1.2-alpha.2",
|
|
46
|
+
"@deepseek-ai/dsh-session-persistence": "^0.1.2-alpha.2",
|
|
47
|
+
"@deepseek-ai/cordis": "^4.0.2",
|
|
48
|
+
"@deepseek-ai/dsh-session": "^0.1.2-alpha.2"
|
|
49
49
|
}
|
|
50
50
|
}
|