@deepseek-ai/dsh-storage-json 0.1.1-rc.2 → 0.1.2-alpha.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.i18n.yaml +2 -2
- package/README.md +126 -11
- package/README.zh.md +130 -15
- package/lib/index.js +293 -16
- package/lib/types/format.d.ts +25 -1
- package/lib/types/index.d.ts +5 -4
- package/lib/types/per-record-unit.d.ts +71 -0
- package/lib/types/single-unit.d.ts +21 -0
- package/package.json +8 -8
- package/lib/types/unit.d.ts +0 -18
package/README.i18n.yaml
CHANGED
|
@@ -2,5 +2,5 @@
|
|
|
2
2
|
# side as of the last confirmed-consistent state. Both languages carry equal authority;
|
|
3
3
|
# after editing either side, bring the other along and re-record with:
|
|
4
4
|
# pnpm run verify-translation-pairing --write packages/storage/storage-json/README.md
|
|
5
|
-
README.md:
|
|
6
|
-
README.zh.md:
|
|
5
|
+
README.md: 76df80de70cb58c78ff4339b3f1cbd137ca49ef0
|
|
6
|
+
README.zh.md: 7cbf63d14e3c257ef7670f559792129dea6ff497
|
package/README.md
CHANGED
|
@@ -1,21 +1,120 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: "JSON storage backend for hosts and maintainers choosing, configuring, or debugging whole-unit and per-record files under a configured root."
|
|
3
|
+
kind: "package-reference"
|
|
4
|
+
---
|
|
5
|
+
|
|
1
6
|
# @deepseek-ai/dsh-storage-json
|
|
2
7
|
|
|
3
8
|
English | [中文](README.zh.md)
|
|
4
9
|
|
|
5
|
-
|
|
10
|
+
## Summary
|
|
11
|
+
|
|
12
|
+
`dsh-storage-json` stores domain data as readable JSON under a configured root and registers as backend `json`. Its default `single` layout keeps one complete `<unit>.json` file per unit; its `per-record` layout keeps one version-stamped document per record. Both layouts publish each changed file atomically, while the domain layer orders calls. Choose it when operators need inspectable files and the selected layout fits the write volume; choose SQLite for larger or highly concurrent data. The backend is host-side only and contributes no prompt, tool, or schema.
|
|
13
|
+
|
|
14
|
+
## Table of Contents
|
|
15
|
+
|
|
16
|
+
- [Use this package](#use-this-package)
|
|
17
|
+
- [Understand the implementation](#understand-the-implementation)
|
|
18
|
+
- [Further Exploration](#further-exploration)
|
|
19
|
+
- [Model Experience](#model-experience)
|
|
20
|
+
- [Known Limitations and Deferred Work](#known-limitations-and-deferred-work)
|
|
21
|
+
- [Dev Note](#dev-note)
|
|
22
|
+
|
|
23
|
+
-----
|
|
24
|
+
|
|
25
|
+
<a id="use-this-package"></a>
|
|
26
|
+
## Use this package
|
|
27
|
+
|
|
28
|
+
Use this package when a composition needs readable, editable JSON storage. Route the relevant domains to backend `json`; each domain specification selects the `single` or `per-record` layout.
|
|
29
|
+
|
|
30
|
+
### When to choose it
|
|
31
|
+
|
|
32
|
+
Choose the default `single` layout for small units that benefit from one complete, pretty-printed file. Choose `per-record` when point writes should replace only one record document. Choose the SQLite backend when data is large, writes are frequent, or multiple records need transactional updates.
|
|
33
|
+
|
|
34
|
+
### Configuration
|
|
35
|
+
|
|
36
|
+
The only plugin field is `root`, which holds the unit files and directories. It is required because the backend does not fall back to `process.cwd()`. The backend creates the root with mode `0o700` on demand. A domain specification selects its layout; this plugin has no layout override.
|
|
37
|
+
|
|
38
|
+
```yaml
|
|
39
|
+
- name: '@deepseek-ai/dsh-storage'
|
|
40
|
+
- name: '@deepseek-ai/dsh-storage-json'
|
|
41
|
+
config:
|
|
42
|
+
root: /var/lib/dsh/data
|
|
43
|
+
- name: '@deepseek-ai/dsh-storage-domain'
|
|
44
|
+
config:
|
|
45
|
+
backend: json
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
| Field | Default | Meaning |
|
|
49
|
+
|---|---|---|
|
|
50
|
+
| `root` | required | Directory holding `<unit>.json` files and `<unit>/` trees; created `0o700` on demand |
|
|
51
|
+
|
|
52
|
+
The generated [configuration catalog](../../../docs/config-catalog.md#deepseek-aidsh-storage-json) is the exhaustive source for every accepted field and its JSDoc.
|
|
53
|
+
|
|
54
|
+
### Observable behavior
|
|
55
|
+
|
|
56
|
+
A missing `single` file or `per-record` directory opens as an empty unit and materializes on the first write. In `single`, malformed content rejects with `malformed-medium`, and a different stored version rejects with `version-mismatch`. In `per-record`, each malformed, unreadable, or differently versioned document reads as an absent record, so one bad document does not reject the unit. Record keys must match `[a-zA-Z0-9_-]+`; an unsafe key rejects before any file operation. Every resolved write is durable, and operations after close reject with `closed`.
|
|
57
|
+
|
|
58
|
+
An empty `per-record` tree can initialize its declared tables from a valid `<root>/<unit>.json` whole-unit document. The backend leaves that source file unchanged. Any document path in a declared table, or a declared `global.json`, suppresses this initialization for the complete unit, even if that document is unreadable or stale.
|
|
59
|
+
|
|
60
|
+
-----
|
|
61
|
+
|
|
62
|
+
<a id="understand-the-implementation"></a>
|
|
63
|
+
## Understand the implementation
|
|
6
64
|
|
|
7
|
-
|
|
65
|
+
<details>
|
|
66
|
+
<summary>Implementation internals — click to expand</summary>
|
|
8
67
|
|
|
9
|
-
|
|
10
|
-
- A missing file opens as an empty unit and materializes on the first write. A foreign or unparsable file rejects with `malformed-medium`; a stored version differing from the descriptor rejects with `version-mismatch` (no migration, pre-release stance).
|
|
11
|
-
- Write ordering across calls belongs to the caller (the domain layer's write chain); each single call is atomic and durable once resolved.
|
|
68
|
+
The two layouts share atomic publication but assign state ownership differently. `single` owns an in-memory unit projection; `per-record` treats its directory tree as authoritative.
|
|
12
69
|
|
|
13
|
-
|
|
70
|
+
### Design concept
|
|
14
71
|
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
72
|
+
- **`single` keeps memory authoritative.** Each write changes the in-memory unit, serializes its complete state, and atomically replaces `<unit>.json`. A failed publish restores the prior in-memory value.
|
|
73
|
+
- **`per-record` keeps the directory authoritative.** Each put or delete changes one `<unit>/<table>/<key>.json` document, and `loadAll()` rereads the tree. Each document stamps the unit version and carries one record value.
|
|
74
|
+
- **Publication is durable per call.** A write uses a temporary file, fsync, atomic `rename()` replacement, and a parent-directory fsync on POSIX. The domain layer's write chain supplies ordering across calls.
|
|
18
75
|
|
|
76
|
+
### File formats
|
|
77
|
+
|
|
78
|
+
A `single` document carries the unit identity, global singleton, and all tables:
|
|
79
|
+
|
|
80
|
+
```json
|
|
81
|
+
{
|
|
82
|
+
"unit": { "name": "workspace", "version": 1 },
|
|
83
|
+
"global": null,
|
|
84
|
+
"tables": { "workspaces": { "<key>": { "path": "/work/demo" } } }
|
|
85
|
+
}
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
A `per-record` table document at `<root>/<unit>/<table>/<key>.json` has the form `{ "version": 1, "record": <value> }`; the optional global value uses `<root>/<unit>/global.json`. The format version comes from the domain specification.
|
|
89
|
+
|
|
90
|
+
### Source map
|
|
91
|
+
|
|
92
|
+
| File | Role |
|
|
93
|
+
|---|---|
|
|
94
|
+
| [`src/index.ts`](src/index.ts) | Plugin entry: backend registration, `root` config, unit open/close table |
|
|
95
|
+
| [`src/single-unit.ts`](src/single-unit.ts) | One `single` unit: authoritative memory, write primitives, publish rollback |
|
|
96
|
+
| [`src/per-record-unit.ts`](src/per-record-unit.ts) | One `per-record` unit: tree reads, path-safe records, and one-document writes |
|
|
97
|
+
| [`src/format.ts`](src/format.ts) | Whole-unit and record serialization with version validation |
|
|
98
|
+
| [`src/atomic.ts`](src/atomic.ts) | Atomic file replacement: temp write, fsync, rename, directory fsync |
|
|
99
|
+
| [`src/invariant.ts`](src/invariant.ts) | Invariant companion (no runtime invariant: correctness is round-trip durability) |
|
|
100
|
+
|
|
101
|
+
</details>
|
|
102
|
+
|
|
103
|
+
-----
|
|
104
|
+
|
|
105
|
+
<a id="further-exploration"></a>
|
|
106
|
+
## Further Exploration
|
|
107
|
+
|
|
108
|
+
Read these pages when this backend's view is not enough: the subsystem reference is the authoritative contract, and the sibling backend shows the alternative medium.
|
|
109
|
+
|
|
110
|
+
- [Storage subsystem](../../../docs/subsystems/storage.md) — the backend contract, domain semantics, and generated API.
|
|
111
|
+
- [Storage package map](../README.md) — the family's packages and their repository position.
|
|
112
|
+
- [SQLite storage backend](../storage-sqlite/README.md) — the point-update medium for high-frequency data.
|
|
113
|
+
- [domain KV storage Agent Note](../../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.md) — the design behind the backend family and its deferred work.
|
|
114
|
+
|
|
115
|
+
-----
|
|
116
|
+
|
|
117
|
+
<a id="model-experience"></a>
|
|
19
118
|
## Model Experience
|
|
20
119
|
|
|
21
120
|
### Stored domain records
|
|
@@ -34,5 +133,21 @@ None — the backend never touches live request prefixes.
|
|
|
34
133
|
|
|
35
134
|
## Known Limitations and Deferred Work
|
|
36
135
|
|
|
37
|
-
|
|
38
|
-
|
|
136
|
+
<a id="known-limitations-and-deferred-work"></a>
|
|
137
|
+
|
|
138
|
+
|
|
139
|
+
These limits define when this backend is a poor fit or needs special operational care. They are current package constraints, not a task backlog.
|
|
140
|
+
|
|
141
|
+
- **`single` rewrites the whole unit** — each write republishes the complete unit file; use `per-record` or route the domain to SQLite when this cost is too high.
|
|
142
|
+
- **No cross-process write locking** — two processes writing the same unit can interleave replacements; writes to the same file use last-completion wins.
|
|
143
|
+
- **Windows rename without explicit write-through** — durability relies on libuv's `rename()` (`MoveFileExW` with replacement); the stricter Win32 write-through publish helper from the session-log backend is planned to move down here when the `log` facet lands.
|
|
144
|
+
|
|
145
|
+
<a id="dev-note"></a>
|
|
146
|
+
### Dev Note
|
|
147
|
+
|
|
148
|
+
<details>
|
|
149
|
+
<summary>Working context for maintainers — click to expand</summary>
|
|
150
|
+
|
|
151
|
+
The Agent Note flags the whole-unit rewrite scale premise as a risk: if a second consumer lands on this backend at thousand-record scale before being routed to SQLite, rewrite cost surfaces earlier than expected. The mitigation is configuration — point `routes` at the SQLite backend — not a change to this package.
|
|
152
|
+
|
|
153
|
+
</details>
|
package/README.zh.md
CHANGED
|
@@ -1,28 +1,127 @@
|
|
|
1
|
+
---
|
|
2
|
+
description: "JSON 存储后端：面向在配置根目录下选择、配置或排查整单元文件与逐记录文件的宿主与维护者。"
|
|
3
|
+
kind: "package-reference"
|
|
4
|
+
---
|
|
5
|
+
|
|
1
6
|
# @deepseek-ai/dsh-storage-json
|
|
2
7
|
|
|
3
8
|
[English](README.md) | 中文
|
|
4
9
|
|
|
5
|
-
|
|
10
|
+
## 概述
|
|
11
|
+
|
|
12
|
+
`dsh-storage-json` 在配置的根目录下把领域数据存为可读 JSON，并注册为后端 `json`。默认的 `single` 布局为每个单元保存一份完整的 `<unit>.json` 文件；`per-record` 布局为每条记录保存一份带版本戳的文档。两种布局都以原子方式发布每个变更文件，领域层负责安排调用顺序。当运维方需要可检查文件且所选布局适合写入量时选择它；对于更大或高并发的数据则选择 SQLite。本后端只面向宿主侧，不贡献提示词、工具或 schema。
|
|
13
|
+
|
|
14
|
+
## 目录
|
|
15
|
+
|
|
16
|
+
- [使用本包](#use-this-package)
|
|
17
|
+
- [理解实现](#understand-the-implementation)
|
|
18
|
+
- [进一步探索](#further-exploration)
|
|
19
|
+
- [模型体验](#model-experience)
|
|
20
|
+
- [已知限制与延期工作](#known-limitations-and-deferred-work)
|
|
21
|
+
- [开发备注](#dev-note)
|
|
22
|
+
|
|
23
|
+
-----
|
|
24
|
+
|
|
25
|
+
<a id="use-this-package"></a>
|
|
26
|
+
## 使用本包
|
|
27
|
+
|
|
28
|
+
当组合需要可读、可编辑的 JSON 存储时使用本包。把相关领域路由到 `json` 后端；每个领域规范选择 `single` 或 `per-record` 布局。
|
|
29
|
+
|
|
30
|
+
### 何时选择
|
|
31
|
+
|
|
32
|
+
小型单元需要一份完整、美化打印的文件时，选择默认的 `single` 布局。定点写入只应替换一份记录文档时，选择 `per-record`。当数据量大、写入频繁或多条记录需要事务更新时，选择 SQLite 后端。
|
|
33
|
+
|
|
34
|
+
### 配置
|
|
35
|
+
|
|
36
|
+
唯一的插件字段是 `root`，用于保存单元文件与目录。它是必填项，因为本后端不回退到 `process.cwd()`。后端按需以 `0o700` 模式创建根目录。领域规范选择其布局；本插件不提供布局覆盖项。
|
|
37
|
+
|
|
38
|
+
```yaml
|
|
39
|
+
- name: '@deepseek-ai/dsh-storage'
|
|
40
|
+
- name: '@deepseek-ai/dsh-storage-json'
|
|
41
|
+
config:
|
|
42
|
+
root: /var/lib/dsh/data
|
|
43
|
+
- name: '@deepseek-ai/dsh-storage-domain'
|
|
44
|
+
config:
|
|
45
|
+
backend: json
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
| 字段 | 默认值 | 含义 |
|
|
49
|
+
|---|---|---|
|
|
50
|
+
| `root` | 必填 | 保存 `<unit>.json` 文件与 `<unit>/` 目录树的目录；按需以 `0o700` 创建 |
|
|
51
|
+
|
|
52
|
+
生成的[配置目录](../../../docs/config-catalog.zh.md#deepseek-aidsh-storage-json)是每个受支持字段及其 JSDoc 的穷尽式真源。
|
|
53
|
+
|
|
54
|
+
### 可观察行为
|
|
55
|
+
|
|
56
|
+
缺失的 `single` 文件或 `per-record` 目录会作为空单元打开，并在第一次写入时物化。在 `single` 中，畸形内容以 `malformed-medium` 拒绝，不同的已存版本以 `version-mismatch` 拒绝。在 `per-record` 中，每份畸形、不可读或版本不同的文档都读作记录不存在，因此单个坏文档不会使单元被拒绝。记录键必须匹配 `[a-zA-Z0-9_-]+`；不安全的键在任何文件操作前被拒绝。每次已完成的写入都已持久化，关闭后的操作以 `closed` 拒绝。
|
|
57
|
+
|
|
58
|
+
空的 `per-record` 目录树可以从有效的 `<root>/<unit>.json` 整单元文档初始化其已声明表。后端保持该源文件不变。已声明表中只要存在任意文档路径，或存在已声明的 `global.json`，就会对整个单元禁止该初始化，即使该文档不可读或版本陈旧。
|
|
59
|
+
|
|
60
|
+
-----
|
|
61
|
+
|
|
62
|
+
<a id="understand-the-implementation"></a>
|
|
63
|
+
## 理解实现
|
|
6
64
|
|
|
7
|
-
|
|
65
|
+
<details>
|
|
66
|
+
<summary>实现细节——点击展开</summary>
|
|
8
67
|
|
|
9
|
-
-
|
|
10
|
-
- 缺失文件会作为空单元打开，并在第一次写入时物化。外来或无法解析的文件以 `malformed-medium` 拒绝；已存版本与描述符不同时以 `version-mismatch` 拒绝（预发布立场，不迁移）。
|
|
11
|
-
- 跨调用的写入顺序属于调用方（领域层的写入链）；每次调用都具备原子性，并在完成时已达到持久状态。
|
|
68
|
+
两种布局共享原子发布机制，但以不同方式确定状态所有权。`single` 拥有一份内存单元投影；`per-record` 把目录树视为权威状态。
|
|
12
69
|
|
|
13
|
-
|
|
70
|
+
### 设计理念
|
|
14
71
|
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
72
|
+
- **`single` 以内存为权威状态。** 每次写入都会更改内存单元、序列化其完整状态，并以原子方式替换 `<unit>.json`。发布失败会恢复先前的内存值。
|
|
73
|
+
- **`per-record` 以目录为权威状态。** 每次 put 或 delete 都会更改一个 `<unit>/<table>/<key>.json` 文档，`loadAll()` 则重新读取目录树。每份文档都带有单元版本戳与一条记录值。
|
|
74
|
+
- **每次调用都持久发布。** 写入过程使用临时文件、fsync、原子 `rename()` 替换，并在 POSIX 上 fsync 父目录。领域层写入链负责安排跨调用的顺序。
|
|
18
75
|
|
|
76
|
+
### 文件格式
|
|
77
|
+
|
|
78
|
+
`single` 文档携带单元标识、全局单例与所有表：
|
|
79
|
+
|
|
80
|
+
```json
|
|
81
|
+
{
|
|
82
|
+
"unit": { "name": "workspace", "version": 1 },
|
|
83
|
+
"global": null,
|
|
84
|
+
"tables": { "workspaces": { "<key>": { "path": "/work/demo" } } }
|
|
85
|
+
}
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
`per-record` 表文档位于 `<root>/<unit>/<table>/<key>.json`，形式为 `{ "version": 1, "record": <value> }`；可选的全局值使用 `<root>/<unit>/global.json`。格式版本来自领域规范。
|
|
89
|
+
|
|
90
|
+
### 源码地图
|
|
91
|
+
|
|
92
|
+
| 文件 | 职责 |
|
|
93
|
+
|---|---|
|
|
94
|
+
| [`src/index.ts`](src/index.ts) | 插件入口：后端注册、`root` 配置、单元打开／关闭表 |
|
|
95
|
+
| [`src/single-unit.ts`](src/single-unit.ts) | 一个 `single` 单元：权威内存、写入原语与发布回滚 |
|
|
96
|
+
| [`src/per-record-unit.ts`](src/per-record-unit.ts) | 一个 `per-record` 单元：目录树读取、路径安全记录与单文档写入 |
|
|
97
|
+
| [`src/format.ts`](src/format.ts) | 带版本校验的整单元与记录序列化 |
|
|
98
|
+
| [`src/atomic.ts`](src/atomic.ts) | 原子文件替换：临时文件写入、fsync、rename、目录 fsync |
|
|
99
|
+
| [`src/invariant.ts`](src/invariant.ts) | 不变式伴生插件（无运行时不变式：正确性靠往返持久性） |
|
|
100
|
+
|
|
101
|
+
</details>
|
|
102
|
+
|
|
103
|
+
-----
|
|
104
|
+
|
|
105
|
+
<a id="further-exploration"></a>
|
|
106
|
+
## 进一步探索
|
|
107
|
+
|
|
108
|
+
当本后端视角不够用时阅读以下页面：子系统参考是权威约定，兄弟后端展示了另一种介质。
|
|
109
|
+
|
|
110
|
+
- [存储子系统](../../../docs/subsystems/storage.zh.md)——后端约定、领域语义与生成的 API。
|
|
111
|
+
- [存储包映射](../README.zh.md)——家族的各包及其在仓库中的位置。
|
|
112
|
+
- [SQLite 存储后端](../storage-sqlite/README.zh.md)——面向高频数据的定点更新介质。
|
|
113
|
+
- [领域 KV 存储 Agent Note](../../../.agents/notes/proposed/architecture/2026-07-24-domain-kv-storage-and-workspace.zh.md)——后端家族背后的设计及其延期工作。
|
|
114
|
+
|
|
115
|
+
-----
|
|
116
|
+
|
|
117
|
+
<a id="model-experience"></a>
|
|
19
118
|
## 模型体验
|
|
20
119
|
|
|
21
120
|
### 已存领域记录
|
|
22
121
|
|
|
23
|
-
####
|
|
122
|
+
#### 模型看到什么
|
|
24
123
|
|
|
25
|
-
|
|
124
|
+
无。本后端不贡献提示词、工具或 schema；它在 `ctx.storage` 后面持久化非会话领域数据，只供宿主侧消费方使用。
|
|
26
125
|
|
|
27
126
|
#### Token 影响
|
|
28
127
|
|
|
@@ -30,9 +129,25 @@
|
|
|
30
129
|
|
|
31
130
|
#### KV Cache 影响
|
|
32
131
|
|
|
33
|
-
|
|
132
|
+
无：本后端从不触碰实时请求前缀。
|
|
133
|
+
|
|
134
|
+
## 已知限制与延期工作
|
|
135
|
+
|
|
136
|
+
<a id="known-limitations-and-deferred-work"></a>
|
|
137
|
+
|
|
138
|
+
|
|
139
|
+
这些限制说明本后端何时不合适，或何时需要特别的运维注意。它们是当前包约束，不是任务积压。
|
|
140
|
+
|
|
141
|
+
- **`single` 会重写整个单元**——每次写入都重新发布完整单元文件；当此成本过高时，使用 `per-record` 或把领域路由到 SQLite。
|
|
142
|
+
- **没有跨进程写锁**——两个进程写入同一单元时可能交错执行替换；对同一文件的写入以最后完成者为准。
|
|
143
|
+
- **Windows rename 没有显式 write-through**——持久性依赖 libuv 的 `rename()`（`MoveFileExW` 并启用替换）；`log` 分面落地时，计划把会话日志后端更严格的 Win32 write-through 发布辅助函数下移到此处。
|
|
144
|
+
|
|
145
|
+
<a id="dev-note"></a>
|
|
146
|
+
### 开发备注
|
|
147
|
+
|
|
148
|
+
<details>
|
|
149
|
+
<summary>维护者的工作上下文——点击展开</summary>
|
|
34
150
|
|
|
35
|
-
|
|
151
|
+
Agent Note 把整单元重写的规模前提标记为风险：如果在被路由到 SQLite 之前，第二个消费方以千条记录规模落到本后端，重写成本会比预期更早显现。缓解办法是配置——把 `routes` 指向 SQLite 后端——而不是修改本包。
|
|
36
152
|
|
|
37
|
-
|
|
38
|
-
- 没有跨进程写锁：两个进程写入同一根目录时，可能交错执行整文件替换（最后写入者胜出）。当前消费方采用单一宿主进程部署；多进程方案按 Agent Note 的范围外事项表暂缓。
|
|
153
|
+
</details>
|
package/lib/index.js
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
|
-
import { mkdir, open, readFile, rename, rm } from "node:fs/promises";
|
|
2
|
-
import { dirname, join } from "node:path";
|
|
1
|
+
import { mkdir, open, readFile, readdir, rename, rm } from "node:fs/promises";
|
|
3
2
|
import z from "@deepseek-ai/schemastery";
|
|
4
3
|
import { StorageError, UNIT_NAME_RE, storageBackendServiceKey } from "@deepseek-ai/dsh-storage";
|
|
4
|
+
import { dirname, join } from "node:path";
|
|
5
5
|
import { randomUUID } from "node:crypto";
|
|
6
6
|
//#region lib/types/atomic.js
|
|
7
7
|
/**
|
|
@@ -56,7 +56,11 @@ async function fsyncDirectory(path) {
|
|
|
56
56
|
/**
|
|
57
57
|
* On-disk JSON unit format: the file is always the current net state, kept
|
|
58
58
|
* human-readable (pretty-printed, stable key order from insertion) — that
|
|
59
|
-
* legibility is this backend's reason to exist.
|
|
59
|
+
* legibility is this backend's reason to exist. `single`-layout units are
|
|
60
|
+
* one document with a unit header; `per-record`-layout units are a directory
|
|
61
|
+
* with one version-stamped document per record (`<table>/<key>.json`) plus a
|
|
62
|
+
* `global.json` for the global slot, so a write rewrites one record instead
|
|
63
|
+
* of the whole unit.
|
|
60
64
|
* @module @deepseek-ai/dsh-storage-json/src/format
|
|
61
65
|
*/
|
|
62
66
|
/**
|
|
@@ -113,37 +117,77 @@ function parse(text, descriptor) {
|
|
|
113
117
|
}
|
|
114
118
|
return state;
|
|
115
119
|
}
|
|
120
|
+
/**
|
|
121
|
+
* Serialize one per-record document: the unit's version stamp plus the
|
|
122
|
+
* record value, pretty-printed like the whole-unit document.
|
|
123
|
+
* @param version - Unit format version, stamped into the header.
|
|
124
|
+
* @param value - The record value (or the global singleton value).
|
|
125
|
+
* @returns pretty-printed JSON document with a trailing newline.
|
|
126
|
+
*/
|
|
127
|
+
function serializeRecord(version, value) {
|
|
128
|
+
return `${JSON.stringify({
|
|
129
|
+
version,
|
|
130
|
+
record: value
|
|
131
|
+
}, null, 2)}\n`;
|
|
132
|
+
}
|
|
133
|
+
/**
|
|
134
|
+
* Parse one per-record document, validating its version stamp. A document
|
|
135
|
+
* that is malformed or stamped with a different version is FOREIGN and reads
|
|
136
|
+
* as absent — the per-record contract: one bad or stale record file must not
|
|
137
|
+
* brick the whole unit, and a version bump discards stale records instead of
|
|
138
|
+
* migrating them (the whole-unit format rejects instead, because there is
|
|
139
|
+
* exactly one document).
|
|
140
|
+
* @param text - Raw per-record document content.
|
|
141
|
+
* @param version - Expected unit version; a mismatch discards the document.
|
|
142
|
+
* @returns the record value, or `undefined` for a foreign document.
|
|
143
|
+
*/
|
|
144
|
+
function parseRecord(text, version) {
|
|
145
|
+
let document;
|
|
146
|
+
try {
|
|
147
|
+
document = JSON.parse(text);
|
|
148
|
+
} catch {
|
|
149
|
+
return;
|
|
150
|
+
}
|
|
151
|
+
if (typeof document !== "object" || document === null) return void 0;
|
|
152
|
+
const { version: stamped, record } = document;
|
|
153
|
+
if (stamped !== version) return void 0;
|
|
154
|
+
return record;
|
|
155
|
+
}
|
|
116
156
|
//#endregion
|
|
117
|
-
//#region lib/types/unit.js
|
|
157
|
+
//#region lib/types/single-unit.js
|
|
118
158
|
/**
|
|
119
|
-
* One opened JSON unit
|
|
159
|
+
* One opened JSON unit in `single` layout: the whole unit is one document at
|
|
160
|
+
* `<root>/<name>.json`. The in-memory state is authoritative; every write
|
|
120
161
|
* primitive mutates it and republishes the whole file atomically. Writes are
|
|
121
162
|
* NOT queued here — per the backend contract, write ordering belongs to the
|
|
122
163
|
* caller (the domain layer's write chain); this unit only guarantees that
|
|
123
|
-
* each single call publishes a complete, durable file.
|
|
124
|
-
*
|
|
164
|
+
* each single call publishes a complete, durable file. The `per-record`
|
|
165
|
+
* layout is a separate unit class in `per-record-unit.ts`.
|
|
166
|
+
* @module @deepseek-ai/dsh-storage-json/src/single-unit
|
|
125
167
|
*/
|
|
126
168
|
/**
|
|
127
|
-
* Open (load or lazily create) one unit
|
|
169
|
+
* Open (load or lazily create) one `single`-layout unit under `root`: the
|
|
170
|
+
* unit file is `<root>/<name>.json`.
|
|
128
171
|
* @param descriptor - Static identity and shape of the unit.
|
|
129
|
-
* @param
|
|
172
|
+
* @param root - Absolute backend root directory.
|
|
130
173
|
* @param onClose - Backend callback releasing the unit's open-slot.
|
|
131
174
|
* @returns the opened unit.
|
|
132
175
|
*/
|
|
133
|
-
async function
|
|
176
|
+
async function openSingleUnit(descriptor, root, onClose) {
|
|
177
|
+
const path = join(root, `${descriptor.name}.json`);
|
|
134
178
|
let text;
|
|
135
179
|
try {
|
|
136
180
|
text = await readFile(path, "utf8");
|
|
137
181
|
} catch (error) {
|
|
138
182
|
if (error.code !== "ENOENT") throw error;
|
|
139
183
|
}
|
|
140
|
-
return new
|
|
184
|
+
return new SingleJsonUnit(descriptor, path, text === void 0 ? {
|
|
141
185
|
version: descriptor.version,
|
|
142
186
|
global: null,
|
|
143
187
|
tables: new Map(descriptor.tables.map((table) => [table, /* @__PURE__ */ new Map()]))
|
|
144
188
|
} : parse(text, descriptor), onClose);
|
|
145
189
|
}
|
|
146
|
-
var
|
|
190
|
+
var SingleJsonUnit = class {
|
|
147
191
|
descriptor;
|
|
148
192
|
path;
|
|
149
193
|
state;
|
|
@@ -224,11 +268,243 @@ var JsonKvUnit = class {
|
|
|
224
268
|
}
|
|
225
269
|
};
|
|
226
270
|
//#endregion
|
|
271
|
+
//#region lib/types/per-record-unit.js
|
|
272
|
+
/**
|
|
273
|
+
* One opened JSON unit in `per-record` layout: the unit is a directory at
|
|
274
|
+
* `dir`, holding one document per record under `<dir>/<table>/<key>.json`
|
|
275
|
+
* plus `global.json` for the global slot. The directory is the state — this
|
|
276
|
+
* unit holds NO in-memory state of its own: `loadAll` re-reads the tree and
|
|
277
|
+
* every write is one durable file operation. The domain layer owns the live
|
|
278
|
+
* in-memory tables (seeded by the open-time `loadAll`) and serializes writes
|
|
279
|
+
* through its write chain, so this unit never mutates memory and needs no
|
|
280
|
+
* rollback — a failed write simply leaves both the file and the domain's
|
|
281
|
+
* memory unchanged.
|
|
282
|
+
*
|
|
283
|
+
* Per-record contract: a record document that is malformed or stamped with a
|
|
284
|
+
* different version reads as an absent record — one bad or stale file never
|
|
285
|
+
* bricks the whole unit, and a version bump discards stale records instead
|
|
286
|
+
* of migrating them. Record keys become path segments, so they must be
|
|
287
|
+
* path-safe (`[a-zA-Z0-9_-]+`); an unsafe key rejects at write.
|
|
288
|
+
*
|
|
289
|
+
* Legacy bootstrap: when the new tree has no document path, a legacy
|
|
290
|
+
* whole-unit file `<root>/<name>.json` (the pre-per-record layout) seeds
|
|
291
|
+
* per-record documents. Any new document path, including one whose contents
|
|
292
|
+
* are unreadable or stale, suppresses the bootstrap for the whole unit. The
|
|
293
|
+
* legacy file is never changed or deleted.
|
|
294
|
+
* @module @deepseek-ai/dsh-storage-json/src/per-record-unit
|
|
295
|
+
*/
|
|
296
|
+
/** Keys become path segments in this layout; this set is path-safe on every OS. */
|
|
297
|
+
const SAFE_KEY_RE = /^[a-zA-Z0-9_-]+$/;
|
|
298
|
+
/**
|
|
299
|
+
* Open one `per-record`-layout unit under `root`: the unit directory is
|
|
300
|
+
* `<root>/<name>/`. Loads lazily on the first `loadAll` — this unit holds no
|
|
301
|
+
* state, so opening touches nothing on the medium.
|
|
302
|
+
* @param descriptor - Static identity and shape of the unit.
|
|
303
|
+
* @param root - Absolute backend root directory.
|
|
304
|
+
* @param onClose - Backend callback releasing the unit's open-slot.
|
|
305
|
+
* @returns the opened unit.
|
|
306
|
+
*/
|
|
307
|
+
async function openPerRecordUnit(descriptor, root, onClose) {
|
|
308
|
+
return new PerRecordJsonUnit(descriptor, join(root, descriptor.name), onClose);
|
|
309
|
+
}
|
|
310
|
+
/**
|
|
311
|
+
* Read every record document under the unit directory: each declared table's
|
|
312
|
+
* `<key>.json` files plus `global.json`. A missing directory is the empty
|
|
313
|
+
* unit (materialization defers to the first write); a foreign document
|
|
314
|
+
* (missing, malformed, or stamped with another version) reads as an absent
|
|
315
|
+
* record, per the per-record contract.
|
|
316
|
+
* @param descriptor - Static identity and shape of the unit.
|
|
317
|
+
* @param dir - Absolute unit directory path.
|
|
318
|
+
* @returns the authoritative state reconstructed from the tree.
|
|
319
|
+
*/
|
|
320
|
+
async function loadPerRecordState(descriptor, dir) {
|
|
321
|
+
const state = {
|
|
322
|
+
version: descriptor.version,
|
|
323
|
+
global: null,
|
|
324
|
+
tables: new Map(descriptor.tables.map((table) => [table, /* @__PURE__ */ new Map()]))
|
|
325
|
+
};
|
|
326
|
+
let entries;
|
|
327
|
+
try {
|
|
328
|
+
entries = await readdir(dir, { withFileTypes: true });
|
|
329
|
+
} catch (error) {
|
|
330
|
+
if (error.code !== "ENOENT") throw error;
|
|
331
|
+
}
|
|
332
|
+
if (!(entries === void 0 ? false : (await Promise.all(entries.map(async (entry) => {
|
|
333
|
+
if (entry.isDirectory()) {
|
|
334
|
+
const records = state.tables.get(entry.name);
|
|
335
|
+
if (records !== void 0) return loadTableRecords(records, descriptor.version, join(dir, entry.name));
|
|
336
|
+
}
|
|
337
|
+
if (entry.name === "global.json" && descriptor.hasGlobal) {
|
|
338
|
+
const global = await readRecord(join(dir, entry.name), descriptor.version);
|
|
339
|
+
if (global !== void 0) state.global = global;
|
|
340
|
+
return true;
|
|
341
|
+
}
|
|
342
|
+
return false;
|
|
343
|
+
}))).some(Boolean))) await bootstrapLegacyUnit(descriptor, dir, state);
|
|
344
|
+
return state;
|
|
345
|
+
}
|
|
346
|
+
/**
|
|
347
|
+
* Bootstrap an empty per-record tree from a legacy whole-unit file
|
|
348
|
+
* (`<root>/<name>.json`, the pre-per-record layout). Every declared-table
|
|
349
|
+
* record is copied into a current-version document, while the legacy file is
|
|
350
|
+
* retained unchanged. A missing, foreign (another unit's name), malformed,
|
|
351
|
+
* or non-unit legacy file is left alone; other read failures propagate.
|
|
352
|
+
* @param descriptor - Static identity and shape of the unit.
|
|
353
|
+
* @param dir - The per-record unit directory (`<root>/<name>`).
|
|
354
|
+
* @param state - The empty tree state; bootstrapped records are added.
|
|
355
|
+
*/
|
|
356
|
+
async function bootstrapLegacyUnit(descriptor, dir, state) {
|
|
357
|
+
const legacyPath = join(dirname(dir), `${descriptor.name}.json`);
|
|
358
|
+
let text;
|
|
359
|
+
try {
|
|
360
|
+
text = await readFile(legacyPath, "utf8");
|
|
361
|
+
} catch (error) {
|
|
362
|
+
if (error.code !== "ENOENT") throw error;
|
|
363
|
+
return;
|
|
364
|
+
}
|
|
365
|
+
let document;
|
|
366
|
+
try {
|
|
367
|
+
document = JSON.parse(text);
|
|
368
|
+
} catch {
|
|
369
|
+
return;
|
|
370
|
+
}
|
|
371
|
+
if (document.unit?.name !== descriptor.name) return;
|
|
372
|
+
const tables = document.tables;
|
|
373
|
+
if (typeof tables !== "object" || tables === null) return;
|
|
374
|
+
const recordsByTable = tables;
|
|
375
|
+
for (const [table, records] of Object.entries(recordsByTable)) {
|
|
376
|
+
const target = state.tables.get(table);
|
|
377
|
+
if (target === void 0) continue;
|
|
378
|
+
for (const [key, value] of Object.entries(records)) {
|
|
379
|
+
const path = join(dir, table, `${key}.json`);
|
|
380
|
+
await mkdir(dirname(path), {
|
|
381
|
+
recursive: true,
|
|
382
|
+
mode: 448
|
|
383
|
+
});
|
|
384
|
+
await writeAtomic(path, serializeRecord(descriptor.version, value));
|
|
385
|
+
target.set(key, value);
|
|
386
|
+
}
|
|
387
|
+
}
|
|
388
|
+
}
|
|
389
|
+
/**
|
|
390
|
+
* Read one declared table's record documents into `records`.
|
|
391
|
+
* @returns whether the directory contains any `.json` document path,
|
|
392
|
+
* independently of key safety, readability, or stored version.
|
|
393
|
+
*/
|
|
394
|
+
async function loadTableRecords(records, version, dir) {
|
|
395
|
+
const files = await readdir(dir, { withFileTypes: true });
|
|
396
|
+
const hasDocuments = files.some((file) => file.name.endsWith(".json"));
|
|
397
|
+
const loaded = await Promise.all(files.map(async (file) => {
|
|
398
|
+
if (!file.name.endsWith(".json")) return;
|
|
399
|
+
const key = file.name.slice(0, -5);
|
|
400
|
+
if (!SAFE_KEY_RE.test(key)) return;
|
|
401
|
+
const record = await readRecord(join(dir, file.name), version);
|
|
402
|
+
if (record !== void 0) return [key, record];
|
|
403
|
+
}));
|
|
404
|
+
for (const record of loaded) if (record !== void 0) records.set(...record);
|
|
405
|
+
return hasDocuments;
|
|
406
|
+
}
|
|
407
|
+
/** Read one record document; a foreign (unreadable or stale) one reads as absent. */
|
|
408
|
+
async function readRecord(path, version) {
|
|
409
|
+
try {
|
|
410
|
+
return parseRecord(await readFile(path, "utf8"), version);
|
|
411
|
+
} catch {
|
|
412
|
+
return;
|
|
413
|
+
}
|
|
414
|
+
}
|
|
415
|
+
/**
|
|
416
|
+
* One opened `per-record`-layout unit. Stateless by design: the directory is
|
|
417
|
+
* the medium, the domain layer owns the live memory, and each method here is
|
|
418
|
+
* a single durable file operation. Write ordering belongs to the caller (the
|
|
419
|
+
* domain layer's write chain), exactly like the `single`-layout unit.
|
|
420
|
+
*/
|
|
421
|
+
var PerRecordJsonUnit = class {
|
|
422
|
+
descriptor;
|
|
423
|
+
dir;
|
|
424
|
+
onClose;
|
|
425
|
+
closed = false;
|
|
426
|
+
/** In-flight durable writes; close() drains them before releasing the unit. */
|
|
427
|
+
inFlight = /* @__PURE__ */ new Set();
|
|
428
|
+
constructor(descriptor, dir, onClose) {
|
|
429
|
+
this.descriptor = descriptor;
|
|
430
|
+
this.dir = dir;
|
|
431
|
+
this.onClose = onClose;
|
|
432
|
+
}
|
|
433
|
+
/** Re-read the tree: the directory is the authoritative state. */
|
|
434
|
+
async loadAll() {
|
|
435
|
+
this.assertOpen();
|
|
436
|
+
const state = await loadPerRecordState(this.descriptor, this.dir);
|
|
437
|
+
const tables = {};
|
|
438
|
+
for (const [table, records] of state.tables) tables[table] = Object.fromEntries(records);
|
|
439
|
+
return {
|
|
440
|
+
tables,
|
|
441
|
+
global: state.global
|
|
442
|
+
};
|
|
443
|
+
}
|
|
444
|
+
/** Durably replace one record: its own document, atomically. */
|
|
445
|
+
async putRecord(table, key, value) {
|
|
446
|
+
this.assertOpen();
|
|
447
|
+
assertSafeKey(this.descriptor.name, key);
|
|
448
|
+
await this.tracked(this.writeDocument(join(this.tableDir(table), `${key}.json`), value));
|
|
449
|
+
}
|
|
450
|
+
/** Durably delete one record. Idempotent: a missing key is a no-op. */
|
|
451
|
+
async deleteRecord(table, key) {
|
|
452
|
+
this.assertOpen();
|
|
453
|
+
assertSafeKey(this.descriptor.name, key);
|
|
454
|
+
await this.tracked(rm(join(this.tableDir(table), `${key}.json`), { force: true }));
|
|
455
|
+
}
|
|
456
|
+
/** Durably replace the global singleton. Only valid when declared. */
|
|
457
|
+
async setGlobal(value) {
|
|
458
|
+
this.assertOpen();
|
|
459
|
+
if (!this.descriptor.hasGlobal) throw new Error(`unit '${this.descriptor.name}' does not declare a global slot`);
|
|
460
|
+
await this.tracked(this.writeDocument(join(this.dir, "global.json"), value));
|
|
461
|
+
}
|
|
462
|
+
/** Drain in-flight writes and release the unit. Idempotent. */
|
|
463
|
+
async close() {
|
|
464
|
+
if (this.closed) {
|
|
465
|
+
await Promise.allSettled(this.inFlight);
|
|
466
|
+
return;
|
|
467
|
+
}
|
|
468
|
+
this.closed = true;
|
|
469
|
+
await Promise.allSettled(this.inFlight);
|
|
470
|
+
this.onClose();
|
|
471
|
+
}
|
|
472
|
+
assertOpen() {
|
|
473
|
+
if (this.closed) throw new StorageError("closed", `unit '${this.descriptor.name}' is closed`);
|
|
474
|
+
}
|
|
475
|
+
/** Resolve a declared table's directory; an undeclared table is a caller bug and throws. */
|
|
476
|
+
tableDir(table) {
|
|
477
|
+
if (!this.descriptor.tables.includes(table)) throw new Error(`unit '${this.descriptor.name}' does not declare table '${table}'`);
|
|
478
|
+
return join(this.dir, table);
|
|
479
|
+
}
|
|
480
|
+
/** Durably replace one document, creating its parent directory. */
|
|
481
|
+
writeDocument(path, value) {
|
|
482
|
+
return (async () => {
|
|
483
|
+
await mkdir(dirname(path), {
|
|
484
|
+
recursive: true,
|
|
485
|
+
mode: 448
|
|
486
|
+
});
|
|
487
|
+
await writeAtomic(path, serializeRecord(this.descriptor.version, value));
|
|
488
|
+
})();
|
|
489
|
+
}
|
|
490
|
+
/** Track one durable write so close() drains it. */
|
|
491
|
+
tracked(write) {
|
|
492
|
+
this.inFlight.add(write);
|
|
493
|
+
write.catch(() => {}).finally(() => this.inFlight.delete(write));
|
|
494
|
+
return write;
|
|
495
|
+
}
|
|
496
|
+
};
|
|
497
|
+
/** Reject a record key that would be unsafe as a path segment. */
|
|
498
|
+
function assertSafeKey(unit, key) {
|
|
499
|
+
if (!SAFE_KEY_RE.test(key)) throw new Error(`unit '${unit}': per-record key '${key}' is not path-safe (must match ${SAFE_KEY_RE})`);
|
|
500
|
+
}
|
|
501
|
+
//#endregion
|
|
227
502
|
//#region lib/types/index.js
|
|
228
503
|
/**
|
|
229
|
-
* JSON storage backend: one human-readable
|
|
230
|
-
* root
|
|
231
|
-
*
|
|
504
|
+
* JSON storage backend: one human-readable document per unit under a
|
|
505
|
+
* configured root — a whole-unit file (`single` layout) or one document per
|
|
506
|
+
* record (`per-record` layout), published by atomic rewrite. Registers as
|
|
507
|
+
* backend `json` on the storage hub.
|
|
232
508
|
* @module @deepseek-ai/dsh-storage-json
|
|
233
509
|
*/
|
|
234
510
|
/** Cordis plugin name. */
|
|
@@ -259,7 +535,8 @@ var JsonStorageBackend = class {
|
|
|
259
535
|
recursive: true,
|
|
260
536
|
mode: 448
|
|
261
537
|
});
|
|
262
|
-
const
|
|
538
|
+
const onClose = () => this.open.delete(descriptor.name);
|
|
539
|
+
const unit = descriptor.layout === "per-record" ? await openPerRecordUnit(descriptor, this.root, onClose) : await openSingleUnit(descriptor, this.root, onClose);
|
|
263
540
|
if (this.closed) {
|
|
264
541
|
await unit.close();
|
|
265
542
|
throw new StorageError("closed", "json backend is closed");
|
package/lib/types/format.d.ts
CHANGED
|
@@ -1,7 +1,11 @@
|
|
|
1
1
|
/**
|
|
2
2
|
* On-disk JSON unit format: the file is always the current net state, kept
|
|
3
3
|
* human-readable (pretty-printed, stable key order from insertion) — that
|
|
4
|
-
* legibility is this backend's reason to exist.
|
|
4
|
+
* legibility is this backend's reason to exist. `single`-layout units are
|
|
5
|
+
* one document with a unit header; `per-record`-layout units are a directory
|
|
6
|
+
* with one version-stamped document per record (`<table>/<key>.json`) plus a
|
|
7
|
+
* `global.json` for the global slot, so a write rewrites one record instead
|
|
8
|
+
* of the whole unit.
|
|
5
9
|
* @module @deepseek-ai/dsh-storage-json/src/format
|
|
6
10
|
*/
|
|
7
11
|
import type { KvUnitDescriptor } from '@deepseek-ai/dsh-storage';
|
|
@@ -25,4 +29,24 @@ export declare function serialize(name: string, state: UnitState): string;
|
|
|
25
29
|
* @returns the parsed state.
|
|
26
30
|
*/
|
|
27
31
|
export declare function parse(text: string, descriptor: KvUnitDescriptor): UnitState;
|
|
32
|
+
/**
|
|
33
|
+
* Serialize one per-record document: the unit's version stamp plus the
|
|
34
|
+
* record value, pretty-printed like the whole-unit document.
|
|
35
|
+
* @param version - Unit format version, stamped into the header.
|
|
36
|
+
* @param value - The record value (or the global singleton value).
|
|
37
|
+
* @returns pretty-printed JSON document with a trailing newline.
|
|
38
|
+
*/
|
|
39
|
+
export declare function serializeRecord(version: number, value: unknown): string;
|
|
40
|
+
/**
|
|
41
|
+
* Parse one per-record document, validating its version stamp. A document
|
|
42
|
+
* that is malformed or stamped with a different version is FOREIGN and reads
|
|
43
|
+
* as absent — the per-record contract: one bad or stale record file must not
|
|
44
|
+
* brick the whole unit, and a version bump discards stale records instead of
|
|
45
|
+
* migrating them (the whole-unit format rejects instead, because there is
|
|
46
|
+
* exactly one document).
|
|
47
|
+
* @param text - Raw per-record document content.
|
|
48
|
+
* @param version - Expected unit version; a mismatch discards the document.
|
|
49
|
+
* @returns the record value, or `undefined` for a foreign document.
|
|
50
|
+
*/
|
|
51
|
+
export declare function parseRecord(text: string, version: number): unknown;
|
|
28
52
|
//# sourceMappingURL=format.d.ts.map
|
package/lib/types/index.d.ts
CHANGED
|
@@ -1,7 +1,8 @@
|
|
|
1
1
|
/**
|
|
2
|
-
* JSON storage backend: one human-readable
|
|
3
|
-
* root
|
|
4
|
-
*
|
|
2
|
+
* JSON storage backend: one human-readable document per unit under a
|
|
3
|
+
* configured root — a whole-unit file (`single` layout) or one document per
|
|
4
|
+
* record (`per-record` layout), published by atomic rewrite. Registers as
|
|
5
|
+
* backend `json` on the storage hub.
|
|
5
6
|
* @module @deepseek-ai/dsh-storage-json
|
|
6
7
|
*/
|
|
7
8
|
import type { Context } from '@deepseek-ai/cordis';
|
|
@@ -18,7 +19,7 @@ export declare const inject: string[];
|
|
|
18
19
|
* location explicitly.
|
|
19
20
|
*/
|
|
20
21
|
export interface Config {
|
|
21
|
-
/** Directory holding one `<unit>.json` file per unit. */
|
|
22
|
+
/** Directory holding one `<unit>.json` file (or `<unit>/` tree) per unit. */
|
|
22
23
|
root: string;
|
|
23
24
|
}
|
|
24
25
|
/** Config schema. */
|
|
@@ -0,0 +1,71 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* One opened JSON unit in `per-record` layout: the unit is a directory at
|
|
3
|
+
* `dir`, holding one document per record under `<dir>/<table>/<key>.json`
|
|
4
|
+
* plus `global.json` for the global slot. The directory is the state — this
|
|
5
|
+
* unit holds NO in-memory state of its own: `loadAll` re-reads the tree and
|
|
6
|
+
* every write is one durable file operation. The domain layer owns the live
|
|
7
|
+
* in-memory tables (seeded by the open-time `loadAll`) and serializes writes
|
|
8
|
+
* through its write chain, so this unit never mutates memory and needs no
|
|
9
|
+
* rollback — a failed write simply leaves both the file and the domain's
|
|
10
|
+
* memory unchanged.
|
|
11
|
+
*
|
|
12
|
+
* Per-record contract: a record document that is malformed or stamped with a
|
|
13
|
+
* different version reads as an absent record — one bad or stale file never
|
|
14
|
+
* bricks the whole unit, and a version bump discards stale records instead
|
|
15
|
+
* of migrating them. Record keys become path segments, so they must be
|
|
16
|
+
* path-safe (`[a-zA-Z0-9_-]+`); an unsafe key rejects at write.
|
|
17
|
+
*
|
|
18
|
+
* Legacy bootstrap: when the new tree has no document path, a legacy
|
|
19
|
+
* whole-unit file `<root>/<name>.json` (the pre-per-record layout) seeds
|
|
20
|
+
* per-record documents. Any new document path, including one whose contents
|
|
21
|
+
* are unreadable or stale, suppresses the bootstrap for the whole unit. The
|
|
22
|
+
* legacy file is never changed or deleted.
|
|
23
|
+
* @module @deepseek-ai/dsh-storage-json/src/per-record-unit
|
|
24
|
+
*/
|
|
25
|
+
import type { KvUnit, KvUnitDescriptor } from '@deepseek-ai/dsh-storage';
|
|
26
|
+
/**
|
|
27
|
+
* Open one `per-record`-layout unit under `root`: the unit directory is
|
|
28
|
+
* `<root>/<name>/`. Loads lazily on the first `loadAll` — this unit holds no
|
|
29
|
+
* state, so opening touches nothing on the medium.
|
|
30
|
+
* @param descriptor - Static identity and shape of the unit.
|
|
31
|
+
* @param root - Absolute backend root directory.
|
|
32
|
+
* @param onClose - Backend callback releasing the unit's open-slot.
|
|
33
|
+
* @returns the opened unit.
|
|
34
|
+
*/
|
|
35
|
+
export declare function openPerRecordUnit(descriptor: KvUnitDescriptor, root: string, onClose: () => void): Promise<KvUnit>;
|
|
36
|
+
/**
|
|
37
|
+
* One opened `per-record`-layout unit. Stateless by design: the directory is
|
|
38
|
+
* the medium, the domain layer owns the live memory, and each method here is
|
|
39
|
+
* a single durable file operation. Write ordering belongs to the caller (the
|
|
40
|
+
* domain layer's write chain), exactly like the `single`-layout unit.
|
|
41
|
+
*/
|
|
42
|
+
export declare class PerRecordJsonUnit implements KvUnit {
|
|
43
|
+
private readonly descriptor;
|
|
44
|
+
private readonly dir;
|
|
45
|
+
private readonly onClose;
|
|
46
|
+
private closed;
|
|
47
|
+
/** In-flight durable writes; close() drains them before releasing the unit. */
|
|
48
|
+
private readonly inFlight;
|
|
49
|
+
constructor(descriptor: KvUnitDescriptor, dir: string, onClose: () => void);
|
|
50
|
+
/** Re-read the tree: the directory is the authoritative state. */
|
|
51
|
+
loadAll(): Promise<{
|
|
52
|
+
tables: Record<string, Record<string, unknown>>;
|
|
53
|
+
global: unknown;
|
|
54
|
+
}>;
|
|
55
|
+
/** Durably replace one record: its own document, atomically. */
|
|
56
|
+
putRecord(table: string, key: string, value: unknown): Promise<void>;
|
|
57
|
+
/** Durably delete one record. Idempotent: a missing key is a no-op. */
|
|
58
|
+
deleteRecord(table: string, key: string): Promise<void>;
|
|
59
|
+
/** Durably replace the global singleton. Only valid when declared. */
|
|
60
|
+
setGlobal(value: unknown): Promise<void>;
|
|
61
|
+
/** Drain in-flight writes and release the unit. Idempotent. */
|
|
62
|
+
close(): Promise<void>;
|
|
63
|
+
private assertOpen;
|
|
64
|
+
/** Resolve a declared table's directory; an undeclared table is a caller bug and throws. */
|
|
65
|
+
private tableDir;
|
|
66
|
+
/** Durably replace one document, creating its parent directory. */
|
|
67
|
+
private writeDocument;
|
|
68
|
+
/** Track one durable write so close() drains it. */
|
|
69
|
+
private tracked;
|
|
70
|
+
}
|
|
71
|
+
//# sourceMappingURL=per-record-unit.d.ts.map
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* One opened JSON unit in `single` layout: the whole unit is one document at
|
|
3
|
+
* `<root>/<name>.json`. The in-memory state is authoritative; every write
|
|
4
|
+
* primitive mutates it and republishes the whole file atomically. Writes are
|
|
5
|
+
* NOT queued here — per the backend contract, write ordering belongs to the
|
|
6
|
+
* caller (the domain layer's write chain); this unit only guarantees that
|
|
7
|
+
* each single call publishes a complete, durable file. The `per-record`
|
|
8
|
+
* layout is a separate unit class in `per-record-unit.ts`.
|
|
9
|
+
* @module @deepseek-ai/dsh-storage-json/src/single-unit
|
|
10
|
+
*/
|
|
11
|
+
import type { KvUnit, KvUnitDescriptor } from '@deepseek-ai/dsh-storage';
|
|
12
|
+
/**
|
|
13
|
+
* Open (load or lazily create) one `single`-layout unit under `root`: the
|
|
14
|
+
* unit file is `<root>/<name>.json`.
|
|
15
|
+
* @param descriptor - Static identity and shape of the unit.
|
|
16
|
+
* @param root - Absolute backend root directory.
|
|
17
|
+
* @param onClose - Backend callback releasing the unit's open-slot.
|
|
18
|
+
* @returns the opened unit.
|
|
19
|
+
*/
|
|
20
|
+
export declare function openSingleUnit(descriptor: KvUnitDescriptor, root: string, onClose: () => void): Promise<KvUnit>;
|
|
21
|
+
//# sourceMappingURL=single-unit.d.ts.map
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@deepseek-ai/dsh-storage-json",
|
|
3
3
|
"description": "JSON file KV storage backend for the DeepSeek Harness storage hub",
|
|
4
|
-
"version": "0.1.
|
|
4
|
+
"version": "0.1.2-alpha.2",
|
|
5
5
|
"publishConfig": {
|
|
6
6
|
"access": "public"
|
|
7
7
|
},
|
|
@@ -32,16 +32,16 @@
|
|
|
32
32
|
],
|
|
33
33
|
"license": "MIT",
|
|
34
34
|
"peerDependencies": {
|
|
35
|
-
"@deepseek-ai/dsh-
|
|
36
|
-
"@deepseek-ai/
|
|
37
|
-
"@deepseek-ai/
|
|
35
|
+
"@deepseek-ai/dsh-invariants": "^0.1.2-alpha.2",
|
|
36
|
+
"@deepseek-ai/dsh-storage": "^0.1.2-alpha.2",
|
|
37
|
+
"@deepseek-ai/cordis": "^4.0.2"
|
|
38
38
|
},
|
|
39
39
|
"dependencies": {
|
|
40
|
-
"@deepseek-ai/schemastery": "^3.18.
|
|
40
|
+
"@deepseek-ai/schemastery": "^3.18.2"
|
|
41
41
|
},
|
|
42
42
|
"devDependencies": {
|
|
43
|
-
"@deepseek-ai/dsh-invariants": "^0.1.
|
|
44
|
-
"@deepseek-ai/dsh-storage": "^0.1.
|
|
45
|
-
"@deepseek-ai/cordis": "^4.0.
|
|
43
|
+
"@deepseek-ai/dsh-invariants": "^0.1.2-alpha.2",
|
|
44
|
+
"@deepseek-ai/dsh-storage": "^0.1.2-alpha.2",
|
|
45
|
+
"@deepseek-ai/cordis": "^4.0.2"
|
|
46
46
|
}
|
|
47
47
|
}
|
package/lib/types/unit.d.ts
DELETED
|
@@ -1,18 +0,0 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* One opened JSON unit. The in-memory state is authoritative; every write
|
|
3
|
-
* primitive mutates it and republishes the whole file atomically. Writes are
|
|
4
|
-
* NOT queued here — per the backend contract, write ordering belongs to the
|
|
5
|
-
* caller (the domain layer's write chain); this unit only guarantees that
|
|
6
|
-
* each single call publishes a complete, durable file.
|
|
7
|
-
* @module @deepseek-ai/dsh-storage-json/src/unit
|
|
8
|
-
*/
|
|
9
|
-
import type { KvUnit, KvUnitDescriptor } from '@deepseek-ai/dsh-storage';
|
|
10
|
-
/**
|
|
11
|
-
* Open (load or lazily create) one unit backed by `path`.
|
|
12
|
-
* @param descriptor - Static identity and shape of the unit.
|
|
13
|
-
* @param path - Absolute unit file path under the backend root.
|
|
14
|
-
* @param onClose - Backend callback releasing the unit's open-slot.
|
|
15
|
-
* @returns the opened unit.
|
|
16
|
-
*/
|
|
17
|
-
export declare function openJsonUnit(descriptor: KvUnitDescriptor, path: string, onClose: () => void): Promise<KvUnit>;
|
|
18
|
-
//# sourceMappingURL=unit.d.ts.map
|