@sublang/playbook 17.11.1 → 18.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/docs/cli.md +25 -15
- package/docs/embedding.md +12 -2
- package/package.json +4 -1
- package/reference/sdlc/code.playbook/bin/session-chunks.js +448 -0
- package/reference/sdlc/code.playbook/bin/session-replay.js +291 -0
- package/reference/sdlc/code.playbook/bin/session-store.js +340 -612
- package/reference/sdlc/code.playbook/bin/storage-manifest.js +36 -0
- package/reference/sdlc/code.playbook/session-store.d.ts +2 -0
package/docs/cli.md
CHANGED
|
@@ -321,20 +321,25 @@ Unsupported settings and ambiguous failures do not trigger that fallback
|
|
|
321
321
|
|
|
322
322
|
### Following replay history
|
|
323
323
|
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
one
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
|
|
332
|
-
|
|
333
|
-
|
|
324
|
+
Both front ends record observed host records in order through the shared store.
|
|
325
|
+
New sessions use a small schema-8 `<session-id>.json` root, a
|
|
326
|
+
`<session-id>.replay.json` root and immutable files in `<session-id>.chunks/`.
|
|
327
|
+
Every generated root and chunk is at most 8 MiB. Equal content within a session
|
|
328
|
+
is reused; one oversized record is split without discarding its contents.
|
|
329
|
+
Use `readHistory()` or `readStream()` rather than opening a former
|
|
330
|
+
`<session-id>.records.jsonl` file. Logical replay still contains version-1
|
|
331
|
+
envelopes with contiguous writer-assigned sequences and optional local roles.
|
|
332
|
+
|
|
333
|
+
The hydrated manifest remains logical schema 7 and contains token-free recovery.
|
|
334
|
+
Replay carries immutable configuration and graph records, so history does not
|
|
334
335
|
require installed modules. Provider continuation hints live separately in
|
|
335
336
|
`<session-id>.hints.json` and stay local. A hint is consumed before use and
|
|
336
337
|
accepted only for the exact checkpoint that created it.
|
|
337
338
|
|
|
339
|
+
The file bound does not cap total disk usage or in-memory session size.
|
|
340
|
+
Superseded chunks are retained for safe concurrent reads and recovery; they are
|
|
341
|
+
removed with the session. This release does not automatically prune evidence.
|
|
342
|
+
|
|
338
343
|
The manifest hashes its exact durable replay prefix. Missing or changed bytes,
|
|
339
344
|
saved incompleteness, or unsupported required context block continuation while
|
|
340
345
|
preserving readable history. A partial final line waits for completion; valid
|
|
@@ -346,7 +351,9 @@ history warning without authorizing a repeated action. Recovery and uncertainty
|
|
|
346
351
|
remain durable; the incomplete marker persists across process restarts.
|
|
347
352
|
Release retains the lease if that marker cannot be saved.
|
|
348
353
|
|
|
349
|
-
Copy the
|
|
354
|
+
Copy the complete file inventory from `exportBundle().files`, including chunks
|
|
355
|
+
and assets. Do not copy hints or leases, or serialize the hydrated manifest
|
|
356
|
+
back into one large file.
|
|
350
357
|
Continuation also requires matching working-directory/module paths and
|
|
351
358
|
compatible runtimes; changed paths allow history only. Prompts, replies and
|
|
352
359
|
tool content remain sensitive even after structured continuation fields have
|
|
@@ -354,18 +361,21 @@ been removed. See the [shared storage contract](https://github.com/sublang-ai/pl
|
|
|
354
361
|
|
|
355
362
|
### Migrating older sessions
|
|
356
363
|
|
|
357
|
-
Stop
|
|
358
|
-
|
|
364
|
+
Stop every CLI and Spex writer sharing a store and snapshot its home before
|
|
365
|
+
upgrading. Upgrade all hosts to the coordinated bounded-storage release before
|
|
366
|
+
new writes. Earlier Playbook versions and direct file readers cannot use schema 8.
|
|
367
|
+
When using the ordinary default, the CLI imports supported schema 2–6 sessions from the former `${XDG_STATE_HOME:-$HOME/.local/state}/playbook/sessions` directory into `~/.spex/sessions`, retaining history even when recovery is unavailable. Explicit store overrides bypass this import; existing schema 7 sessions are not converted automatically.
|
|
359
368
|
Migration retains original files under `local/migrations/` beside the destination sessions directory, validates the new bundle, then removes the former replay and manifest.
|
|
360
369
|
Unknown or unsafe inputs stay in place with a diagnostic.
|
|
361
370
|
|
|
362
|
-
For
|
|
371
|
+
For each selected monolithic session, including logical schema 7, run explicit
|
|
372
|
+
migration before continuing it. Use the same configuration that selects its store:
|
|
363
373
|
|
|
364
374
|
```sh
|
|
365
375
|
playbook migrate-session <id> [--with <path>]...
|
|
366
376
|
```
|
|
367
377
|
|
|
368
|
-
Schema 6 may retain execution; older CLI records and desktop sidecars lacking complete recovery remain readable history.
|
|
378
|
+
Schema 7 retains its validated logical recovery and exact replay; schema 6 may retain execution; older CLI records and desktop sidecars lacking complete recovery remain readable history.
|
|
369
379
|
Discovery skips these histories and unmigrated records with a reason; they do not block other sessions.
|
|
370
380
|
|
|
371
381
|
### Reconciling possible repository effects
|
package/docs/embedding.md
CHANGED
|
@@ -201,7 +201,8 @@ remain isolated. Child return, frame disposal, and a later root engagement do
|
|
|
201
201
|
not clear the session ledger.
|
|
202
202
|
|
|
203
203
|
Runtime and complete shell snapshots are schema 4. The shared store projects
|
|
204
|
-
provider continuations out of nested snapshots and writes schema-7
|
|
204
|
+
provider continuations out of nested snapshots and writes logical schema-7
|
|
205
|
+
manifests inside bounded physical schema-8 roots.
|
|
205
206
|
Local hints may rehydrate the current checkpoint; retained generations never
|
|
206
207
|
restore provider tokens. Use the shared migrator for older records instead of
|
|
207
208
|
guessing identity or effect evidence. On a compatible
|
|
@@ -472,7 +473,16 @@ only when `isUncertainTurnDiscardable(record)` returns true: no abandonment, no
|
|
|
472
473
|
clearly marked synthetic projection when a validated legacy journal has no
|
|
473
474
|
stream. `validate()` separates byte integrity from resumability.
|
|
474
475
|
`readManifest()` may return an older or unknown object; use
|
|
475
|
-
`validateSessionManifest()` before interpreting schema-7 recovery.
|
|
476
|
+
`validateSessionManifest()` before interpreting logical schema-7 recovery.
|
|
477
|
+
The store hydrates schema-8 roots transparently; do not parse the physical root
|
|
478
|
+
as a recovery snapshot. `exportBundle().manifest` contains the exact physical
|
|
479
|
+
root bytes, and `exportBundle().files` lists its verified portable file closure
|
|
480
|
+
relative to `sessionsDir`. Copy that closure as one session unit. The
|
|
481
|
+
`replay` result remains its checkpoint-proven logical prefix.
|
|
482
|
+
|
|
483
|
+
Every new storage root and chunk is at most 8 MiB; total retained disk usage and
|
|
484
|
+
hydrated memory are not capped. Stop all old writers and upgrade the CLI and
|
|
485
|
+
embedded host together before creating new sessions or migrating old ones.
|
|
476
486
|
`migrate()` preserves original inputs before conversion. `delete()` removes the
|
|
477
487
|
bundle under the shared lease, with the manifest last. Management needs no
|
|
478
488
|
playbook module. A coordination-only `acquireManagement()` lease reserves even
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@sublang/playbook",
|
|
3
|
-
"version": "
|
|
3
|
+
"version": "18.0.0",
|
|
4
4
|
"type": "module",
|
|
5
5
|
"description": "Composable XState v5 playbook runtime with compiled Captain, CODE, REVIEW, DECIDE, DEV, BRANCH, PR, and INSPECT workflows driven by GEARS specs.",
|
|
6
6
|
"license": "Apache-2.0",
|
|
@@ -92,6 +92,9 @@
|
|
|
92
92
|
"reference/sdlc/code.playbook/bin/replay-observer.js",
|
|
93
93
|
"reference/sdlc/code.playbook/bin/session-store.js",
|
|
94
94
|
"reference/sdlc/code.playbook/bin/private-paths.js",
|
|
95
|
+
"reference/sdlc/code.playbook/bin/session-chunks.js",
|
|
96
|
+
"reference/sdlc/code.playbook/bin/session-replay.js",
|
|
97
|
+
"reference/sdlc/code.playbook/bin/storage-manifest.js",
|
|
95
98
|
"reference/sdlc/code.playbook/bin/machine-identity.js",
|
|
96
99
|
"reference/sdlc/code.playbook/bin/provision.js",
|
|
97
100
|
"reference/sdlc/code.playbook/bin/adapter-sdk.js",
|
|
@@ -0,0 +1,448 @@
|
|
|
1
|
+
// SPDX-License-Identifier: Apache-2.0
|
|
2
|
+
// SPDX-FileCopyrightText: 2026 SubLang International <https://sublang.ai>
|
|
3
|
+
|
|
4
|
+
// Physical storage only: callers retain the session lease and validate the
|
|
5
|
+
// materialized recovery schema. No published chunk is replaced or collected.
|
|
6
|
+
import { createHash, randomUUID } from 'node:crypto';
|
|
7
|
+
import { AsyncLocalStorage } from 'node:async_hooks';
|
|
8
|
+
import { constants } from 'node:fs';
|
|
9
|
+
import * as nativeFs from 'node:fs/promises';
|
|
10
|
+
import { dirname, isAbsolute, join } from 'node:path';
|
|
11
|
+
import { sameFileIdentity, tightenPrivateEntry, verifyPrivateEntry } from './private-paths.js';
|
|
12
|
+
|
|
13
|
+
export const MAX_SESSION_CHUNK_BYTES = 8 * 1024 * 1024;
|
|
14
|
+
const INLINE_BYTES = 16 * 1024;
|
|
15
|
+
const FANOUT = 1024;
|
|
16
|
+
const CACHE_BYTES = 32 * 1024 * 1024;
|
|
17
|
+
const CACHE_NODES = 4096;
|
|
18
|
+
const DIGEST = /^[0-9a-f]{64}$/;
|
|
19
|
+
const hash = (bytes) => createHash('sha256').update(bytes).digest('hex');
|
|
20
|
+
const jsonBytes = (value) => Buffer.from(JSON.stringify(value));
|
|
21
|
+
const parseJson = (bytes) => JSON.parse(new TextDecoder('utf-8', { fatal: true }).decode(bytes));
|
|
22
|
+
const own = (object, key) => Object.prototype.hasOwnProperty.call(object, key);
|
|
23
|
+
const fail = () => { throw new Error('invalid session chunk encoding'); };
|
|
24
|
+
function reference(value) {
|
|
25
|
+
if (!value || typeof value !== 'object' || Array.isArray(value) ||
|
|
26
|
+
Object.keys(value).length !== 2 || !own(value, 'sha256') || !own(value, 'byteLength') ||
|
|
27
|
+
!DIGEST.test(value.sha256) || !Number.isSafeInteger(value.byteLength) ||
|
|
28
|
+
value.byteLength < 1 || value.byteLength > MAX_SESSION_CHUNK_BYTES) fail();
|
|
29
|
+
return value;
|
|
30
|
+
}
|
|
31
|
+
function privateStat(stat, directory) {
|
|
32
|
+
if (typeof stat.mode === 'bigint') {
|
|
33
|
+
const original = stat;
|
|
34
|
+
stat = { ...stat, mode: Number(stat.mode), uid: Number(stat.uid), nlink: Number(stat.nlink),
|
|
35
|
+
isFile: () => original.isFile(), isDirectory: () => original.isDirectory(), isSymbolicLink: () => original.isSymbolicLink() };
|
|
36
|
+
}
|
|
37
|
+
verifyPrivateEntry(stat, directory, 'session chunk');
|
|
38
|
+
if ((stat.mode & 0o7777) !== (directory ? 0o700 : 0o600)) throw new Error('session chunk permissions must be private');
|
|
39
|
+
}
|
|
40
|
+
function unchanged(before, after) {
|
|
41
|
+
const precise = before.mtimeNs !== undefined && after.mtimeNs !== undefined;
|
|
42
|
+
return BigInt(before.dev) === BigInt(after.dev) && BigInt(before.ino) === BigInt(after.ino) && BigInt(before.size) === BigInt(after.size) &&
|
|
43
|
+
(precise ? before.mtimeNs === after.mtimeNs && before.ctimeNs === after.ctimeNs :
|
|
44
|
+
typeof before.size === typeof after.size ? before.mtimeMs === after.mtimeMs && before.ctimeMs === after.ctimeMs :
|
|
45
|
+
Math.trunc(Number(before.mtimeMs)) === Math.trunc(Number(after.mtimeMs)) && Math.trunc(Number(before.ctimeMs)) === Math.trunc(Number(after.ctimeMs)));
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
export function createSessionChunks({ directory, fsOps, createDirectory = true }) {
|
|
49
|
+
if (typeof directory !== 'string' || !isAbsolute(directory) || directory.includes('\0')) throw new TypeError('session chunk directory must be absolute');
|
|
50
|
+
if (typeof createDirectory !== 'boolean') throw new TypeError('createDirectory must be boolean');
|
|
51
|
+
const fs = { ...nativeFs, ...fsOps };
|
|
52
|
+
// Reuse immutable bytes only after a fresh private-path proof with exact
|
|
53
|
+
// nanosecond metadata. Public export proofs always read and hash the files.
|
|
54
|
+
const verified = new Map();
|
|
55
|
+
let cachedBytes = 0;
|
|
56
|
+
const forget = (digest) => {
|
|
57
|
+
const entry = verified.get(digest);
|
|
58
|
+
if (entry) { cachedBytes -= entry.bytes.length; verified.delete(digest); }
|
|
59
|
+
};
|
|
60
|
+
const remember = (digest, bytes, stat, owner) => {
|
|
61
|
+
if (stat.mtimeNs === undefined || stat.ctimeNs === undefined) return;
|
|
62
|
+
forget(digest);
|
|
63
|
+
verified.set(digest, { bytes, stat, owner }); cachedBytes += bytes.length;
|
|
64
|
+
while (cachedBytes > CACHE_BYTES || verified.size > CACHE_NODES) forget(verified.keys().next().value);
|
|
65
|
+
};
|
|
66
|
+
const operations = new AsyncLocalStorage();
|
|
67
|
+
const freshDirectory = async () => { const stat = await fs.lstat(directory); privateStat(stat, true); return stat; };
|
|
68
|
+
const checkDirectory = async () => {
|
|
69
|
+
const context = operations.getStore();
|
|
70
|
+
if (context?.owner) return context.owner;
|
|
71
|
+
const owner = await freshDirectory();
|
|
72
|
+
if (context) context.owner = owner;
|
|
73
|
+
return owner;
|
|
74
|
+
};
|
|
75
|
+
const operation = (work, force = false) => operations.run({ reads: new Map(), writes: new Map(), force }, async () => {
|
|
76
|
+
try { return await work(); }
|
|
77
|
+
finally {
|
|
78
|
+
const owner = operations.getStore().owner;
|
|
79
|
+
if (owner && !sameFileIdentity(owner, await freshDirectory())) throw new Error('session chunk directory changed during operation');
|
|
80
|
+
}
|
|
81
|
+
});
|
|
82
|
+
const sync = async (path, expected) => {
|
|
83
|
+
const handle = await fs.open(path, constants.O_RDONLY | (constants.O_NOFOLLOW ?? 0) | (constants.O_DIRECTORY ?? 0));
|
|
84
|
+
try {
|
|
85
|
+
const opened = await handle.stat(); privateStat(opened, true);
|
|
86
|
+
if (expected && !sameFileIdentity(expected, opened)) throw new Error('session chunk directory changed');
|
|
87
|
+
await handle.sync();
|
|
88
|
+
const current = await fs.lstat(path); privateStat(current, true);
|
|
89
|
+
if (!sameFileIdentity(opened, current)) throw new Error('session chunk directory changed');
|
|
90
|
+
} finally { await handle.close(); }
|
|
91
|
+
};
|
|
92
|
+
const ensureDirectory = async () => {
|
|
93
|
+
// Never create the sessions directory or its ancestors implicitly.
|
|
94
|
+
let stat;
|
|
95
|
+
try { stat = await fs.lstat(directory); }
|
|
96
|
+
catch (error) {
|
|
97
|
+
if (error.code !== 'ENOENT' || !createDirectory) throw error;
|
|
98
|
+
const parent = await fs.lstat(dirname(directory)); privateStat(parent, true);
|
|
99
|
+
try { await fs.mkdir(directory, { mode: 0o700 }); }
|
|
100
|
+
catch (cause) { if (cause.code !== 'EEXIST') throw cause; }
|
|
101
|
+
stat = await fs.lstat(directory);
|
|
102
|
+
await sync(dirname(directory), parent);
|
|
103
|
+
}
|
|
104
|
+
await tightenPrivateEntry(directory, stat, true, fs, 'session chunk directory');
|
|
105
|
+
};
|
|
106
|
+
const prepare = async ({ recoverInterrupted = false } = {}) => {
|
|
107
|
+
if (typeof recoverInterrupted !== 'boolean') throw new TypeError('recoverInterrupted must be boolean');
|
|
108
|
+
await ensureDirectory();
|
|
109
|
+
const pendingAliases = new Map();
|
|
110
|
+
const names = await fs.readdir(directory);
|
|
111
|
+
for (const name of names) {
|
|
112
|
+
const match = /^\.write-([0-9a-f]{64})-[0-9a-f-]{36}\.tmp$/.exec(name);
|
|
113
|
+
if (!match) continue;
|
|
114
|
+
const path = join(directory, name);
|
|
115
|
+
const temporary = await fs.lstat(path);
|
|
116
|
+
if (temporary.nlink === 1) privateStat(temporary, false);
|
|
117
|
+
else {
|
|
118
|
+
// A crash between exclusive link publication and alias removal may
|
|
119
|
+
// leave precisely these two names for one private store-owned inode.
|
|
120
|
+
if (temporary.isSymbolicLink() || !temporary.isFile() || temporary.nlink !== 2 ||
|
|
121
|
+
(process.getuid && temporary.uid !== process.getuid()) || (temporary.mode & 0o7777) !== 0o600) throw new Error('unsafe interrupted session chunk publication');
|
|
122
|
+
const destination = await fs.lstat(join(directory, match[1]));
|
|
123
|
+
if (!unchanged(temporary, destination) || destination.isSymbolicLink()) throw new Error('session chunk temporary alias differs from publication');
|
|
124
|
+
pendingAliases.set(match[1], temporary);
|
|
125
|
+
}
|
|
126
|
+
if (recoverInterrupted) {
|
|
127
|
+
// The caller must hold exclusive recovery authority. Unowned prepare
|
|
128
|
+
// must never unlink a temporary still being written by another host.
|
|
129
|
+
if (!unchanged(temporary, await fs.lstat(path))) throw new Error('session chunk temporary changed');
|
|
130
|
+
await fs.unlink(path);
|
|
131
|
+
}
|
|
132
|
+
}
|
|
133
|
+
await sync(directory, await checkDirectory());
|
|
134
|
+
for (const name of names) {
|
|
135
|
+
if (!DIGEST.test(name)) continue;
|
|
136
|
+
const path = join(directory, name);
|
|
137
|
+
const stat = await fs.lstat(path);
|
|
138
|
+
if (stat.size > MAX_SESSION_CHUNK_BYTES) throw new Error('session chunk exceeds file bound');
|
|
139
|
+
if (!recoverInterrupted && pendingAliases.has(name) && unchanged(pendingAliases.get(name), stat)) continue;
|
|
140
|
+
await tightenPrivateEntry(path, stat, false, fs, 'session chunk file');
|
|
141
|
+
}
|
|
142
|
+
};
|
|
143
|
+
const readNodeUncached = async (ref) => {
|
|
144
|
+
reference(ref);
|
|
145
|
+
const owner = await checkDirectory();
|
|
146
|
+
const path = join(directory, ref.sha256);
|
|
147
|
+
const before = await fs.lstat(path, { bigint: true }); privateStat(before, false);
|
|
148
|
+
if (BigInt(before.size) !== BigInt(ref.byteLength)) throw new Error('session chunk length mismatch');
|
|
149
|
+
const cached = verified.get(ref.sha256);
|
|
150
|
+
if (!operations.getStore()?.force && cached && before.mtimeNs !== undefined &&
|
|
151
|
+
sameFileIdentity(owner, cached.owner) && unchanged(cached.stat, before)) {
|
|
152
|
+
verified.delete(ref.sha256); verified.set(ref.sha256, cached);
|
|
153
|
+
return cached.bytes;
|
|
154
|
+
}
|
|
155
|
+
forget(ref.sha256);
|
|
156
|
+
const handle = await fs.open(path, constants.O_RDONLY | (constants.O_NOFOLLOW ?? 0) | (constants.O_NONBLOCK ?? 0));
|
|
157
|
+
try {
|
|
158
|
+
const opened = await handle.stat({ bigint: true }); privateStat(opened, false);
|
|
159
|
+
if (!unchanged(before, opened)) throw new Error('session chunk changed before read');
|
|
160
|
+
// Read the proven bounded length, rather than readFile on a growing file.
|
|
161
|
+
const bytes = Buffer.alloc(ref.byteLength);
|
|
162
|
+
let offset = 0;
|
|
163
|
+
while (offset < bytes.length) {
|
|
164
|
+
const { bytesRead } = await handle.read(bytes, offset, bytes.length - offset, offset);
|
|
165
|
+
if (!bytesRead) throw new Error('session chunk truncated during read');
|
|
166
|
+
offset += bytesRead;
|
|
167
|
+
}
|
|
168
|
+
const after = await handle.stat({ bigint: true }); privateStat(after, false);
|
|
169
|
+
const current = await fs.lstat(path, { bigint: true }); privateStat(current, false);
|
|
170
|
+
if (!unchanged(opened, after) || !unchanged(after, current) || !sameFileIdentity(owner, await checkDirectory())) throw new Error('session chunk changed during read');
|
|
171
|
+
if (hash(bytes) !== ref.sha256) throw new Error('session chunk digest mismatch');
|
|
172
|
+
if ([before, opened, after, current].every((stat) => stat.mtimeNs !== undefined && stat.ctimeNs !== undefined)) remember(ref.sha256, bytes, current, owner);
|
|
173
|
+
return bytes;
|
|
174
|
+
} finally { await handle.close(); }
|
|
175
|
+
};
|
|
176
|
+
const readNode = (ref) => {
|
|
177
|
+
reference(ref);
|
|
178
|
+
const reads = operations.getStore()?.reads;
|
|
179
|
+
const previous = reads?.get(ref.sha256);
|
|
180
|
+
if (previous) {
|
|
181
|
+
if (previous.byteLength !== ref.byteLength) fail();
|
|
182
|
+
return previous.bytes;
|
|
183
|
+
}
|
|
184
|
+
const bytes = readNodeUncached(ref);
|
|
185
|
+
reads?.set(ref.sha256, { byteLength: ref.byteLength, bytes });
|
|
186
|
+
return bytes;
|
|
187
|
+
};
|
|
188
|
+
const writeNode = async (bytes) => {
|
|
189
|
+
if (!bytes.length || bytes.length > MAX_SESSION_CHUNK_BYTES) throw new Error('session chunk exceeds file bound');
|
|
190
|
+
const ref = Object.freeze({ sha256: hash(bytes), byteLength: bytes.length });
|
|
191
|
+
const writes = operations.getStore()?.writes;
|
|
192
|
+
if (writes?.has(ref.sha256)) return writes.get(ref.sha256);
|
|
193
|
+
const owner = await checkDirectory();
|
|
194
|
+
try { await readNode(ref); writes?.set(ref.sha256, ref); return ref; }
|
|
195
|
+
catch (error) {
|
|
196
|
+
if (error.code !== 'ENOENT') throw error;
|
|
197
|
+
operations.getStore()?.reads.delete(ref.sha256);
|
|
198
|
+
}
|
|
199
|
+
const temporary = join(directory, `.write-${ref.sha256}-${randomUUID()}.tmp`);
|
|
200
|
+
const destination = join(directory, ref.sha256);
|
|
201
|
+
let handle;
|
|
202
|
+
try {
|
|
203
|
+
handle = await fs.open(temporary, 'wx', 0o600);
|
|
204
|
+
await handle.chmod(0o600);
|
|
205
|
+
const opened = await handle.stat(); privateStat(opened, false);
|
|
206
|
+
await handle.writeFile(bytes); await handle.sync();
|
|
207
|
+
const after = await handle.stat(); privateStat(after, false);
|
|
208
|
+
const current = await fs.lstat(temporary); privateStat(current, false);
|
|
209
|
+
if (!sameFileIdentity(opened, after) || !unchanged(after, current) || after.size !== bytes.length || !sameFileIdentity(owner, await checkDirectory())) throw new Error('session chunk changed during write');
|
|
210
|
+
await handle.close(); handle = undefined;
|
|
211
|
+
// link is exclusive publication; remove its temporary alias before any
|
|
212
|
+
// private reader sees it, then sync both the file and directory.
|
|
213
|
+
try { await fs.link(temporary, destination); }
|
|
214
|
+
catch (error) { if (error.code !== 'EEXIST') throw error; }
|
|
215
|
+
await fs.unlink(temporary);
|
|
216
|
+
await readNode(ref);
|
|
217
|
+
writes?.set(ref.sha256, ref);
|
|
218
|
+
return ref;
|
|
219
|
+
} finally {
|
|
220
|
+
await handle?.close();
|
|
221
|
+
// Failed publication leaves only unreachable bytes, never a new root.
|
|
222
|
+
try { await fs.unlink(temporary); } catch (error) { if (error.code !== 'ENOENT') throw error; }
|
|
223
|
+
}
|
|
224
|
+
};
|
|
225
|
+
const put = (tag, value) => writeNode(Buffer.concat([Buffer.from([tag]), jsonBytes(value)]));
|
|
226
|
+
const writeBytes = async (input) => {
|
|
227
|
+
const bytes = Buffer.from(input);
|
|
228
|
+
let level = [];
|
|
229
|
+
for (let offset = 0; offset < bytes.length || offset === 0; offset += MAX_SESSION_CHUNK_BYTES - 1) {
|
|
230
|
+
level.push(await writeNode(Buffer.concat([Buffer.from([0]), bytes.subarray(offset, offset + MAX_SESSION_CHUNK_BYTES - 1)])));
|
|
231
|
+
}
|
|
232
|
+
while (level.length > 1) {
|
|
233
|
+
const next = [];
|
|
234
|
+
for (let offset = 0; offset < level.length; offset += FANOUT) next.push(await put(1, level.slice(offset, offset + FANOUT)));
|
|
235
|
+
level = next;
|
|
236
|
+
}
|
|
237
|
+
return level[0];
|
|
238
|
+
};
|
|
239
|
+
const readBytes = async (ref, depth = 0) => {
|
|
240
|
+
if (depth > 512) fail();
|
|
241
|
+
const bytes = await readNode(ref);
|
|
242
|
+
if (bytes[0] === 0) return bytes.subarray(1);
|
|
243
|
+
if (bytes[0] !== 1) fail();
|
|
244
|
+
const children = parseJson(bytes.subarray(1));
|
|
245
|
+
if (!Array.isArray(children) || !children.length || children.length > FANOUT) fail();
|
|
246
|
+
const pieces = [];
|
|
247
|
+
for (const child of children) pieces.push(await readBytes(reference(child), depth + 1));
|
|
248
|
+
return Buffer.concat(pieces);
|
|
249
|
+
};
|
|
250
|
+
const writeEncodedJson = async (encoded, bytes = jsonBytes(encoded)) => {
|
|
251
|
+
return bytes.length + 1 <= MAX_SESSION_CHUNK_BYTES
|
|
252
|
+
? writeNode(Buffer.concat([Buffer.from([2]), bytes]))
|
|
253
|
+
: put(3, await writeBytes(bytes));
|
|
254
|
+
};
|
|
255
|
+
// Inline JSON has no asynchronous work. Yield only at actual chunk IO,
|
|
256
|
+
// retaining depth-first ordering even when a child needs persistence.
|
|
257
|
+
const eachSequential = (values, visit, start = 0) => {
|
|
258
|
+
for (let index = start; index < values.length; index++) {
|
|
259
|
+
const pending = visit(values[index], index);
|
|
260
|
+
if (pending instanceof Promise) return pending.then(() => eachSequential(values, visit, index + 1));
|
|
261
|
+
}
|
|
262
|
+
};
|
|
263
|
+
const writeJson = async (value) => {
|
|
264
|
+
const active = new Set();
|
|
265
|
+
const encode = (node, depth) => {
|
|
266
|
+
if (depth > 512) throw new Error('session JSON nesting exceeds supported depth');
|
|
267
|
+
if (node === null || typeof node === 'string' || typeof node === 'boolean' || (typeof node === 'number' && Number.isFinite(node))) {
|
|
268
|
+
const serialized = jsonBytes(node);
|
|
269
|
+
return serialized.length <= INLINE_BYTES ? ['v', node] : writeBytes(serialized).then((ref) => ['s', ref]);
|
|
270
|
+
}
|
|
271
|
+
if (!node || typeof node !== 'object' || (!Array.isArray(node) && ![Object.prototype, null].includes(Object.getPrototypeOf(node))) || active.has(node)) throw new TypeError('session chunks require plain acyclic JSON');
|
|
272
|
+
active.add(node);
|
|
273
|
+
try {
|
|
274
|
+
const entries = [];
|
|
275
|
+
for (const key of Object.keys(node)) {
|
|
276
|
+
const descriptor = Object.getOwnPropertyDescriptor(node, key);
|
|
277
|
+
if (!descriptor || !own(descriptor, 'value')) throw new TypeError('session chunks reject JSON accessors');
|
|
278
|
+
entries.push([key, descriptor.value]);
|
|
279
|
+
}
|
|
280
|
+
if (Array.isArray(node) && (entries.length !== node.length || entries.some(([key], index) => key !== String(index)))) throw new TypeError('session chunks reject sparse or extended arrays');
|
|
281
|
+
const encoded = [];
|
|
282
|
+
const finish = () => {
|
|
283
|
+
const plain = Array.isArray(node) ? encoded.map(([, child]) => child) : encoded;
|
|
284
|
+
const representation = [Array.isArray(node) ? 'a' : 'o', plain];
|
|
285
|
+
const bytes = jsonBytes(representation);
|
|
286
|
+
return bytes.length <= INLINE_BYTES ? representation : writeEncodedJson(representation, bytes).then((ref) => ['r', ref]);
|
|
287
|
+
};
|
|
288
|
+
const children = eachSequential(entries, ([key, child]) => {
|
|
289
|
+
const result = encode(child, depth + 1);
|
|
290
|
+
if (result instanceof Promise) return result.then((value) => { encoded.push([key, value]); });
|
|
291
|
+
encoded.push([key, result]);
|
|
292
|
+
});
|
|
293
|
+
const result = children instanceof Promise ? children.then(finish) : finish();
|
|
294
|
+
if (result instanceof Promise) return result.finally(() => active.delete(node));
|
|
295
|
+
active.delete(node);
|
|
296
|
+
return result;
|
|
297
|
+
} catch (error) { active.delete(node); throw error; }
|
|
298
|
+
};
|
|
299
|
+
return writeEncodedJson(await encode(value, 0));
|
|
300
|
+
};
|
|
301
|
+
const decode = (node, depth, collect) => {
|
|
302
|
+
if (depth > 2048 || !Array.isArray(node) || node.length !== 2) fail();
|
|
303
|
+
const [kind, value] = node;
|
|
304
|
+
if (kind === 'v') {
|
|
305
|
+
if (value !== null && !['string', 'boolean', 'number'].includes(typeof value) || typeof value === 'number' && !Number.isFinite(value)) fail();
|
|
306
|
+
return value;
|
|
307
|
+
}
|
|
308
|
+
if (kind === 'r') { collect?.(reference(value)); return loadJson(value, depth + 1, collect); }
|
|
309
|
+
if (kind === 's') {
|
|
310
|
+
collect?.(reference(value));
|
|
311
|
+
return readBytes(value).then((bytes) => { const result = parseJson(bytes); if (typeof result !== 'string') fail(); return result; });
|
|
312
|
+
}
|
|
313
|
+
if (kind !== 'a' && kind !== 'o' || !Array.isArray(value)) fail();
|
|
314
|
+
const result = kind === 'a' ? [] : {};
|
|
315
|
+
const children = eachSequential(value, (entry) => {
|
|
316
|
+
if (kind === 'o' && (!Array.isArray(entry) || entry.length !== 2 || typeof entry[0] !== 'string' || own(result, entry[0]))) fail();
|
|
317
|
+
const set = (decoded) => {
|
|
318
|
+
if (kind === 'a') result.push(decoded);
|
|
319
|
+
else Object.defineProperty(result, entry[0], { value: decoded, enumerable: true, writable: true, configurable: true });
|
|
320
|
+
};
|
|
321
|
+
const decoded = decode(kind === 'a' ? entry : entry[1], depth + 1, collect);
|
|
322
|
+
if (decoded instanceof Promise) return decoded.then(set);
|
|
323
|
+
set(decoded);
|
|
324
|
+
});
|
|
325
|
+
return children instanceof Promise ? children.then(() => result) : result;
|
|
326
|
+
};
|
|
327
|
+
const loadJson = async (ref, depth = 0, collect) => {
|
|
328
|
+
if (depth > 2048) fail();
|
|
329
|
+
const bytes = await readNode(ref);
|
|
330
|
+
if (bytes[0] === 2) return decode(parseJson(bytes.subarray(1)), depth + 1, collect);
|
|
331
|
+
if (bytes[0] !== 3) fail();
|
|
332
|
+
const body = reference(parseJson(bytes.subarray(1)));
|
|
333
|
+
collect?.(body);
|
|
334
|
+
return decode(parseJson(await readBytes(body)), depth + 1, collect);
|
|
335
|
+
};
|
|
336
|
+
const walk = async (ref) => {
|
|
337
|
+
const seen = new Map();
|
|
338
|
+
const visit = async (value) => {
|
|
339
|
+
reference(value);
|
|
340
|
+
if (seen.has(value.sha256)) { if (seen.get(value.sha256).byteLength !== value.byteLength) fail(); return; }
|
|
341
|
+
const bytes = await readNode(value);
|
|
342
|
+
seen.set(value.sha256, Object.freeze({ path: value.sha256, sha256: value.sha256, byteLength: value.byteLength }));
|
|
343
|
+
if (bytes[0] === 0) return;
|
|
344
|
+
if (bytes[0] === 1) {
|
|
345
|
+
const children = parseJson(bytes.subarray(1));
|
|
346
|
+
if (!Array.isArray(children) || !children.length || children.length > FANOUT) fail();
|
|
347
|
+
for (const child of children) await visit(child);
|
|
348
|
+
} else if (bytes[0] === 2 || bytes[0] === 3) {
|
|
349
|
+
const refs = [];
|
|
350
|
+
await loadJson(value, 0, (child) => refs.push(child));
|
|
351
|
+
for (const child of refs) await visit(child);
|
|
352
|
+
} else fail();
|
|
353
|
+
};
|
|
354
|
+
await visit(ref);
|
|
355
|
+
return Object.freeze([...seen.values()]);
|
|
356
|
+
};
|
|
357
|
+
const validateEncoding = (encoded, depth = 0) => {
|
|
358
|
+
if (depth > 2048 || !Array.isArray(encoded) || encoded.length !== 2) fail();
|
|
359
|
+
const [kind, value] = encoded;
|
|
360
|
+
if (kind === 'v') {
|
|
361
|
+
if (value !== null && !['string', 'boolean', 'number'].includes(typeof value) || typeof value === 'number' && !Number.isFinite(value)) fail();
|
|
362
|
+
} else if (kind === 'r' || kind === 's') reference(value);
|
|
363
|
+
else if (kind === 'a') {
|
|
364
|
+
if (!Array.isArray(value)) fail();
|
|
365
|
+
for (const child of value) validateEncoding(child, depth + 1);
|
|
366
|
+
} else if (kind === 'o') {
|
|
367
|
+
if (!Array.isArray(value)) fail();
|
|
368
|
+
const keys = new Set();
|
|
369
|
+
for (const entry of value) {
|
|
370
|
+
if (!Array.isArray(entry) || entry.length !== 2 || typeof entry[0] !== 'string' || keys.has(entry[0])) fail();
|
|
371
|
+
keys.add(entry[0]); validateEncoding(entry[1], depth + 1);
|
|
372
|
+
}
|
|
373
|
+
} else fail();
|
|
374
|
+
};
|
|
375
|
+
const validateNamespace = async () => {
|
|
376
|
+
const owner = await checkDirectory();
|
|
377
|
+
const entries = [];
|
|
378
|
+
for (const name of await fs.readdir(directory)) {
|
|
379
|
+
const path = join(directory, name);
|
|
380
|
+
const stat = await fs.lstat(path);
|
|
381
|
+
if (!DIGEST.test(name)) {
|
|
382
|
+
// A namespace proof never repairs temporary aliases. The same proof
|
|
383
|
+
// used by prepare permits a pending store-owned publication alias.
|
|
384
|
+
const temporary = /^\.write-([0-9a-f]{64})-[0-9a-f-]{36}\.tmp$/.exec(name);
|
|
385
|
+
if (temporary && stat.nlink === 2) {
|
|
386
|
+
if (stat.isSymbolicLink() || !stat.isFile() ||
|
|
387
|
+
(process.getuid && stat.uid !== process.getuid()) || (stat.mode & 0o7777) !== 0o600 ||
|
|
388
|
+
!unchanged(stat, await fs.lstat(join(directory, temporary[1])))) throw new Error('unsafe interrupted session chunk publication');
|
|
389
|
+
} else privateStat(stat, false);
|
|
390
|
+
continue;
|
|
391
|
+
}
|
|
392
|
+
if (stat.size > MAX_SESSION_CHUNK_BYTES) throw new Error('session chunk exceeds file bound');
|
|
393
|
+
const bytes = await readNode({ sha256: name, byteLength: stat.size });
|
|
394
|
+
if (bytes[0] === 1) {
|
|
395
|
+
const children = parseJson(bytes.subarray(1));
|
|
396
|
+
if (!Array.isArray(children) || !children.length || children.length > FANOUT) fail();
|
|
397
|
+
for (const child of children) reference(child);
|
|
398
|
+
} else if (bytes[0] === 2) validateEncoding(parseJson(bytes.subarray(1)));
|
|
399
|
+
else if (bytes[0] === 3) reference(parseJson(bytes.subarray(1)));
|
|
400
|
+
else if (bytes[0] !== 0) fail();
|
|
401
|
+
entries.push(Object.freeze({ path: name, byteLength: bytes.length }));
|
|
402
|
+
}
|
|
403
|
+
if (!sameFileIdentity(owner, await checkDirectory())) throw new Error('session chunk directory changed during validation');
|
|
404
|
+
return Object.freeze(entries);
|
|
405
|
+
};
|
|
406
|
+
const remove = async () => {
|
|
407
|
+
let owner;
|
|
408
|
+
try { owner = await checkDirectory(); } catch (error) { if (error.code === 'ENOENT') return; throw error; }
|
|
409
|
+
const names = await fs.readdir(directory);
|
|
410
|
+
// Validate every candidate before removing any; links and subdirectories
|
|
411
|
+
// are never followed or recursively removed.
|
|
412
|
+
for (const name of names) privateStat(await fs.lstat(join(directory, name)), false);
|
|
413
|
+
for (const name of names) {
|
|
414
|
+
if (!sameFileIdentity(owner, await checkDirectory())) throw new Error('session chunk directory changed during deletion');
|
|
415
|
+
const path = join(directory, name); privateStat(await fs.lstat(path), false); await fs.unlink(path);
|
|
416
|
+
}
|
|
417
|
+
await sync(directory, owner);
|
|
418
|
+
await fs.rmdir(directory);
|
|
419
|
+
verified.clear(); cachedBytes = 0;
|
|
420
|
+
await sync(dirname(directory));
|
|
421
|
+
};
|
|
422
|
+
const writeOperation = (work) => operation(async () => {
|
|
423
|
+
await ensureDirectory();
|
|
424
|
+
const owner = await checkDirectory();
|
|
425
|
+
const result = await work();
|
|
426
|
+
// Files have each been synced before publication. One directory sync
|
|
427
|
+
// proves the complete dependency graph before its reference is returned.
|
|
428
|
+
await sync(directory, owner);
|
|
429
|
+
return result;
|
|
430
|
+
});
|
|
431
|
+
return Object.freeze({
|
|
432
|
+
prepare,
|
|
433
|
+
writeJson: (value) => writeOperation(() => writeJson(value)),
|
|
434
|
+
writeBytesAndJson: (bytes, makeJson) => writeOperation(async () => {
|
|
435
|
+
if (typeof makeJson !== 'function') throw new TypeError('session chunk JSON builder must be a function');
|
|
436
|
+
// This fixed dependency pair cannot return an acknowledged head until
|
|
437
|
+
// both files and the containing directory have been synced.
|
|
438
|
+
const content = await writeBytes(bytes);
|
|
439
|
+
return writeJson(makeJson(content));
|
|
440
|
+
}),
|
|
441
|
+
readJson: (ref) => operation(() => loadJson(ref)),
|
|
442
|
+
writeBytes: (bytes) => writeOperation(() => writeBytes(bytes)),
|
|
443
|
+
readBytes: (ref) => operation(async () => Buffer.from(await readBytes(ref))),
|
|
444
|
+
walk: (ref) => operation(() => walk(ref), true),
|
|
445
|
+
validateNamespace: () => operation(validateNamespace, true),
|
|
446
|
+
delete: remove,
|
|
447
|
+
});
|
|
448
|
+
}
|