@gregiteen/ssss-cli 0.12.1 → 0.13.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -7,6 +7,61 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
7
7
 
8
8
  ## [Unreleased]
9
9
 
10
+ ## [0.13.0] - 2026-11-02
11
+ ### Fixed
12
+ - A committed operation is no longer executed twice. Stage 2 now falls back to
13
+ the event log when the idempotency store has no record, because every event
14
+ already carries its `idempotency_key` and `request_hash`. Previously a retry
15
+ re-executed after a lost or expired record, or from any new process using the
16
+ engine façade (each process had its own memory store); an append-type patch
17
+ landed twice with two events. Replays rebuilt from the log omit post-commit
18
+ warnings and are otherwise identical.
19
+ - A crash between the VFS write and the event append no longer leaves the vault
20
+ ahead of the log. The kernel records a commit intent (prior content, intended
21
+ digest, pre-assigned `event_id`) and holds a per-path commit lock until the
22
+ append ends. A leftover intent is resolved before the next commit on its path
23
+ and before a kernel's first operation: committed if its event is in the log,
24
+ rolled back otherwise, and reported as a conflict if the document was changed
25
+ outside the kernel. A live writer's intent is never touched. Verified with
26
+ child processes killed by `SIGKILL`.
27
+ - `FileSystemVfs` and `JsonlEventStore` now `fsync` files and directories so
28
+ the intent, the document and the event reach disk in that order.
29
+ - Bundle import keys include the file's content digest, so re-importing a
30
+ changed bundle commits the change instead of failing `idempotency_conflict`.
31
+ - Spec §6.4 no longer claims the reference idempotency stores never expire.
32
+
33
+ ### Added
34
+ - Event stores: `findByIdempotencyKey(workspaceId, key)` (memory and JSONL,
35
+ including rotated archives).
36
+ - `@gregiteen/ssss-cli/intents`: `MemoryIntentJournal`, `FileIntentJournal`,
37
+ `runIntentJournalContract`. `FileSystemVfs#intentJournal()` keeps intents in
38
+ the reserved `.ssss-intents/` directory, so hosts that already pass a
39
+ `FileSystemVfs` get recovery without code changes.
40
+ - `kernel.recover()` and `engine.recover(vaultRoot)`.
41
+ - `ssss adapter conformance` reports intent journal suites.
42
+
43
+ ### Behavior changes hosts should check
44
+ - `FileSystemVfs` reserves `.ssss-intents/` beside `.ssss-locks/`: document
45
+ paths under it are rejected and `list` skips it.
46
+ - A kernel's first operation resolves leftover intents. If recovery itself
47
+ fails, operations return `internal_error` ("Commit recovery failed") rather
48
+ than writing over an unresolved commit. Intents that cannot be resolved are
49
+ moved to `.ssss-intents/unresolved/` and reported by `recover()`.
50
+ - Commits now wait on `fsync`. The conformance suite went from about 9 s to
51
+ 13 s on the Mac mini; per-commit cost depends on the disk.
52
+ - Bundle import keys change format. Keys are opaque per §6.4; a host that
53
+ stored them should not compare old and new keys.
54
+ - Custom event stores without `findByIdempotencyKey` keep the previous
55
+ behavior; §6.4 then requires their idempotency stores to retain records
56
+ indefinitely.
57
+ - Custom VFS adapters without `intentJournal()` get an in-process journal and no
58
+ crash recovery; pass `intentJournal` to `createKernel` to add one.
59
+
60
+ ### Known issue
61
+ - Delivering bundle content that this host never imported, and that is older
62
+ than a local edit, still overwrites the edit. It needs an expected-base
63
+ precondition on import and is tracked in `CAPABILITY_PROVISIONING_CONTRACT`.
64
+
10
65
  ## [0.12.1] - 2026-10-29
11
66
  ### Fixed
12
67
  - Republish under a new version: npm rejected 0.12.0 as a previously staged
package/VERSION CHANGED
@@ -1 +1 @@
1
- 0.12.1
1
+ 0.13.0
@@ -106,7 +106,7 @@
106
106
  "file_count": 14,
107
107
  "provenance": {
108
108
  "content_hash": "sha256:7febd673938a3dc04c068e0bf08097a9783c09e612606d2df9a4db8dcb8907f5",
109
- "exporter": "@gregiteen/ssss-cli@0.12.1"
109
+ "exporter": "@gregiteen/ssss-cli@0.13.0"
110
110
  }
111
111
  },
112
112
  "files": [
package/docs/ssss-spec.md CHANGED
@@ -1022,7 +1022,7 @@ Kernels MUST NOT reorder or skip stages. Adapters (§6.7) implement mechanics on
1022
1022
  | 6 | **Lease** — if the primitive is `lease_required` or the host requires a lease for this path, verify it (§7). | `lease_conflict` |
1023
1023
  | 7 | **Resource prepare** — if the host coordinates an external resource (a domain, a phone number), reserve it. | `internal_error` |
1024
1024
  | 8 | **Dry-run exit** — if `dry_run`, release anything prepared in stage 7 and return success with `committed_at: null`. | — |
1025
- | 9 | **Commit** — write the document with a compare-and-swap on the version read in stage 3 (create-if-absent for a new document), or remove it for `delete`. `event` envelopes write nothing here. | `version_conflict`, `internal_error` |
1025
+ | 9 | **Commit** — under the path's commit intent (below), write the document with a compare-and-swap on the version read in stage 3 (create-if-absent for a new document), or remove it for `delete`. `event` envelopes write nothing here. | `version_conflict`, `internal_error` |
1026
1026
  | 10 | **Event append — the commit point.** Append the canonical event (§8.2). If the append fails, undo stage 9 and release stage 7. | `internal_error` |
1027
1027
  | 11 | **Resource finalize** — complete the external effect. A failure here MUST NOT undo the commit. The host reconciles and the response carries a warning. | warning only |
1028
1028
  | 12 | **Projections** — dispatch the event to projections (§10). A failure is a warning, never a rollback. | warning only |
@@ -1033,6 +1033,30 @@ durably appended. Before that, every failure leaves no trace in the vault. After
1033
1033
  it, nothing may undo the mutation, because the append-only log (§8) already says it
1034
1034
  happened. Work after the commit point is compensated forward, never rolled back.
1035
1035
 
1036
+ **Commit intents.** A process can die between stage 9 and stage 10, leaving the
1037
+ vault ahead of the log. A host whose vault outlives the process MUST therefore,
1038
+ before stage 9, durably record a *commit intent* for the path and hold the path's
1039
+ commit lock until stage 10 ends. The intent names the `event_id` the commit will
1040
+ append, the prior content (or its absence), and a digest of the intended content.
1041
+ Durably means the intent, then the stage-9 write, then the stage-10 append each
1042
+ reach stable storage before the next begins.
1043
+
1044
+ An intent whose lock is free belongs to a writer that died. Before the next commit
1045
+ on that path, and before a kernel's first operation, the host resolves it:
1046
+
1047
+ | The log holds the intent's `event_id` | The document now | Outcome |
1048
+ |---|---|---|
1049
+ | yes | anything | `committed` — the commit stands |
1050
+ | no | has the intended digest (absent, for `delete`) | `rolled_back` — the prior content is restored |
1051
+ | no | has the prior digest (absent, if it had none) | `discarded` — stage 9 never landed |
1052
+ | no | anything else | `conflict` — written outside the kernel; the intent is kept aside and reported |
1053
+
1054
+ An intent whose writer is alive is never resolved. A writer that read uncommitted
1055
+ state fails its stage-9 compare-and-swap after the rollback and retries on the real
1056
+ state, so nothing is ever committed on top of a write that did not commit. The
1057
+ reference `FileSystemVfs` supplies a journal under the reserved `.ssss-intents/`
1058
+ directory; `kernel.recover()` runs resolution on demand.
1059
+
1036
1060
  **Resource lifecycle.** If a host coordinates external resources, every successful
1037
1061
  `prepare` MUST be followed by exactly one `finalize` or `reconcile`. `reconcile`
1038
1062
  receives the phase it runs in (`dry_run`, `commit`, or `finalize`), so the host
@@ -1060,12 +1084,27 @@ Every envelope carries an `idempotency_key`, scoped to its `workspace_id`.
1060
1084
  fails with `idempotency_conflict`.
1061
1085
  - Because the principal is part of the hash, a key replayed by a *different*
1062
1086
  principal is a conflict, never a disclosure of another principal's result.
1063
- - A host MUST retain records for at least 24 hours and MAY retain them
1064
- indefinitely. The reference stores never expire.
1087
+ - **The event log is the durable record.** Every committed envelope's event
1088
+ carries its `idempotency_key` and `request_hash` (§8.2), written at the commit
1089
+ point, so it survives anything the commit survives. The idempotency store is a
1090
+ cache of full responses in front of it. When the store has no record, a kernel
1091
+ MUST look the key up in the log if its event store supports that lookup
1092
+ (§6.7): the first event under the key decides. A matching hash replays a
1093
+ response rebuilt from the event, identical to the original except that
1094
+ post-commit warnings (stages 11–13) are absent; any other hash, including a
1095
+ missing one, is `idempotency_conflict`.
1096
+ - **Retention.** A host whose event store answers key lookups MAY expire store
1097
+ records; the reference `FileIdempotencyStore` keeps them 24 hours by default.
1098
+ A host whose event store cannot answer them MUST retain records indefinitely.
1099
+ - An `unchanged` response (an `operation` identical to the stored document)
1100
+ commits nothing and appends no event, so only the store holds it.
1065
1101
 
1066
1102
  Keys are opaque strings. Use a random UUID for ad-hoc requests. Use a
1067
1103
  deterministic derivation where retries must converge, as with runtime-created work
1068
- (§11.8) and bundle import (§17).
1104
+ (§11.8) and bundle import (§17). A committed key is never reusable for a different
1105
+ request, so a derivation MUST cover everything that distinguishes one intended
1106
+ request from another: the reference bundle import derives its key from the
1107
+ workspace, the target path, and the file's content digest.
1069
1108
 
1070
1109
  ### 6.5 The Response & Error Codes
1071
1110
 
@@ -1164,8 +1203,9 @@ each with a shared contract test in the conformance suite (§12):
1164
1203
 
1165
1204
  | Adapter | Contract |
1166
1205
  |---------|----------|
1167
- | **VFS** | `read(path) → {bytes, version, hash} \| null`; `writeAtomic(path, bytes, {version \| ifAbsent})` and `remove(path, {version})`, both compare-and-swap, failing with a version conflict and never a partial write. Rejects unsafe paths and symlink traversal. |
1168
- | **Event store** | `append(event)` durably and in order, rejecting a duplicate `event_id` across every writer; `replay({cursor, workspaceId})` in append order. |
1206
+ | **VFS** | `read(path) → {bytes, version, hash} \| null`; `writeAtomic(path, bytes, {version \| ifAbsent})` and `remove(path, {version})`, both compare-and-swap, failing with a version conflict and never a partial write. Rejects unsafe paths and symlink traversal. A VFS whose state outlives the process SHOULD provide `intentJournal()`, its co-located intent journal. |
1207
+ | **Event store** | `append(event)` durably and in order, rejecting a duplicate `event_id` across every writer; `replay({cursor, workspaceId})` in append order. SHOULD provide `findByIdempotencyKey(workspaceId, key) → event \| null`, the first event recorded under the key, across every writer and every rotated segment (§6.4). |
1208
+ | **Intent journal** | `run(path, intent, commit, {resolve})` holds the path's commit lock across `commit`, first passing any leftover intent for the path to `resolve`, and removes the intent afterward unless `commit` asks to keep it; `recover(resolve)` resolves every intent whose writer is gone and reports live ones as `in_flight` (§6.3). |
1169
1209
  | **Idempotency store** | `get(workspace, key)`; `put(workspace, key, record)`, which rejects a second `put` for the same key across every writer. |
1170
1210
  | **Lease store** | `acquire`, `verify`, `renew`, `release` per §7, with at most one live lease per target across every writer. |
1171
1211
  | **Authorizer** | `authorize({principal, workspaceId, definition, action}) → {allowed, reason}`, per §6.6. Fails closed. |
@@ -1218,7 +1258,10 @@ flat physical *log*, and a relational *event graph* layered on top of it.
1218
1258
  a kernel SHOULD derive `event_id` deterministically from (`workspace_id`,
1219
1259
  `idempotency_key`). The reference uses UUIDv5 over their JSON array. That way,
1220
1260
  concurrent duplicates of one event collide on the uniqueness check instead of
1221
- both landing.
1261
+ both landing. Other envelopes get their `event_id` before stage 9, so a commit
1262
+ intent (§6.3) can name the event that decides it.
1263
+ - The log is also the durable idempotency record (§6.4): the first event recorded
1264
+ under an `idempotency_key` decides every later request with that key.
1222
1265
  - Order is append order within a log.
1223
1266
  - The event log is the canonical history of *what happened*; the vault is the
1224
1267
  canonical state of *what is true now*. Both are source-of-truth; projections are
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@gregiteen/ssss-cli",
3
- "version": "0.12.1",
3
+ "version": "0.13.0",
4
4
  "description": "CLI tools and the reference engine for the Structured Semantic Syntax System",
5
5
  "main": "src/engine.mjs",
6
6
  "exports": {
@@ -23,6 +23,7 @@
23
23
  "./authorization": "./src/authorization.mjs",
24
24
  "./leases": "./src/leases.mjs",
25
25
  "./idempotency": "./src/idempotency.mjs",
26
+ "./intents": "./src/intents.mjs",
26
27
  "./events": "./src/events.mjs",
27
28
  "./projections": "./src/projections.mjs",
28
29
  "./ui": "./src/ui.mjs",
@@ -56,7 +57,7 @@
56
57
  ],
57
58
  "scripts": {
58
59
  "conformance": "node ./scripts/conformance.mjs conformance",
59
- "test": "node ./scripts/conformance.mjs conformance --engine && node ./scripts/check-events-streaming.mjs && node ./scripts/check-frontmatter.mjs"
60
+ "test": "node ./scripts/conformance.mjs conformance --engine && node ./scripts/check-events-streaming.mjs && node ./scripts/check-frontmatter.mjs && node ./scripts/check-commit-durability.mjs"
60
61
  },
61
62
  "keywords": [
62
63
  "ssss",
@@ -0,0 +1,392 @@
1
+ /**
2
+ * Commit durability checks (spec §6.3, §6.4, §8.1): a committed key is never
3
+ * executed twice, and the vault and event log agree after a crash.
4
+ * Crash cases kill real child processes; nothing here is simulated in-process.
5
+ */
6
+ import fs from 'node:fs';
7
+ import os from 'node:os';
8
+ import path from 'node:path';
9
+ import { spawn, spawnSync } from 'node:child_process';
10
+ import { fileURLToPath } from 'node:url';
11
+ import { createKernel } from '../src/kernel.mjs';
12
+ import { createEngine } from '../src/engine.mjs';
13
+ import { FileSystemVfs, MemoryVfs } from '../src/vfs.mjs';
14
+ import { JsonlEventStore, MemoryEventStore, createCanonicalEvent } from '../src/events.mjs';
15
+ import { FileIdempotencyStore, MemoryIdempotencyStore } from '../src/idempotency.mjs';
16
+ import { contentHash, importBundle, provisionBundle } from '../src/bundle.mjs';
17
+ import { FileIntentJournal, MemoryIntentJournal, runIntentJournalContract } from '../src/intents.mjs';
18
+
19
+ const REGISTRY_DIR = fileURLToPath(new URL('../registry', import.meta.url));
20
+ const SRC = (file) => new URL(`../src/${file}`, import.meta.url).href;
21
+
22
+ const checks = [];
23
+ const check = (name, passed, detail = '') => checks.push({ name, passed: !!passed, detail });
24
+
25
+ const allow = async () => ({ allowed: true });
26
+ const principal = { id: 'durability', kind: 'service' };
27
+ const context = { principal };
28
+ const runDoc = (title) => [
29
+ '---', 'type: run', `title: ${title}`, 'description: Durability check run.', 'run_id: r1', 'workflow_id: w1',
30
+ 'status: running', 'timestamp: 2026-10-31T00:00:00Z', '---', '# Log', '',
31
+ ].join('\n');
32
+ const create = (key, file, title = 'created') => ({ type: 'operation', workspace_id: 'ws', idempotency_key: key, path: file, content: runDoc(title) });
33
+ // `run` is append-only; replace and delete cases use a replaceable `rule`.
34
+ const ruleDoc = (name) => [
35
+ '---', 'type: rule', `name: ${name}`, `title: ${name}`, 'description: Durability check rule.', 'timestamp: 2026-10-31T00:00:00Z', '---', `# ${name}`, '',
36
+ ].join('\n');
37
+ const put = (key, file, name) => ({ type: 'operation', workspace_id: 'ws', idempotency_key: key, path: file, content: ruleDoc(name) });
38
+ const append = (key, file, line) => ({ type: 'patch', workspace_id: 'ws', idempotency_key: key, path: file, patches: { __body__: line } });
39
+ const occurrences = (text, needle) => text.split(needle).length - 1;
40
+
41
+ async function eventsFor(store, key) {
42
+ const found = [];
43
+ for await (const { event } of store.replay({ workspaceId: 'ws' })) if (event.idempotency_key === key) found.push(event);
44
+ return found;
45
+ }
46
+
47
+ async function checkLostRecord() {
48
+ const vfs = new MemoryVfs(), eventStore = new MemoryEventStore(), cache = new MemoryIdempotencyStore();
49
+ let failNextPut = false;
50
+ const flaky = {
51
+ get: (...args) => cache.get(...args),
52
+ put: async (...args) => { if (failNextPut) { failNextPut = false; throw new Error('ENOSPC'); } return cache.put(...args); },
53
+ };
54
+ const kernel = createKernel({ vfs, eventStore, idempotencyStore: flaky, authorize: allow });
55
+ await kernel.execute(create('lost-create', 'runs/lost.md'), context);
56
+ failNextPut = true;
57
+ const first = await kernel.execute(append('lost-1', 'runs/lost.md', 'step one'), context);
58
+ const retry = await kernel.execute(append('lost-1', 'runs/lost.md', 'step one'), context);
59
+ const body = (await vfs.read('runs/lost.md')).bytes.toString();
60
+ check('a retry after a lost idempotency record replays from the event log',
61
+ first.success && retry.replay === true && retry.event_id === first.event_id, JSON.stringify(retry));
62
+ check('a retry after a lost idempotency record applies the append once',
63
+ occurrences(body, 'step one') === 1 && (await eventsFor(eventStore, 'lost-1')).length === 1);
64
+ }
65
+
66
+ async function checkExpiredRecord(root) {
67
+ const vault = path.join(root, 'vault');
68
+ const kernelWith = (cacheDir) => createKernel({
69
+ vfs: new FileSystemVfs(vault), eventStore: new JsonlEventStore(path.join(vault, '.events')),
70
+ idempotencyStore: new FileIdempotencyStore(path.join(root, cacheDir)), authorize: allow,
71
+ });
72
+ const before = kernelWith('cache-a');
73
+ await before.execute(create('exp-create', 'runs/expired.md'), context);
74
+ const first = await before.execute(append('exp-1', 'runs/expired.md', 'step one'), context);
75
+ // A different cache directory is a cache whose entries have all expired.
76
+ const after = kernelWith('cache-b');
77
+ const retry = await after.execute(append('exp-1', 'runs/expired.md', 'step one'), context);
78
+ const conflict = await after.execute(append('exp-1', 'runs/expired.md', 'something else'), context);
79
+ const body = fs.readFileSync(path.join(vault, 'runs/expired.md'), 'utf8');
80
+ check('a retry after the idempotency record expired replays the commit',
81
+ retry.replay === true && retry.event_id === first.event_id && occurrences(body, 'step one') === 1, JSON.stringify(retry));
82
+ check('a replay from the log matches the original response',
83
+ retry.operation_id === first.operation_id && retry.committed_at === first.committed_at
84
+ && retry.validation.type === first.validation.type && retry.type === 'patch' && retry.path === 'runs/expired.md');
85
+ check('a replay from the log backfills the idempotency cache',
86
+ (await new FileIdempotencyStore(path.join(root, 'cache-b')).get('ws', 'exp-1'))?.request_hash === (await new FileIdempotencyStore(path.join(root, 'cache-a')).get('ws', 'exp-1'))?.request_hash);
87
+ check('reusing an expired key for a different request is a conflict', conflict.error?.code === 'idempotency_conflict', JSON.stringify(conflict.error));
88
+ }
89
+
90
+ async function checkReplayAfterLocalEdit() {
91
+ const vfs = new MemoryVfs(), eventStore = new MemoryEventStore();
92
+ const before = createKernel({ vfs, eventStore, authorize: allow });
93
+ await before.execute(put('import-v1', 'rules/imported.md', 'v1'), context);
94
+ await before.execute(put('local-edit', 'rules/imported.md', 'v2 local edit'), context);
95
+ // A new kernel has an empty cache: every record has expired.
96
+ const replay = await createKernel({ vfs, eventStore, authorize: allow }).execute(put('import-v1', 'rules/imported.md', 'v1'), context);
97
+ check('replaying an imported key after expiry does not revert a later edit',
98
+ replay.replay === true && (await vfs.read('rules/imported.md')).bytes.toString() === ruleDoc('v2 local edit'), JSON.stringify(replay));
99
+ }
100
+
101
+ async function checkSeparateEngines(root) {
102
+ const vault = path.join(root, 'engine');
103
+ const envelope = (extra) => ({ workspace_id: 'ws', actor: { role: 'system' }, ...extra });
104
+ await createEngine().processOperation(envelope(create('eng-create', 'runs/engine.md')), vault);
105
+ const first = await createEngine().processOperation(envelope(append('eng-1', 'runs/engine.md', 'step one')), vault);
106
+ const second = await createEngine().processOperation(envelope(append('eng-1', 'runs/engine.md', 'step one')), vault);
107
+ const body = fs.readFileSync(path.join(vault, 'runs/engine.md'), 'utf8');
108
+ check('a second engine instance replays a key the first committed',
109
+ first.success && second.replay === true && occurrences(body, 'step one') === 1, JSON.stringify(second));
110
+ }
111
+
112
+ async function checkJsonlLookup(root) {
113
+ const make = (key, extra = {}) => createCanonicalEvent({
114
+ workspace_id: 'ws', action: 'event', subject: 'lookup', principal, operation_id: `op-${key}`,
115
+ idempotency_key: key, request_hash: `sha256:${key}`, payload: { pad: 'x'.repeat(200) }, ...extra,
116
+ });
117
+ const store = new JsonlEventStore(path.join(root, 'lookup'));
118
+ await store.append(make('first-key'));
119
+ const duplicate = make('first-key', { operation_id: 'op-again' });
120
+ await store.append(duplicate);
121
+ check('event store lookup returns the first event recorded under a key',
122
+ (await store.findByIdempotencyKey('ws', 'first-key'))?.operation_id === 'op-first-key');
123
+ check('event store lookup returns null for an unknown key', (await store.findByIdempotencyKey('ws', 'missing')) === null);
124
+ check('event store lookup is scoped to the workspace', (await store.findByIdempotencyKey('other', 'first-key')) === null);
125
+
126
+ const child = spawnSync(process.execPath, ['--input-type=module', '-e', [
127
+ `import { JsonlEventStore, createCanonicalEvent } from ${JSON.stringify(SRC('events.mjs'))};`,
128
+ `await new JsonlEventStore(${JSON.stringify(path.join(root, 'lookup'))}).append(createCanonicalEvent(${JSON.stringify({
129
+ workspace_id: 'ws', action: 'event', subject: 'lookup', principal, operation_id: 'op-child', idempotency_key: 'child-key', request_hash: 'sha256:child',
130
+ })}));`,
131
+ ].join('\n')], { encoding: 'utf8' });
132
+ check('event store lookup finds a key another process appended',
133
+ child.status === 0 && (await store.findByIdempotencyKey('ws', 'child-key'))?.operation_id === 'op-child', child.stderr);
134
+
135
+ const rotating = new JsonlEventStore(path.join(root, 'rotating'), { maxBytes: 800 });
136
+ for (let index = 0; index < 8; index++) await rotating.append(make(`rot-${index}`));
137
+ const segments = fs.readdirSync(path.join(root, 'rotating')).filter((name) => /^ws\.\d+\.jsonl$/.test(name));
138
+ const fresh = new JsonlEventStore(path.join(root, 'rotating'), { maxBytes: 800 });
139
+ check('event store lookup finds keys in rotated archives',
140
+ segments.length > 1 && (await fresh.findByIdempotencyKey('ws', 'rot-0'))?.operation_id === 'op-rot-0'
141
+ && (await rotating.findByIdempotencyKey('ws', 'rot-7'))?.operation_id === 'op-rot-7', `segments=${segments.length}`);
142
+
143
+ // Another writer's line in flight: complete lines stay findable, and the
144
+ // append path still refuses to write after bytes it cannot parse.
145
+ const live = path.join(root, 'lookup', 'ws.jsonl');
146
+ const pending = JSON.stringify(make('pending-key'));
147
+ fs.appendFileSync(live, pending.slice(0, 40));
148
+ const midWrite = await store.findByIdempotencyKey('ws', 'first-key');
149
+ const appendRefused = await store.append(make('after-partial')).then(() => false, (error) => error instanceof SyntaxError);
150
+ fs.appendFileSync(live, `${pending.slice(40)}\n`);
151
+ check('event store lookup tolerates another writer\'s line in flight',
152
+ midWrite?.operation_id === 'op-first-key' && appendRefused && (await store.findByIdempotencyKey('ws', 'pending-key'))?.operation_id === 'op-pending-key');
153
+
154
+ const memory = new MemoryEventStore();
155
+ await memory.append(make('mem-key'));
156
+ await memory.append(make('mem-key', { operation_id: 'op-mem-again' }));
157
+ check('memory event store lookup returns the first event recorded under a key',
158
+ (await memory.findByIdempotencyKey('ws', 'mem-key'))?.operation_id === 'op-mem-key' && (await memory.findByIdempotencyKey('ws', 'nope')) === null);
159
+ }
160
+
161
+ async function checkBundleReimport(root) {
162
+ const bundle = JSON.parse(fs.readFileSync(new URL('../conformance/reference-bundle.ucw.json', import.meta.url), 'utf8'));
163
+ const options = { workspaceId: 'ws-conf', parameters: { business_name: 'Demo', domain: 'demo.example' } };
164
+ const vault = path.join(root, 'bundle');
165
+ const first = await importBundle(provisionBundle(bundle, options).plan, vault, createEngine({ registryDir: REGISTRY_DIR }));
166
+ const changed = structuredClone(bundle);
167
+ const target = changed.files.find((file) => file.path === 'assistants/concierge.md');
168
+ target.content = `${target.content.replace(/\s+$/, '')}\n\nUpdated in a later bundle version.\n`;
169
+ changed.manifest.provenance.content_hash = contentHash(changed.files);
170
+ if (changed.manifest.dependencies?.integrity) changed.manifest.dependencies.integrity.content_hash = changed.manifest.provenance.content_hash;
171
+ const provisioned = provisionBundle(changed, options);
172
+ const { plan } = provisioned;
173
+ const updated = await importBundle(plan, vault, createEngine({ registryDir: REGISTRY_DIR }));
174
+ const again = await importBundle(plan, vault, createEngine({ registryDir: REGISTRY_DIR }));
175
+ check('re-importing a changed bundle commits the changed file',
176
+ first.ok && provisioned.ok && updated.ok && updated.committed === 1,
177
+ JSON.stringify({ provision: provisioned.errors, committed: updated.committed, failed: updated.results.filter((result) => !result.success).map((result) => result.error) }));
178
+ check('re-importing an identical bundle from a new engine commits nothing', again.ok && again.committed === 0, `committed=${again.committed}`);
179
+ }
180
+
181
+ const fileKernel = (vault) => createKernel({
182
+ vfs: new FileSystemVfs(vault), eventStore: new JsonlEventStore(path.join(vault, '.events')), authorize: allow,
183
+ });
184
+ const read = (vault, file) => (fs.existsSync(path.join(vault, file)) ? fs.readFileSync(path.join(vault, file), 'utf8') : null);
185
+ const intentFiles = (vault) => {
186
+ const dir = path.join(vault, '.ssss-intents');
187
+ return fs.existsSync(dir) ? fs.readdirSync(dir).filter((name) => name.endsWith('.json')) : [];
188
+ };
189
+
190
+ // A child process that runs one envelope through a filesystem kernel and dies
191
+ // with SIGKILL at a chosen point: `before` the event append, `after` it, or
192
+ // `hang` inside it (printing READY) until the parent kills it.
193
+ function childScript(vault, envelope, mode) {
194
+ return [
195
+ `import { createKernel } from ${JSON.stringify(SRC('kernel.mjs'))};`,
196
+ `import { FileSystemVfs } from ${JSON.stringify(SRC('vfs.mjs'))};`,
197
+ `import { JsonlEventStore } from ${JSON.stringify(SRC('events.mjs'))};`,
198
+ `const real = new JsonlEventStore(${JSON.stringify(path.join(vault, '.events'))});`,
199
+ `const mode = ${JSON.stringify(mode)};`,
200
+ 'const eventStore = {',
201
+ ' findByIdempotencyKey: (...args) => real.findByIdempotencyKey(...args),',
202
+ ' replay: (options) => real.replay(options),',
203
+ ' async append(event) {',
204
+ " if (mode === 'before') process.kill(process.pid, 'SIGKILL');",
205
+ " if (mode === 'after') { await real.append(event); process.kill(process.pid, 'SIGKILL'); }",
206
+ " if (mode === 'hang') { console.log('READY'); await new Promise(() => {}); }",
207
+ ' return real.append(event);',
208
+ ' },',
209
+ '};',
210
+ `const kernel = createKernel({ vfs: new FileSystemVfs(${JSON.stringify(vault)}), eventStore, authorize: async () => ({ allowed: true }) });`,
211
+ `await kernel.execute(${JSON.stringify(envelope)}, { principal: ${JSON.stringify(principal)} });`,
212
+ "console.log('SURVIVED');",
213
+ ].join('\n');
214
+ }
215
+
216
+ function crashIn(vault, envelope, mode) {
217
+ return spawnSync(process.execPath, ['--input-type=module', '-e', childScript(vault, envelope, mode)], { encoding: 'utf8' });
218
+ }
219
+
220
+ function hangIn(vault, envelope) {
221
+ const child = spawn(process.execPath, ['--input-type=module', '-e', childScript(vault, envelope, 'hang')]);
222
+ const ready = new Promise((resolve, reject) => {
223
+ let out = '';
224
+ child.stdout.on('data', (chunk) => { out += chunk; if (out.includes('READY')) resolve(); });
225
+ child.on('exit', () => reject(new Error(`child exited before READY: ${out}`)));
226
+ });
227
+ const exited = new Promise((resolve) => child.on('exit', (code, signal) => resolve(signal)));
228
+ return { child, ready, exited };
229
+ }
230
+
231
+ async function checkCrashRecovery(root) {
232
+ const vault = path.join(root, 'crash');
233
+ const setup = fileKernel(vault);
234
+ for (const file of ['rules/replace.md', 'rules/delete.md', 'rules/after.md']) {
235
+ const setupResult = await setup.execute(put(`setup-${file}`, file, 'original'), context);
236
+ if (!setupResult.success) throw new Error(`setup failed: ${JSON.stringify(setupResult.error)}`);
237
+ }
238
+ await setup.execute(create('setup-patch', 'runs/patch.md', 'original'), context);
239
+
240
+ const cases = [
241
+ { name: 'create', file: 'rules/created.md', envelope: put('crash-create', 'rules/created.md', 'never committed'), expected: null },
242
+ { name: 'replace', file: 'rules/replace.md', envelope: put('crash-replace', 'rules/replace.md', 'never committed'), expected: read(vault, 'rules/replace.md') },
243
+ { name: 'patch', file: 'runs/patch.md', envelope: append('crash-patch', 'runs/patch.md', 'never committed'), expected: read(vault, 'runs/patch.md') },
244
+ { name: 'delete', file: 'rules/delete.md', envelope: { type: 'delete', workspace_id: 'ws', idempotency_key: 'crash-delete', path: 'rules/delete.md' }, expected: read(vault, 'rules/delete.md') },
245
+ ];
246
+ // One case at a time: a crashing child's own first operation would
247
+ // otherwise recover the previous child's crash before the parent looks.
248
+ const events = new JsonlEventStore(path.join(vault, '.events'));
249
+ for (const item of cases) {
250
+ const result = crashIn(vault, item.envelope, 'before');
251
+ item.killed = result.signal === 'SIGKILL';
252
+ item.orphaned = read(vault, item.file) !== item.expected;
253
+ const recovered = await fileKernel(vault).recover();
254
+ item.outcome = recovered.find((entry) => entry.path === item.file)?.outcome;
255
+ item.restored = read(vault, item.file) === item.expected;
256
+ item.noEvent = !(await events.findByIdempotencyKey('ws', item.envelope.idempotency_key));
257
+ }
258
+ check('a SIGKILL between the VFS write and the event append leaves uncommitted state (reproduced)',
259
+ cases.every((item) => item.killed && item.orphaned), JSON.stringify(cases.map(({ name, killed, orphaned }) => ({ name, killed, orphaned }))));
260
+ for (const item of cases) {
261
+ check(`recovery rolls back a crashed ${item.name}`, item.outcome === 'rolled_back' && item.restored && item.noEvent,
262
+ JSON.stringify({ outcome: item.outcome, restored: item.restored, noEvent: item.noEvent }));
263
+ }
264
+ check('recovery leaves no intent behind', intentFiles(vault).length === 0, JSON.stringify(intentFiles(vault)));
265
+
266
+ const committedLate = crashIn(vault, put('crash-after', 'rules/after.md', 'committed before the crash'), 'after');
267
+ const afterRecovery = await fileKernel(vault).recover();
268
+ const retried = await fileKernel(vault).execute(put('crash-after', 'rules/after.md', 'committed before the crash'), context);
269
+ check('a crash after the event append keeps the commit',
270
+ committedLate.signal === 'SIGKILL' && afterRecovery.find((entry) => entry.path === 'rules/after.md')?.outcome === 'committed'
271
+ && read(vault, 'rules/after.md').includes('committed before the crash') && retried.replay === true,
272
+ JSON.stringify({ afterRecovery, retried: retried.error || retried.replay }));
273
+ }
274
+
275
+ async function checkRecoveryTriggers(root) {
276
+ // A kernel's first operation recovers the whole vault, even other paths.
277
+ const vault = path.join(root, 'first-op');
278
+ await fileKernel(vault).execute(put('first-setup', 'rules/one.md', 'original'), context);
279
+ const original = read(vault, 'rules/one.md');
280
+ crashIn(vault, put('first-crash', 'rules/one.md', 'never committed'), 'before');
281
+ const orphaned = read(vault, 'rules/one.md') !== original;
282
+ await fileKernel(vault).execute(put('first-other', 'rules/two.md', 'unrelated'), context);
283
+ check('a kernel\'s first operation rolls back a crashed write on another path', orphaned && read(vault, 'rules/one.md') === original);
284
+
285
+ // A kernel already past its first operation resolves the leftover when it
286
+ // next commits on that path. It read the uncommitted state, so its first
287
+ // attempt loses the compare-and-swap; the retry commits on the real base.
288
+ const busy = path.join(root, 'next-commit');
289
+ const kernel = fileKernel(busy);
290
+ await kernel.execute(create('busy-setup', 'runs/log.md', 'original'), context);
291
+ crashIn(busy, append('busy-crash', 'runs/log.md', 'never committed'), 'before');
292
+ const stale = await kernel.execute(append('busy-1', 'runs/log.md', 'real step'), context);
293
+ const retry = await kernel.execute(append('busy-1', 'runs/log.md', 'real step'), context);
294
+ const body = read(busy, 'runs/log.md');
295
+ check('the next commit on a crashed path rolls the leftover back first',
296
+ stale.error?.code === 'version_conflict' && retry.success && !retry.replay && !body.includes('never committed') && occurrences(body, 'real step') === 1,
297
+ JSON.stringify({ stale: stale.error, retry: retry.error || retry.success }));
298
+
299
+ // A live writer's intent is never touched; once it is killed, it is.
300
+ const live = path.join(root, 'live');
301
+ await fileKernel(live).execute(put('live-setup', 'rules/live.md', 'original'), context);
302
+ const before = read(live, 'rules/live.md');
303
+ const holder = hangIn(live, put('live-hang', 'rules/live.md', 'in flight'));
304
+ await holder.ready;
305
+ const whileAlive = await fileKernel(live).recover();
306
+ const untouched = read(live, 'rules/live.md').includes('in flight');
307
+ holder.child.kill('SIGKILL');
308
+ const signal = await holder.exited;
309
+ const afterKill = await fileKernel(live).recover();
310
+ check('recovery skips an intent whose writer is alive',
311
+ whileAlive.length === 1 && whileAlive[0].outcome === 'in_flight' && untouched, JSON.stringify(whileAlive));
312
+ check('recovery rolls back the same intent once its writer is killed',
313
+ signal === 'SIGKILL' && afterKill[0]?.outcome === 'rolled_back' && read(live, 'rules/live.md') === before, JSON.stringify(afterKill));
314
+
315
+ // A document changed outside the kernel after a crash is reported, not overwritten.
316
+ const external = path.join(root, 'external');
317
+ await fileKernel(external).execute(put('ext-setup', 'rules/ext.md', 'original'), context);
318
+ crashIn(external, put('ext-crash', 'rules/ext.md', 'never committed'), 'before');
319
+ fs.writeFileSync(path.join(external, 'rules/ext.md'), ruleDoc('edited by hand'));
320
+ const conflict = await fileKernel(external).recover();
321
+ const quarantined = conflict[0]?.quarantined && fs.existsSync(path.join(external, '.ssss-intents', conflict[0].quarantined));
322
+ const keptExternal = read(external, 'rules/ext.md') === ruleDoc('edited by hand');
323
+ const writable = await fileKernel(external).execute(put('ext-after', 'rules/ext.md', 'later write'), context);
324
+ check('recovery reports an external write as a conflict and keeps the evidence',
325
+ conflict[0]?.outcome === 'conflict' && quarantined && keptExternal && intentFiles(external).length === 0,
326
+ JSON.stringify(conflict));
327
+ check('a path whose intent was quarantined stays writable', writable.success, JSON.stringify(writable.error));
328
+
329
+ const engineVault = path.join(root, 'engine-recover');
330
+ await createEngine().processOperation({ ...put('eng-setup', 'rules/e.md', 'original'), actor: { role: 'system' } }, engineVault);
331
+ const engineOriginal = read(engineVault, 'rules/e.md');
332
+ crashIn(engineVault, put('eng-crash', 'rules/e.md', 'never committed'), 'before');
333
+ const engineRecovered = await createEngine().recover(engineVault);
334
+ check('the engine façade recovers a crashed write', engineRecovered[0]?.outcome === 'rolled_back' && read(engineVault, 'rules/e.md') === engineOriginal, JSON.stringify(engineRecovered));
335
+ }
336
+
337
+ async function checkFlushOrder(root) {
338
+ const vault = path.join(root, 'flush');
339
+ const kernel = fileKernel(vault);
340
+ await kernel.execute(create('flush-setup', 'runs/setup.md'), context);
341
+ const original = { openSync: fs.openSync, closeSync: fs.closeSync, fsyncSync: fs.fsyncSync };
342
+ const paths = new Map(), flushed = [];
343
+ fs.openSync = (target, ...rest) => { const fd = original.openSync(target, ...rest); paths.set(fd, String(target)); return fd; };
344
+ fs.closeSync = (fd) => { paths.delete(fd); return original.closeSync(fd); };
345
+ fs.fsyncSync = (fd) => { flushed.push(paths.get(fd) || `fd:${fd}`); return original.fsyncSync(fd); };
346
+ let result;
347
+ try { result = await kernel.execute(create('flush-1', 'runs/flushed.md'), context); }
348
+ finally { Object.assign(fs, original); }
349
+ const first = (test) => flushed.findIndex(test);
350
+ const intentFile = first((file) => file.includes('.ssss-intents') && file.endsWith('.tmp'));
351
+ const intentDir = first((file) => file.endsWith('.ssss-intents'));
352
+ const documentFile = first((file) => file.includes(`${path.sep}runs${path.sep}.flushed.md.`));
353
+ const documentDir = first((file) => file.endsWith(`${path.sep}runs`));
354
+ const eventLog = first((file) => file.endsWith('ws.jsonl'));
355
+ check('a filesystem commit flushes the intent, then the document, then the event',
356
+ result.success && intentFile >= 0 && intentFile < intentDir && intentDir < documentFile && documentFile < documentDir && documentDir < eventLog,
357
+ JSON.stringify(flushed.map((file) => path.relative(vault, file))));
358
+ }
359
+
360
+ // A section that throws is a failed check, so one crash cannot hide the rest.
361
+ async function section(name, fn) {
362
+ try { await fn(); } catch (error) { check(`${name} completes`, false, error.stack || error.message); }
363
+ }
364
+
365
+ const root = fs.mkdtempSync(path.join(os.tmpdir(), 'ssss-durability-'));
366
+ try {
367
+ await section('lost record', () => checkLostRecord());
368
+ await section('expired record', () => checkExpiredRecord(path.join(root, 'expired')));
369
+ await section('replay after a local edit', () => checkReplayAfterLocalEdit());
370
+ await section('separate engines', () => checkSeparateEngines(root));
371
+ await section('event store lookup', () => checkJsonlLookup(root));
372
+ await section('bundle re-import', () => checkBundleReimport(root));
373
+ await section('crash recovery', () => checkCrashRecovery(root));
374
+ await section('recovery triggers', () => checkRecoveryTriggers(root));
375
+ await section('flush order', () => checkFlushOrder(root));
376
+ await section('intent journal contracts', async () => {
377
+ for (const [name, journal] of [['memory', new MemoryIntentJournal()], ['filesystem', new FileIntentJournal(path.join(root, 'journal'))]]) {
378
+ const contract = await runIntentJournalContract(journal);
379
+ check(`${name} intent journal passes the shared contract`, contract.passed, JSON.stringify(contract.checks.filter((item) => !item.passed)));
380
+ }
381
+ });
382
+ } finally {
383
+ fs.rmSync(root, { recursive: true, force: true });
384
+ }
385
+
386
+ let passed = 0;
387
+ for (const item of checks) {
388
+ if (item.passed) { passed++; console.log(` ✅ ${item.name}`); }
389
+ else console.log(` ❌ ${item.name}${item.detail ? ` — ${item.detail}` : ''}`);
390
+ }
391
+ console.log(`\n ${passed}/${checks.length} commit durability checks passed (§6.3/§6.4/§8.1)`);
392
+ if (passed !== checks.length) process.exitCode = 1;
@@ -5,6 +5,7 @@ import path from 'node:path';
5
5
  import { MemoryVfs, FileSystemVfs, runVfsContract } from '../src/vfs.mjs';
6
6
  import { MemoryLeaseStore, FileLeaseStore, runLeaseContract } from '../src/leases.mjs';
7
7
  import { MemoryIdempotencyStore, FileIdempotencyStore, runIdempotencyContract } from '../src/idempotency.mjs';
8
+ import { MemoryIntentJournal, FileIntentJournal, runIntentJournalContract } from '../src/intents.mjs';
8
9
  import { parseArgs, die, wantsHelp } from './lib/cli.mjs';
9
10
 
10
11
  export async function run(argv) {
@@ -20,6 +21,8 @@ export async function run(argv) {
20
21
  lease_filesystem: await runLeaseContract(new FileLeaseStore(path.join(root, 'leases'))),
21
22
  idempotency_memory: await runIdempotencyContract(new MemoryIdempotencyStore()),
22
23
  idempotency_filesystem: await runIdempotencyContract(new FileIdempotencyStore(path.join(root, 'idempotency'))),
24
+ intent_journal_memory: await runIntentJournalContract(new MemoryIntentJournal()),
25
+ intent_journal_filesystem: await runIntentJournalContract(new FileIntentJournal(path.join(root, 'intents'))),
23
26
  };
24
27
  const report = { version: 1, passed: Object.values(suites).every((suite) => suite.passed), suites };
25
28
  const text = `${JSON.stringify(report, null, 2)}\n`;