swarm-deploy 0.3.1 → 0.4.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +65 -0
- package/README.md +89 -1257
- package/RELEASING.md +15 -8
- package/dist/cli.d.ts +7 -0
- package/dist/cli.js +127 -2
- package/dist/cli.js.map +1 -1
- package/dist/hooks-module.d.ts +3 -2
- package/dist/hooks-module.js +4 -4
- package/dist/hooks-module.js.map +1 -1
- package/dist/hooks.d.ts +75 -2
- package/dist/hooks.js +57 -3
- package/dist/hooks.js.map +1 -1
- package/dist/index.d.ts +1 -1
- package/dist/index.js.map +1 -1
- package/dist/server.d.ts +58 -8
- package/dist/server.js +287 -57
- package/dist/server.js.map +1 -1
- package/dist/storage/layout.d.ts +29 -0
- package/dist/storage/layout.js +53 -4
- package/dist/storage/layout.js.map +1 -1
- package/dist/storage/link-store.d.ts +48 -7
- package/dist/storage/link-store.js +119 -11
- package/dist/storage/link-store.js.map +1 -1
- package/dist/storage/reset.d.ts +95 -0
- package/dist/storage/reset.js +667 -0
- package/dist/storage/reset.js.map +1 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -21,6 +21,37 @@ accepts only client public keys in its immutable startup allowlist.
|
|
|
21
21
|
The receiver is upload-only. It does not execute, serve, or provide a download
|
|
22
22
|
protocol for stored artifacts.
|
|
23
23
|
|
|
24
|
+
## Features
|
|
25
|
+
|
|
26
|
+
- **Direct and authenticated:** a fresh direct HyperDHT connection per file, with
|
|
27
|
+
Noise authentication and encryption, a pinned server key, and a static client
|
|
28
|
+
allowlist. There is no topic or swarm discovery.
|
|
29
|
+
- **Resumable:** incomplete uploads keep durable TAR progress and resume after
|
|
30
|
+
the client verifies the stored prefix.
|
|
31
|
+
- **Verified:** files and directories are sent as deterministic TAR and checked
|
|
32
|
+
with SHA-256 before an explicit terminal `COMMITTED` result.
|
|
33
|
+
- **Safe storage:** names are create-only by default, named files can opt in to
|
|
34
|
+
replacement with history, and unmanaged files are never overwritten or deleted.
|
|
35
|
+
- **Retention and rotation:** age and storage-quota limits, plus count and
|
|
36
|
+
SemVer rotation driven by artifact patterns.
|
|
37
|
+
- **Managed symlinks:** automatic newest-match aliases and explicit client
|
|
38
|
+
promotions such as `latest`.
|
|
39
|
+
- **Trusted hooks:** server-side `beforeCommit`, `afterCommit`, `afterLink`, and
|
|
40
|
+
`onFailure` callbacks.
|
|
41
|
+
- **CLI and API:** the `swarm-deploy` command and a strict-TypeScript API on
|
|
42
|
+
Node.js and Bare.
|
|
43
|
+
|
|
44
|
+
## Contents
|
|
45
|
+
|
|
46
|
+
- [Requirements](#requirements)
|
|
47
|
+
- [Install](#install)
|
|
48
|
+
- [Security and identity model](#security-and-identity-model)
|
|
49
|
+
- [Provision identities](#provision-identities)
|
|
50
|
+
- [Quick start](#quick-start)
|
|
51
|
+
- [Documentation](#documentation)
|
|
52
|
+
- [Examples](#examples)
|
|
53
|
+
- [Project documents](#project-documents)
|
|
54
|
+
|
|
24
55
|
## Requirements
|
|
25
56
|
|
|
26
57
|
- Node.js 22 or 24, or the current stable Bare runtime.
|
|
@@ -68,6 +99,16 @@ Keep seeds stable to preserve identity. Copying one client seed to several
|
|
|
68
99
|
machines intentionally gives all of them the same uploader identity; use
|
|
69
100
|
separate client seeds when independent authorization is required.
|
|
70
101
|
|
|
102
|
+
Operator essentials:
|
|
103
|
+
|
|
104
|
+
- Run the server under a dedicated non-root account with a dedicated storage
|
|
105
|
+
root, and restrict the seed and storage root to that account.
|
|
106
|
+
- Protect the startup allowlist as security-sensitive configuration.
|
|
107
|
+
- Never edit `.swarm-deploy/` while the server is running, and stop the server
|
|
108
|
+
cleanly before backing up or restoring the complete storage root.
|
|
109
|
+
- Report vulnerabilities privately as described in
|
|
110
|
+
[SECURITY.md](https://github.com/tetherto/swarm-deploy/blob/main/SECURITY.md).
|
|
111
|
+
|
|
71
112
|
## Provision identities
|
|
72
113
|
|
|
73
114
|
Generate separate server and client seed files:
|
|
@@ -123,1260 +164,51 @@ artifact.bin COMMITTED
|
|
|
123
164
|
Uploading the same managed content again returns `ALREADY_COMMITTED` without
|
|
124
165
|
retransmitting its TAR bytes.
|
|
125
166
|
|
|
126
|
-
##
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
140
|
-
|
|
141
|
-
|
|
142
|
-
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
-
|
|
174
|
-
- `--allow-key`: authorized client public key. Repeat for multiple identities;
|
|
175
|
-
duplicates and malformed keys are rejected.
|
|
176
|
-
- `--max-file-bytes`: maximum extracted artifact size.
|
|
177
|
-
- `--max-staging-bytes`: aggregate persistent staging reservation. An admitted
|
|
178
|
-
transfer reserves its deterministic TAR size plus extracted file size, so
|
|
179
|
-
this commonly needs to be at least twice the largest simultaneously staged
|
|
180
|
-
payload.
|
|
181
|
-
|
|
182
|
-
Optional options:
|
|
183
|
-
|
|
184
|
-
- `--max-storage-bytes`: maximum total managed committed storage. Oldest
|
|
185
|
-
eligible artifacts are removed first.
|
|
186
|
-
- `--max-age-days`: remove eligible committed artifacts at or beyond this age.
|
|
187
|
-
- `--replace-name`: permit replacement of this exact basename. Repeat for
|
|
188
|
-
multiple mutable names.
|
|
189
|
-
- `--artifact-pattern <template>`: identify releases and make matching
|
|
190
|
-
mandatory. Repeat for multiple templates; declaration order is match order.
|
|
191
|
-
See [Artifact patterns and rotation](#artifact-patterns-and-rotation).
|
|
192
|
-
- `--max-count <count>`: keep the newest `<count>` releases per series. Requires
|
|
193
|
-
at least one `--artifact-pattern`.
|
|
194
|
-
- `--max-versions <count>`: keep every release in the newest `<count>` distinct
|
|
195
|
-
version groups per series. Requires a pattern containing `{version}` and
|
|
196
|
-
`--version-granularity`.
|
|
197
|
-
- `--version-granularity <major|minor>`: how versions are grouped for
|
|
198
|
-
`--max-versions`. It is rejected without `--max-versions`, and there is no
|
|
199
|
-
default.
|
|
200
|
-
- `--symlink <selector> [<link-name>]`: repeatable managed-symlink policy.
|
|
201
|
-
With two arguments, the server automatically points `<link-name>` at the
|
|
202
|
-
newest matching managed artifact. With one argument, the rule creates no
|
|
203
|
-
link by itself; it authorizes authenticated clients to choose any matching
|
|
204
|
-
current managed artifact as a manual-link target. A selector that begins and
|
|
205
|
-
ends with `/` is an unflagged regular expression matched against managed
|
|
206
|
-
artifact basenames; anything else is an exact managed basename.
|
|
207
|
-
- `--hooks <module>`: JavaScript module (`.js`, `.mjs`, or `.cjs`) exporting
|
|
208
|
-
trusted lifecycle callbacks. The path is resolved against the working
|
|
209
|
-
directory. See [Deployment hooks](#deployment-hooks).
|
|
210
|
-
|
|
211
|
-
All counts are positive safe integers. Invalid combinations, malformed or
|
|
212
|
-
duplicate templates, and unloadable hook modules are configuration errors that
|
|
213
|
-
exit `2` before the server listens. Each hooks-module error names only the
|
|
214
|
-
module basename with a fixed reason, never module content.
|
|
215
|
-
|
|
216
|
-
The CLI requires at least one `--allow-key`. Its snapshot is immutable for the
|
|
217
|
-
life of the process.
|
|
218
|
-
|
|
219
|
-
### `upload`
|
|
220
|
-
|
|
221
|
-
```sh
|
|
222
|
-
swarm-deploy upload \
|
|
223
|
-
--seed-file <seed-file> \
|
|
224
|
-
--server-key <64-lower-hex> \
|
|
225
|
-
[--idle-timeout <milliseconds>] \
|
|
226
|
-
[--no-source-parent] \
|
|
227
|
-
<file-or-directory>
|
|
228
|
-
```
|
|
229
|
-
|
|
230
|
-
Replace `--seed-file <seed-file>` with `--seed <64-lower-hex>` to provide the
|
|
231
|
-
seed inline.
|
|
232
|
-
|
|
233
|
-
- `--server-key` is the full pinned server public key.
|
|
234
|
-
- `--idle-timeout` defaults to 60 seconds and bounds inactive protocol reads and
|
|
235
|
-
backpressured writes.
|
|
236
|
-
- The input may be any regular binary file; Swarm Deploy creates the canonical
|
|
237
|
-
one-entry USTAR stream automatically. Pre-tarring is not required.
|
|
238
|
-
- A direct file retains its basename.
|
|
239
|
-
- Each upload also carries the immediate local parent directory name as
|
|
240
|
-
optional `sourceParent` metadata, but only when that name is a safe single
|
|
241
|
-
component (see [Source parent](#source-parent)).
|
|
242
|
-
- `--no-source-parent` takes no value and suppresses that metadata for every
|
|
243
|
-
file in the run. The upload then keeps the legacy transfer identity and
|
|
244
|
-
cannot match a server pattern with a parent segment.
|
|
245
|
-
- A directory input commits exactly one recursive managed directory artifact
|
|
246
|
-
named after the input basename. Every member must be a safe regular file or
|
|
247
|
-
directory; symlinks, hard links, devices, sockets, FIFOs, unsafe names,
|
|
248
|
-
over-depth, over-count, or over-length members reject the whole upload.
|
|
249
|
-
Nothing is silently skipped.
|
|
250
|
-
- On success the CLI prints one line: `<name> <kind> <status>` (for example
|
|
251
|
-
`0.18.1 directory COMMITTED`).
|
|
252
|
-
|
|
253
|
-
Accepted names start with an ASCII letter or digit, contain only letters,
|
|
254
|
-
digits, `.`, `_`, and `-`, and occupy at most 100 UTF-8 bytes. `history-` is
|
|
255
|
-
reserved for server-managed replacement history.
|
|
256
|
-
|
|
257
|
-
If a basename is exactly 64 lowercase hexadecimal characters, pass it with a
|
|
258
|
-
directory component such as `./<name>` so the CLI does not treat it as an
|
|
259
|
-
accidentally pasted seed.
|
|
260
|
-
|
|
261
|
-
### `link`
|
|
262
|
-
|
|
263
|
-
```sh
|
|
264
|
-
swarm-deploy link \
|
|
265
|
-
--seed-file <seed-file> \
|
|
266
|
-
--server-key <64-lower-hex> \
|
|
267
|
-
[--idle-timeout <milliseconds>] \
|
|
268
|
-
<target> <link-name>
|
|
269
|
-
```
|
|
270
|
-
|
|
271
|
-
Replace `--seed-file <seed-file>` with `--seed <64-lower-hex>`, or use
|
|
272
|
-
`SWARM_DEPLOY_CLIENT_SEED`, exactly as for `upload`. `--server-key` pins the
|
|
273
|
-
server identity, and `--idle-timeout` has the same 60-second default and
|
|
274
|
-
inactivity semantics as upload.
|
|
275
|
-
|
|
276
|
-
The command asks the authenticated server to point `<link-name>` at the
|
|
277
|
-
existing managed file or directory `<target>`. The target must match a
|
|
278
|
-
one-argument `--symlink` rule. Success prints exactly:
|
|
279
|
-
|
|
280
|
-
```text
|
|
281
|
-
<link-name> -> <target> <LINKED|UNCHANGED>
|
|
282
|
-
```
|
|
283
|
-
|
|
284
|
-
`LINKED` means the link was created or repointed. `UNCHANGED` means the same
|
|
285
|
-
managed link already points at that target; the operation is idempotent.
|
|
286
|
-
|
|
287
|
-
### Seed sources
|
|
288
|
-
|
|
289
|
-
Server, upload, and link commands accept exactly one of:
|
|
290
|
-
|
|
291
|
-
- `--seed-file <seed-file>`
|
|
292
|
-
- `--seed <64-lower-hex>`
|
|
293
|
-
- the role-specific `SWARM_DEPLOY_SERVER_SEED` or
|
|
294
|
-
`SWARM_DEPLOY_CLIENT_SEED` environment variable
|
|
295
|
-
|
|
296
|
-
String values must contain exactly 64 lowercase hexadecimal characters.
|
|
297
|
-
Combining seed sources is an error.
|
|
298
|
-
|
|
299
|
-
The `public-key` command accepts `--seed-file` or `--seed`; it does not consume a
|
|
300
|
-
role-specific environment variable.
|
|
301
|
-
|
|
302
|
-
Prefer seed files or protected environment variables in production. Use
|
|
303
|
-
`--seed` only when exposure through command history, process inspection, and
|
|
304
|
-
tooling logs is acceptable.
|
|
305
|
-
|
|
306
|
-
### CLI exit codes
|
|
307
|
-
|
|
308
|
-
- `0`: the upload committed or was already committed, or the link is linked or
|
|
309
|
-
unchanged.
|
|
310
|
-
- `1`: upload, network, protocol, storage, cleanup, or runtime failure.
|
|
311
|
-
- `2`: usage or configuration error.
|
|
312
|
-
|
|
313
|
-
## CI uploader example
|
|
314
|
-
|
|
315
|
-
Store the client seed as a protected CI secret and the server public key as a
|
|
316
|
-
nonsecret variable:
|
|
317
|
-
|
|
318
|
-
```yaml
|
|
319
|
-
- name: Install uploader
|
|
320
|
-
run: npm install --global swarm-deploy
|
|
321
|
-
|
|
322
|
-
- name: Upload artifact
|
|
323
|
-
env:
|
|
324
|
-
SWARM_DEPLOY_CLIENT_SEED: ${{ secrets.SWARM_DEPLOY_CLIENT_SEED }}
|
|
325
|
-
SWARM_DEPLOY_SERVER_KEY: ${{ vars.SWARM_DEPLOY_SERVER_KEY }}
|
|
326
|
-
run: |
|
|
327
|
-
swarm-deploy upload \
|
|
328
|
-
--server-key "$SWARM_DEPLOY_SERVER_KEY" \
|
|
329
|
-
--idle-timeout 60000 \
|
|
330
|
-
./dist/artifact-linux-x64.tar.gz
|
|
331
|
-
```
|
|
332
|
-
|
|
333
|
-
Do not enable shell tracing around commands that read seed environment
|
|
334
|
-
variables or pass `--seed`.
|
|
335
|
-
|
|
336
|
-
## Transfer and resume behavior
|
|
337
|
-
|
|
338
|
-
Each file follows this lifecycle:
|
|
339
|
-
|
|
340
|
-
1. The client connects directly to the pinned server public key with its seeded
|
|
341
|
-
HyperDHT identity.
|
|
342
|
-
2. The server firewall and connection handler verify the authenticated client
|
|
343
|
-
key against the startup allowlist.
|
|
344
|
-
3. The client sends bounded metadata containing the name, file size and digest,
|
|
345
|
-
deterministic TAR size and digest, transfer ID, and optional `sourceParent`.
|
|
346
|
-
4. The server responds with `ACCEPT`, `RESUME`, `VERIFIED`,
|
|
347
|
-
`ALREADY_COMMITTED`, or a stable rejection.
|
|
348
|
-
5. The client sends exactly the required deterministic one-entry USTAR bytes.
|
|
349
|
-
6. The server validates the canonical archive, extracted size, and both
|
|
350
|
-
SHA-256 values before durable commit.
|
|
351
|
-
7. Success requires an explicit terminal `COMMITTED` result. EOF or socket
|
|
352
|
-
closure is never success.
|
|
353
|
-
|
|
354
|
-
Incomplete uploads retain only durable TAR progress. On reconnect, the server
|
|
355
|
-
returns a TAR-byte offset and SHA-256 of that prefix. The client regenerates and
|
|
356
|
-
compares the prefix before sending the suffix. A mismatch closes that connection
|
|
357
|
-
and retries once from zero with explicit reset intent.
|
|
358
|
-
|
|
359
|
-
The server coalesces network fragments into 1 MiB durability batches plus the
|
|
360
|
-
final remainder. A disconnect can require retransmitting only the uncheckpointed
|
|
361
|
-
in-memory tail; it never advertises bytes that were not durably published.
|
|
362
|
-
|
|
363
|
-
Inactive sessions expire after seven days by default. Active receives and
|
|
364
|
-
verification are protected from expiry.
|
|
365
|
-
|
|
366
|
-
## Storage, replacement, and retention
|
|
367
|
-
|
|
368
|
-
The storage root contains visible current and historical artifacts plus the
|
|
369
|
-
reserved `.swarm-deploy/` internal directory. Do not modify that directory
|
|
370
|
-
while the server is running.
|
|
371
|
-
|
|
372
|
-
Names are create-only by default:
|
|
373
|
-
|
|
374
|
-
- New content for an unused name is committed atomically.
|
|
375
|
-
- Identical managed content returns `ALREADY_COMMITTED`.
|
|
376
|
-
- Different content for an occupied create-only name returns `FILE_EXISTS`.
|
|
377
|
-
- Unmanaged files and paths are never overwritten or deleted.
|
|
378
|
-
|
|
379
|
-
Names configured with `--replace-name` or `ServerOptions.replaceNames` are
|
|
380
|
-
mutable:
|
|
381
|
-
|
|
382
|
-
- Different verified content atomically becomes current.
|
|
383
|
-
- The prior managed inode and record are preserved as
|
|
384
|
-
`history-<full-old-transfer-id>`.
|
|
385
|
-
- The current mutable name is pinned against age and quota retention.
|
|
386
|
-
- Historical versions remain eligible for retention.
|
|
387
|
-
|
|
388
|
-
Commit journals and inode checks recover interrupted create and replacement
|
|
389
|
-
operations. Recovery rolls back mutations before the durable new current
|
|
390
|
-
sidecar and rolls forward operations after that linearization point.
|
|
391
|
-
|
|
392
|
-
Optional retention applies to managed artifacts only:
|
|
393
|
-
|
|
394
|
-
- `maxAge`/`--max-age-days` removes eligible artifacts by age.
|
|
395
|
-
- `maxStorageBytes`/`--max-storage-bytes` removes the oldest eligible artifacts
|
|
396
|
-
until under quota.
|
|
397
|
-
- `maxCount`/`--max-count` and `maxVersions`/`--max-versions` rotate released
|
|
398
|
-
artifacts by series; see
|
|
399
|
-
[Artifact patterns and rotation](#artifact-patterns-and-rotation).
|
|
400
|
-
- Directory artifacts are create-only. A directory offer for a configured
|
|
401
|
-
`replaceNames` entry, a file-to-directory kind change, or a directory-to-file
|
|
402
|
-
kind change is rejected. Directory replacement and history are deferred.
|
|
403
|
-
- Committed artifacts, including directories, are immutable on disk. Do not write
|
|
404
|
-
into a committed directory after publish: hooks must not run `npm ci`, create
|
|
405
|
-
`.cache`, dotfiles, or extra symlinks inside the tree. If the tree changes,
|
|
406
|
-
startup hash scrub drops the sidecar while leaving the bytes in place, the
|
|
407
|
-
name stays occupied (`FILE_EXISTS`), the tree falls outside quota and rotation,
|
|
408
|
-
and a managed `latest` link can move to an older release.
|
|
409
|
-
- Managed directories are deleted by renaming into `.swarm-deploy/trash` before
|
|
410
|
-
recursive removal. Startup sweeps proven trash residue.
|
|
411
|
-
- Startup recovery re-hashes managed files and recursively verifies directory
|
|
412
|
-
tree digests. Scheduled and pre-commit passes validate type, sidecar, and root
|
|
413
|
-
identity without re-hashing entire trees on every tick.
|
|
414
|
-
|
|
415
|
-
Runtime defaults:
|
|
416
|
-
|
|
417
|
-
- 64 authenticated connections.
|
|
418
|
-
- 8 active uploads.
|
|
419
|
-
- 60-second upload inactivity timeout.
|
|
420
|
-
- 1 GiB minimum free-disk reserve.
|
|
421
|
-
- 15-minute cleanup interval.
|
|
422
|
-
- 7-day resumable-session lifetime.
|
|
423
|
-
|
|
424
|
-
The advanced runtime API can override these values; the server CLI intentionally
|
|
425
|
-
exposes only its required limits, committed retention and rotation, replacement
|
|
426
|
-
policy, artifact patterns, and hooks module.
|
|
427
|
-
|
|
428
|
-
## Artifact patterns and rotation
|
|
429
|
-
|
|
430
|
-
Artifact patterns teach the server which uploads belong to which release
|
|
431
|
-
series and, optionally, which SemVer version each one carries. They drive count
|
|
432
|
-
and version rotation and are configured only on the server:
|
|
433
|
-
`ServerOptions.artifactPatterns` or repeatable `--artifact-pattern`.
|
|
434
|
-
|
|
435
|
-
### Pattern templates
|
|
436
|
-
|
|
437
|
-
A template is literal text plus the placeholders `{series}` and `{version}`:
|
|
438
|
-
|
|
439
|
-
- At least one placeholder is required; each may appear at most once.
|
|
440
|
-
- Two placeholders in one path segment must be separated by literal text.
|
|
441
|
-
- `{` and `}` are reserved. A template whose literal text still contains a
|
|
442
|
-
brace after the exact `{series}` and `{version}` placeholders are removed
|
|
443
|
-
(`{serie}-{version}.tar.gz`, `{{series}.zip`) is rejected at startup rather
|
|
444
|
-
than matched literally.
|
|
445
|
-
- A template has one segment (matched against the basename) or two segments
|
|
446
|
-
separated by one `/` (`<source-parent>/<basename>`). Empty segments, further
|
|
447
|
-
`/` characters, and duplicate templates are rejected at startup.
|
|
448
|
-
- `{series}` captures one safe component. A template without `{series}` is a
|
|
449
|
-
fixed series whose key is `fixed-` followed by the lowercase hex SHA-256 of
|
|
450
|
-
the exact template text. `{version}/payments.tar.gz` therefore has the series
|
|
451
|
-
`fixed-594ef13d9808171190f7827f7d2fb1dc33f000f531b69d55c1bd485a70ff8c4b`. The
|
|
452
|
-
key is a 70-character safe basename, so it persists in commit records and
|
|
453
|
-
stays identical across restarts and between Node.js and Bare. The root export
|
|
454
|
-
`fixedSeriesKey(template)` derives the same value, so an operator can map a
|
|
455
|
-
series in a sidecar or log line back to the template that produced it.
|
|
456
|
-
- That derived shape is a reserved namespace: a captured `{series}` of exactly
|
|
457
|
-
`fixed-` plus 64 lowercase hex characters never matches, so an uploaded
|
|
458
|
-
filename cannot place itself in a version-only pattern's rotation group. The
|
|
459
|
-
offer is simply unmatched, which with mandatory matching means
|
|
460
|
-
`INVALID_FILENAME`. Look-alikes (different length, uppercase, non-hex) are
|
|
461
|
-
unaffected.
|
|
462
|
-
- `{version}` must be strict SemVer 2.0.0: no `v` prefix, no leading zeros, no
|
|
463
|
-
padding. The stored version is the normalized `major.minor.patch[-prerelease]`;
|
|
464
|
-
build metadata is accepted in a folder name but dropped, and it never
|
|
465
|
-
influences precedence.
|
|
466
|
-
- Basenames never contain `+`, so build metadata can only appear in a
|
|
467
|
-
source-parent segment.
|
|
468
|
-
- A malformed version is a non-match, not a partial match.
|
|
469
|
-
|
|
470
|
-
Examples:
|
|
471
|
-
|
|
472
|
-
| Template | Offer (`sourceParent`, basename) | Series | Version |
|
|
473
|
-
| --------------------------- | ----------------------------------- | ------------- | ------------ |
|
|
474
|
-
| `{series}-{version}.tar.gz` | `payments-2.4.1.tar.gz` | `payments` | `2.4.1` |
|
|
475
|
-
| `{series}-{version}.tar.gz` | `payments-3.0.0-rc.1.tar.gz` | `payments` | `3.0.0-rc.1` |
|
|
476
|
-
| `{version}/{series}.tar.gz` | `2.4.1+build.7` / `payments.tar.gz` | `payments` | `2.4.1` |
|
|
477
|
-
| `releases/{series}.zip` | `releases` / `payments.zip` | `payments` | none |
|
|
478
|
-
| `{version}/payments.tar.gz` | `1.8.0` / `payments.tar.gz` | `fixed-594e…` | `1.8.0` |
|
|
479
|
-
| `{series}.tar.gz` | `payments.tar.gz` | `payments` | none |
|
|
480
|
-
|
|
481
|
-
With a series-first template such as `{series}-{version}.tar.gz`, the split
|
|
482
|
-
chosen is the right-most `-` that yields a valid SemVer, so hyphenated series
|
|
483
|
-
names and prereleases both work. A version-first template such as
|
|
484
|
-
`{version}-{series}.tar.gz` splits at the left-most valid position and cannot
|
|
485
|
-
tell a prerelease hyphen from the series separator; prefer series-first or a
|
|
486
|
-
folder-based version for prereleases.
|
|
487
|
-
|
|
488
|
-
Patterns are tried in declaration order and the first template that produces
|
|
489
|
-
coordinates wins. A template with a parent segment never matches an offer that
|
|
490
|
-
carries no `sourceParent`.
|
|
491
|
-
|
|
492
|
-
### Source parent
|
|
493
|
-
|
|
494
|
-
The client sends the immediate parent directory name of the resolved input as
|
|
495
|
-
optional authenticated metadata, never an absolute path or a higher component.
|
|
496
|
-
Uploading `/ci/2.4.1+build.7/payments.tar.gz` sends `sourceParent`
|
|
497
|
-
`2.4.1+build.7`; uploading a directory sends that directory's name for each
|
|
498
|
-
child.
|
|
499
|
-
|
|
500
|
-
The parent is sent only when it is a safe single component
|
|
501
|
-
(`[A-Za-z0-9][A-Za-z0-9._+-]*`, at most 100 UTF-8 bytes). A parent with a space,
|
|
502
|
-
a leading dot, a non-ASCII character, an overlong name, or the filesystem root
|
|
503
|
-
is omitted, and the upload keeps the pre-existing metadata shape. Consequently a
|
|
504
|
-
folder pattern rejects uploads whose parent is unsafe or omitted: stage release
|
|
505
|
-
files in a conforming directory. `sourceParent` is part of the transfer ID, so
|
|
506
|
-
it cannot change between resume attempts.
|
|
507
|
-
|
|
508
|
-
A client can also suppress it deliberately with `includeSourceParent: false`
|
|
509
|
-
(CLI `--no-source-parent`) where the staging folder name itself is sensitive.
|
|
510
|
-
The upload then sends no parent and derives the same transfer ID as a client
|
|
511
|
-
that predates the field, so a server pattern needing a parent segment rejects
|
|
512
|
-
it with `INVALID_FILENAME`.
|
|
513
|
-
|
|
514
|
-
### Mandatory matching
|
|
515
|
-
|
|
516
|
-
When `artifactPatterns` is non-empty, matching is mandatory for new uploads. An
|
|
517
|
-
offer that matches no pattern is rejected with `INVALID_FILENAME` before
|
|
518
|
-
session admission, staging, verification, commit, `beforeCommit`, and
|
|
519
|
-
`afterCommit`; the server then calls `onFailure` once with phase `offer`, a
|
|
520
|
-
`null` path, and no `release`. A server with no patterns accepts every
|
|
521
|
-
otherwise-valid name, and records get no release identity.
|
|
522
|
-
|
|
523
|
-
Release coordinates (`series` and, if the pattern has one, `version`) are
|
|
524
|
-
decided at commit time and persisted in the commit record, replacement history
|
|
525
|
-
inherits them, and they survive restarts and later pattern changes. A commit
|
|
526
|
-
record written before this feature, or by a server without patterns, has no
|
|
527
|
-
release and is never deleted by count or version rotation; it remains subject
|
|
528
|
-
to age and quota retention.
|
|
529
|
-
|
|
530
|
-
A new upload whose matched release differs from the persisted release of the
|
|
531
|
-
same transfer fails closed instead of reporting `ALREADY_COMMITTED`.
|
|
532
|
-
|
|
533
|
-
### Selection rules
|
|
534
|
-
|
|
535
|
-
Retention runs under the root lease at startup, on schedule, around every
|
|
536
|
-
commit, and manually. After session expiry and a committed-state scrub, its
|
|
537
|
-
stages run in this order, each working from the records the previous stage
|
|
538
|
-
left:
|
|
539
|
-
|
|
540
|
-
1. age (`maxAge`);
|
|
541
|
-
2. count (`maxCount`);
|
|
542
|
-
3. version (`maxVersions` with `versionGranularity`);
|
|
543
|
-
4. storage quota (`maxStorageBytes`).
|
|
544
|
-
|
|
545
|
-
**Count.** Records are grouped by series and ordered by commit order: newest
|
|
546
|
-
`committedAt` first, with transfer ID and then name as deterministic
|
|
547
|
-
tie-breakers. The newest `maxCount` records in each series are kept. Replacement
|
|
548
|
-
history records count like any other release.
|
|
549
|
-
|
|
550
|
-
**Version.** Records with a version are grouped by series and ordered by SemVer
|
|
551
|
-
precedence, not by commit time. With `major`, every record in the newest
|
|
552
|
-
`maxVersions` distinct majors is kept; with `minor`, in the newest
|
|
553
|
-
`maxVersions` distinct `major.minor` groups. Prereleases and history records in
|
|
554
|
-
a retained group are kept, so `2.0.0-rc.1` and `2.0.0` share group `2`. Series-only
|
|
555
|
-
records (no `{version}`) are never selected by version rotation.
|
|
556
|
-
|
|
557
|
-
**Combined limits.** A record may be removed by either configured bound, so
|
|
558
|
-
`maxCount` plus `maxVersions` retains the intersection of the two keep sets, and
|
|
559
|
-
age and quota then apply to what remains.
|
|
560
|
-
|
|
561
|
-
**Pinned mutable names.** The current artifact of a `--replace-name` name is
|
|
562
|
-
never deleted by any stage, but it still takes part in the accounting. If it
|
|
563
|
-
occupies a slot or falls outside a keep set, the limit is best-effort: the
|
|
564
|
-
series can hold more than the configured number until a later upload moves the
|
|
565
|
-
name on.
|
|
566
|
-
|
|
567
|
-
**Immediate effect.** Rotation also runs right after each commit, so an upload
|
|
568
|
-
that is older than the retained window (for example version `1.0.0` when
|
|
569
|
-
`maxVersions` is `2` and `3.x` and `2.x` exist) is committed and then removed in
|
|
570
|
-
that same pass. The exact ordering depends on hooks:
|
|
571
|
-
|
|
572
|
-
- Without an `afterCommit` hook, the post-commit pass runs immediately after the
|
|
573
|
-
commit becomes durable, before `COMMITTED` is sent.
|
|
574
|
-
- With an `afterCommit` hook, the post-commit pass is deferred. Order:
|
|
575
|
-
commit (the pre-commit quota and age checks still run first), `afterCommit`
|
|
576
|
-
with the final path and sidecar present, and only after the callback
|
|
577
|
-
succeeds the post-commit pass, then `COMMITTED` (or `ALREADY_COMMITTED`).
|
|
578
|
-
Because the deferred pass runs before the terminal reply, a slow retention
|
|
579
|
-
pass delays `COMMITTED` or `ALREADY_COMMITTED`. `beforeCommit`-only and
|
|
580
|
-
`onFailure`-only servers do not defer.
|
|
581
|
-
- The final path exists when `afterCommit` begins in the sequential server
|
|
582
|
-
flow, including for an out-of-window release, and for replacement commits
|
|
583
|
-
(a replaced artifact's history record is also present). It is not a lock:
|
|
584
|
-
a concurrent commit's retention pass, a scheduled retention pass, or a manual
|
|
585
|
-
retention pass can remove the file while the callback runs. A hook that needs stable bytes should open or
|
|
586
|
-
copy the file promptly at the start of the callback.
|
|
587
|
-
- If `afterCommit` throws, no post-commit pass runs for that connection. The
|
|
588
|
-
artifact and its record stay, and the server remembers in memory that this
|
|
589
|
-
transfer still owes a pass. An immediate retry normally reaches
|
|
590
|
-
`ALREADY_COMMITTED`, runs `afterCommit` again with the existing path, and runs
|
|
591
|
-
the owed pass only after that call succeeds. A failing already-committed
|
|
592
|
-
`afterCommit` likewise skips the pass and keeps the transfer owed. See the
|
|
593
|
-
retry caveat below.
|
|
594
|
-
- An ordinary duplicate offer — one whose `afterCommit` already succeeded, or
|
|
595
|
-
that this process never committed — still calls `afterCommit` with
|
|
596
|
-
`alreadyCommitted: true`, but starts **no** retention pass. Only a connection
|
|
597
|
-
that committed in this process, or a retry of a transfer whose hook failed
|
|
598
|
-
here, can trigger the deferred pass, so repeated duplicate offers cannot be
|
|
599
|
-
used to force repeated full retention scans.
|
|
600
|
-
- The owed-transfer set lives only in memory. A server restart forgets it, and
|
|
601
|
-
closing the server clears it; the artifact is then rotated by the next
|
|
602
|
-
startup, scheduled, or commit-triggered pass instead.
|
|
603
|
-
- Post-commit retention failures stay non-fatal: they are logged and reported as
|
|
604
|
-
`retention` events, and never fail the upload.
|
|
605
|
-
|
|
606
|
-
**Retry caveat.** There is no persistent "hook pending" marker. After
|
|
607
|
-
`afterCommit` fails for an out-of-window artifact, any intervening retention
|
|
608
|
-
pass can remove it before the client retries: a startup pass after a restart, a
|
|
609
|
-
scheduled or manual cleanup, or the retention pass of another concurrent commit.
|
|
610
|
-
The retry is then a normal fresh upload (transfer, verification, `beforeCommit`,
|
|
611
|
-
commit, `afterCommit` with `alreadyCommitted: false`), not `ALREADY_COMMITTED`.
|
|
612
|
-
Every statement below that a failed hook is retried as `ALREADY_COMMITTED`
|
|
613
|
-
assumes the artifact was not removed in between.
|
|
614
|
-
|
|
615
|
-
`retention` events and results report `ageDeleted`, `countDeleted`,
|
|
616
|
-
`versionDeleted`, and `storageDeleted` next to `expiredSessions` and `scrubbed`.
|
|
617
|
-
Logged deletions use the reasons `MAX_AGE`, `MAX_COUNT`, `MAX_VERSIONS`, and
|
|
618
|
-
`MAX_STORAGE`.
|
|
619
|
-
|
|
620
|
-
### Configuration and CLI examples
|
|
621
|
-
|
|
622
|
-
Filename versions, keeping the newest 5 builds per product and the newest two
|
|
623
|
-
minor lines:
|
|
624
|
-
|
|
625
|
-
```sh
|
|
626
|
-
swarm-deploy server --seed-file server.seed --storage /srv/artifacts \
|
|
627
|
-
--allow-key "$CLIENT_KEY" --max-file-bytes 1073741824 --max-staging-bytes 2147483648 \
|
|
628
|
-
--artifact-pattern '{series}-{version}.tar.gz' \
|
|
629
|
-
--max-count 5 --max-versions 2 --version-granularity minor
|
|
630
|
-
```
|
|
631
|
-
|
|
632
|
-
Folder versions (the source parent carries the version, build metadata
|
|
633
|
-
included), keeping the two newest majors:
|
|
634
|
-
|
|
635
|
-
```sh
|
|
636
|
-
swarm-deploy server --seed-file server.seed --storage /srv/artifacts \
|
|
637
|
-
--allow-key "$CLIENT_KEY" --max-file-bytes 1073741824 --max-staging-bytes 2147483648 \
|
|
638
|
-
--artifact-pattern '{version}/{series}.tar.gz' \
|
|
639
|
-
--max-versions 2 --version-granularity major
|
|
640
|
-
|
|
641
|
-
# client: /ci/out/2.4.1+build.7/payments.tar.gz is offered as
|
|
642
|
-
# sourceParent "2.4.1+build.7", name "payments.tar.gz"
|
|
643
|
-
swarm-deploy upload --seed-file client.seed --server-key "$SERVER_KEY" \
|
|
644
|
-
/ci/out/2.4.1+build.7/payments.tar.gz
|
|
645
|
-
```
|
|
646
|
-
|
|
647
|
-
The runtime API takes the same values:
|
|
648
|
-
|
|
649
|
-
```ts
|
|
650
|
-
new Server({
|
|
651
|
-
// ...required options
|
|
652
|
-
artifactPatterns: ['{series}-{version}.tar.gz', 'releases/{series}.zip'],
|
|
653
|
-
maxCount: 5,
|
|
654
|
-
maxVersions: 2,
|
|
655
|
-
versionGranularity: 'minor'
|
|
656
|
-
})
|
|
657
|
-
```
|
|
658
|
-
|
|
659
|
-
`maxCount` requires at least one pattern. `maxVersions` requires at least one
|
|
660
|
-
pattern containing `{version}` and a `versionGranularity`; a granularity without
|
|
661
|
-
`maxVersions` is invalid. These errors are raised at construction or CLI startup
|
|
662
|
-
(exit `2`). The pattern list is snapshotted at construction.
|
|
663
|
-
|
|
664
|
-
### Rollout and compatibility
|
|
665
|
-
|
|
666
|
-
Upgrade every server before any client. An older server decodes upload metadata
|
|
667
|
-
with an exact key set and rejects the new `sourceParent` field, so a new client
|
|
668
|
-
that sends a parent to an old server fails.
|
|
669
|
-
|
|
670
|
-
A new server remains compatible with older clients: they never send
|
|
671
|
-
`sourceParent`, no-pattern servers are unaffected, filename-only patterns still
|
|
672
|
-
match, and patterns that need a parent reject their offers as unmatched. Older
|
|
673
|
-
sessions and commit records stay readable, and older commit records are not
|
|
674
|
-
count or version rotated.
|
|
675
|
-
|
|
676
|
-
Roll out the server, then the pattern configuration, then clients that stage
|
|
677
|
-
into conforming folders.
|
|
678
|
-
|
|
679
|
-
#### Managed-link rollback warning
|
|
680
|
-
|
|
681
|
-
**Do not roll a server back to v0.2.0 after an upgraded server has created,
|
|
682
|
-
repaired, or repointed any managed link.** Those operations write a v2
|
|
683
|
-
ownership record, which v0.2.0 cannot read. Rollback requires restoring the
|
|
684
|
-
complete pre-upgrade storage backup (including visible links and
|
|
685
|
-
`.swarm-deploy/links`), or keeping the upgraded server in place.
|
|
686
|
-
|
|
687
|
-
`swarm-deploy link` requires an upgraded server; v0.2.0 does not recognize
|
|
688
|
-
link control requests.
|
|
689
|
-
|
|
690
|
-
#### Overlapping retries now get `FILE_BUSY`
|
|
691
|
-
|
|
692
|
-
**Behavior change.** A transfer ID may have only one in-flight commit
|
|
693
|
-
lifecycle. If a client (or a job runner that fans the same artifact out to
|
|
694
|
-
parallel workers) opens a second connection for a transfer that is still
|
|
695
|
-
verifying or committing, that second connection is now rejected with
|
|
696
|
-
`FILE_BUSY` instead of being admitted alongside the first. Sequential retries,
|
|
697
|
-
resumes, and verified reconnects are unaffected, and so is a retry issued after
|
|
698
|
-
the previous attempt failed — the server releases the transfer before running
|
|
699
|
-
its observational `onFailure`, so a slow callback never blocks the retry.
|
|
700
|
-
|
|
701
|
-
Audit any automation that uploads the same artifact from more than one worker
|
|
702
|
-
at a time, and treat `FILE_BUSY` on an upload as "retry after the in-flight
|
|
703
|
-
attempt finishes" rather than a permanent failure.
|
|
704
|
-
|
|
705
|
-
#### Enabling or changing patterns
|
|
706
|
-
|
|
707
|
-
A commit record's release identity is durable and is never rewritten in place.
|
|
708
|
-
Enabling patterns, or editing one that an in-flight upload already matched,
|
|
709
|
-
therefore changes the identity a retry computes for bytes that are already
|
|
710
|
-
committed, and the server fails closed rather than relabelling them:
|
|
711
|
-
|
|
712
|
-
- A create-only name whose record carries a different release (or none) is no
|
|
713
|
-
longer recognised as the same commit, so the retry is rejected with
|
|
714
|
-
`FILE_EXISTS`.
|
|
715
|
-
- A replaceable name retried under the same transfer ID with a different
|
|
716
|
-
release is rejected as a release-identity conflict.
|
|
717
|
-
|
|
718
|
-
Drain or complete in-flight uploads before enabling or changing
|
|
719
|
-
`artifactPatterns`. Uploads started after the change are unaffected, and a
|
|
720
|
-
rejected legacy retry is resolved by re-uploading under a new transfer (new
|
|
721
|
-
content or a new staged name), not by editing records on disk.
|
|
722
|
-
|
|
723
|
-
#### Session records and rollback
|
|
724
|
-
|
|
725
|
-
A resumable session is written at on-disk version 3 only when it carries a
|
|
726
|
-
`sourceParent`; a session without one is written at version 2, exactly as an
|
|
727
|
-
older server would. The new server reads both. An older server cannot read
|
|
728
|
-
version 3, so only parent-bearing resumable sessions are lost to a downgrade:
|
|
729
|
-
they cannot resume and must be re-uploaded. Draining in-flight uploads before
|
|
730
|
-
rolling a server back avoids this entirely.
|
|
731
|
-
|
|
732
|
-
Commit-record compatibility with older code is covered only by what this
|
|
733
|
-
version's tests assert: an older record without release coordinates is read by
|
|
734
|
-
this version and is not count or version rotated. Whether older code tolerates
|
|
735
|
-
the release coordinates this version writes is untested.
|
|
736
|
-
|
|
737
|
-
## Managed symlinks
|
|
738
|
-
|
|
739
|
-
Managed symlinks support both automatic newest-match aliases and explicit
|
|
740
|
-
client promotions. Configure either form with repeatable `--symlink` options:
|
|
741
|
-
|
|
742
|
-
```sh
|
|
743
|
-
swarm-deploy server \
|
|
744
|
-
--seed-file ./server.seed \
|
|
745
|
-
--storage /srv/artifacts \
|
|
746
|
-
--allow-key <64-lower-hex> \
|
|
747
|
-
--max-file-bytes 1073741824 \
|
|
748
|
-
--max-staging-bytes 4294967296 \
|
|
749
|
-
--symlink '/^\d+\.\d+\.\d+$/' newest \
|
|
750
|
-
--symlink '/^app-\d+\.\d+\.\d+\.tar\.gz$/'
|
|
751
|
-
```
|
|
752
|
-
|
|
753
|
-
The two-argument rule automatically reconciles `newest` to the newest matching
|
|
754
|
-
artifact. The one-argument rule is a target allowlist: it creates no link until
|
|
755
|
-
an authenticated client explicitly chooses a matching current artifact:
|
|
756
|
-
|
|
757
|
-
```sh
|
|
758
|
-
swarm-deploy link --seed-file ./client.seed --server-key "$SERVER_KEY" \
|
|
759
|
-
app-2.4.1.tar.gz latest-app
|
|
760
|
-
|
|
761
|
-
# app-2.4.1.tar.gz is the tested target
|
|
762
|
-
# output: latest-app -> app-2.4.1.tar.gz LINKED
|
|
763
|
-
```
|
|
764
|
-
|
|
765
|
-
Runtime equivalent:
|
|
766
|
-
|
|
767
|
-
```js
|
|
768
|
-
const server = new Server({
|
|
769
|
-
seed,
|
|
770
|
-
storageDir: '/srv/artifacts',
|
|
771
|
-
allowedKeys,
|
|
772
|
-
maxFileBytes: 1024 ** 3,
|
|
773
|
-
maxStagingBytes: 4 * 1024 ** 3,
|
|
774
|
-
symlinks: [
|
|
775
|
-
{ selector: '/^\\d+\\.\\d+\\.\\d+$/', name: 'newest' },
|
|
776
|
-
{ selector: '/^app-\\d+\\.\\d+\\.\\d+\\.tar\\.gz$/' }
|
|
777
|
-
]
|
|
778
|
-
})
|
|
779
|
-
```
|
|
780
|
-
|
|
781
|
-
Selectors are exact safe managed-artifact basenames or unflagged `/regex/`
|
|
782
|
-
expressions. Automatic selection uses only validated managed commit records.
|
|
783
|
-
The newest `committedAt` wins, tie-broken by transfer ID then name; replacement
|
|
784
|
-
history never matches, and a rule is dormant until a target exists. Automatic
|
|
785
|
-
link names are unique and reserved even while their rule is dormant, so a
|
|
786
|
-
manual request cannot claim one.
|
|
787
|
-
|
|
788
|
-
Manual targets must be current managed top-level files or directories, must
|
|
789
|
-
match a one-argument rule, and cannot be history names. Manual link names are
|
|
790
|
-
safe top-level basenames and cannot equal the target, an automatic link name,
|
|
791
|
-
or any managed artifact name. A missing destination may be created, and an
|
|
792
|
-
existing manual managed link may be repointed. An unmanaged file, directory,
|
|
793
|
-
unrecorded symlink, changed symlink, or foreign ownership record is never
|
|
794
|
-
replaced, moved, or deleted: the operation fails closed.
|
|
795
|
-
|
|
796
|
-
Manual links retain their selected target across restart and automatic
|
|
797
|
-
reconciliation. Removing their authorization rule prevents future requests
|
|
798
|
-
that need that rule but does not remove or rewrite the existing link.
|
|
799
|
-
Automatic and manual targets are pinned against age, count, SemVer, and quota
|
|
800
|
-
retention and still count toward those limits. Reconciliation and manual
|
|
801
|
-
mutation run under the storage-root lease, and uploads cannot claim any
|
|
802
|
-
persisted managed-link name.
|
|
803
|
-
|
|
804
|
-
There is no unlink RPC. To remove a managed link safely, stop the server,
|
|
805
|
-
remove the visible symlink and its matching
|
|
806
|
-
`.swarm-deploy/links/<sha256(link-name)>.json` ownership record, then restart
|
|
807
|
-
the server. Never edit the ledger while the server runs. Removing all automatic
|
|
808
|
-
rules intentionally reconciles proven stale automatic links away; malformed or
|
|
809
|
-
conflicting ledgers still fail closed on the default symlink-capable storage.
|
|
810
|
-
|
|
811
|
-
A safe promotion workflow is:
|
|
812
|
-
|
|
813
|
-
1. Upload a versioned artifact such as `app-2.4.1.tar.gz`.
|
|
814
|
-
2. QA that exact version through its deployed versioned URL.
|
|
815
|
-
3. Run `swarm-deploy link app-2.4.1.tar.gz latest` to point `latest` at the
|
|
816
|
-
tested artifact.
|
|
817
|
-
|
|
818
|
-
The explicit link operation is a control request, not an upload. It supports
|
|
819
|
-
files and directories, is idempotent, and does not invoke deployment hooks.
|
|
820
|
-
`symlinks` requires `StorageAdapter.symlink` and `readlink`; the Node and Bare
|
|
821
|
-
default adapters provide them; a custom adapter without symlink rules stays
|
|
822
|
-
source-compatible.
|
|
823
|
-
|
|
824
|
-
## Deployment hooks
|
|
825
|
-
|
|
826
|
-
Hooks let a trusted server operator run code around the commit. They are
|
|
827
|
-
server-side only, run in the server process with its privileges, and receive no
|
|
828
|
-
seeds, secret keys, TAR bytes, or session material.
|
|
829
|
-
|
|
830
|
-
### Lifecycle points
|
|
831
|
-
|
|
832
|
-
- `beforeCommit(context)`: after verification, before any commit mutation. For a
|
|
833
|
-
file, `path` is the verified `.part` staging file; for a directory, `path` is
|
|
834
|
-
the verified `.tree` staging directory. Its failure aborts the commit.
|
|
835
|
-
- `afterCommit(context)`: after the artifact is durably committed and configured
|
|
836
|
-
links are reconciled, before the terminal success reply. For a directory,
|
|
837
|
-
`path` is the committed directory tree. Its failure fails the upload even
|
|
838
|
-
though the artifact is already stored. When it is configured, post-commit
|
|
839
|
-
rotation is deferred until it succeeds (see "Immediate effect" above).
|
|
840
|
-
- `onFailure(context)`: once per failed connection whose metadata was decoded,
|
|
841
|
-
after the client has been answered.
|
|
842
|
-
|
|
843
|
-
Callbacks may return `void` or a promise, are called with no receiver, and
|
|
844
|
-
receive a frozen context. The artifact is
|
|
845
|
-
`{ name, kind, size, sha256, transferId, entryCount?, sourceParent?, release? }`,
|
|
846
|
-
where `entryCount` is present only for directories and `release` is
|
|
847
|
-
`{ series, version? }` when patterns are configured. These descriptive fields
|
|
848
|
-
come from decoded offer metadata, not yet from verified content.
|
|
849
|
-
|
|
850
|
-
| Callback | Extra context |
|
|
851
|
-
| -------------- | ---------------------------------------------------------------------------- |
|
|
852
|
-
| `beforeCommit` | `path` (verified staging file or tree), `resumed`, `alreadyCommitted: false` |
|
|
853
|
-
| `afterCommit` | `path` (committed artifact), `resumed`, `alreadyCommitted` |
|
|
854
|
-
| `onFailure` | `path` (`string \| null`), `phase`, `resumed`, `alreadyCommitted`, `error` |
|
|
855
|
-
|
|
856
|
-
`resumed` is `true` when the server admitted the connection as `RESUME` or
|
|
857
|
-
`VERIFIED`. Closing the server aborts the wait; a callback that outlives the
|
|
858
|
-
abort continues detached and its later result is ignored.
|
|
859
|
-
|
|
860
|
-
There is no hook timeout, and a hung callback holds more than its own
|
|
861
|
-
connection. On a fresh or resumed upload it also holds a `maxActiveUploads`
|
|
862
|
-
slot and the session's staging reservation for as long as it runs, so enough
|
|
863
|
-
simultaneously hung callbacks stop the server from admitting new uploads
|
|
864
|
-
(`ACTIVE_UPLOAD_LIMIT`) until the server is closed and the waits are aborted.
|
|
865
|
-
Only the already-committed path holds no upload slot. Write callbacks that
|
|
866
|
-
finish or throw; bounded hook timeout and anti-spam controls are tracked in
|
|
867
|
-
issue #8.
|
|
868
|
-
|
|
869
|
-
The **lifecycle** hooks `beforeCommit` and `afterCommit` never run concurrently
|
|
870
|
-
for one transfer. Once a transfer ID authenticates, that connection owns it for
|
|
871
|
-
the whole verify/commit sequence; a second connection offering the same transfer
|
|
872
|
-
ID — fresh, resumed, verified, or already committed — is rejected with
|
|
873
|
-
`FILE_BUSY` before any lifecycle hook runs, and that rejection is reported to
|
|
874
|
-
`onFailure` once with `phase: 'offer'`.
|
|
875
|
-
|
|
876
|
-
`onFailure` is observational and is **not** covered by that guarantee. A failed
|
|
877
|
-
connection hands the transfer back before awaiting its `onFailure`, because the
|
|
878
|
-
callback only reports an outcome and the connection has already finished
|
|
879
|
-
mutating state. A slow callback must not reject the client's legitimate retry,
|
|
880
|
-
so a retry of the same transfer can be admitted — and can run its own
|
|
881
|
-
`beforeCommit`/`afterCommit` — while the previous `onFailure` is still running.
|
|
882
|
-
Each connection still calls `onFailure` at most once, so write it to tolerate
|
|
883
|
-
overlapping with the next attempt and key any external effect on
|
|
884
|
-
`artifact.transferId`.
|
|
885
|
-
|
|
886
|
-
Hooks for **different** transfers always run concurrently.
|
|
887
|
-
|
|
888
|
-
### Invocation sequences
|
|
889
|
-
|
|
890
|
-
Fresh upload:
|
|
891
|
-
|
|
892
|
-
1. Offer inspected and admitted (`ACCEPT`), TAR received.
|
|
893
|
-
2. Verification.
|
|
894
|
-
3. `beforeCommit` (`resumed: false`).
|
|
895
|
-
4. Commit.
|
|
896
|
-
5. `afterCommit` (`resumed: false`, `alreadyCommitted: false`).
|
|
897
|
-
6. `COMMITTED`.
|
|
898
|
-
|
|
899
|
-
Partial resume (`RESUME`): the same steps, with `resumed: true` in both hooks.
|
|
900
|
-
|
|
901
|
-
Verified reconnect (`VERIFIED`): the staged data was already verified, so the
|
|
902
|
-
server re-reads it, calls `beforeCommit` again with `resumed: true`, commits, and
|
|
903
|
-
calls `afterCommit` with `resumed: true`. A previous connection may have ended
|
|
904
|
-
after `beforeCommit` returned but before the commit finished.
|
|
905
|
-
|
|
906
|
-
Already committed: identical content is detected during offer inspection, so
|
|
907
|
-
there is no transfer, verification, or `beforeCommit`. The server calls
|
|
908
|
-
`afterCommit` with `resumed: false` and `alreadyCommitted: true`, then replies
|
|
909
|
-
`ALREADY_COMMITTED`. A deferred retention pass follows only when this process
|
|
910
|
-
committed that transfer and its `afterCommit` has not yet succeeded. This path
|
|
911
|
-
is reached only while the artifact still exists; see the
|
|
912
|
-
[retry caveat](#artifact-patterns-and-rotation).
|
|
913
|
-
|
|
914
|
-
Failure sequences:
|
|
915
|
-
|
|
916
|
-
- `beforeCommit` throws: nothing is committed and the verified session remains.
|
|
917
|
-
The client receives `HOOK_FAILED`; `onFailure` runs (`phase: 'beforeCommit'`).
|
|
918
|
-
A retry reconnects as `VERIFIED` and calls `beforeCommit` again.
|
|
919
|
-
- `afterCommit` throws on a fresh or resumed upload: the artifact **stays
|
|
920
|
-
durably committed** and its session is retired. The client receives
|
|
921
|
-
`HOOK_FAILED`; `onFailure` runs (`phase: 'afterCommit'`, final path). Unless
|
|
922
|
-
retention removed the artifact first (see the
|
|
923
|
-
[retry caveat](#artifact-patterns-and-rotation)), a retry takes the
|
|
924
|
-
already-committed path and calls `afterCommit` with `alreadyCommitted: true`;
|
|
925
|
-
otherwise it is a fresh upload.
|
|
926
|
-
- `afterCommit` throws on an already-committed retry: the offer is rejected with
|
|
927
|
-
`HOOK_FAILED` and a later retry repeats `afterCommit`, subject to the same
|
|
928
|
-
retry caveat.
|
|
929
|
-
- A failure after both hooks succeeded (for example the terminal reply cannot be
|
|
930
|
-
written) does not call `onFailure`.
|
|
931
|
-
|
|
932
|
-
Retries therefore can call hooks more than once for one transfer. **Make hooks
|
|
933
|
-
idempotent and key external side effects on `artifact.transferId`.** Treat
|
|
934
|
-
`afterCommit` as at-least-once: record the transfer ID when the deployment step
|
|
935
|
-
finishes and skip repeated calls.
|
|
936
|
-
|
|
937
|
-
### onFailure
|
|
938
|
-
|
|
939
|
-
`phase` is one of `offer`, `transfer`, `verification`, `beforeCommit`, `commit`,
|
|
940
|
-
or `afterCommit`. `offer` covers every failure before a staging path exists:
|
|
941
|
-
metadata or transfer-ID problems, unmatched patterns, file-size, capacity and
|
|
942
|
-
destination rejections, and inspection errors.
|
|
943
|
-
|
|
944
|
-
An `offer`-phase context is built from decoded metadata **before** the transfer
|
|
945
|
-
ID is authenticated, and it is also reported when that authentication is exactly
|
|
946
|
-
what failed. Its `transferId`, `name`, `size`, `sha256`, and `sourceParent` are
|
|
947
|
-
therefore attacker-chosen values from an allowlisted key, not verified identity.
|
|
948
|
-
Use them for logging and alerting only; never as audit records or idempotency
|
|
949
|
-
keys. Every later phase runs after authentication, so only `offer` carries this
|
|
950
|
-
caveat.
|
|
951
|
-
|
|
952
|
-
`error` is the original failure. For an exception thrown by `beforeCommit` or
|
|
953
|
-
`afterCommit`, hooks receive the **raw thrown value** (not a wrapper) because
|
|
954
|
-
hooks are trusted. The client and server events see only the stable wire code
|
|
955
|
-
`HOOK_FAILED` with a fixed message, so callback text never reaches the wire. Other
|
|
956
|
-
failures arrive as `SwarmDeployError` (or raw storage errors).
|
|
957
|
-
A failing `onFailure` is logged as a secondary warning and never replaces the
|
|
958
|
-
original failure.
|
|
959
|
-
|
|
960
|
-
`path` by phase:
|
|
961
|
-
|
|
962
|
-
| Situation | `path` |
|
|
963
|
-
| ------------------------------------------------------------------ | ------------------------------------------------------- |
|
|
964
|
-
| `offer` | `null` |
|
|
965
|
-
| `transfer`; fresh-upload `verification` | `<storage>/.swarm-deploy/staging/<transferId>.tar.part` |
|
|
966
|
-
| `beforeCommit`, `commit`; `verification` of a `VERIFIED` reconnect | `<storage>/.swarm-deploy/staging/<transferId>.part` |
|
|
967
|
-
| `afterCommit` (including already committed) | `<storage>/<name>` |
|
|
968
|
-
|
|
969
|
-
### Hook modules
|
|
970
|
-
|
|
971
|
-
`--hooks <module>` accepts `.js`, `.mjs`, and `.cjs`; any other extension is
|
|
972
|
-
rejected at startup with exit code `2` before the module is loaded. ESM and
|
|
973
|
-
CommonJS examples:
|
|
974
|
-
|
|
975
|
-
```js
|
|
976
|
-
// hooks.mjs
|
|
977
|
-
export async function afterCommit({ artifact, path, alreadyCommitted }) {
|
|
978
|
-
// Idempotent: keyed by artifact.transferId.
|
|
979
|
-
await deploy(artifact.transferId, path)
|
|
980
|
-
}
|
|
981
|
-
|
|
982
|
-
export function onFailure({ phase, artifact, error }) {
|
|
983
|
-
console.error('deploy failed', phase, artifact.transferId, error)
|
|
984
|
-
}
|
|
985
|
-
```
|
|
986
|
-
|
|
987
|
-
```js
|
|
988
|
-
// hooks.cjs
|
|
989
|
-
module.exports = {
|
|
990
|
-
beforeCommit({ artifact, path }) {
|
|
991
|
-
// Return or throw; a throw rejects the upload with HOOK_FAILED.
|
|
992
|
-
},
|
|
993
|
-
async afterCommit({ artifact, path }) {
|
|
994
|
-
await deploy(artifact.transferId, path)
|
|
995
|
-
}
|
|
996
|
-
}
|
|
997
|
-
```
|
|
998
|
-
|
|
999
|
-
Loader rules:
|
|
1000
|
-
|
|
1001
|
-
- Only the own properties `beforeCommit`, `afterCommit`, and `onFailure` are
|
|
1002
|
-
read. Other exports (helpers, configuration) are ignored, including a
|
|
1003
|
-
harmless non-object default export.
|
|
1004
|
-
- Named exports take precedence over callbacks on a default export object; a
|
|
1005
|
-
named export set to `undefined` does not override it.
|
|
1006
|
-
- A hook name that is present must be a function, the callbacks must be plain
|
|
1007
|
-
own properties, and at least one must exist; otherwise startup fails.
|
|
1008
|
-
- One level of compiled-CommonJS interop (`__esModule` with `default`) is
|
|
1009
|
-
unwrapped; deeper nesting is ignored.
|
|
1010
|
-
- Modules are loaded once with `import()` and cached by the runtime. Top-level
|
|
1011
|
-
code runs with the server's privileges during startup, so the module is fully
|
|
1012
|
-
trusted.
|
|
1013
|
-
|
|
1014
|
-
`ServerOptions.hooks` is stricter: it must be a hook object whose own keys are
|
|
1015
|
-
only the three callbacks, every callback a function. Unknown keys and
|
|
1016
|
-
non-function values throw `PROTOCOL_INVALID`. The object is snapshotted at
|
|
1017
|
-
construction, so later mutation has no effect.
|
|
1018
|
-
|
|
1019
|
-
## Runtime API
|
|
1020
|
-
|
|
1021
|
-
The package is strict TypeScript and exposes the same root API to ESM and
|
|
1022
|
-
CommonJS consumers.
|
|
1023
|
-
|
|
1024
|
-
### Identity helpers
|
|
1025
|
-
|
|
1026
|
-
```ts
|
|
1027
|
-
import {
|
|
1028
|
-
generateSeed,
|
|
1029
|
-
keyPairFromSeed,
|
|
1030
|
-
parsePublicKey,
|
|
1031
|
-
parseSeed,
|
|
1032
|
-
publicKeyFromSeed
|
|
1033
|
-
} from 'swarm-deploy'
|
|
1034
|
-
|
|
1035
|
-
const seed = generateSeed() // 32 random bytes
|
|
1036
|
-
const restored = parseSeed(process.env.SEED!) // strict lowercase hex
|
|
1037
|
-
const keyPair = keyPairFromSeed(restored)
|
|
1038
|
-
const publicKey = publicKeyFromSeed(restored)
|
|
1039
|
-
const peer = parsePublicKey(process.env.PEER_PUBLIC_KEY!)
|
|
1040
|
-
```
|
|
1041
|
-
|
|
1042
|
-
`parseSeed` and `parsePublicKey` require exactly 64 lowercase hexadecimal
|
|
1043
|
-
characters. `keyPairFromSeed` is deterministic.
|
|
1044
|
-
|
|
1045
|
-
`parseAllowlist(text)` parses lowercase client public keys separated by
|
|
1046
|
-
newlines. Blank lines and lines beginning with `#` are ignored; malformed or
|
|
1047
|
-
duplicate keys throw. It returns a `Set<string>` suitable for
|
|
1048
|
-
`ServerOptions.allowedKeys`.
|
|
1049
|
-
|
|
1050
|
-
`fixedSeriesKey(template)` returns the series a template without `{series}`
|
|
1051
|
-
persists, so an operator reading `fixed-…` in a commit sidecar, retention log,
|
|
1052
|
-
or hook context can identify which configured pattern owns it:
|
|
1053
|
-
|
|
1054
|
-
```ts
|
|
1055
|
-
import { fixedSeriesKey } from 'swarm-deploy'
|
|
1056
|
-
|
|
1057
|
-
const owner = patterns.find((template) => fixedSeriesKey(template) === record.release?.series)
|
|
1058
|
-
```
|
|
1059
|
-
|
|
1060
|
-
### Server
|
|
1061
|
-
|
|
1062
|
-
```ts
|
|
1063
|
-
import { Server, parsePublicKey } from 'swarm-deploy'
|
|
1064
|
-
|
|
1065
|
-
const server = new Server({
|
|
1066
|
-
seed: process.env.SWARM_DEPLOY_SERVER_SEED!,
|
|
1067
|
-
storageDir: '/srv/artifacts',
|
|
1068
|
-
allowedKeys: [
|
|
1069
|
-
parsePublicKey(process.env.CLIENT_A_PUBLIC_KEY!),
|
|
1070
|
-
parsePublicKey(process.env.CLIENT_B_PUBLIC_KEY!)
|
|
1071
|
-
],
|
|
1072
|
-
maxFileBytes: 1024 ** 3,
|
|
1073
|
-
maxStagingBytes: 4 * 1024 ** 3,
|
|
1074
|
-
maxConnections: 64,
|
|
1075
|
-
maxActiveUploads: 8,
|
|
1076
|
-
idleTimeout: 60_000,
|
|
1077
|
-
cleanupInterval: 15 * 60_000,
|
|
1078
|
-
resumeTtl: 7 * 24 * 60 * 60_000,
|
|
1079
|
-
minFreeBytes: 1024 ** 3,
|
|
1080
|
-
maxAge: 15 * 24 * 60 * 60_000,
|
|
1081
|
-
maxStorageBytes: 100 * 1024 ** 3,
|
|
1082
|
-
replaceNames: ['release.tar.gz', 'latest.json']
|
|
1083
|
-
})
|
|
1084
|
-
|
|
1085
|
-
await server.listen()
|
|
1086
|
-
console.log(server.publicKey.toString('hex'))
|
|
1087
|
-
|
|
1088
|
-
// Later, after draining or on process shutdown:
|
|
1089
|
-
await server.close()
|
|
1090
|
-
```
|
|
1091
|
-
|
|
1092
|
-
Required `ServerOptions`:
|
|
1093
|
-
|
|
1094
|
-
- `seed: Buffer | string`
|
|
1095
|
-
- `storageDir: string`
|
|
1096
|
-
- `allowedKeys: Iterable<Buffer | string>`
|
|
1097
|
-
- `maxFileBytes: number`
|
|
1098
|
-
- `maxStagingBytes: number`
|
|
1099
|
-
|
|
1100
|
-
Optional operational limits and policies:
|
|
1101
|
-
|
|
1102
|
-
- `maxConnections`, `maxActiveUploads`, `idleTimeout`
|
|
1103
|
-
- `cleanupInterval`, `resumeTtl`, `minFreeBytes`
|
|
1104
|
-
- `maxAge`, `maxStorageBytes`, `replaceNames`
|
|
1105
|
-
- `artifactPatterns?: Iterable<string>`, `maxCount?: number`,
|
|
1106
|
-
`maxVersions?: number`, and `versionGranularity?: 'major' | 'minor'`; see
|
|
1107
|
-
[Artifact patterns and rotation](#artifact-patterns-and-rotation).
|
|
1108
|
-
- `symlinks?: Iterable<SymlinkRule>` with automatic
|
|
1109
|
-
`{ selector, name }` and manual-authorization `{ selector }` entries; see
|
|
1110
|
-
[Managed symlinks](#managed-symlinks).
|
|
1111
|
-
- `hooks?: ServerHooks | null`; see [Deployment hooks](#deployment-hooks).
|
|
1112
|
-
|
|
1113
|
-
The root package exports the `ServerHooks`, `BeforeCommitContext`,
|
|
1114
|
-
`AfterCommitContext`, `HookFailureContext`, `HookFailurePhase`, `HookArtifact`,
|
|
1115
|
-
`ReleaseCoordinates`, and `VersionGranularity` types:
|
|
1116
|
-
|
|
1117
|
-
```ts
|
|
1118
|
-
import type { ServerHooks } from 'swarm-deploy'
|
|
1119
|
-
|
|
1120
|
-
const hooks: ServerHooks = {
|
|
1121
|
-
async afterCommit({ artifact, path, alreadyCommitted }) {
|
|
1122
|
-
// Idempotent: skip work already recorded for artifact.transferId.
|
|
1123
|
-
},
|
|
1124
|
-
onFailure({ phase, path, error }) {
|
|
1125
|
-
console.error(phase, path, error)
|
|
1126
|
-
}
|
|
1127
|
-
}
|
|
1128
|
-
```
|
|
1129
|
-
|
|
1130
|
-
Advanced integration and test seams:
|
|
1131
|
-
|
|
1132
|
-
- `dht` or `dhtFactory` for an injected HyperDHT node.
|
|
1133
|
-
- `storage` for a compatible filesystem adapter.
|
|
1134
|
-
- `scheduler` for timeout and interval control.
|
|
1135
|
-
- `logger` with optional `info`, `warn`, and `error` methods.
|
|
1136
|
-
|
|
1137
|
-
### Client
|
|
1138
|
-
|
|
1139
|
-
```ts
|
|
1140
|
-
import { Client, parsePublicKey } from 'swarm-deploy'
|
|
1141
|
-
|
|
1142
|
-
const client = new Client({
|
|
1143
|
-
seed: process.env.SWARM_DEPLOY_CLIENT_SEED!,
|
|
1144
|
-
serverPublicKey: parsePublicKey(process.env.SWARM_DEPLOY_SERVER_KEY!),
|
|
1145
|
-
connectTimeout: 30_000,
|
|
1146
|
-
idleTimeout: 60_000
|
|
1147
|
-
})
|
|
1148
|
-
|
|
1149
|
-
const result = await client.upload('./dist/release.tar.gz')
|
|
1150
|
-
console.log(result.status, result.name, result.size)
|
|
1151
|
-
|
|
1152
|
-
await client.close()
|
|
1153
|
-
```
|
|
1154
|
-
|
|
1155
|
-
`ClientOptions.seed` accepts a Buffer or canonical lowercase 64-character hex
|
|
1156
|
-
string. `serverPublicKey` is a Buffer produced by `parsePublicKey`. Optional
|
|
1157
|
-
values are `connectTimeout`, `idleTimeout`, `includeSourceParent`, `dht`,
|
|
1158
|
-
`dhtFactory`, and `logger`.
|
|
1159
|
-
|
|
1160
|
-
`connectTimeout` defaults to 30 seconds. `idleTimeout` defaults to 60 seconds.
|
|
1161
|
-
`includeSourceParent` defaults to `true`; set it to `false` to suppress the
|
|
1162
|
-
[source parent](#source-parent) for every upload the client makes.
|
|
1163
|
-
Calling `close()` aborts pending work, closes active sockets, and is idempotent.
|
|
1164
|
-
|
|
1165
|
-
### Link results
|
|
1166
|
-
|
|
1167
|
-
`Client.link(target, name)` resolves to:
|
|
1168
|
-
|
|
1169
|
-
```ts
|
|
1170
|
-
interface ClientLinkResult {
|
|
1171
|
-
target: string
|
|
1172
|
-
name: string
|
|
1173
|
-
status: 'LINKED' | 'UNCHANGED'
|
|
1174
|
-
}
|
|
1175
|
-
```
|
|
1176
|
-
|
|
1177
|
-
The client validates both names before connecting. Link operations share the
|
|
1178
|
-
client's pinned server identity, authenticated client identity, connection and
|
|
1179
|
-
idle timeouts, serialized operation queue, cancellation, and `close()`
|
|
1180
|
-
behavior with uploads.
|
|
1181
|
-
|
|
1182
|
-
### Upload results
|
|
1183
|
-
|
|
1184
|
-
A direct file resolves to:
|
|
1185
|
-
|
|
1186
|
-
```ts
|
|
1187
|
-
interface UploadResult {
|
|
1188
|
-
status: 'COMMITTED' | 'ALREADY_COMMITTED'
|
|
1189
|
-
kind: 'file' | 'directory'
|
|
1190
|
-
name: string
|
|
1191
|
-
size: number
|
|
1192
|
-
digest: Buffer
|
|
1193
|
-
transferId: Buffer
|
|
1194
|
-
entryCount?: number
|
|
1195
|
-
}
|
|
1196
|
-
```
|
|
1197
|
-
|
|
1198
|
-
Files and directories share the same result shape. For a directory, `size` is
|
|
1199
|
-
aggregate payload bytes and `entryCount` counts stored entries (files and
|
|
1200
|
-
directories).
|
|
1201
|
-
|
|
1202
|
-
### Events
|
|
1203
|
-
|
|
1204
|
-
`Server` and `Client` are event emitters. Listener and logger exceptions are
|
|
1205
|
-
contained and cannot change protocol correctness.
|
|
1206
|
-
|
|
1207
|
-
Public peer and transfer correlation fields use 12-character SHA-256
|
|
1208
|
-
fingerprints. Events never expose seeds, secret keys, full remote public keys,
|
|
1209
|
-
TAR contents, or resumable session material. Internal storage warnings may
|
|
1210
|
-
include a full SHA-256 uploader fingerprint, but never the uploader key itself.
|
|
1211
|
-
|
|
1212
|
-
Server events:
|
|
1213
|
-
|
|
1214
|
-
- `authentication`: accepted client fingerprint.
|
|
1215
|
-
- `connection`, `connection-open`, `connection-close`: authenticated connection
|
|
1216
|
-
lifecycle and current count.
|
|
1217
|
-
- `offer`: accepted, resumed, reset, rejected, or already-committed state.
|
|
1218
|
-
Includes artifact `kind`.
|
|
1219
|
-
- `progress`: durable TAR bytes received and total TAR bytes. Includes artifact
|
|
1220
|
-
`kind`.
|
|
1221
|
-
- `verification`: started, succeeded, or failed.
|
|
1222
|
-
- `commit`: succeeded or failed.
|
|
1223
|
-
- `recovery`: startup, per-journal, corruption, resumable, and completion
|
|
1224
|
-
outcomes.
|
|
1225
|
-
- `retention`: startup, scheduled, manual, commit, or post-commit outcomes,
|
|
1226
|
-
with `expiredSessions`, `scrubbed`, `ageDeleted`, `countDeleted`,
|
|
1227
|
-
`versionDeleted`, and `storageDeleted` counters.
|
|
1228
|
-
- `link`: one terminal manual-link outcome with `target`, `name`, client
|
|
1229
|
-
fingerprint, and status `linked`, `unchanged`, `rejected`, or `failed`.
|
|
1230
|
-
Rejected and failed outcomes also include a stable `reason`.
|
|
1231
|
-
- `failure`: stable failure code and peer fingerprint.
|
|
1232
|
-
- `listening`: local server-key fingerprint.
|
|
1233
|
-
- `close`: closed or failed outcome.
|
|
1234
|
-
|
|
1235
|
-
Client events:
|
|
1236
|
-
|
|
1237
|
-
- `connection`, `connection-open`, `connection-close`
|
|
1238
|
-
- `offer`: offered, accepted, resumed, reset, rejected, or already committed.
|
|
1239
|
-
- `progress`: cumulative TAR `bytesSent` and full `totalBytes`, including a
|
|
1240
|
-
durable resume offset.
|
|
1241
|
-
- `verification`, `commit`
|
|
1242
|
-
- `result`: terminal upload result with artifact `kind`.
|
|
1243
|
-
- `link`: after the request is written, `requested` with `final: false`,
|
|
1244
|
-
followed by exactly one `LINKED`, `UNCHANGED`, or stable error code with
|
|
1245
|
-
`final: true`. A request-write failure after connection can emit only the
|
|
1246
|
-
terminal failed event. Local argument validation, an already closed or
|
|
1247
|
-
cancelled client, and connection failure can reject before any `link` event.
|
|
1248
|
-
- `failure`, `close`
|
|
1249
|
-
|
|
1250
|
-
Use the exported `ServerEventMap`, `ClientEventMap`, `ServerEventName`, and
|
|
1251
|
-
`ClientEventName` types for event-name-specific payload narrowing:
|
|
1252
|
-
|
|
1253
|
-
```ts
|
|
1254
|
-
client.on('progress', ({ name, bytesSent, totalBytes }) => {
|
|
1255
|
-
console.log(name, `${bytesSent}/${totalBytes}`)
|
|
1256
|
-
})
|
|
1257
|
-
|
|
1258
|
-
server.on('recovery', (event) => {
|
|
1259
|
-
if (event.status === 'failed') console.error(event.phase, event.reason)
|
|
1260
|
-
})
|
|
1261
|
-
```
|
|
1262
|
-
|
|
1263
|
-
### Logging
|
|
1264
|
-
|
|
1265
|
-
Both constructors accept:
|
|
1266
|
-
|
|
1267
|
-
```ts
|
|
1268
|
-
interface Logger {
|
|
1269
|
-
info?(message: string, details?: Record<string, unknown>): void
|
|
1270
|
-
warn?(message: string, details?: Record<string, unknown>): void
|
|
1271
|
-
error?(message: string, details?: Record<string, unknown>): void
|
|
1272
|
-
}
|
|
1273
|
-
```
|
|
1274
|
-
|
|
1275
|
-
Logger exceptions are ignored. Logger detail objects are diagnostic rather than
|
|
1276
|
-
a stable ingestion schema; use typed event payloads or `SwarmDeployError.code`
|
|
1277
|
-
for automation. Do not use fingerprints as credentials.
|
|
1278
|
-
|
|
1279
|
-
### Errors
|
|
1280
|
-
|
|
1281
|
-
Configuration, authentication, protocol, transfer, and managed-storage failures
|
|
1282
|
-
generally reject with `SwarmDeployError`. Raw operating-system or adapter errors
|
|
1283
|
-
may propagate while selecting or opening a local input, initializing or locking
|
|
1284
|
-
the server storage root, or performing filesystem operations:
|
|
1285
|
-
|
|
1286
|
-
```ts
|
|
1287
|
-
import { ERRORS, SwarmDeployError } from 'swarm-deploy'
|
|
1288
|
-
|
|
1289
|
-
try {
|
|
1290
|
-
await client.upload('./artifact.tgz')
|
|
1291
|
-
} catch (error) {
|
|
1292
|
-
if (error instanceof SwarmDeployError) {
|
|
1293
|
-
console.error(error.code, error.message)
|
|
1294
|
-
if (error.code === ERRORS.CONNECT_TIMEOUT) {
|
|
1295
|
-
// Server could not be reached and authenticated before the deadline.
|
|
1296
|
-
}
|
|
1297
|
-
} else {
|
|
1298
|
-
console.error(error)
|
|
1299
|
-
}
|
|
1300
|
-
}
|
|
1301
|
-
```
|
|
1302
|
-
|
|
1303
|
-
`HOOK_FAILED` is the stable code for a failed `beforeCommit` or `afterCommit`
|
|
1304
|
-
callback. Its message is fixed and the original exception is available only as
|
|
1305
|
-
`cause` and to `onFailure`.
|
|
1306
|
-
|
|
1307
|
-
Stable codes include authentication and server-key rejection, invalid
|
|
1308
|
-
configuration and protocol records, file and staging limits, disk reserve,
|
|
1309
|
-
filename and replacement conflicts, checksum failures, connection or upload
|
|
1310
|
-
timeouts, aborts, commit failures, cleanup failures, and managed-link failures
|
|
1311
|
-
(`LINK_NOT_ALLOWED`, `LINK_TARGET_NOT_FOUND`, `LINK_CONFLICT`, `LINK_FAILED`,
|
|
1312
|
-
`UNSUPPORTED_STORAGE`). Manual requests use `LINK_NOT_ALLOWED` for policy,
|
|
1313
|
-
self-link, and automatic-name rejection; `LINK_TARGET_NOT_FOUND` for an absent
|
|
1314
|
-
current managed target; `LINK_CONFLICT` for managed-namespace or unmanaged-path
|
|
1315
|
-
collisions; and `LINK_FAILED` for other link-operation failures. A link
|
|
1316
|
-
reconciliation failure after a durable commit behaves like other post-commit
|
|
1317
|
-
deployment failures: the artifact stays committed, the client receives a
|
|
1318
|
-
stable failure, and an already-committed retry reruns reconciliation before
|
|
1319
|
-
succeeding. Import `ERRORS` rather than matching exception messages.
|
|
1320
|
-
|
|
1321
|
-
## Production operations
|
|
1322
|
-
|
|
1323
|
-
- Run the server under a dedicated non-root account.
|
|
1324
|
-
- Restrict the server seed and storage root to that account.
|
|
1325
|
-
- Supervise the process and wait for the final `ready` line before marking it
|
|
1326
|
-
healthy.
|
|
1327
|
-
- Upgrade servers before clients; see
|
|
1328
|
-
[Rollout and compatibility](#rollout-and-compatibility).
|
|
1329
|
-
- Restart with the same seed, allowlist, limits, replacement names, artifact
|
|
1330
|
-
patterns, rotation limits, hooks module, and storage root so interrupted sessions and commit journals can recover.
|
|
1331
|
-
- Alert on nonzero CLI exits and failed authentication, recovery, verification,
|
|
1332
|
-
commit, retention, and cleanup events.
|
|
1333
|
-
- Never edit `.swarm-deploy/` while the server is running.
|
|
1334
|
-
- Stop the server cleanly before backing up or restoring the complete storage
|
|
1335
|
-
root.
|
|
1336
|
-
- Publish stored artifacts through a separately configured artifact service or
|
|
1337
|
-
web server.
|
|
1338
|
-
- Roll out an exact package version to a canary before wider deployment.
|
|
1339
|
-
|
|
1340
|
-
See [SECURITY.md](SECURITY.md) for vulnerability reporting and operator
|
|
1341
|
-
precautions.
|
|
1342
|
-
|
|
1343
|
-
## Contributor development
|
|
1344
|
-
|
|
1345
|
-
Production and tests are strict TypeScript. Build output is generated under
|
|
1346
|
-
untracked `dist/` and `.test-dist/`. Tests generate binary payloads in temporary
|
|
1347
|
-
directories and use an isolated local HyperDHT testnet, not the public DHT.
|
|
1348
|
-
|
|
1349
|
-
```sh
|
|
1350
|
-
npm install
|
|
1351
|
-
npm run build
|
|
1352
|
-
npm run build:test
|
|
1353
|
-
npm run test:types
|
|
1354
|
-
npm run format:check
|
|
1355
|
-
npm run lint
|
|
1356
|
-
npm run test:node
|
|
1357
|
-
npm run test:bare
|
|
1358
|
-
npm run test:property
|
|
1359
|
-
npm run test:package
|
|
1360
|
-
npm run test:release-tag
|
|
1361
|
-
```
|
|
1362
|
-
|
|
1363
|
-
Exercise the built CLI directly:
|
|
1364
|
-
|
|
1365
|
-
```sh
|
|
1366
|
-
node dist/bin/swarm-deploy.js --help
|
|
1367
|
-
bare dist/bin/swarm-deploy.js --help
|
|
1368
|
-
```
|
|
1369
|
-
|
|
1370
|
-
## Release
|
|
1371
|
-
|
|
1372
|
-
Version tags use `v<package-version>`. The tag workflow validates the version,
|
|
1373
|
-
builds the untracked distribution, checks package contents and types, and
|
|
1374
|
-
publishes through the configured npm environment with provenance.
|
|
1375
|
-
|
|
1376
|
-
See [RELEASING.md](RELEASING.md) for the release and rollback procedure.
|
|
1377
|
-
|
|
1378
|
-
## Protocol specification
|
|
1379
|
-
|
|
1380
|
-
See [docs/spec/swarm-deploy.md](docs/spec/swarm-deploy.md) for the complete
|
|
1381
|
-
transport, deterministic TAR, resumability, storage, replacement, recovery,
|
|
1382
|
-
retention, threat-model, and package requirements.
|
|
167
|
+
## Documentation
|
|
168
|
+
|
|
169
|
+
The detailed guides live in `docs/` in the repository and are not included in
|
|
170
|
+
the npm package, so these links point to GitHub.
|
|
171
|
+
|
|
172
|
+
- [CLI reference](https://github.com/tetherto/swarm-deploy/blob/main/docs/cli.md):
|
|
173
|
+
every command and option (`keygen`, `public-key`, `server`, `upload`, `link`,
|
|
174
|
+
`reset-storage`), seed sources, exit codes, and a
|
|
175
|
+
[CI uploader example](https://github.com/tetherto/swarm-deploy/blob/main/docs/cli.md#ci-uploader-example).
|
|
176
|
+
- [Runtime API](https://github.com/tetherto/swarm-deploy/blob/main/docs/api.md):
|
|
177
|
+
identity helpers, `Server`, `Client`, link and upload results, events,
|
|
178
|
+
logging, and errors.
|
|
179
|
+
- [Operations, storage, and rollback](https://github.com/tetherto/swarm-deploy/blob/main/docs/operations.md):
|
|
180
|
+
the production checklist, transfer and resume behavior, storage, replacement,
|
|
181
|
+
and retention, resetting a storage root, and rollout, compatibility, and
|
|
182
|
+
rollback.
|
|
183
|
+
- [Artifact patterns and rotation](https://github.com/tetherto/swarm-deploy/blob/main/docs/artifact-patterns.md):
|
|
184
|
+
pattern templates, source parent, mandatory matching, and count and version
|
|
185
|
+
rotation.
|
|
186
|
+
- [Managed symlinks](https://github.com/tetherto/swarm-deploy/blob/main/docs/managed-links.md):
|
|
187
|
+
automatic and manual links, ownership rules, removal, and the promotion
|
|
188
|
+
workflow.
|
|
189
|
+
- [Deployment hooks](https://github.com/tetherto/swarm-deploy/blob/main/docs/hooks.md):
|
|
190
|
+
lifecycle points, invocation sequences, link publication, `onFailure`, and
|
|
191
|
+
hook modules.
|
|
192
|
+
- [Protocol specification](https://github.com/tetherto/swarm-deploy/blob/main/docs/spec/swarm-deploy.md):
|
|
193
|
+
the complete transport, deterministic TAR, resumability, storage,
|
|
194
|
+
replacement, recovery, retention, threat-model, and package requirements.
|
|
195
|
+
|
|
196
|
+
## Examples
|
|
197
|
+
|
|
198
|
+
[`examples/link-publisher`](https://github.com/tetherto/swarm-deploy/tree/main/examples/link-publisher)
|
|
199
|
+
is a runnable, Node-only `afterLink` hooks module and publisher. See
|
|
200
|
+
[Link publication hooks](https://github.com/tetherto/swarm-deploy/blob/main/docs/hooks.md#link-publication-hooks)
|
|
201
|
+
for how it behaves.
|
|
202
|
+
|
|
203
|
+
## Project documents
|
|
204
|
+
|
|
205
|
+
- [Contributing](https://github.com/tetherto/swarm-deploy/blob/main/docs/contributing.md):
|
|
206
|
+
build, test, and lint commands.
|
|
207
|
+
- [RELEASING.md](https://github.com/tetherto/swarm-deploy/blob/main/RELEASING.md):
|
|
208
|
+
the release and rollback procedure. Version tags use `v<package-version>`.
|
|
209
|
+
- [SECURITY.md](https://github.com/tetherto/swarm-deploy/blob/main/SECURITY.md):
|
|
210
|
+
vulnerability reporting and operator precautions.
|
|
211
|
+
- [CHANGELOG.md](https://github.com/tetherto/swarm-deploy/blob/main/CHANGELOG.md):
|
|
212
|
+
release history.
|
|
213
|
+
- [LICENSE.md](https://github.com/tetherto/swarm-deploy/blob/main/LICENSE.md):
|
|
214
|
+
Apache-2.0.
|