swarm-deploy 0.0.0 → 0.2.0

package/CHANGELOG.md

@@ -0,0 +1,76 @@
+# Changelog
+
+All notable changes to this package are documented here.
+
+## 0.2.0
+
+- **Behavior change.** A directory upload now commits exactly one recursive managed directory artifact instead of a batch of immediate-child file uploads. `BatchUploadResult`, `BatchUploadFailure`, `SkippedUploadEntry`, `SkippedUploadReason`, and the client `skipped` event are removed, `ClientUploadResult` is now `UploadResult`, and `UploadResult` gains `kind` and, for a directory, `entryCount`. Direct file upload stays byte-compatible. Automation that relied on per-child results must now treat the directory as one artifact.
+- Add recursive directory artifacts: a canonical deterministic multi-entry USTAR archive with normalized file mode `0644`, directory mode `0755`, fixed uid, gid, mtime, uname, and gname; bytewise entry ordering with parents before children; empty directories preserved; and symlinks, hardlinks, devices, sockets, FIFOs, cycles, traversal, duplicate, and case-fold-colliding paths rejected. Limits are 100 UTF-8 bytes per component, 100 UTF-8 bytes per stored TAR name, depth 32, 10,000 entries, `maxFileBytes` aggregate payload, and `maxStagingBytes` aggregate staging.
+- Add directory offer metadata with `kind: 'directory'`, `entryCount`, `payloadBytes`, and a canonical `treeSha256`, and the `swarm-deploy/direct-tree/v1` transfer-ID domain. The key set is disjoint from the file record's, so an older server rejects it without mutating storage. Upgrade servers before clients.
+- Add directory sessions at persisted version 4, verified into `.swarm-deploy/staging/<transfer-id>.tree/` by rebuilding and byte-comparing every canonical header and independently recomputing the tree digest. File sessions keep versions 2 and 3.
+- Add directory commit records at version 3 and directory journals at version 3. A directory is published by renaming its verified staging tree under the name and root leases, with explicit recovery for crashes before the rename, after the rename, after the sidecar, and during symlink reconciliation.
+- Directory artifacts are create-only. A directory offer for a configured `replaceNames` entry, a file-to-directory kind change, and a directory-to-file kind change are all rejected. Directory replacement and history are deferred because a directory cannot use the file hardlink replacement transaction safely.
+- Delete a managed directory by renaming it into `.swarm-deploy/trash/<transfer-id>.tree` and then removing it recursively with `lstat`, `unlink`, and `rmdir` without following symlinks. Startup sweeps proven trash residue.
+- Make storage scrubbing kind aware: scheduled and pre-commit passes validate type, sidecar, and root identity, while startup recovery recursively hashes a directory artifact and verifies its canonical tree digest. A mutated managed directory is preserved as unknown and its invalid metadata removed.
+- Add server-managed symlinks: `ServerOptions.symlinks` and the repeatable two-value CLI option `--symlink <selector> <link-name>`. A selector that begins and ends with `/` is an unflagged regular expression matched against managed artifact basenames; anything else is an exact managed basename. Selection uses only validated commit records and picks the newest `committedAt`, tie-broken by transfer ID then name. A rule is dormant until its target exists and replacement history names never match.
+- Persist managed-link ownership in `.swarm-deploy/links/<sha256(link-name)>.json`. A destination is replaced only when a valid record exists, the destination is a symbolic link, and `readlink()` returns the recorded or desired target. A pre-existing file, directory, unrecorded symlink, changed symlink, or foreign ownership record is unmanaged: it is never replaced, moved, or deleted, and the server fails closed. Reconciliation is level-triggered from startup, commit, already-committed retry, recovery, and retention.
+- Reject an upload whose artifact name equals a configured symlink name before session admission, so a configured link never masks a managed artifact.
+- Pin every selected symlink target against age, count, SemVer, and quota retention. Pinned targets still count toward the quota, and the desired-link set is recomputed under the root lease before any deletion, so a repoint always precedes the old target becoming eligible. Removing a rule removes its ownership record and visible link only while ownership is still proven.
+- Add optional `StorageAdapter.symlink` and `readlink`. Configuring symlink rules requires both; the Node and Bare default adapters provide them, and a custom adapter without symlink rules stays source-compatible.
+- Add `HookArtifact.kind` and optional `entryCount`, and the artifact kind to server transfer, offer, progress, and client result events. For a directory, `beforeCommit.path` is the verified `.tree` staging directory and `afterCommit.path` is the final committed directory; configured links are reconciled before `afterCommit`, so a hook observes the new link.
+- Add the `LINK_CONFLICT`, `LINK_FAILED`, and `UNSUPPORTED_STORAGE` error codes. A link reconciliation failure after a durable commit behaves like other post-commit deployment failures: the artifact stays committed, the client receives a stable failure, and an already-committed retry reruns reconciliation before succeeding.
+- Add the private `.swarm-deploy/links` and `.swarm-deploy/trash` directories, protected, identity-checked, and synchronized like the existing internal directories.
+- Directory metadata, session version 4, commit record version 3, directory journals, and directory trash are not downgrade-compatible. Drain uploads before upgrading and do not roll back after committing a directory artifact without restoring from backup.
+- The package exposes no user-facing "atomic directory" term or option; atomicity is an internal commit and recovery guarantee.
+- Add server-configured artifact patterns (`artifactPatterns`, repeatable `--artifact-pattern`) with `{series}` and strict SemVer `{version}` placeholders over basenames and the immediate source parent. Configuring patterns makes matching mandatory: unmatched offers are rejected with `INVALID_FILENAME` before admission.
+- Give a template without `{series}` the fixed series key `fixed-<lowercase hex SHA-256 of the template text>`. The previous key was the raw template, whose braces and slashes failed commit-record validation and made every version-only pattern fail at runtime.
+- Reject an artifact pattern whose literal text still contains `{` or `}` after the exact `{series}` and `{version}` placeholders are removed.
+- Add optional authenticated `sourceParent` offer metadata, sent only when it is a safe single component. Upgrade servers before clients: older servers reject the new metadata field.
+- Add `ClientOptions.includeSourceParent` (default `true`) and the valueless upload CLI flag `--no-source-parent` to suppress `sourceParent` where the staging folder name is itself sensitive. An opted-out upload keeps the legacy transfer identity and is rejected by server patterns that need a parent segment.
+- Persist normalized release coordinates in commit records, replacement history, journals, and recovery. Legacy records without coordinates are not count or version rotated.
+- Add count rotation (`maxCount`, `--max-count`) and SemVer major/minor rotation (`maxVersions`, `versionGranularity`, `--max-versions`, `--version-granularity`) running after age and before storage-quota retention, with `countDeleted` and `versionDeleted` retention counters.
+- Add `beforeCommit`, `afterCommit`, and `onFailure` server hooks (`ServerOptions.hooks` and `--hooks <module>` for `.js`, `.mjs`, and `.cjs` modules), the `HOOK_FAILED` error code, and exported hook and release types. Hooks run again on retries and must be idempotent by transfer ID; `afterCommit` can fail after the artifact is durably committed.
+- Defer post-commit rotation until a configured `afterCommit` hook succeeds. The final path exists when `afterCommit` begins in the sequential server flow, but a concurrent commit's retention pass, a scheduled pass, or a manual pass can remove it while the callback runs, so hooks needing stable bytes should open or copy it promptly. The deferred pass runs before `COMMITTED` or `ALREADY_COMMITTED`, so a slow retention pass can delay the reply. Servers without `afterCommit` rotate immediately as before.
+- Run the deferred post-commit retention pass from an already-committed offer only for a transfer this process committed whose `afterCommit` has not yet succeeded. Other duplicate offers still invoke `afterCommit` but start no retention pass, so they cannot force repeated full scans. The owed-transfer set is in memory only and is cleared by server close.
+- Reject a `--hooks` path whose extension is not exactly `.js`, `.mjs`, or `.cjs` before the module is loaded, so an unsupported extension gives a precise startup error instead of a runtime loader message.
+- **Behavior change.** Give an authenticated transfer ID a single in-flight owner. A second connection offering a transfer another connection already owns is rejected with `FILE_BUSY` during the offer phase, so staging, commit, `beforeCommit`, and `afterCommit` never run concurrently for one transfer identity. Sequential retries, resumes, and verified reconnects are unaffected, but automation that uploads one artifact from several workers at once must now treat `FILE_BUSY` as "retry once the in-flight attempt finishes". The guard is per process, bounded by the connection limit, and released when the connection ends or the server closes.
+- Release the transfer guard before awaiting `onFailure`, so a slow or hung failure callback cannot reject the client's legitimate retry. `onFailure` is observational and may therefore overlap the next attempt's lifecycle hooks; each connection still reports at most once, and the serialization of `beforeCommit`/`afterCommit` per transfer is unchanged.
+- Bound the owed post-commit retention set at 1024 transfers, matching the connection ceiling, evicting the oldest owed transfer first. An evicted transfer keeps its committed artifact and leaves rotation to the next startup, scheduled, or commit-triggered pass.
+- Reserve the derived fixed-series namespace: a captured `{series}` of exactly `fixed-` plus 64 lowercase hex characters never matches, so an uploaded filename cannot place itself in a version-only pattern's rotation group.
+- Export `fixedSeriesKey` from the root API so operators can map a series in a commit sidecar or log line back to the template that produced it.
+- Retrying a failed `afterCommit` normally takes the `ALREADY_COMMITTED` path, but there is no persistent hook-pending marker: any intervening retention pass (startup after a restart, scheduled or manual cleanup, or a concurrent commit's retention) can remove an out-of-window artifact first, and the retry is then a fresh upload.
+- Write a resumable session at on-disk version 3 only when it carries a `sourceParent`, and at version 2 otherwise. Both are readable. A downgrade therefore loses the resumability of parent-bearing sessions only, instead of every session written by this version; draining in-flight uploads before a rollback still avoids it.
+- Never rewrite the durable release identity of a commit record. Enabling or changing `artifactPatterns` while uploads are in flight makes a retry of already-committed bytes fail closed (`FILE_EXISTS` for a create-only name, a release-identity conflict for a replaceable name retried under the same transfer ID), so drain in-flight uploads before changing patterns.
+- Commit-record compatibility is asserted only in the reading direction: records without release coordinates are read and excluded from count and version rotation. Whether older code tolerates records this version writes is untested.
+- Document that a hung hook callback on a fresh or resumed upload also holds an active-upload slot and its staging reservation, so enough hung callbacks exhaust upload capacity until the server closes. Bounded timeout and anti-spam controls are tracked in issue #8.
+- Document that an `offer`-phase `onFailure` context is built before the transfer ID is authenticated and is delivered even when that authentication failed, so its artifact fields must not be used as audit or idempotency keys.
+- Document rollout, retry, path, and compatibility semantics.
+
+## 0.1.0
+
+- Initial public release.
+- Publish under the unscoped `swarm-deploy` npm package name.
+- Add authenticated, encrypted direct-HyperDHT uploads pinned to the server public key and a static server allowlist.
+- Add bounded one-file and immediate-directory uploads with canonical TAR verification and TAR-byte-offset resumable staging.
+- Add crash-safe create-only commits, startup recovery, storage scrubbing, and retention cleanup.
+- Add opt-in atomic replacement for exact configured names with retained version history and v2 journal recovery.
+- Add Node.js and Bare APIs, CLI identity provisioning, structured lifecycle events, and safe exit codes.
+- Accept canonical 64-character seed strings in runtime constructors and through
+  the CLI `--seed` option, alongside seed files and role-specific environment
+  variables.
+- Wrap arbitrary regular binary inputs in byte-stable canonical one-entry USTAR
+  archives automatically.
+- Reserve direct `history-*` inputs and report batch children as `reserved-history` skips.
+- Reclaim expired inactive staging before offer admission while preserving active uploads.
+- Add stable `ACTIVE_UPLOAD_LIMIT` and `CONNECT_TIMEOUT` errors.
+- Require an explicit terminal result; EOF and close are not success.
+- Keep recovery logs on 12-character transfer fingerprints and narrow the supported root API to Client/Server usage.
+- Pin the release publish action to the reviewed immutable Holepunch actions revision.
+- Release a session's staging reservation even when its files cannot be unlinked, so a failed cleanup no longer exhausts staging capacity or leaves the name permanently busy.
+- Keep expired-session sweeps from aborting on the first failure, so one unremovable session no longer rejects every later upload; the count is exposed as `strandedSessions`.
+- Report a peer that half-closes mid-phase as a typed `PROTOCOL_INVALID` truncation instead of an untyped runtime error.
+- Report an unsafe staging path as a typed error instead of dereferencing a null error cause.
+- Validate received archives against the canonical framing directly and take the payload by offset, removing the redundant second parse and its unreachable entry checks.
+- Consolidate the canonical USTAR constants, header, and size math into one module, with a test pinning the header against the packer.
+- Keep a failed post-commit session cleanup from reporting a durably committed upload as a failure.
+- Share one null-safe error-code helper instead of seven private copies.

package/LICENSE.md

@@ -0,0 +1,201 @@
+Apache License
+Version 2.0, January 2004
+http://www.apache.org/licenses/
+
+TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+1. Definitions.
+
+"License" shall mean the terms and conditions for use, reproduction,
+and distribution as defined by Sections 1 through 9 of this document.
+
+"Licensor" shall mean the copyright owner or entity authorized by
+the copyright owner that is granting the License.
+
+"Legal Entity" shall mean the union of the acting entity and all
+other entities that control, are controlled by, or are under common
+control with that entity. For the purposes of this definition,
+"control" means (i) the power, direct or indirect, to cause the
+direction or management of such entity, whether by contract or
+otherwise, or (ii) ownership of fifty percent (50%) or more of the
+outstanding shares, or (iii) beneficial ownership of such entity.
+
+"You" (or "Your") shall mean an individual or Legal Entity
+exercising permissions granted by this License.
+
+"Source" form shall mean the preferred form for making modifications,
+including but not limited to software source code, documentation
+source, and configuration files.
+
+"Object" form shall mean any form resulting from mechanical
+transformation or translation of a Source form, including but
+not limited to compiled object code, generated documentation,
+and conversions to other media types.
+
+"Work" shall mean the work of authorship, whether in Source or
+Object form, made available under the License, as indicated by a
+copyright notice that is included in or attached to the work
+(an example is provided in the Appendix below).
+
+"Derivative Works" shall mean any work, whether in Source or Object
+form, that is based on (or derived from) the Work and for which the
+editorial revisions, annotations, elaborations, or other modifications
+represent, as a whole, an original work of authorship. For the purposes
+of this License, Derivative Works shall not include works that remain
+separable from, or merely link (or bind by name) to the interfaces of,
+the Work and Derivative Works thereof.
+
+"Contribution" shall mean any work of authorship, including
+the original version of the Work and any modifications or additions
+to that Work or Derivative Works thereof, that is intentionally
+submitted to Licensor for inclusion in the Work by the copyright owner
+or by an individual or Legal Entity authorized to submit on behalf of
+the copyright owner. For the purposes of this definition, "submitted"
+means any form of electronic, verbal, or written communication sent
+to the Licensor or its representatives, including but not limited to
+communication on electronic mailing lists, source code control systems,
+and issue tracking systems that are managed by, or on behalf of, the
+Licensor for the purpose of discussing and improving the Work, but
+excluding communication that is conspicuously marked or otherwise
+designated in writing by the copyright owner as "Not a Contribution."
+
+"Contributor" shall mean Licensor and any individual or Legal Entity
+on behalf of whom a Contribution has been received by Licensor and
+subsequently incorporated within the Work.
+
+2. Grant of Copyright License. Subject to the terms and conditions of
+   this License, each Contributor hereby grants to You a perpetual,
+   worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+   copyright license to reproduce, prepare Derivative Works of,
+   publicly display, publicly perform, sublicense, and distribute the
+   Work and such Derivative Works in Source or Object form.
+
+3. Grant of Patent License. Subject to the terms and conditions of
+   this License, each Contributor hereby grants to You a perpetual,
+   worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+   (except as stated in this section) patent license to make, have made,
+   use, offer to sell, sell, import, and otherwise transfer the Work,
+   where such license applies only to those patent claims licensable
+   by such Contributor that are necessarily infringed by their
+   Contribution(s) alone or by combination of their Contribution(s)
+   with the Work to which such Contribution(s) was submitted. If You
+   institute patent litigation against any entity (including a
+   cross-claim or counterclaim in a lawsuit) alleging that the Work
+   or a Contribution incorporated within the Work constitutes direct
+   or contributory patent infringement, then any patent licenses
+   granted to You under this License for that Work shall terminate
+   as of the date such litigation is filed.
+
+4. Redistribution. You may reproduce and distribute copies of the
+   Work or Derivative Works thereof in any medium, with or without
+   modifications, and in Source or Object form, provided that You
+   meet the following conditions:
+
+(a) You must give any other recipients of the Work or
+Derivative Works a copy of this License; and
+
+(b) You must cause any modified files to carry prominent notices
+stating that You changed the files; and
+
+(c) You must retain, in the Source form of any Derivative Works
+that You distribute, all copyright, patent, trademark, and
+attribution notices from the Source form of the Work,
+excluding those notices that do not pertain to any part of
+the Derivative Works; and
+
+(d) If the Work includes a "NOTICE" text file as part of its
+distribution, then any Derivative Works that You distribute must
+include a readable copy of the attribution notices contained
+within such NOTICE file, excluding those notices that do not
+pertain to any part of the Derivative Works, in at least one
+of the following places: within a NOTICE text file distributed
+as part of the Derivative Works; within the Source form or
+documentation, if provided along with the Derivative Works; or,
+within a display generated by the Derivative Works, if and
+wherever such third-party notices normally appear. The contents
+of the NOTICE file are for informational purposes only and
+do not modify the License. You may add Your own attribution
+notices within Derivative Works that You distribute, alongside
+or as an addendum to the NOTICE text from the Work, provided
+that such additional attribution notices cannot be construed
+as modifying the License.
+
+You may add Your own copyright statement to Your modifications and
+may provide additional or different license terms and conditions
+for use, reproduction, or distribution of Your modifications, or
+for any such Derivative Works as a whole, provided Your use,
+reproduction, and distribution of the Work otherwise complies with
+the conditions stated in this License.
+
+5. Submission of Contributions. Unless You explicitly state otherwise,
+   any Contribution intentionally submitted for inclusion in the Work
+   by You to the Licensor shall be under the terms and conditions of
+   this License, without any additional terms or conditions.
+   Notwithstanding the above, nothing herein shall supersede or modify
+   the terms of any separate license agreement you may have executed
+   with Licensor regarding such Contributions.
+
+6. Trademarks. This License does not grant permission to use the trade
+   names, trademarks, service marks, or product names of the Licensor,
+   except as required for reasonable and customary use in describing the
+   origin of the Work and reproducing the content of the NOTICE file.
+
+7. Disclaimer of Warranty. Unless required by applicable law or
+   agreed to in writing, Licensor provides the Work (and each
+   Contributor provides its Contributions) on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+   implied, including, without limitation, any warranties or conditions
+   of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+   PARTICULAR PURPOSE. You are solely responsible for determining the
+   appropriateness of using or redistributing the Work and assume any
+   risks associated with Your exercise of permissions under this License.
+
+8. Limitation of Liability. In no event and under no legal theory,
+   whether in tort (including negligence), contract, or otherwise,
+   unless required by applicable law (such as deliberate and grossly
+   negligent acts) or agreed to in writing, shall any Contributor be
+   liable to You for damages, including any direct, indirect, special,
+   incidental, or consequential damages of any character arising as a
+   result of this License or out of the use or inability to use the
+   Work (including but not limited to damages for loss of goodwill,
+   work stoppage, computer failure or malfunction, or any and all
+   other commercial damages or losses), even if such Contributor
+   has been advised of the possibility of such damages.
+
+9. Accepting Warranty or Additional Liability. While redistributing
+   the Work or Derivative Works thereof, You may choose to offer,
+   and charge a fee for, acceptance of support, warranty, indemnity,
+   or other liability obligations and/or rights consistent with this
+   License. However, in accepting such obligations, You may act only
+   on Your own behalf and on Your sole responsibility, not on behalf
+   of any other Contributor, and only if You agree to indemnify,
+   defend, and hold each Contributor harmless for any liability
+   incurred by, or claims asserted against, such Contributor by reason
+   of your accepting any such warranty or additional liability.
+
+END OF TERMS AND CONDITIONS
+
+APPENDIX: How to apply the Apache License to your work.
+
+To apply the Apache License to your work, attach the following
+boilerplate notice, with the fields enclosed by brackets "[]"
+replaced with your own identifying information. (Don't include
+the brackets!) The text should be enclosed in the appropriate
+comment syntax for the file format. We also recommend that a
+file or class name and description of purpose be included on the
+same "printed page" as the copyright notice for easier
+identification within third-party archives.
+
+Copyright [yyyy] [name of copyright owner]
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.

package/NOTICE.md

@@ -0,0 +1,13 @@
+Copyright 2026 Tether Inc
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.