@openclaw/fs-safe 0.4.7 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +45 -0
- package/README.md +82 -43
- package/dist/archive-entry.d.ts.map +1 -1
- package/dist/archive-entry.js +5 -4
- package/dist/archive-errors.d.ts +11 -0
- package/dist/archive-errors.d.ts.map +1 -0
- package/dist/archive-errors.js +16 -0
- package/dist/archive-input.d.ts +12 -0
- package/dist/archive-input.d.ts.map +1 -0
- package/dist/archive-input.js +72 -0
- package/dist/archive-kind.d.ts +1 -1
- package/dist/archive-kind.d.ts.map +1 -1
- package/dist/archive-kind.js +18 -0
- package/dist/archive-limits.d.ts +10 -0
- package/dist/archive-limits.d.ts.map +1 -1
- package/dist/archive-limits.js +18 -0
- package/dist/archive-native.d.ts +18 -0
- package/dist/archive-native.d.ts.map +1 -0
- package/dist/archive-native.js +115 -0
- package/dist/archive-options.d.ts +21 -0
- package/dist/archive-options.d.ts.map +1 -0
- package/dist/archive-options.js +1 -0
- package/dist/archive-policy.d.ts +20 -0
- package/dist/archive-policy.d.ts.map +1 -0
- package/dist/archive-policy.js +29 -0
- package/dist/archive-read.d.ts +6 -0
- package/dist/archive-read.d.ts.map +1 -0
- package/dist/archive-read.js +200 -0
- package/dist/archive-staging.d.ts +2 -5
- package/dist/archive-staging.d.ts.map +1 -1
- package/dist/archive-staging.js +19 -10
- package/dist/archive-tar-meta.d.ts +6 -0
- package/dist/archive-tar-meta.d.ts.map +1 -0
- package/dist/archive-tar-meta.js +143 -0
- package/dist/archive-tar-runtime.d.ts +43 -0
- package/dist/archive-tar-runtime.d.ts.map +1 -0
- package/dist/archive-tar-runtime.js +8 -0
- package/dist/archive-tar.d.ts +5 -1
- package/dist/archive-tar.d.ts.map +1 -1
- package/dist/archive-tar.js +24 -6
- package/dist/archive-zip-entry.d.ts +15 -0
- package/dist/archive-zip-entry.d.ts.map +1 -0
- package/dist/archive-zip-entry.js +17 -0
- package/dist/archive.d.ts +7 -17
- package/dist/archive.d.ts.map +1 -1
- package/dist/archive.js +111 -108
- package/dist/atomic.d.ts +1 -1
- package/dist/atomic.d.ts.map +1 -1
- package/dist/config.d.ts +1 -1
- package/dist/config.d.ts.map +1 -1
- package/dist/config.js +1 -1
- package/dist/containment.d.ts +2 -0
- package/dist/containment.d.ts.map +1 -0
- package/dist/containment.js +1 -0
- package/dist/durability.d.ts +2 -0
- package/dist/durability.d.ts.map +1 -1
- package/dist/durability.js +2 -0
- package/dist/errors.d.ts +4 -1
- package/dist/errors.d.ts.map +1 -1
- package/dist/errors.js +2 -0
- package/dist/file-hash.d.ts +10 -0
- package/dist/file-hash.d.ts.map +1 -0
- package/dist/file-hash.js +73 -0
- package/dist/file-lock-sync.d.ts +35 -0
- package/dist/file-lock-sync.d.ts.map +1 -0
- package/dist/file-lock-sync.js +233 -0
- package/dist/file-lock.d.ts +2 -0
- package/dist/file-lock.d.ts.map +1 -1
- package/dist/file-lock.js +1 -0
- package/dist/filename.d.ts.map +1 -1
- package/dist/filename.js +4 -1
- package/dist/guarded-mkdir.d.ts +7 -1
- package/dist/guarded-mkdir.d.ts.map +1 -1
- package/dist/guarded-mkdir.js +45 -3
- package/dist/index.d.ts +4 -3
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +1 -1
- package/dist/json-document-store.d.ts.map +1 -1
- package/dist/json-document-store.js +30 -15
- package/dist/move-path.d.ts.map +1 -1
- package/dist/move-path.js +101 -17
- package/dist/native/darwin-arm64/fs-safe-native.node +0 -0
- package/dist/native/darwin-x64/fs-safe-native.node +0 -0
- package/dist/native/linux-arm64-gnu/fs-safe-native.node +0 -0
- package/dist/native/linux-arm64-musl/fs-safe-native.node +0 -0
- package/dist/native/linux-x64-gnu/fs-safe-native.node +0 -0
- package/dist/native/linux-x64-musl/fs-safe-native.node +0 -0
- package/dist/native/win32-x64-msvc/fs-safe-native.node +0 -0
- package/dist/native-binding.d.ts +79 -0
- package/dist/native-binding.d.ts.map +1 -0
- package/dist/native-binding.js +1 -0
- package/dist/native-config.d.ts +17 -0
- package/dist/native-config.d.ts.map +1 -0
- package/dist/native-config.js +71 -0
- package/dist/native-operations.d.ts +21 -0
- package/dist/native-operations.d.ts.map +1 -0
- package/dist/native-operations.js +115 -0
- package/dist/native-pinned-write.d.ts +5 -0
- package/dist/native-pinned-write.d.ts.map +1 -0
- package/dist/native-pinned-write.js +115 -0
- package/dist/native.d.ts +14 -0
- package/dist/native.d.ts.map +1 -0
- package/dist/native.js +204 -0
- package/dist/output-sibling.d.ts +8 -0
- package/dist/output-sibling.d.ts.map +1 -0
- package/dist/output-sibling.js +122 -0
- package/dist/output.d.ts +2 -0
- package/dist/output.d.ts.map +1 -1
- package/dist/output.js +32 -5
- package/dist/owner-dacl.d.ts +31 -0
- package/dist/owner-dacl.d.ts.map +1 -0
- package/dist/owner-dacl.js +27 -0
- package/dist/path-policy.js +1 -1
- package/dist/permissions-public.d.ts +2 -0
- package/dist/permissions-public.d.ts.map +1 -1
- package/dist/permissions-public.js +2 -0
- package/dist/permissions.d.ts.map +1 -1
- package/dist/permissions.js +6 -0
- package/dist/pinned-operation.d.ts +2 -0
- package/dist/pinned-operation.d.ts.map +1 -0
- package/dist/pinned-operation.js +35 -0
- package/dist/pinned-write.d.ts +3 -16
- package/dist/pinned-write.d.ts.map +1 -1
- package/dist/pinned-write.js +20 -85
- package/dist/private-directory.d.ts +5 -0
- package/dist/private-directory.d.ts.map +1 -0
- package/dist/private-directory.js +13 -0
- package/dist/private-temp-workspace.d.ts +6 -2
- package/dist/private-temp-workspace.d.ts.map +1 -1
- package/dist/private-temp-workspace.js +40 -13
- package/dist/publish-file-failure.d.ts +27 -0
- package/dist/publish-file-failure.d.ts.map +1 -0
- package/dist/publish-file-failure.js +21 -0
- package/dist/publish-file.d.ts +21 -0
- package/dist/publish-file.d.ts.map +1 -0
- package/dist/publish-file.js +351 -0
- package/dist/read-opened-file.d.ts +3 -0
- package/dist/read-opened-file.d.ts.map +1 -1
- package/dist/read-opened-file.js +1 -0
- package/dist/replace-file-copy-fallback.d.ts +34 -0
- package/dist/replace-file-copy-fallback.d.ts.map +1 -0
- package/dist/replace-file-copy-fallback.js +339 -0
- package/dist/replace-file.d.ts +6 -2
- package/dist/replace-file.d.ts.map +1 -1
- package/dist/replace-file.js +39 -90
- package/dist/root-context.d.ts +4 -1
- package/dist/root-context.d.ts.map +1 -1
- package/dist/root-context.js +20 -1
- package/dist/root-impl.d.ts +5 -0
- package/dist/root-impl.d.ts.map +1 -1
- package/dist/root-impl.js +128 -225
- package/dist/root-path-existing.d.ts +3 -0
- package/dist/root-path-existing.d.ts.map +1 -0
- package/dist/root-path-existing.js +69 -0
- package/dist/root-path.d.ts +1 -2
- package/dist/root-path.d.ts.map +1 -1
- package/dist/root-path.js +67 -81
- package/dist/root-walk.d.ts +41 -0
- package/dist/root-walk.d.ts.map +1 -0
- package/dist/root-walk.js +114 -0
- package/dist/root.d.ts +2 -0
- package/dist/root.d.ts.map +1 -1
- package/dist/secret-file.d.ts +5 -2
- package/dist/secret-file.d.ts.map +1 -1
- package/dist/secret-file.js +20 -2
- package/dist/secret-read-async.d.ts +4 -0
- package/dist/secret-read-async.d.ts.map +1 -0
- package/dist/secret-read-async.js +95 -0
- package/dist/secret.d.ts +2 -1
- package/dist/secret.d.ts.map +1 -1
- package/dist/secret.js +2 -1
- package/dist/sibling-temp.d.ts.map +1 -1
- package/dist/sibling-temp.js +1 -0
- package/dist/sidecar-lock-handle.d.ts +20 -0
- package/dist/sidecar-lock-handle.d.ts.map +1 -0
- package/dist/sidecar-lock-handle.js +28 -0
- package/dist/sidecar-lock-policy.d.ts +10 -0
- package/dist/sidecar-lock-policy.d.ts.map +1 -0
- package/dist/sidecar-lock-policy.js +29 -0
- package/dist/sidecar-lock-reclaim.d.ts +22 -6
- package/dist/sidecar-lock-reclaim.d.ts.map +1 -1
- package/dist/sidecar-lock-reclaim.js +106 -20
- package/dist/sidecar-lock-types.d.ts +56 -0
- package/dist/sidecar-lock-types.d.ts.map +1 -0
- package/dist/sidecar-lock-types.js +1 -0
- package/dist/sidecar-lock.d.ts +2 -45
- package/dist/sidecar-lock.d.ts.map +1 -1
- package/dist/sidecar-lock.js +94 -55
- package/dist/temp-cleanup.d.ts +7 -1
- package/dist/temp-cleanup.d.ts.map +1 -1
- package/dist/temp-cleanup.js +35 -5
- package/dist/temp.d.ts +2 -1
- package/dist/temp.d.ts.map +1 -1
- package/dist/test-hooks.d.ts +3 -0
- package/dist/test-hooks.d.ts.map +1 -1
- package/dist/windows-permissions-native.d.ts +9 -0
- package/dist/windows-permissions-native.d.ts.map +1 -0
- package/dist/windows-permissions-native.js +31 -0
- package/docs/archive.md +131 -13
- package/docs/atomic.md +30 -1
- package/docs/config.md +51 -21
- package/docs/durability.md +211 -1
- package/docs/errors.md +36 -6
- package/docs/filename.md +0 -0
- package/docs/index.md +11 -8
- package/docs/install.md +27 -26
- package/docs/json-store.md +18 -3
- package/docs/migrating-to-0.5.md +196 -0
- package/docs/native-helper.md +85 -0
- package/docs/native.md +134 -0
- package/docs/output.md +37 -11
- package/docs/path.md +1 -1
- package/docs/permissions.md +84 -1
- package/docs/quickstart.md +5 -2
- package/docs/reading.md +4 -4
- package/docs/root.md +31 -15
- package/docs/secret-file.md +50 -1
- package/docs/security-model.md +29 -8
- package/docs/sidecar-lock.md +92 -7
- package/docs/temp.md +24 -1
- package/docs/test-hooks.md +22 -2
- package/docs/testing.md +18 -7
- package/docs/types.md +3 -1
- package/docs/walk.md +55 -0
- package/docs/writing.md +3 -3
- package/package.json +8 -2
- package/dist/pinned-helper.d.ts +0 -9
- package/dist/pinned-helper.d.ts.map +0 -1
- package/dist/pinned-helper.js +0 -19
- package/dist/pinned-path.d.ts +0 -7
- package/dist/pinned-path.d.ts.map +0 -1
- package/dist/pinned-path.js +0 -21
- package/dist/pinned-python-config.d.ts +0 -9
- package/dist/pinned-python-config.d.ts.map +0 -1
- package/dist/pinned-python-config.js +0 -38
- package/dist/pinned-python.d.ts +0 -12
- package/dist/pinned-python.d.ts.map +0 -1
- package/dist/pinned-python.js +0 -688
- package/docs/python-helper.md +0 -103
package/docs/archive.md
CHANGED
|
@@ -1,12 +1,11 @@
|
|
|
1
1
|
# Archive extraction
|
|
2
2
|
|
|
3
|
-
`@openclaw/fs-safe/archive` extracts ZIP and TAR archives behind one API, with traversal checks, blocked-link-type rejection, and entry-count and byte budgets.
|
|
3
|
+
`@openclaw/fs-safe/archive` extracts ZIP and TAR archives behind one API, with traversal checks, blocked-link-type rejection, and entry-count and byte budgets. When the bundled native binding is available for the current platform, Rust streams ZIP, TAR, gzip, zstd, and bzip2 while TypeScript remains the sole policy owner; every accepted output is created fd-relative in a private staging root. Extraction then merges through the same safe-open boundary used by direct writes — a symlinked entry can't trick the merge into following an out-of-tree path.
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
for TAR.
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
installed.
|
|
5
|
+
The guarded JavaScript fallback uses optional runtime dependencies: `jszip` for
|
|
6
|
+
ZIP and `tar` for TAR/gzip. The native path does not use those packages. Installs
|
|
7
|
+
that omit optional dependencies can still import this subpath and use the
|
|
8
|
+
native path or pure path/limit helpers.
|
|
10
9
|
|
|
11
10
|
Some package managers and CI installs skip optional dependencies
|
|
12
11
|
(`--no-optional`, `--omit=optional`, or equivalent). If an archive helper throws
|
|
@@ -26,11 +25,16 @@ await extractArchive({
|
|
|
26
25
|
kind: "zip", // optional; resolveArchiveKind() can infer
|
|
27
26
|
timeoutMs: 15_000, // hard ceiling for the whole extraction
|
|
28
27
|
stripComponents: 0, // tar-style strip-leading-dirs
|
|
28
|
+
entryModes: "clamp", // default; use "preserve" for archive rwx bits
|
|
29
|
+
entryFilter: ({ path, kind, size }) => "extract",
|
|
30
|
+
onFiltered: "reject-archive", // default; opt into "skip-entry" explicitly
|
|
29
31
|
limits: {
|
|
30
32
|
maxArchiveBytes: 256 * 1024 * 1024,
|
|
31
33
|
maxEntries: 50_000,
|
|
32
34
|
maxExtractedBytes: 512 * 1024 * 1024,
|
|
33
35
|
maxEntryBytes: 256 * 1024 * 1024,
|
|
36
|
+
maxMetaEntryBytes: 1024 * 1024,
|
|
37
|
+
maxEntryPathComponents: 256,
|
|
34
38
|
},
|
|
35
39
|
});
|
|
36
40
|
```
|
|
@@ -42,14 +46,67 @@ type ExtractArchiveParams = {
|
|
|
42
46
|
archivePath: string; // absolute path to the archive
|
|
43
47
|
destDir: string; // absolute destination directory; must already exist
|
|
44
48
|
timeoutMs: number; // wall-clock cap; throws on overrun
|
|
45
|
-
kind?: ArchiveKind; // "zip" | "tar"
|
|
49
|
+
kind?: ArchiveKind; // "zip" | "tar" | "tar-zstd" | "tar-bzip2"
|
|
46
50
|
stripComponents?: number; // strip N leading dirs from entry paths
|
|
47
51
|
tarGzip?: boolean; // when archive is .tar.gz/.tgz
|
|
48
52
|
limits?: ArchiveExtractLimits;
|
|
49
53
|
logger?: ArchiveLogger; // { info?, warn? }
|
|
54
|
+
entryModes?: "clamp" | "preserve";
|
|
55
|
+
entryFilter?: (entry: { path: string; kind: ArchiveEntryKind; size: number }) =>
|
|
56
|
+
"extract" | "skip";
|
|
57
|
+
onFiltered?: "reject-archive" | "skip-entry";
|
|
50
58
|
};
|
|
51
59
|
```
|
|
52
60
|
|
|
61
|
+
`entryModes` defaults to `"clamp"`: directories become `0o755`; files become
|
|
62
|
+
`0o644`, or `0o755` when the archived owner-execute bit is set. `"preserve"`
|
|
63
|
+
keeps archived read/write/execute bits. Both policies strip setuid, setgid, and
|
|
64
|
+
sticky bits, and neither applies archived ownership. TAR extraction disables
|
|
65
|
+
`tar`'s ownership and mode restoration and applies the selected modes in the
|
|
66
|
+
private staging tree; ZIP applies the same policy to `unixPermissions`.
|
|
67
|
+
|
|
68
|
+
Native extraction is deliberately split into two phases. Rust first reports an
|
|
69
|
+
entry manifest without creating paths. TypeScript validates paths, applies
|
|
70
|
+
`stripComponents`, filters, limits, and mode policy, then passes an explicit
|
|
71
|
+
accepted-entry plan back to Rust. Rust only performs decompression and the
|
|
72
|
+
fd-relative `mkdirBeneath`/exclusive-open writes. This keeps policy identical
|
|
73
|
+
between native and JavaScript paths rather than reimplementing it in Rust.
|
|
74
|
+
|
|
75
|
+
An `entryFilter` sees the validated archive path, entry kind, and declared
|
|
76
|
+
size. Returning `"skip"` rejects the whole archive unless `onFiltered` is
|
|
77
|
+
explicitly `"skip-entry"`. Path traversal and archive-wide entry-count checks
|
|
78
|
+
still apply to skipped entries.
|
|
79
|
+
|
|
80
|
+
For example, a fleet restore can omit regenerated cache entries while rejecting
|
|
81
|
+
any other policy mismatch by default:
|
|
82
|
+
|
|
83
|
+
```ts
|
|
84
|
+
await extractArchive({
|
|
85
|
+
archivePath: snapshotPath,
|
|
86
|
+
destDir: restoreRoot,
|
|
87
|
+
timeoutMs: 30_000,
|
|
88
|
+
entryFilter: ({ path: entryPath, kind }) =>
|
|
89
|
+
kind === "directory" && entryPath === "state/cache"
|
|
90
|
+
? "skip"
|
|
91
|
+
: entryPath.startsWith("state/cache/")
|
|
92
|
+
? "skip"
|
|
93
|
+
: "extract",
|
|
94
|
+
onFiltered: "skip-entry",
|
|
95
|
+
limits: { maxEntries: 50_000, maxEntryPathComponents: 64 },
|
|
96
|
+
});
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
If skipping was not explicitly part of the restore contract, omit
|
|
100
|
+
`onFiltered`; the first `"skip"` then rejects the complete archive with
|
|
101
|
+
`ArchiveSecurityError("entry-filtered")`.
|
|
102
|
+
|
|
103
|
+
Policy rejection is prompt on both implementations. The JavaScript TAR path
|
|
104
|
+
owns the file stream and aborts node-tar through a pipeline on filter, path,
|
|
105
|
+
link, limit, validation, or timeout failure, which destroys both ends instead
|
|
106
|
+
of leaving a paused parser to drain indefinitely. The native path finishes its
|
|
107
|
+
bounded manifest read before TypeScript policy evaluation, so a rejected plan
|
|
108
|
+
never starts the extraction worker.
|
|
109
|
+
|
|
53
110
|
If `kind` is omitted, the helper calls `resolveArchiveKind(archivePath)` and throws if the extension is not recognized. Pass `kind` explicitly when the archive name doesn't carry the type (e.g. content-addressed names).
|
|
54
111
|
|
|
55
112
|
### Limits
|
|
@@ -60,29 +117,48 @@ type ArchiveExtractLimits = {
|
|
|
60
117
|
maxEntries?: number; // refuse before extracting if entry count > this
|
|
61
118
|
maxExtractedBytes?: number; // refuse mid-stream if total extracted bytes > this
|
|
62
119
|
maxEntryBytes?: number; // refuse a single entry larger than this
|
|
120
|
+
maxMetaEntryBytes?: number; // refuse one PAX/GNU metadata body above this
|
|
121
|
+
maxEntryPathComponents?: number; // bound output path depth after stripComponents
|
|
63
122
|
};
|
|
64
123
|
```
|
|
65
124
|
|
|
66
|
-
Defaults exist for each (`DEFAULT_MAX_ARCHIVE_BYTES_ZIP`, `DEFAULT_MAX_ENTRIES`, `DEFAULT_MAX_EXTRACTED_BYTES`, `DEFAULT_MAX_ENTRY_BYTES`).
|
|
125
|
+
Defaults exist for each (`DEFAULT_MAX_ARCHIVE_BYTES_ZIP`, `DEFAULT_MAX_ENTRIES`, `DEFAULT_MAX_EXTRACTED_BYTES`, `DEFAULT_MAX_ENTRY_BYTES`, `DEFAULT_MAX_META_ENTRY_BYTES`, `DEFAULT_MAX_ENTRY_PATH_COMPONENTS`). The path-component default is 256. It is evaluated after `stripComponents` and before TypeScript accepts an entry for either JavaScript or native extraction, so rejected entries cannot cause implicit parent-directory creation. The 1 MiB metadata default matches node-tar's `maxMetaEntrySize`; fs-safe passes the same resolved value to node-tar and the native TAR meter.
|
|
67
126
|
|
|
68
127
|
A limit violation throws `ArchiveLimitError`. The error's code is one of:
|
|
69
128
|
|
|
70
129
|
```ts
|
|
71
130
|
ARCHIVE_LIMIT_ERROR_CODE.ARCHIVE_SIZE_EXCEEDS_LIMIT
|
|
72
131
|
ARCHIVE_LIMIT_ERROR_CODE.ENTRY_COUNT_EXCEEDS_LIMIT
|
|
73
|
-
ARCHIVE_LIMIT_ERROR_CODE.
|
|
74
|
-
ARCHIVE_LIMIT_ERROR_CODE.
|
|
132
|
+
ARCHIVE_LIMIT_ERROR_CODE.EXTRACTED_SIZE_EXCEEDS_LIMIT
|
|
133
|
+
ARCHIVE_LIMIT_ERROR_CODE.ENTRY_EXTRACTED_SIZE_EXCEEDS_LIMIT
|
|
134
|
+
ARCHIVE_LIMIT_ERROR_CODE.META_ENTRY_SIZE_EXCEEDS_LIMIT
|
|
135
|
+
ARCHIVE_LIMIT_ERROR_CODE.ENTRY_PATH_COMPONENTS_EXCEEDS_LIMIT
|
|
75
136
|
```
|
|
76
137
|
|
|
77
138
|
Catch and branch on the code to surface a meaningful response to the caller.
|
|
78
139
|
|
|
140
|
+
Entry policy failures throw `ArchiveSecurityError`. Its entry-related codes
|
|
141
|
+
are `"entry-path"`, `"entry-link"`, and `"entry-filtered"`; destination-race
|
|
142
|
+
codes remain `"destination-not-directory"`, `"destination-symlink"`, and
|
|
143
|
+
`"destination-symlink-traversal"`.
|
|
144
|
+
|
|
79
145
|
## What it defends against
|
|
80
146
|
|
|
81
147
|
- **Path traversal:** entries with `..`, absolute paths, or Windows drive prefixes are rejected (`ArchiveSecurityError`).
|
|
82
148
|
- **Symlink/hardlink entries:** rejected by default. Some archives ship symlink/hardlink entries that point outside the destination once resolved; `extractArchive` does not follow them.
|
|
83
|
-
- **TOCTOU during merge:** extraction first writes to a private temp dir, then merges into `destDir` using the same boundary checks as `root().write()`.
|
|
149
|
+
- **TOCTOU during merge:** extraction first writes to a private temp dir, then merges into `destDir` using the same boundary checks as `root().write()`. Destination symlink swaps are checked with the selected platform mechanism; non-Linux routes retain the best-effort race window documented in the [security model](security-model.md#containment-guarantees-by-platform).
|
|
84
150
|
- **Zip bombs:** `maxExtractedBytes` and `maxEntryBytes` apply to *post-decompression* bytes, so highly-compressed payloads hit the cap before they exhaust disk.
|
|
85
151
|
- **Slow-loris archives:** `timeoutMs` is a hard wall-clock budget. Extraction is aborted on overrun.
|
|
152
|
+
- **Metadata bombs:** a fixed-header pass-through reader rejects oversized PAX, GNU long-name, and GNU long-link bodies before either TAR implementation buffers them. It understands octal and base-256 size fields without interpreting metadata content.
|
|
153
|
+
|
|
154
|
+
PAX headers can override the next entry's size from inside their content. The
|
|
155
|
+
fixed-header meter deliberately never interprets that content, so PAX and GNU
|
|
156
|
+
sparse entries are rejected with
|
|
157
|
+
`ArchiveFormatError("archive-header-invalid")` rather than guessing. GNU sparse
|
|
158
|
+
extension blocks are still metered in 512-byte units before rejection, ensuring
|
|
159
|
+
malformed or excessive chains cannot bypass the metadata ceiling. GNU long-name
|
|
160
|
+
and long-link entries remain supported because their fixed header size fully
|
|
161
|
+
determines their layout.
|
|
86
162
|
|
|
87
163
|
## `resolveArchiveKind`
|
|
88
164
|
|
|
@@ -91,15 +167,56 @@ import { resolveArchiveKind, type ArchiveKind } from "@openclaw/fs-safe/archive"
|
|
|
91
167
|
|
|
92
168
|
const kind = resolveArchiveKind("upload.zip"); // "zip"
|
|
93
169
|
const tar = resolveArchiveKind("upload.tar.gz"); // "tar"
|
|
170
|
+
const zstd = resolveArchiveKind("upload.tar.zst"); // "tar-zstd" when native is available
|
|
94
171
|
const unknown = resolveArchiveKind("upload.bin"); // undefined
|
|
95
172
|
```
|
|
96
173
|
|
|
97
174
|
Recognizes:
|
|
98
175
|
|
|
99
176
|
- `*.zip` → `"zip"`
|
|
100
|
-
- `*.tar`, `*.tar.gz`, `*.tgz
|
|
177
|
+
- `*.tar`, `*.tar.gz`, `*.tgz` → `"tar"`
|
|
178
|
+
- `*.tar.zst`, `*.tar.zstd`, `*.tzst` → `"tar-zstd"` (native only)
|
|
179
|
+
- `*.tar.bz2`, `*.tbz2`, `*.tbz` → `"tar-bzip2"` (native only)
|
|
101
180
|
|
|
102
|
-
Returns `undefined` for unknown extensions; check the result before calling
|
|
181
|
+
Returns `undefined` for unknown extensions; check the result before calling
|
|
182
|
+
`extractArchive` if the filename is caller-controlled. A recognized zstd or
|
|
183
|
+
bzip2 TAR extension with no native binding throws the typed
|
|
184
|
+
`FsSafeError("helper-unavailable")` with installation guidance. This includes
|
|
185
|
+
`mode: "off"`; those two formats have no JavaScript fallback.
|
|
186
|
+
|
|
187
|
+
For a service whose input contract requires zstd, configure native mode before
|
|
188
|
+
the first archive call so a packaging mistake fails at the boundary:
|
|
189
|
+
|
|
190
|
+
```ts
|
|
191
|
+
import { configureFsSafeNative } from "@openclaw/fs-safe/config";
|
|
192
|
+
import { extractArchive } from "@openclaw/fs-safe/archive";
|
|
193
|
+
|
|
194
|
+
configureFsSafeNative({ mode: "require" });
|
|
195
|
+
await extractArchive({
|
|
196
|
+
archivePath: "/srv/restore/snapshot.tar.zst",
|
|
197
|
+
destDir: "/srv/restore/staging",
|
|
198
|
+
kind: "tar-zstd",
|
|
199
|
+
timeoutMs: 60_000,
|
|
200
|
+
});
|
|
201
|
+
```
|
|
202
|
+
|
|
203
|
+
## `readArchiveEntry`
|
|
204
|
+
|
|
205
|
+
`readArchiveEntry(archivePath, entryPath, { maxBytes, kind? })` reads one
|
|
206
|
+
regular-file entry into a bounded `Buffer` without extracting a tree. It pins
|
|
207
|
+
and privately stages the archive input, rejects link and directory entries,
|
|
208
|
+
and throws `ArchiveLimitError` if decompressed bytes exceed `maxBytes`. ZIP
|
|
209
|
+
inputs retain the archive subpath's 256 MiB compressed-input ceiling.
|
|
210
|
+
With a native binding it uses the same Rust decoders as extraction, including
|
|
211
|
+
zstd and bzip2 TAR. Without native it retains the JS ZIP/TAR/gzip implementation.
|
|
212
|
+
|
|
213
|
+
```ts
|
|
214
|
+
const rawManifest = await readArchiveEntry(uploadPath, "package/manifest.json", {
|
|
215
|
+
maxBytes: 64 * 1024,
|
|
216
|
+
});
|
|
217
|
+
const manifest = JSON.parse(rawManifest.toString("utf8")) as PluginManifest;
|
|
218
|
+
validatePluginManifest(manifest);
|
|
219
|
+
```
|
|
103
220
|
|
|
104
221
|
## Lower-level building blocks
|
|
105
222
|
|
|
@@ -196,4 +313,5 @@ await withTempWorkspace({ rootDir: "/srv/site/tmp", prefix: "extract-" }, async
|
|
|
196
313
|
- [Atomic writes](atomic.md) — `replaceDirectoryAtomic` for staged directory replacement.
|
|
197
314
|
- [Temp workspaces](temp.md) — extract into a private workspace and commit as one step.
|
|
198
315
|
- [Errors](errors.md) — `FsSafeError` codes the underlying writes can raise.
|
|
316
|
+
- [Migrating to 0.5](migrating-to-0.5.md) — clamp-default and native-format upgrade checklist.
|
|
199
317
|
- [`extractArchive` source](https://github.com/openclaw/fs-safe/blob/main/src/archive.ts).
|
package/docs/atomic.md
CHANGED
|
@@ -43,6 +43,9 @@ type ReplaceFileAtomicOptions = {
|
|
|
43
43
|
renameMaxRetries?: number;
|
|
44
44
|
renameRetryBaseDelayMs?: number;
|
|
45
45
|
copyFallbackOnPermissionError?: boolean;
|
|
46
|
+
copyFallbackRestore?: "restore-original" | "none"; // default: "none"
|
|
47
|
+
maxRestoreBytes?: number; // required with "restore-original"
|
|
48
|
+
destinationHardlinks?: "reject";
|
|
46
49
|
syncTempFile?: boolean; // fsync(temp) before rename
|
|
47
50
|
syncParentDir?: boolean; // fsync(parent) after rename (POSIX only)
|
|
48
51
|
beforeRename?: (params: { filePath: string; tempPath: string }) => Promise<void>;
|
|
@@ -75,6 +78,25 @@ with exclusive/no-follow flags where the platform supports them, and refuses
|
|
|
75
78
|
known symlink destinations so it does not write through a replaced destination
|
|
76
79
|
link.
|
|
77
80
|
|
|
81
|
+
Set `destinationHardlinks: "reject"` when an existing regular-file destination
|
|
82
|
+
must not have aliases. The policy reads `nlink` from a pinned destination
|
|
83
|
+
descriptor, not pathname metadata, before rename and rechecks it in the copy
|
|
84
|
+
fallback.
|
|
85
|
+
|
|
86
|
+
The default `copyFallbackRestore: "none"` preserves the existing fallback
|
|
87
|
+
contract: a failed copy can leave a partial destination. For state files where
|
|
88
|
+
preserving the old bytes is more important, choose `"restore-original"` and set
|
|
89
|
+
an explicit `maxRestoreBytes` memory budget. If the destination exists, fs-safe
|
|
90
|
+
snapshots it through a pinned descriptor, overwrites through that same
|
|
91
|
+
descriptor, and synchronizes the result. Any write or sync failure triggers a
|
|
92
|
+
restore and another sync through the same descriptor.
|
|
93
|
+
|
|
94
|
+
Restore failures are `FsSafeError("helper-failed")` values with typed
|
|
95
|
+
`details.cleanup` set to `"restored"` or `"restore-failed"`. An original larger
|
|
96
|
+
than `maxRestoreBytes` fails with `too-large` before mutation. A missing
|
|
97
|
+
destination has no original to restore and follows the exclusive-create copy
|
|
98
|
+
fallback.
|
|
99
|
+
|
|
78
100
|
### Sync variant
|
|
79
101
|
|
|
80
102
|
`replaceFileAtomicSync` accepts the same options shape, with the obvious removal of the async-only hooks. Use it inside synchronous boot paths or test setup code.
|
|
@@ -148,6 +170,13 @@ await movePathWithCopyFallback({
|
|
|
148
170
|
```
|
|
149
171
|
|
|
150
172
|
Use it when source and destination might live on different filesystems (containers, tmpfs, separate volumes).
|
|
173
|
+
`sourceHardlinks: "reject"` performs a recursive preflight capped at 50,000
|
|
174
|
+
entries before any mutation. Because link count and rename cannot be one atomic
|
|
175
|
+
portable operation, this mode always commits a fresh inode/tree through the
|
|
176
|
+
staged-copy route, even on one filesystem. Each regular file is checked again
|
|
177
|
+
after open and after copying, so a post-scan hardlink cannot become the
|
|
178
|
+
published target. A hardlink fails with `FsSafeError("hardlink")`; exceeding
|
|
179
|
+
the preflight cap fails with `FsSafeError("too-large")`.
|
|
151
180
|
If another writer changes source entries during the fallback, the staged copy
|
|
152
181
|
throws `ESTALE` before commit when possible. If the destination has already
|
|
153
182
|
been committed, cleanup still preserves the changed source entries and throws
|
|
@@ -160,7 +189,7 @@ been committed, cleanup still preserves the changed source entries and throws
|
|
|
160
189
|
| Take relative paths, bound to a `rootDir`. | Take absolute paths, no boundary. |
|
|
161
190
|
| Throw `FsSafeError` with `code`. | Throw `FsSafeError` *or* the underlying `NodeJS.ErrnoException`, depending on failure point. |
|
|
162
191
|
| Atomicity, mode, hooks, fsync are sane defaults. | Caller controls all of the above. |
|
|
163
|
-
| `mkdir`, identity check, hardlink reject built in. | No
|
|
192
|
+
| `mkdir`, identity check, hardlink reject built in. | No root boundary; `movePathWithCopyFallback` has explicit `sourceHardlinks` policy, while other helpers expose their own narrower checks. |
|
|
164
193
|
|
|
165
194
|
Use `Root` when the path is caller-controlled. Use `atomic` when the path is fully under your control and you want explicit knobs.
|
|
166
195
|
|
package/docs/config.md
CHANGED
|
@@ -5,47 +5,46 @@ description: "Process-global defaults for optional fs-safe helpers."
|
|
|
5
5
|
|
|
6
6
|
# `@openclaw/fs-safe/config`
|
|
7
7
|
|
|
8
|
-
Process-global configuration knobs for optional fs-safe helpers. The
|
|
8
|
+
Process-global configuration knobs for optional fs-safe helpers. The native helper policy is described in the [native helper policy](native-helper.md); this page is the API reference.
|
|
9
9
|
|
|
10
10
|
```ts
|
|
11
11
|
import {
|
|
12
|
-
|
|
12
|
+
configureFsSafeNative,
|
|
13
13
|
configureFsSafeLocks,
|
|
14
|
-
|
|
14
|
+
getFsSafeNativeConfig,
|
|
15
15
|
getFsSafeLockConfig,
|
|
16
16
|
type FsSafeLockConfig,
|
|
17
|
-
type
|
|
18
|
-
type
|
|
17
|
+
type FsSafeNativeConfig,
|
|
18
|
+
type FsSafeNativeMode,
|
|
19
19
|
} from "@openclaw/fs-safe/config";
|
|
20
20
|
```
|
|
21
21
|
|
|
22
22
|
These functions are also re-exported from the main entry point. Prefer the subpath when you only need helper configuration and want the smallest import surface.
|
|
23
23
|
|
|
24
|
-
## `
|
|
24
|
+
## `configureFsSafeNative(config)`
|
|
25
25
|
|
|
26
26
|
```ts
|
|
27
|
-
function
|
|
27
|
+
function configureFsSafeNative(config: Partial<FsSafeNativeConfig>): void;
|
|
28
28
|
|
|
29
|
-
type
|
|
30
|
-
mode:
|
|
31
|
-
pythonPath?: string;
|
|
29
|
+
type FsSafeNativeConfig = {
|
|
30
|
+
mode: FsSafeNativeMode;
|
|
32
31
|
};
|
|
33
32
|
|
|
34
|
-
type
|
|
33
|
+
type FsSafeNativeMode = "auto" | "off" | "require";
|
|
35
34
|
```
|
|
36
35
|
|
|
37
|
-
Set the process-global policy.
|
|
36
|
+
Set the process-global loading policy. Configure once at startup, before the first filesystem operation. The binding is loaded lazily and the result is cached.
|
|
38
37
|
|
|
39
38
|
| Mode | Behavior |
|
|
40
39
|
|---|---|
|
|
41
|
-
| `auto` | Default.
|
|
42
|
-
| `off` |
|
|
43
|
-
| `require` |
|
|
40
|
+
| `auto` | Default. Prefer the platform binding and use guarded JavaScript when it is unavailable. |
|
|
41
|
+
| `off` | Do not load the binding; use guarded JavaScript deterministically. |
|
|
42
|
+
| `require` | Operations that need the binding raise `FsSafeError("helper-unavailable")` when it cannot load. |
|
|
44
43
|
|
|
45
|
-
## `
|
|
44
|
+
## `getFsSafeNativeConfig()`
|
|
46
45
|
|
|
47
46
|
```ts
|
|
48
|
-
function
|
|
47
|
+
function getFsSafeNativeConfig(): FsSafeNativeConfig;
|
|
49
48
|
```
|
|
50
49
|
|
|
51
50
|
Return the effective configuration: programmatic overrides win, then env vars, then the package default (`auto`).
|
|
@@ -67,6 +66,22 @@ Set process-wide defaults for sidecar lock options. This does **not** turn locki
|
|
|
67
66
|
|
|
68
67
|
`staleRecovery` defaults to `"fail-closed"`. The opt-in `"remove-if-unchanged"` mode requires caller approval and serializes the final snapshot check and unlink with an exclusive `.reclaim` guard. A reclaim guard left by a killed reclaimer fails closed and requires externally coordinated cleanup.
|
|
69
68
|
|
|
69
|
+
For a daemon that should wait briefly for normal contention but never delete a
|
|
70
|
+
stale owner without per-lock approval:
|
|
71
|
+
|
|
72
|
+
```ts
|
|
73
|
+
configureFsSafeLocks({
|
|
74
|
+
staleRecovery: "fail-closed",
|
|
75
|
+
staleMs: 2 * 60_000,
|
|
76
|
+
timeoutMs: 15_000,
|
|
77
|
+
retry: { retries: 30, minTimeout: 50, maxTimeout: 1_000, randomize: true },
|
|
78
|
+
});
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
Individual lock calls can override any default. Switching the global stale
|
|
82
|
+
recovery mode does not provide the application-owned liveness proof required
|
|
83
|
+
by `shouldRemoveStaleLock`.
|
|
84
|
+
|
|
70
85
|
## `getFsSafeLockConfig()`
|
|
71
86
|
|
|
72
87
|
```ts
|
|
@@ -80,15 +95,30 @@ Return the current sidecar lock defaults.
|
|
|
80
95
|
The same policy can be set without code:
|
|
81
96
|
|
|
82
97
|
```bash
|
|
83
|
-
|
|
84
|
-
FS_SAFE_PYTHON=/usr/bin/python3
|
|
98
|
+
FS_SAFE_NATIVE_MODE=auto # auto | off | require | true | false | on | 1 | 0 | never | required
|
|
85
99
|
```
|
|
86
100
|
|
|
87
|
-
|
|
101
|
+
`OPENCLAW_FS_SAFE_NATIVE_MODE` is accepted as an alias. Programmatic overrides via `configureFsSafeNative` always win.
|
|
102
|
+
|
|
103
|
+
### Python-helper migration bridge
|
|
104
|
+
|
|
105
|
+
Version 0.5 detects the former `FS_SAFE_PYTHON_MODE`, `FS_SAFE_PYTHON`,
|
|
106
|
+
`OPENCLAW_FS_SAFE_PYTHON_MODE`, `OPENCLAW_FS_SAFE_PYTHON`,
|
|
107
|
+
`OPENCLAW_PINNED_PYTHON`, and `OPENCLAW_PINNED_WRITE_PYTHON` names. It emits one
|
|
108
|
+
`FS_SAFE_PYTHON_DEPRECATED` warning and maps `auto`, `off`, or `require` to the
|
|
109
|
+
same native mode; interpreter paths are ignored. The deprecated
|
|
110
|
+
`configureFsSafePython()` export behaves the same way.
|
|
111
|
+
|
|
112
|
+
Replace these inputs with `configureFsSafeNative()` or
|
|
113
|
+
`FS_SAFE_NATIVE_MODE` during the 0.5 upgrade. The bridge exists only so shipped
|
|
114
|
+
0.4 configuration fails loudly and maps predictably; it is not a supported
|
|
115
|
+
Python execution path. Follow the [0.5 migration checklist](migrating-to-0.5.md)
|
|
116
|
+
for the full upgrade.
|
|
88
117
|
|
|
89
118
|
## Related pages
|
|
90
119
|
|
|
91
|
-
- [
|
|
120
|
+
- [Native helper policy](native-helper.md) — when to pick `auto`, `off`, or `require`, and what each mode protects.
|
|
92
121
|
- [File lock](sidecar-lock.md) — the per-resource lock API that consumes lock defaults.
|
|
93
122
|
- [Root API](root.md) — the API whose POSIX hardening the helper backs.
|
|
94
123
|
- [Errors](errors.md) — `helper-unavailable` and `helper-failed`.
|
|
124
|
+
- [Migrating to 0.5](migrating-to-0.5.md) — ordered consumer upgrade checklist.
|
package/docs/durability.md
CHANGED
|
@@ -81,9 +81,219 @@ accepted.
|
|
|
81
81
|
the caller before a separate permission or policy check. A missing or replaced
|
|
82
82
|
target fails with `FsSafeError("path-mismatch")`.
|
|
83
83
|
|
|
84
|
+
## Exclusive file publication
|
|
85
|
+
|
|
86
|
+
`publishFileExclusive()` materializes one file without clobbering an existing
|
|
87
|
+
target. It pins the source with `O_NOFOLLOW`, optionally verifies
|
|
88
|
+
`expectedSourceIdentity`, tries a hardlink first, then synchronizes the target
|
|
89
|
+
parent directory.
|
|
90
|
+
|
|
91
|
+
For example, a backup archive is complete before publication. If directory
|
|
92
|
+
sync fails, keeping that complete file is more useful than conditionally
|
|
93
|
+
deleting it by pathname:
|
|
94
|
+
|
|
95
|
+
```ts
|
|
96
|
+
import { FsSafeError } from "@openclaw/fs-safe/errors";
|
|
97
|
+
import { publishFileExclusive } from "@openclaw/fs-safe/durability";
|
|
98
|
+
|
|
99
|
+
try {
|
|
100
|
+
const result = await publishFileExclusive({
|
|
101
|
+
sourcePath: stagedArchive,
|
|
102
|
+
targetPath: finalArchive,
|
|
103
|
+
strategy: "link-or-copy",
|
|
104
|
+
onSyncFailure: "preserve",
|
|
105
|
+
parentReceipt: backupDirectory,
|
|
106
|
+
});
|
|
107
|
+
recordDurableBackup(result.identity, result.directorySync);
|
|
108
|
+
} catch (error) {
|
|
109
|
+
if (
|
|
110
|
+
error instanceof FsSafeError &&
|
|
111
|
+
error.details?.phase === "directory-sync" &&
|
|
112
|
+
error.details.cleanup === "preserved"
|
|
113
|
+
) {
|
|
114
|
+
recordCompleteButPossiblyNonDurableBackup(finalArchive, error.details);
|
|
115
|
+
} else {
|
|
116
|
+
throw error;
|
|
117
|
+
}
|
|
118
|
+
}
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
### Strategies
|
|
122
|
+
|
|
123
|
+
| Strategy | Behavior | Native requirement |
|
|
124
|
+
|---|---|---|
|
|
125
|
+
| `link-required` | Create a same-filesystem hardlink or propagate the failure. | No; guarded JS `link` fallback remains. |
|
|
126
|
+
| `link-or-copy` | Try hardlink, then clone, Linux `copy_file_range`, then the JS byte loop for classified unsupported errors. | No; acceleration is optional. |
|
|
127
|
+
| `rename-noreplace` | Atomically move the source without replacing an existing target. Success consumes `sourcePath`. | Yes. |
|
|
128
|
+
|
|
129
|
+
`"link-required"` propagates an unsupported hardlink failure.
|
|
130
|
+
`"link-or-copy"` falls back only for `EPERM`, `EXDEV`, `ENOTSUP`,
|
|
131
|
+
`EOPNOTSUPP`, or `ENOSYS`; `isHardlinkFallbackError()` exposes that exact
|
|
132
|
+
classifier. The fallback copies from the pinned source into a `wx` target,
|
|
133
|
+
fsyncs it, and fences source and target identity and content before reporting
|
|
134
|
+
success. `parentReceipt`, when supplied, must name the target's direct parent.
|
|
135
|
+
|
|
136
|
+
With a native binding, the copy fallback first attempts a copy-on-write clone
|
|
137
|
+
(`fclonefileat` on macOS, `FICLONE` on Linux), then Linux
|
|
138
|
+
`copy_file_range`, and finally the existing JavaScript byte loop. Every route
|
|
139
|
+
creates the target exclusively, normalizes its mode to `0o600`, and goes
|
|
140
|
+
through the same post-copy identity and SHA-256 fencing. Hashing uses an async
|
|
141
|
+
native task when available, so large verification reads do not occupy the
|
|
142
|
+
JavaScript event loop.
|
|
143
|
+
|
|
144
|
+
On a clone-capable filesystem, publication of a large file becomes mostly a
|
|
145
|
+
metadata operation: data blocks are shared copy-on-write until either file is
|
|
146
|
+
modified. Clone support is filesystem- and mount-dependent, so callers must
|
|
147
|
+
not infer durability or physical independence from timing; an unsupported
|
|
148
|
+
clone or `copy_file_range` transparently continues down the fallback chain.
|
|
149
|
+
|
|
150
|
+
## Recoverable atomic-replace fallback
|
|
151
|
+
|
|
152
|
+
`replaceFileAtomic()` normally publishes a synchronized sibling temp with an
|
|
153
|
+
atomic rename. Some Windows filesystems and file owners reject that rename with
|
|
154
|
+
`EPERM` or `EEXIST`; `copyFallbackOnPermissionError: true` permits a non-atomic
|
|
155
|
+
copy fallback.
|
|
156
|
+
|
|
157
|
+
Callers that cannot tolerate a torn in-place fallback can add:
|
|
158
|
+
|
|
159
|
+
```ts
|
|
160
|
+
await replaceFileAtomic({
|
|
161
|
+
filePath: statePath,
|
|
162
|
+
content: nextState,
|
|
163
|
+
syncTempFile: true,
|
|
164
|
+
syncParentDir: true,
|
|
165
|
+
copyFallbackOnPermissionError: true,
|
|
166
|
+
copyFallbackRestore: "restore-original",
|
|
167
|
+
maxRestoreBytes: 4 * 1024 * 1024,
|
|
168
|
+
destinationHardlinks: "reject",
|
|
169
|
+
});
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
The existing regular-file destination is pinned before its link count is
|
|
173
|
+
accepted. Its original bytes are read within `maxRestoreBytes`, then the new
|
|
174
|
+
bytes are written and synchronized through the same descriptor. If a write or
|
|
175
|
+
sync tears, fs-safe rewrites the snapshot and fsyncs it before throwing. Inspect
|
|
176
|
+
`details.cleanup`: `"restored"` means the original bytes were put back and
|
|
177
|
+
synchronized; `"restore-failed"` means the replacement and recovery both
|
|
178
|
+
failed, so the destination must be treated as indeterminate. This is recovery
|
|
179
|
+
from a live-process I/O failure, not a transaction or a substitute for an
|
|
180
|
+
application backup protocol.
|
|
181
|
+
|
|
182
|
+
## Streaming SHA-256
|
|
183
|
+
|
|
184
|
+
`sha256File()` hashes either a pathname string or an already-open Node
|
|
185
|
+
`FileHandle`. A backup verifier can pin the file itself, compare its size, and
|
|
186
|
+
keep ownership of the handle:
|
|
187
|
+
|
|
188
|
+
```ts
|
|
189
|
+
import { open } from "node:fs/promises";
|
|
190
|
+
import { sha256File } from "@openclaw/fs-safe/durability";
|
|
191
|
+
|
|
192
|
+
const snapshot = await open(stagedArchive, "r");
|
|
193
|
+
try {
|
|
194
|
+
const before = await snapshot.stat();
|
|
195
|
+
const hash = await sha256File(snapshot);
|
|
196
|
+
if (hash.bytes !== before.size || hash.digest !== manifest.sha256) {
|
|
197
|
+
throw new Error("staged backup does not match its manifest");
|
|
198
|
+
}
|
|
199
|
+
} finally {
|
|
200
|
+
await snapshot.close();
|
|
201
|
+
}
|
|
202
|
+
```
|
|
203
|
+
|
|
204
|
+
The result is `{ bytes, digest }`, where `digest` is lowercase hexadecimal.
|
|
205
|
+
The handle overload never closes the caller's descriptor and uses positioned
|
|
206
|
+
reads, so it does not alter the descriptor's current offset. The path overload
|
|
207
|
+
rejects symbolic links and non-regular files, verifies the opened descriptor
|
|
208
|
+
still names the requested path, and closes its own handle. POSIX opens are
|
|
209
|
+
nonblocking, so a raced FIFO or device is rejected after descriptor inspection
|
|
210
|
+
rather than waiting for a writer.
|
|
211
|
+
|
|
212
|
+
When the optional binding is active, hashing runs as an async native task and
|
|
213
|
+
does not occupy the JavaScript event loop with digest updates. With native mode
|
|
214
|
+
`off`, or in `auto` when no binding loads, the fallback performs asynchronous
|
|
215
|
+
positioned reads in 64 KiB chunks but updates Node's `Hash` on the JavaScript
|
|
216
|
+
thread. Both paths stream constant-size buffers rather than loading the file
|
|
217
|
+
into memory. Native mode `require` keeps its usual fail-closed loader semantics.
|
|
218
|
+
|
|
219
|
+
If publication fails after this call created the target, it throws an
|
|
220
|
+
`FsSafeError` with a `details` receipt:
|
|
221
|
+
|
|
222
|
+
```ts
|
|
223
|
+
type PublishFileExclusiveFailureDetails = {
|
|
224
|
+
phase:
|
|
225
|
+
| "hardlink-create" | "hardlink-verify"
|
|
226
|
+
| "copy-create" | "copy-verify"
|
|
227
|
+
| "rename-create" | "rename-verify"
|
|
228
|
+
| "directory-sync";
|
|
229
|
+
targetCreated: boolean;
|
|
230
|
+
targetIdentity?: { dev: number | bigint; ino: number | bigint };
|
|
231
|
+
cleanup: "removed" | "preserved" | "unknown";
|
|
232
|
+
directorySync?: { status: "failed"; code?: string };
|
|
233
|
+
};
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
`"removed"` means the path still matched the identity created by this call and
|
|
237
|
+
was unlinked (or was already absent). `"preserved"` means it was deliberately
|
|
238
|
+
retained—for example after a successful no-replace rename—or the pathname had
|
|
239
|
+
been replaced and therefore was not safe to remove. `"unknown"` means cleanup
|
|
240
|
+
could not verify or remove the created identity. Callers that run a second
|
|
241
|
+
application-level guard, such as SQLite snapshot validation, should branch on
|
|
242
|
+
this receipt instead of inferring ownership from path existence. The original
|
|
243
|
+
failure remains available as `cause`. Failures before target creation retain
|
|
244
|
+
their existing error shape and do not claim a cleanup result.
|
|
245
|
+
|
|
246
|
+
### Directory-sync failure policy
|
|
247
|
+
|
|
248
|
+
`onSyncFailure` applies only after target creation and content/identity fencing
|
|
249
|
+
have succeeded but synchronizing the containing directory throws:
|
|
250
|
+
|
|
251
|
+
```ts
|
|
252
|
+
type PublishFileExclusiveSyncFailurePolicy = "rollback" | "preserve";
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
A returned `{ status: "unsupported", code? }` is an explicit successful
|
|
256
|
+
publication outcome, not a thrown sync failure, so this option does not rewrite
|
|
257
|
+
or clean up that target.
|
|
258
|
+
|
|
259
|
+
- `rollback` is the default. fs-safe removes the target only if its current
|
|
260
|
+
identity still matches the file created by this call. A replacement is never
|
|
261
|
+
removed. The error reports `cleanup: "removed"`, `"preserved"`, or
|
|
262
|
+
`"unknown"` and `directorySync: { status: "failed", code? }`.
|
|
263
|
+
- `preserve` never attempts that unlink. The error reports
|
|
264
|
+
`targetCreated: true`, `cleanup: "preserved"`, the created identity, and the
|
|
265
|
+
failed directory-sync outcome. The file is complete and fenced, but its
|
|
266
|
+
directory entry is not proven crash-durable.
|
|
267
|
+
|
|
268
|
+
Choose `rollback` when the pathname must mean “durably committed” and a failed
|
|
269
|
+
commit should disappear from the live process view. Choose `preserve` when the
|
|
270
|
+
payload itself remains valuable—backup archives are the common case—and the
|
|
271
|
+
caller can record, retry, or independently validate durability. Neither choice
|
|
272
|
+
can make a failed directory sync succeed: rollback deletion is also not proven
|
|
273
|
+
durable, and a preserved name may disappear after a crash. Always use the
|
|
274
|
+
typed receipt rather than inferring ownership from `exists()`.
|
|
275
|
+
|
|
276
|
+
`rename-noreplace` always preserves its target after a successful rename,
|
|
277
|
+
because removing it would discard the source's only remaining name; its typed
|
|
278
|
+
failure receipt makes that explicit regardless of `onSyncFailure`.
|
|
279
|
+
|
|
280
|
+
`"rename-noreplace"` requires the native helper and atomically moves the
|
|
281
|
+
source to the target without replacement. A collision is reported as
|
|
282
|
+
`EEXIST`, both files remain unchanged, and a successful call returns
|
|
283
|
+
`method: "rename-noreplace"` after synchronizing the source and target parent
|
|
284
|
+
directories. Unlike the link/copy strategies, success consumes `sourcePath`.
|
|
285
|
+
|
|
84
286
|
## Scope
|
|
85
287
|
|
|
86
|
-
These primitives establish path identity and filesystem synchronization.
|
|
288
|
+
These primitives establish path identity and filesystem synchronization. One
|
|
289
|
+
`publishFileExclusive()` call is one no-clobber file materialization, not a
|
|
290
|
+
retention policy, multi-file transaction, or application commit protocol. They
|
|
87
291
|
do not decide application commit protocols, marker formats, permission policy,
|
|
88
292
|
or whether an unsupported platform is acceptable. Keep those decisions at the
|
|
89
293
|
owning product boundary.
|
|
294
|
+
|
|
295
|
+
## See also
|
|
296
|
+
|
|
297
|
+
- [Migrating to 0.5](migrating-to-0.5.md) — choosing a publication policy during upgrade.
|
|
298
|
+
- [Native architecture](native.md) — clone/copy/hash mechanisms and fallback guarantees.
|
|
299
|
+
- [Errors](errors.md) — typed operational failure handling.
|