@openclaw/fs-safe 0.5.6 → 0.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (145) hide show
  1. package/CHANGELOG.md +39 -0
  2. package/README.md +16 -10
  3. package/dist/advanced.d.ts +2 -1
  4. package/dist/advanced.d.ts.map +1 -1
  5. package/dist/advanced.js +1 -0
  6. package/dist/archive-crc32.d.ts +2 -0
  7. package/dist/archive-crc32.d.ts.map +1 -0
  8. package/dist/archive-crc32.js +14 -0
  9. package/dist/archive-kind.js +2 -2
  10. package/dist/archive-native.d.ts.map +1 -1
  11. package/dist/archive-native.js +7 -0
  12. package/dist/archive-read.d.ts.map +1 -1
  13. package/dist/archive-read.js +24 -5
  14. package/dist/archive-tar-meta.d.ts.map +1 -1
  15. package/dist/archive-tar-meta.js +53 -6
  16. package/dist/archive-tar-pax.d.ts +8 -0
  17. package/dist/archive-tar-pax.d.ts.map +1 -0
  18. package/dist/archive-tar-pax.js +100 -0
  19. package/dist/archive-tar-runtime.d.ts +1 -0
  20. package/dist/archive-tar-runtime.d.ts.map +1 -1
  21. package/dist/archive-zip-admission.d.ts +7 -0
  22. package/dist/archive-zip-admission.d.ts.map +1 -0
  23. package/dist/archive-zip-admission.js +60 -0
  24. package/dist/archive-zip-count.d.ts +2 -0
  25. package/dist/archive-zip-count.d.ts.map +1 -0
  26. package/dist/archive-zip-count.js +140 -0
  27. package/dist/archive-zip-directory.d.ts +8 -0
  28. package/dist/archive-zip-directory.d.ts.map +1 -0
  29. package/dist/archive-zip-directory.js +224 -0
  30. package/dist/archive-zip-integrity.d.ts.map +1 -1
  31. package/dist/archive-zip-integrity.js +1 -14
  32. package/dist/archive-zip-names.d.ts +12 -0
  33. package/dist/archive-zip-names.d.ts.map +1 -0
  34. package/dist/archive-zip-names.js +99 -0
  35. package/dist/archive-zip-preflight.d.ts +1 -1
  36. package/dist/archive-zip-preflight.d.ts.map +1 -1
  37. package/dist/archive-zip-preflight.js +8 -145
  38. package/dist/archive.d.ts.map +1 -1
  39. package/dist/archive.js +32 -23
  40. package/dist/file-hash.d.ts.map +1 -1
  41. package/dist/file-hash.js +25 -19
  42. package/dist/file-lock-sync.d.ts.map +1 -1
  43. package/dist/file-lock-sync.js +19 -7
  44. package/dist/native-binding.d.ts +3 -0
  45. package/dist/native-binding.d.ts.map +1 -1
  46. package/dist/native-operations.d.ts +4 -1
  47. package/dist/native-operations.d.ts.map +1 -1
  48. package/dist/native-operations.js +22 -6
  49. package/dist/native-pinned-write-windows.d.ts +8 -0
  50. package/dist/native-pinned-write-windows.d.ts.map +1 -0
  51. package/dist/native-pinned-write-windows.js +92 -0
  52. package/dist/native-pinned-write.d.ts.map +1 -1
  53. package/dist/native-pinned-write.js +136 -127
  54. package/dist/native-staged-file.d.ts +24 -0
  55. package/dist/native-staged-file.d.ts.map +1 -0
  56. package/dist/native-staged-file.js +337 -0
  57. package/dist/native.d.ts.map +1 -1
  58. package/dist/native.js +4 -4
  59. package/dist/opened-realpath.d.ts +2 -0
  60. package/dist/opened-realpath.d.ts.map +1 -1
  61. package/dist/opened-realpath.js +12 -7
  62. package/dist/owner-dacl.d.ts.map +1 -1
  63. package/dist/owner-dacl.js +2 -1
  64. package/dist/permission-exec.d.ts +19 -0
  65. package/dist/permission-exec.d.ts.map +1 -1
  66. package/dist/permission-exec.js +57 -11
  67. package/dist/permissions-public.d.ts +1 -1
  68. package/dist/permissions-public.d.ts.map +1 -1
  69. package/dist/permissions-windows.d.ts +3 -0
  70. package/dist/permissions-windows.d.ts.map +1 -1
  71. package/dist/permissions-windows.js +16 -5
  72. package/dist/permissions.d.ts +5 -0
  73. package/dist/permissions.d.ts.map +1 -1
  74. package/dist/pinned-write.d.ts +6 -0
  75. package/dist/pinned-write.d.ts.map +1 -1
  76. package/dist/pinned-write.js +19 -29
  77. package/dist/private-directory.d.ts.map +1 -1
  78. package/dist/private-directory.js +3 -2
  79. package/dist/root-impl.d.ts.map +1 -1
  80. package/dist/root-impl.js +78 -72
  81. package/dist/root-paths.d.ts +11 -14
  82. package/dist/root-paths.d.ts.map +1 -1
  83. package/dist/root-paths.js +36 -27
  84. package/dist/root-write-verification.d.ts +11 -0
  85. package/dist/root-write-verification.d.ts.map +1 -0
  86. package/dist/root-write-verification.js +91 -0
  87. package/dist/secret-file.d.ts +1 -6
  88. package/dist/secret-file.d.ts.map +1 -1
  89. package/dist/secret-file.js +43 -117
  90. package/dist/secret-read-async.d.ts +1 -1
  91. package/dist/secret-read-async.d.ts.map +1 -1
  92. package/dist/secret-read-async.js +45 -67
  93. package/dist/secret-read-policy.d.ts +13 -0
  94. package/dist/secret-read-policy.d.ts.map +1 -0
  95. package/dist/secret-read-policy.js +28 -0
  96. package/dist/secret.d.ts +2 -1
  97. package/dist/secret.d.ts.map +1 -1
  98. package/dist/secret.js +2 -1
  99. package/dist/secure-file.d.ts.map +1 -1
  100. package/dist/secure-file.js +25 -16
  101. package/dist/staged-directory.d.ts +16 -0
  102. package/dist/staged-directory.d.ts.map +1 -0
  103. package/dist/staged-directory.js +60 -0
  104. package/dist/staged-file-types.d.ts +56 -0
  105. package/dist/staged-file-types.d.ts.map +1 -0
  106. package/dist/staged-file-types.js +1 -0
  107. package/dist/staged-file.d.ts +10 -0
  108. package/dist/staged-file.d.ts.map +1 -0
  109. package/dist/staged-file.js +15 -0
  110. package/dist/strict-file-identity.d.ts +6 -0
  111. package/dist/strict-file-identity.d.ts.map +1 -0
  112. package/dist/strict-file-identity.js +48 -0
  113. package/dist/windows-owner.d.ts +3 -0
  114. package/dist/windows-owner.d.ts.map +1 -1
  115. package/dist/windows-owner.js +10 -2
  116. package/docs/advanced.md +11 -1
  117. package/docs/archive.md +62 -15
  118. package/docs/config.md +7 -0
  119. package/docs/contributing.md +45 -7
  120. package/docs/durability.md +17 -4
  121. package/docs/errors.md +16 -1
  122. package/docs/index.md +3 -1
  123. package/docs/install.md +15 -7
  124. package/docs/migrating-to-0.5.md +7 -7
  125. package/docs/migrating-to-0.6.md +43 -0
  126. package/docs/native-helper.md +15 -5
  127. package/docs/native.md +40 -11
  128. package/docs/path-scope.md +28 -2
  129. package/docs/permissions.md +13 -2
  130. package/docs/root.md +1 -1
  131. package/docs/secret-file.md +8 -0
  132. package/docs/secure-file.md +17 -1
  133. package/docs/security-model.md +15 -1
  134. package/docs/sidecar-lock.md +11 -1
  135. package/docs/staged-file.md +178 -0
  136. package/docs/testing.md +1 -1
  137. package/docs/writing.md +19 -0
  138. package/package.json +15 -8
  139. package/dist/native/darwin-arm64/fs-safe-native.node +0 -0
  140. package/dist/native/darwin-x64/fs-safe-native.node +0 -0
  141. package/dist/native/linux-arm64-gnu/fs-safe-native.node +0 -0
  142. package/dist/native/linux-arm64-musl/fs-safe-native.node +0 -0
  143. package/dist/native/linux-x64-gnu/fs-safe-native.node +0 -0
  144. package/dist/native/linux-x64-musl/fs-safe-native.node +0 -0
  145. package/dist/native/win32-x64-msvc/fs-safe-native.node +0 -0
@@ -46,23 +46,55 @@ pnpm check
46
46
  This runs the filesystem boundary checks, build, tests, and package
47
47
  tarball/import validation.
48
48
 
49
+ ### Native consumer installs
50
+
51
+ After `pnpm build` and a fresh `pnpm native:build`, run `pnpm package:smoke`.
52
+ It packs the real root and host binding, then runs root-only npm and the
53
+ declared pnpm version against a disposable loopback registry. The root's exact
54
+ optional dependencies stay unchanged. Each consumer lives outside the workspace
55
+ with isolated configuration, caches, and stores; the registry never proxies to
56
+ the Internet. The smoke verifies root integrity, consumer-local resolution,
57
+ OS/CPU/libc selection, a native-required SHA-256 operation, and fresh-process
58
+ `auto`/`off` fallbacks and `require` failures for missing bindings and omitted
59
+ optionals. Omitted-optionals installs also verify that all public subpaths can
60
+ be imported, without implying every operation remains available.
61
+
62
+ Host-only smoke supplies the six foreign packages using their unchanged real
63
+ manifests and clearly marked synthetic, non-executable payloads. Every foreign
64
+ metadata/tarball endpoint is checked before installation, so a missing fixture
65
+ cannot masquerade as successful platform filtering. These temporary fixtures
66
+ never enter `packages/`, release artifacts, or the publish manifest. They prove
67
+ installer filtering, not foreign native compilation or execution. Full release
68
+ collection uses the actual seven collected native tarballs instead. Run it with
69
+ `pnpm package:collect` after assembling all seven real bindings; missing targets
70
+ fail collection. `pnpm package:collect --allow-host-only` exercises the same
71
+ lifecycle boundary locally but proves only the host. Both collection commands
72
+ require the pnpm lifecycle CLI path; direct `node` invocation is unsupported. Archive
73
+ codecs and their dependencies are packed from the installed dependency graph.
74
+
75
+ PR CI builds and executes four host targets: Linux x64 glibc, Linux x64 musl
76
+ (Alpine), macOS arm64, and Windows x64. The root-only smoke runs on each. The
77
+ seven-target source build matrix runs on release tags; packaging all seven is
78
+ not execution proof for every architecture. The smoke writes manager versions,
79
+ cases, and synthetic-fixture scope to `release-artifacts/consumer-proof.json`.
80
+
49
81
  ## Docs
50
82
 
51
- The docs site is rendered from `docs/*.md` by `scripts/build-docs-site.mjs`. Build locally to preview:
83
+ The docs site is rendered recursively from Markdown files under `docs/` by `scripts/build-docs-site.mjs`. Build locally to preview:
52
84
 
53
85
  ```bash
54
- node scripts/build-docs-site.mjs
86
+ pnpm docs:site
55
87
  open dist/docs-site/index.html
56
88
  ```
57
89
 
58
- The build validates internal links and embedded anchors. Broken links fail the build — fix them before pushing.
90
+ The build validates internal links and embedded anchors. Broken links fail the build — fix them before pushing. Navigation must list every non-excluded Markdown page exactly once: missing, nonexistent, and duplicate entries fail before the build replaces existing site output. The builder and navigation tests share discovery and validation in `scripts/docs-site-navigation.mjs`.
59
91
 
60
92
  Adding a new doc page:
61
93
 
62
94
  1. Create `docs/<page>.md`. Use a leading `# Title` heading.
63
- 2. Add the page to the appropriate section in `scripts/build-docs-site.mjs` (`sections` array near the top).
95
+ 2. Add the page to the appropriate section in `scripts/docs-site-navigation.mjs` (`sections` array near the top). Nested pages use slash-separated paths relative to `docs/`, such as `guides/example.md`.
64
96
  3. Cross-link from `docs/index.md` if it's a major surface.
65
- 4. Re-run the local build.
97
+ 4. Run `pnpm test test/docs-site-navigation.test.ts` and re-run the local build.
66
98
 
67
99
  Internal links use relative `*.md` paths — the builder rewrites them to the rendered HTML. Code fences support GitHub-flavored markdown.
68
100
 
@@ -79,8 +111,14 @@ Small, focused PRs land faster. The general shape:
79
111
 
80
112
  Maintainers publish from a protected `vX.Y.Z` tag on `main` through
81
113
  `.github/workflows/release.yml`. The workflow requires the package version and a
82
- dated `CHANGELOG.md` section to match the tag, then publishes with npm trusted
83
- publishing and provenance before creating the GitHub release.
114
+ dated `CHANGELOG.md` section to match the tag. It builds and publishes all seven
115
+ platform packages before publishing `@openclaw/fs-safe`, verifies every registry
116
+ artifact and provenance statement, and then creates the GitHub release.
117
+
118
+ Each package needs its own npm trusted-publisher configuration for
119
+ `openclaw/fs-safe` and `release.yml`. A new platform package must be created and
120
+ configured on npm before the first tag that references it; npm trust is
121
+ package-specific and cannot be bootstrapped by the tag workflow itself.
84
122
 
85
123
  External contributors do not need to do anything beyond getting the pull
86
124
  request merged. Maintainers must not publish locally or add npm automation
@@ -63,6 +63,12 @@ into proof for a different directory.
63
63
 
64
64
  Call `close()` in `finally`. Closing is idempotent; using a closed pin fails.
65
65
 
66
+ These checks intentionally reject a moved or replaced pathname. For one file's
67
+ abort cleanup through its original directory after a move, use the separate
68
+ [retained-directory staging lifecycle](staged-file.md). Its cleanup authority
69
+ does not weaken `pinDirectory().assertCurrent()` or `.sync()`, and namespace
70
+ cleanup is not proof of crash durability.
71
+
66
72
  ## Durable directory creation
67
73
 
68
74
  `ensureDurableDirectory()` finds and pins the nearest existing ancestor,
@@ -204,10 +210,17 @@ try {
204
210
  The result is `{ bytes, digest }`, where `digest` is lowercase hexadecimal.
205
211
  The handle overload never closes the caller's descriptor and uses positioned
206
212
  reads, so it does not alter the descriptor's current offset. The path overload
207
- rejects symbolic links and non-regular files, verifies the opened descriptor
208
- still names the requested path, and closes its own handle. POSIX opens are
209
- nonblocking, so a raced FIFO or device is rejected after descriptor inspection
210
- rather than waiting for a writer.
213
+ rejects symbolic links and non-regular files, compares lossless bigint identities
214
+ from the pre-open pathname inspection to the opened descriptor and from that
215
+ descriptor to the current pathname, and closes its own handle. All identity
216
+ checks complete before any JavaScript or native hashing. Each inspection allows
217
+ one bounded retry for unknown Windows identity components, retaining known
218
+ components and rejecting known differences immediately. Persistent unknown
219
+ identity fails closed with `path-mismatch`, even for benign files: this trades
220
+ availability for verifiable identity. Retries inspect the same descriptor or
221
+ pathname without reopening the file and repeat the symlink and file-type checks.
222
+ POSIX opens are nonblocking, so a raced FIFO or device is rejected after
223
+ descriptor inspection rather than waiting for a writer.
211
224
 
212
225
  When the optional binding is active, hashing runs as an async native task and
213
226
  does not occupy the JavaScript event loop with digest updates. With native mode
package/docs/errors.md CHANGED
@@ -2,6 +2,15 @@
2
2
 
3
3
  Every failure that's the library's job to surface lands as an `FsSafeError` with a closed `code` union you can branch on. Catch by code, not by message text — messages may change, codes will not.
4
4
 
5
+ Result-based APIs are an exception to throwing: [`pathScope().ensureDir()`](path-scope.md#ensuredir-rel-options)
6
+ and `ensureDirectoryWithinRoot()` return operational failures as
7
+ `{ ok: false, error: string, diagnostic: FsSafeError }`. The diagnostic uses
8
+ `helper-failed` / `operational`, retains the exact native error in `cause`, and
9
+ has the same message as `error`. Its display text names the native code/syscall
10
+ when available but omits raw paths and native messages. Policy failures keep
11
+ the string-only result without `diagnostic`; other `pathScope` methods do not
12
+ gain this field. Directory preparation can partially complete before failing.
13
+
5
14
  Path and archive-entry details embedded in diagnostics escape control characters
6
15
  as `\\uXXXX` sequences. This keeps attacker-controlled names on one log line;
7
16
  the escaped message is for diagnosis, not for reconstructing the original path.
@@ -105,7 +114,7 @@ type FsSafeErrorCode =
105
114
  | `device-path` | A read/open target is a known unsafe device or process-fd path. | `/dev/zero`, `/dev/random`, `/dev/stdin`, `/dev/fd/*`, `/proc/*/fd/*`, or a Windows reserved device name. |
106
115
  | `hardlink` | Read or copy with `hardlinks: "reject"` saw `nlink > 1`. | File is hardlinked — possibly an alias of an out-of-tree inode. |
107
116
  | `helper-failed` | A native mechanism or multi-step operational helper failed. | Inspect `cause` and any operation-specific `details`; retrying may be unsafe if the operation partially completed. |
108
- | `helper-unavailable` | Required native binding could not be loaded. | Unsupported platform, missing/incompatible bundled binary, or `FS_SAFE_NATIVE_MODE=require`. `auto` falls back where possible; `require` fails closed. |
117
+ | `helper-unavailable` | Required native binding could not be loaded. | Unsupported platform, omitted/missing/incompatible platform package, or `FS_SAFE_NATIVE_MODE=require`. `auto` falls back where possible; `require` fails closed. |
109
118
  | `insecure-permissions` | A secure file or path permission check found a mode/ACL that allows broader access than requested. | File or directory is group/world writable/readable; Windows ACL grants broad read. |
110
119
  | `invalid-path` | Input was empty, contained NUL, was an unparseable URL, or otherwise unusable. | Caller didn't validate input; input was a network path on Windows, a drive-relative segment in a portable relative path or store key, or a leading drive-relative spelling such as `C:name` used as a Root destination. Existing-object Root lookups retain legal POSIX drive-like names. |
111
120
  | `not-empty` | `remove()` on a non-empty directory. | Use `replaceDirectoryAtomic` or remove children first. |
@@ -125,6 +134,12 @@ type FsSafeErrorCode =
125
134
  | `too-large` | A read or bounded walk exceeded its configured budget. | Caller gave a too-permissive file or traversal limit. |
126
135
  | `unsupported-platform` | Reserved compatibility code for a platform-specific operation. | No current public helper emits this `FsSafeError` code. Platform-specific APIs currently return a typed unsupported result or use `helper-unavailable`; keep the union member when exhaustively switching across supported package versions. |
127
136
 
137
+ Pathname `sha256File()` also reports `path-mismatch` when pre-open, descriptor,
138
+ or current-path identity remains unknown after one bounded Windows retry, even
139
+ if the file is benign. It never reopens to recover identity. Preview symlinks
140
+ report `symlink`, preview or descriptor non-files report `not-file`, and a
141
+ current-path symlink or non-file reports `path-mismatch`.
142
+
128
143
  ## Branching
129
144
 
130
145
  ```ts
package/docs/index.md CHANGED
@@ -39,6 +39,7 @@ await fs.remove("notes/archive/today.txt");
39
39
 
40
40
  - **First time?** [Install](install.md), then walk through the [Quickstart](quickstart.md). Five minutes from `pnpm add` to a working root.
41
41
  - **Upgrading from 0.4?** Follow [Migrating to 0.5](migrating-to-0.5.md) in order, including the archive clamp-default audit.
42
+ - **Upgrading from 0.5?** Follow [Migrating to 0.6](migrating-to-0.6.md) to adopt platform-native optional packages.
42
43
  - **Designing a workspace feature.** Read the [Security model](security-model.md) before you trust the boundary, the [native helper policy](native-helper.md) before you pick deployment defaults, and the [Errors](errors.md) reference so you know what to catch.
43
44
  - **Replacing ad-hoc atomic writes.** Jump to [Atomic writes](atomic.md) or, for keyed JSON state, [JSON files](json.md).
44
45
  - **Extracting an upload.** Start at [Archive extraction](archive.md) — handles ZIP and TAR with traversal, link, count, and byte limits.
@@ -51,7 +52,7 @@ await fs.remove("notes/archive/today.txt");
51
52
  |---|---|
52
53
  | [`root()`](root.md) | One boundary for read/write/move/remove and bounded recursive walking inside a trusted directory. |
53
54
  | [`@openclaw/fs-safe/config`](config.md) | Process-global native helper and lock-option defaults. |
54
- | [Native helper policy](native-helper.md) | Choose `auto`, `off`, or `require` for bundled native primitives. |
55
+ | [Native helper policy](native-helper.md) | Choose `auto`, `off`, or `require` for platform-native primitives. |
55
56
  | [Native architecture](native.md) | Understand the thin syscall layer, beneath model, platform mechanisms, and fallback boundary. |
56
57
  | [`replaceFileAtomic`](atomic.md) | Sibling-temp + rename, fsync hooks, mode preservation, copy fallback. |
57
58
  | [`@openclaw/fs-safe/durability`](durability.md) | Pinned directory identities, durable creation, exclusive publication, streaming SHA-256, provenance receipts, and sync-failure policy. |
@@ -74,6 +75,7 @@ await fs.remove("notes/archive/today.txt");
74
75
  | [`@openclaw/fs-safe/advanced`](advanced.md) | Directory of lower-level composition helpers (path scopes, regular-file I/O, install paths, sibling-temp writes, …). |
75
76
  | [`@openclaw/fs-safe/test-hooks`](test-hooks.md) | Test-only injection hooks for reproducing open/lstat races. |
76
77
  | [Migrating to 0.5](migrating-to-0.5.md) | End-to-end checklist for Python removal and 0.4 API behavior changes. |
78
+ | [Migrating to 0.6](migrating-to-0.6.md) | Installer-policy checklist for the platform-native package split. |
77
79
 
78
80
  ## Status
79
81
 
package/docs/install.md CHANGED
@@ -85,19 +85,27 @@ Use the main entry for the common surface, or the focused subpaths when you want
85
85
 
86
86
  ## Runtime dependencies
87
87
 
88
- `@openclaw/fs-safe` lists `jszip` and `tar` as optional dependencies for [archive extraction](archive.md). They are loaded lazily and only required when ZIP/TAR helpers run. Installs that omit optional dependencies can still import and use every non-archive subpath; archive calls fail with a clear missing-optional-dependency message.
88
+ `@openclaw/fs-safe` lists `jszip` and `tar` as optional dependencies for JavaScript ZIP/TAR [archive extraction](archive.md). They are loaded lazily; the JavaScript archive fallback requires the corresponding codec and reports a missing-optional-dependency error without it. Public subpaths remain safe to import with all optional dependencies omitted, but imports do not prove native availability.
89
89
 
90
- There are no peer dependencies. The single npm package bundles all seven native binaries, so consumers do not run a native build, download platform code, or execute a postinstall step. Shipping every target increases the tarball size compared with per-platform packages, intentionally trading bandwidth for deterministic installation.
90
+ There are no peer dependencies. Exact-version optional packages carry the seven
91
+ native targets and npm-compatible OS, CPU, and Linux libc filters install only
92
+ the matching binary. Consumers do not run a native build, download code at
93
+ runtime, or execute a postinstall step. Omitting optional dependencies keeps
94
+ non-archive fallback-capable operations working in `auto` or `off`. Native-only
95
+ features, including retained-directory staging, atomic `rename-noreplace`,
96
+ zstd/bzip2 TAR handling, and Windows private-directory creation, remain
97
+ unavailable. Operations needing the binding in `require` mode fail with
98
+ `helper-unavailable` when the matching package is absent or incompatible.
91
99
 
92
- Upgrading an existing consumer? Follow [Migrating to 0.5](migrating-to-0.5.md)
93
- before choosing a native mode or accepting the new archive clamp default.
100
+ Upgrading an existing 0.5 consumer? Follow [Migrating to 0.6](migrating-to-0.6.md)
101
+ before deploying with native mode `require` or native-only features.
94
102
 
95
103
  ## Native helper policy
96
104
 
97
- The bundled native binaries provide fd-relative open/link/mkdir primitives,
105
+ The platform native binaries provide fd-relative open/link/mkdir primitives,
98
106
  atomic no-replace rename, and file identity checks. The default is `auto`: use
99
107
  the matching binary when it loads, otherwise silently keep the guarded
100
- JavaScript path. Platforms without one of the seven bundled targets therefore
108
+ JavaScript path. Platforms without one of the seven published targets therefore
101
109
  continue through the documented fallback in `auto` mode.
102
110
 
103
111
  ```ts
@@ -116,7 +124,7 @@ FS_SAFE_NATIVE_MODE=off # auto | off | require
116
124
 
117
125
  `OPENCLAW_FS_SAFE_NATIVE_MODE` is also accepted.
118
126
 
119
- Disabling native loading keeps the public API working through Node path
127
+ Disabling native loading keeps fallback-capable operations working through Node path
120
128
  operations guarded by lexical and canonical checks plus identity verification.
121
129
  Use `require` when native-backed operations must fail instead of falling back.
122
130
  The exact boundary is documented in [native helper policy](native-helper.md).
@@ -8,8 +8,8 @@ description: "Ordered checklist for moving a 0.4 consumer from the Python helper
8
8
  Use this checklist from top to bottom. Version 0.5 replaces the Python worker,
9
9
  changes the default archive mode policy, and adds explicit contracts for
10
10
  publication, walking, locks, secrets, and native-only features. Nothing in this
11
- guide requires a Rust toolchain: all supported native binaries are prebuilt and
12
- bundled in `@openclaw/fs-safe`.
11
+ guide requires a Rust toolchain: all supported native binaries are prebuilt.
12
+ Current releases install the matching exact-version optional platform package.
13
13
 
14
14
  ## 1. Update the package and runtime
15
15
 
@@ -17,10 +17,10 @@ bundled in `@openclaw/fs-safe`.
17
17
  - Update `@openclaw/fs-safe` and regenerate every lock or shrinkwrap file your
18
18
  deployment consumes.
19
19
  - Keep optional dependencies enabled if you want JavaScript ZIP/TAR support.
20
- Native loading no longer depends on optional packages because all seven
21
- binaries ship in `@openclaw/fs-safe`. An install that omits optional packages
22
- can still import fs-safe, but missing JS archive decoders fail with actionable
23
- errors.
20
+ Native loading no longer depends on Python. Keep optional dependencies enabled
21
+ for the matching native package and JavaScript ZIP/TAR decoders. An install
22
+ that omits them can still import fs-safe, but native-only features and missing
23
+ JS archive decoders fail with actionable errors.
24
24
 
25
25
  If you call `resolveRootPath()` or `resolveRootPathSync()` directly, upgrade to
26
26
  0.5: versions through 0.4.7 could approve an in-root symlink traversal that
@@ -42,7 +42,7 @@ configureFsSafeNative({ mode: "auto" });
42
42
  | `configureFsSafePython({ mode })` | `configureFsSafeNative({ mode })` |
43
43
  | `FS_SAFE_PYTHON_MODE` | `FS_SAFE_NATIVE_MODE` |
44
44
  | `OPENCLAW_FS_SAFE_PYTHON_MODE` | `OPENCLAW_FS_SAFE_NATIVE_MODE` |
45
- | `pythonPath`, `FS_SAFE_PYTHON`, pinned-Python aliases | Nothing; bundled native binaries do not use an interpreter |
45
+ | `pythonPath`, `FS_SAFE_PYTHON`, pinned-Python aliases | Nothing; prebuilt native binaries do not use an interpreter |
46
46
 
47
47
  The old names warn once and map `auto`, `off`, or `require` so a shipped 0.4
48
48
  deployment does not silently change policy. Interpreter paths are ignored and
@@ -0,0 +1,43 @@
1
+ ---
2
+ title: Migrating to 0.6
3
+ description: "Upgrade checklist for the platform-native package split."
4
+ ---
5
+
6
+ # Migrating from 0.5 to 0.6
7
+
8
+ Version 0.6 moves native bindings out of the root `@openclaw/fs-safe` tarball
9
+ and into exact-version platform packages. This removes unrelated operating
10
+ system and architecture binaries from each installation.
11
+
12
+ ## Keep optional dependencies enabled for native mode
13
+
14
+ Package managers select one binding by OS, CPU, and Linux libc. A normal
15
+ install needs no command change:
16
+
17
+ ```bash
18
+ pnpm add @openclaw/fs-safe
19
+ ```
20
+
21
+ If a deployment currently installs 0.5 with `--omit=optional`, `--no-optional`,
22
+ or an equivalent lockfile policy, change that policy before upgrading when it
23
+ uses native mode `require` or any native-only feature. Version 0.5 kept its
24
+ binding in the root tarball; version 0.6 intentionally does not.
25
+
26
+ Omitting optional dependencies remains supported for fallback-capable APIs in
27
+ `auto` mode. It disables native-only features such as zstd/bzip2 TAR handling,
28
+ retained-directory staging, atomic `rename-noreplace`, and Windows private
29
+ directory creation. Native mode `require` reports `helper-unavailable` when the
30
+ matching package is absent or incompatible.
31
+
32
+ ## Deployment checklist
33
+
34
+ 1. Remove any option or policy that omits optional dependencies when native
35
+ support is required.
36
+ 2. Regenerate every lockfile or shrinkwrap file consumed by deployment.
37
+ 3. Verify that the lock contains the matching `@openclaw/fs-safe-*` package.
38
+ 4. Run a native-required operation on every deployed OS/libc target.
39
+ 5. Keep an `FS_SAFE_NATIVE_MODE=off` lane when the guarded JavaScript fallback
40
+ is part of the application contract.
41
+
42
+ No Rust toolchain, postinstall build, or runtime download is introduced. See
43
+ [Native helper policy](native-helper.md) for the exact fallback boundary.
@@ -1,11 +1,15 @@
1
1
  ---
2
2
  title: Native helper policy
3
- description: "How fs-safe loads its bundled native filesystem primitives and how auto, require, and off affect guarded fallbacks."
3
+ description: "How fs-safe loads its platform-specific native filesystem primitives and how auto, require, and off affect guarded fallbacks."
4
4
  ---
5
5
 
6
6
  # Native helper policy
7
7
 
8
- `@openclaw/fs-safe` itself contains seven prebuilt binaries for Linux x64/arm64 (glibc or musl), macOS x64/arm64, and Windows x64. The loader selects `dist/native/<target>/fs-safe-native.node` without optional platform packages, downloads, postinstall scripts, or a consumer Rust build. Carrying every target makes the npm tarball larger, but every installation receives the same complete artifact.
8
+ `@openclaw/fs-safe` declares seven exact-version optional packages for Linux
9
+ x64/arm64 (glibc or musl), macOS x64/arm64, and Windows x64. Package-manager
10
+ OS, CPU, and libc filters install only the matching package. The loader requires
11
+ that package lazily, without runtime downloads, postinstall scripts, or a
12
+ consumer Rust build.
9
13
 
10
14
  ```ts
11
15
  import { configureFsSafeNative } from "@openclaw/fs-safe/config";
@@ -21,15 +25,20 @@ The equivalent environment variables are `FS_SAFE_NATIVE_MODE` and `OPENCLAW_FS_
21
25
 
22
26
  | Mode | Behavior |
23
27
  |---|---|
24
- | `auto` | Prefer native primitives when the current bundled binary loads; otherwise silently use the guarded JavaScript path. |
25
- | `off` | Do not load a bundled binary. Use the guarded JavaScript path deterministically. |
28
+ | `auto` | Prefer native primitives when the current platform package loads; otherwise silently use the guarded JavaScript path. |
29
+ | `off` | Do not load a native package. Use the guarded JavaScript path deterministically. |
26
30
  | `require` | Throw `FsSafeError("helper-unavailable")` instead of falling back when an operation needs the native binding and it cannot load. |
27
31
 
28
32
  Configure the mode once during startup. Loading is lazy and cached; changing from `auto` to `require` after a failed load changes failure policy but does not repeatedly probe the binary.
29
33
 
34
+ [`stageFileInDirectory()`](staged-file.md) always requires native support on
35
+ Linux/macOS and rejects before creation when off, unavailable, or missing the
36
+ required capability. Windows is unsupported for this lifecycle. This does not
37
+ change the mode policy of existing fallback-capable APIs.
38
+
30
39
  ## Native boundary
31
40
 
32
- The bundled native layer exposes policy-free filesystem mechanisms: beneath-root
41
+ The native layer exposes policy-free filesystem mechanisms: beneath-root
33
42
  open/mkdir/link, replace and no-replace rename, identity reads, archive decode/execution,
34
43
  clone/copy/hash workers, and Windows security descriptor calls. The TypeScript
35
44
  layer owns policy, retries, filters, budgets, modes, cleanup, error
@@ -89,3 +98,4 @@ consumer performs its 0.5 upgrade.
89
98
  - [File locks](sidecar-lock.md)
90
99
  - [Durability](durability.md)
91
100
  - [Migrating to 0.5](migrating-to-0.5.md)
101
+ - [Migrating to 0.6](migrating-to-0.6.md)
package/docs/native.md CHANGED
@@ -1,11 +1,11 @@
1
1
  ---
2
2
  title: Native architecture
3
- description: "The bundled native bindings, fd-relative beneath model, platform mechanisms, loader security, and JavaScript fallback contract."
3
+ description: "The platform-specific native bindings, fd-relative beneath model, platform mechanisms, loader security, and JavaScript fallback contract."
4
4
  ---
5
5
 
6
6
  # Native architecture
7
7
 
8
- `@openclaw/fs-safe` bundles native bindings that supply mechanisms Node does
8
+ `@openclaw/fs-safe` uses native bindings that supply mechanisms Node does
9
9
  not expose directly. The Rust layer is deliberately not a second policy engine.
10
10
  TypeScript owns trusted-root selection, path validation, archive filtering,
11
11
  budgets, modes, identity fencing, cleanup decisions, and error normalization.
@@ -14,8 +14,9 @@ platform syscall sequence that can preserve the boundary.
14
14
 
15
15
  Every operation that has an equivalent safe Node implementation keeps that
16
16
  guarded JavaScript path. Native loading is lazy; installs do not compile Rust,
17
- run postinstall code, or fetch binaries. The npm tarball carries all seven
18
- supported targets, so it is larger than a per-platform package by design.
17
+ run postinstall code, or fetch binaries at runtime. Seven exact-version optional
18
+ packages are filtered by OS, CPU, and Linux libc, so an installation receives
19
+ only its matching prebuilt binding.
19
20
  Native-only formats and creation-time Windows DACL guarantees fail explicitly
20
21
  instead of substituting a weaker implementation.
21
22
 
@@ -86,9 +87,36 @@ workers rather than the JavaScript event loop.
86
87
  | `require` | Try once, cache the result | Throw `FsSafeError("helper-unavailable")` |
87
88
  | `off` | Never attempt a binding load | Always use guarded JavaScript |
88
89
 
89
- The one exception is functionality with no safe JavaScript implementation:
90
- zstd/bzip2 TAR and Windows private-directory creation fail with
91
- `helper-unavailable` when native support is absent or off.
90
+ Features without a safe JavaScript implementation, including zstd/bzip2 TAR,
91
+ Windows private-directory creation, and [retained-directory staging](staged-file.md),
92
+ fail with `helper-unavailable` when native support is absent or off. Staging
93
+ is currently Linux/macOS only and rejects Windows with `unsupported-platform`.
94
+
95
+ The staged-file owner also serves POSIX native pinned writes, including streaming.
96
+ Unpublished files remain at `0600`; requested modes are applied through the
97
+ owned file descriptor only after rename and published-entry identity validation.
98
+ Post-rename chmod or sync failures retain the publication receipt and final name.
99
+ Its direct-child exclusive `openat` hands off the descriptor before any fallible
100
+ post-open checks; non-following `statat` compares against that descriptor with
101
+ exact native identities, and cleanup uses `unlinkat` in the retained parent.
102
+ The separate checks and unlink are not atomic conditional deletion. Windows
103
+ pinned writes and other fallback-capable APIs retain their existing mechanisms.
104
+ Native writers share root and parent admission, but keep their platform identity
105
+ checks and leaf ownership. POSIX coordinator disposal uses `SuppressedError` to
106
+ retain both an operation failure and a disposal failure, including their receipts;
107
+ stage preparation and cleanup keep their documented error mappings.
108
+ Root replacement verification borrows the published descriptor after final mode
109
+ application, while a private coordinator retains the staged owner until the
110
+ asynchronous check finishes. The owner never escapes that coordinator; public
111
+ staging methods and receipts expose no descriptor or verification callback.
112
+ Verification failures preserve the published name, and disposal still retains
113
+ both verification and cleanup errors when both fail.
114
+ The private verification channel carries exact bigint identity from the original
115
+ owned descriptor (or the content-accepted FUSE descriptor). Root compares it
116
+ against exact fd and pathname metadata; legacy helper return facts and public
117
+ read metadata behavior are unchanged. Missing Windows pathname identity still
118
+ requires a guarded path reopen and comparison with the original retained file;
119
+ that fallback does not apply to POSIX no-read modes.
92
120
 
93
121
  ## JavaScript fallback guarantees and delta
94
122
 
@@ -116,10 +144,10 @@ infer native loading from timing.
116
144
  Importing fs-safe never executes a child process. Linux libc selection uses
117
145
  the Node process report, conventional musl library filenames, and the ELF
118
146
  `PT_INTERP` field of `process.execPath`. If all probes are inconclusive, the
119
- loader conservatively attempts the bundled glibc binary and lets normal module
120
- loading fail into `auto` fallback. The loader requires only
121
- `dist/native/<target>/fs-safe-native.node`; it never probes optional packages,
122
- downloads code, or runs a postinstall step. A missing or incompatible binary
147
+ loader conservatively attempts the glibc package and lets normal module loading
148
+ fail into `auto` fallback. The loader requires only the package selected from
149
+ the detected target; it never probes unrelated packages, downloads code, or
150
+ runs a postinstall step. A missing or incompatible binary
123
151
  silently selects the JavaScript fallback in `auto`, throws typed
124
152
  `helper-unavailable` in `require`, and is never inspected in `off`. Tests reject
125
153
  `child_process`, `exec`, or `spawn` usage in the loader.
@@ -132,3 +160,4 @@ silently selects the JavaScript fallback in `auto`, throws typed
132
160
  - [Durability](durability.md)
133
161
  - [Permissions](permissions.md)
134
162
  - [Migrating to 0.5](migrating-to-0.5.md)
163
+ - [Migrating to 0.6](migrating-to-0.6.md)
@@ -24,6 +24,8 @@ await sharp(photo.path).resize(800).toFile(/* … */);
24
24
  ## Signature
25
25
 
26
26
  ```ts
27
+ import type { FsSafeError } from "@openclaw/fs-safe/errors";
28
+
27
29
  function pathScope(rootDir: string, options: PathScopeOptions): PathScope;
28
30
 
29
31
  type PathScopeOptions = {
@@ -36,6 +38,10 @@ type PathScopeResolveOptions = {
36
38
 
37
39
  type PathResult = { ok: true; path: string } | { ok: false; error: string };
38
40
  type PathsResult = { ok: true; paths: string[] } | { ok: false; error: string };
41
+ // Illustrative result alias, not a named package export.
42
+ type DirectoryResult =
43
+ | { ok: true; path: string }
44
+ | { ok: false; error: string; diagnostic?: FsSafeError };
39
45
 
40
46
  type PathScope = {
41
47
  rootDir: string;
@@ -45,7 +51,7 @@ type PathScope = {
45
51
  existing(requestedPaths: string[]): Promise<PathsResult>;
46
52
  files(requestedPaths: string[]): Promise<PathsResult>;
47
53
  writable(requestedPath: string, options?: PathScopeResolveOptions): Promise<PathResult>;
48
- ensureDir(requestedPath: string, options?: PathScopeResolveOptions & { mode?: number }): Promise<PathResult>;
54
+ ensureDir(requestedPath: string, options?: PathScopeResolveOptions & { mode?: number }): Promise<DirectoryResult>;
49
55
  };
50
56
  ```
51
57
 
@@ -92,9 +98,29 @@ await fs.writeFile(t.path, body);
92
98
 
93
99
  Async. `mkdir -p` inside the scope. Walks each segment, refuses any symlink in the path, creates missing directories. Optional `mode` sets the directory mode.
94
100
 
101
+ Failures remain nonthrowing results with `error: string`. Policy rejections
102
+ (including traversal, NUL input, symlinks and non-directory segments) omit
103
+ `diagnostic`. Operational failures from `lstat`, `realpath` or `mkdir` include
104
+ an `FsSafeError` with `code: "helper-failed"`, `category: "operational"`, and
105
+ the exact original error in `cause`, including its native `code`, `errno` and
106
+ `syscall` when available. Only `ensureDir()` adds this diagnostic; other scope
107
+ methods retain their existing result shapes.
108
+
109
+ `error` equals `diagnostic.message` for operational failures, for example
110
+ `"Could not prepare uploads directory: ENAMETOOLONG during lstat"`. Display
111
+ text bounds and escapes the label and native code/syscall, without copying
112
+ the requested path or native error message. Treat the raw cause as sensitive
113
+ local diagnostic data, not as text to return to an untrusted caller. A failure
114
+ can leave already-created parent directories; it does not roll them back or
115
+ retry through an unchecked filesystem path.
116
+
95
117
  ```ts
96
118
  const dir = await uploads.ensureDir("inbox", { mode: 0o755 });
97
- if (!dir.ok) return reply(500, dir.error);
119
+ if (!dir.ok) {
120
+ // Pass the cause only to an appropriately restricted local diagnostic sink.
121
+ if (dir.diagnostic) recordLocalFailure(dir.diagnostic.cause);
122
+ return reply(dir.diagnostic ? 500 : 400, dir.error);
123
+ }
98
124
  ```
99
125
 
100
126
  ## Result type vs throwing
@@ -75,10 +75,21 @@ Built-in PowerShell, `icacls.exe`, and `whoami.exe` invocations have a fixed
75
75
  30-second per-process deadline. A command failure or timeout returns an
76
76
  unverified result (`source: "unknown"`) so callers fail closed. Advanced callers
77
77
  that inject a custom `exec` implementation own that executor's deadline.
78
+ Failed owner and ACL inspections retain `error` text and an optional
79
+ `errorDetail: PermissionCommandFailure` with `command`, integer `durationMs`,
80
+ `timedOut`, `exitCode`, `signal`, and `stderr`. The type is exported from both
81
+ `@openclaw/fs-safe/permissions` and `@openclaw/fs-safe/advanced`. Built-in
82
+ execution measures elapsed time; injected execFile-shaped failures receive
83
+ best-effort command diagnostics. Plain errors have no `errorDetail`.
84
+ Display reasons and stderr escape control characters and are limited to 400
85
+ characters, including a trailing `…` when truncated. Diagnostics do not copy
86
+ stdout or read target file contents. The separate `errorCause` retains the
87
+ original exception for restricted local diagnosis; do not serialize or expose
88
+ it as display text.
78
89
  The parser is on the advanced surface so tests and CLIs can process captured
79
90
  `icacls` output without spawning a process.
80
91
 
81
- When the bundled native binding is available, `inspectPathPermissions()`
92
+ When the native binding is available, `inspectPathPermissions()`
82
93
  reads the owner and DACL directly with Windows security APIs. It classifies the
83
94
  current user, LocalSystem, and built-in Administrators as trusted and reports
84
95
  the world/group read/write facts consumed by secure reads. Descriptor forms it
@@ -128,7 +139,7 @@ Object-specific and other ACE layouts are not guessed: they are omitted,
128
139
  `complete` becomes false, and their numeric types appear in
129
140
  `unsupportedAceTypes`, allowing a security-sensitive caller to fail closed.
130
141
  Non-Windows systems return `{ status: "unsupported-platform", platform }`.
131
- Windows requires the bundled native binding; if it is unavailable or forced
142
+ Windows requires the native binding; if it is unavailable or forced
132
143
  off, the call throws `FsSafeError("helper-unavailable")`. The existing coarse
133
144
  `inspectPathPermissions()` API still owns its compatibility fallback and trust
134
145
  classification.
package/docs/root.md CHANGED
@@ -140,7 +140,7 @@ new destination name is subject to the portable guard.
140
140
 
141
141
  ## Native helper mode
142
142
 
143
- Create-only writes prefer the bundled native helper for fd-relative opens and
143
+ Create-only writes prefer the platform native helper for fd-relative opens and
144
144
  atomic no-replace rename. Operations without native wiring retain their guarded
145
145
  JavaScript implementations.
146
146
 
@@ -89,6 +89,14 @@ credential must also fail on broad permissions or unexpected ownership.
89
89
  the same pinned-handle validation, byte cap, trimming, error codes, and strict
90
90
  versus missing-is-undefined naming semantics.
91
91
 
92
+ Both sync and async readers compare lossless bigint identities from the preview,
93
+ opened descriptor, resolved target, and current input path before reading. An
94
+ allowed symlink must still point to the opened file. On Windows, a zero device
95
+ or inode is unverified: that inspection is retried once without reopening the
96
+ file, preserving known identity components and link checks. Definite mismatches
97
+ and persistent ambiguity fail with `path-mismatch`; optional reads do not treat
98
+ these failures as missing files.
99
+
92
100
  If an already validated descriptor fails while reading, both readers throw an
93
101
  operational `FsSafeError` with `code: "read-failed"`; inspect `cause` for the
94
102
  underlying Node filesystem code such as `EIO`.
@@ -28,6 +28,8 @@ The helper:
28
28
 
29
29
  On POSIX, unsafe permissions mean group/world writable, and group/world readable unless `permissions.allowReadableByOthers` is true. On Windows, the helper uses the ACL inspection helpers from [`permissions`](permissions.md) and refuses the read if ACLs cannot be verified.
30
30
 
31
+ Descriptor, pathname, and realpath identity checks use lossless bigint stats internally. The returned `stat` remains a normal Node `Stats` object with numeric fields. A zero Windows device or inode is unverified, never a match: the helper re-inspects that identity once using the same descriptor or pathname, then rejects persistent ambiguity with `path-mismatch`. A definite mismatch rejects immediately; retries retain known identity components and still enforce symlink policy.
32
+
31
33
  ## Options
32
34
 
33
35
  ```ts
@@ -57,6 +59,8 @@ type SecureFileReadOptions = {
57
59
 
58
60
  `permissions.allowInsecure` is a migration escape hatch. Prefer fixing permissions and using [`formatPermissionRemediation`](permissions.md) to show the user what to run. `trust.allowNetworkPath` is off by default because UNC paths are remote authority, not local filesystem input. `inject` is for tests and platform adapters; production callers usually leave it unset.
59
61
 
62
+ `permissions.allowInsecure` bypasses only permission checks. Neither it nor `inject.platform` changes filesystem identity verification, which always uses the actual process platform. `trust.allowSymlink` permits an alias but still requires its target and realpath to match the opened descriptor.
63
+
60
64
  ## Errors
61
65
 
62
66
  `readSecureFile()` throws `FsSafeError` with codes such as:
@@ -67,7 +71,7 @@ type SecureFileReadOptions = {
67
71
  | `not-found` | The path could not be stat'd before open. |
68
72
  | `not-file` | The opened target is not a regular file. |
69
73
  | `symlink` | The path is a symlink and `trust.allowSymlink` is false. |
70
- | `path-mismatch` | The path or realpath changed between open and verification. |
74
+ | `path-mismatch` | The path or realpath changed between open and verification, or filesystem identity could not be verified after bounded re-inspection. |
71
75
  | `outside-workspace` | `realPath` is outside `trust.trustedDirs`. |
72
76
  | `permission-unverified` | Required mode/ACL checks could not be completed. |
73
77
  | `insecure-permissions` | Mode bits or ACLs grant broader access than allowed. |
@@ -75,6 +79,18 @@ type SecureFileReadOptions = {
75
79
  | `too-large` | File size or bytes read exceeded `maxBytes`. |
76
80
  | `timeout` | `timeoutMs` elapsed while reading. |
77
81
 
82
+ Windows inspection failures remain operational `permission-unverified` errors
83
+ and still refuse the read. Their message includes the underlying reason when
84
+ available. `details` includes `ownerError` for owner-query failures and, when
85
+ command diagnostics are available, `command`, `durationMs`, `timedOut`,
86
+ `exitCode`, `signal`, and `stderr`. Reasons and stderr are control-character
87
+ escaped and limited to 400 characters each (including a truncation marker).
88
+ No stdout or target file contents are copied into these display diagnostics.
89
+ The original inspection exception is retained as `cause`; built-in command
90
+ errors also retain their original execFile exception in the cause chain.
91
+ Treat causes as restricted local diagnostic data. No retries are performed,
92
+ and verification order and rejection conditions are unchanged.
93
+
78
94
  ## See also
79
95
 
80
96
  - [Permissions](permissions.md) — standalone POSIX mode and Windows ACL checks.