@openclaw/fs-safe 0.4.6 → 0.4.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +24 -0
- package/README.md +31 -1
- package/dist/directory-durability.d.ts +44 -0
- package/dist/directory-durability.d.ts.map +1 -0
- package/dist/directory-durability.js +299 -0
- package/dist/durability.d.ts +2 -0
- package/dist/durability.d.ts.map +1 -0
- package/dist/durability.js +1 -0
- package/dist/fsync.d.ts +1 -1
- package/dist/fsync.d.ts.map +1 -1
- package/dist/fsync.js +1 -21
- package/dist/sibling-temp.d.ts.map +1 -1
- package/dist/sibling-temp.js +1 -13
- package/docs/advanced.md +1 -1
- package/docs/durability.md +89 -0
- package/docs/index.md +1 -0
- package/docs/security-model.md +13 -0
- package/package.json +5 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,29 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.4.7 - 2026-07-24
|
|
4
|
+
|
|
5
|
+
### Features
|
|
6
|
+
|
|
7
|
+
- Add `@openclaw/fs-safe/durability` with identity-pinned directory handles,
|
|
8
|
+
explicit strict sync outcomes, synchronous and best-effort variants, and
|
|
9
|
+
durable nested-directory creation through every new parent edge.
|
|
10
|
+
|
|
11
|
+
### Security and Correctness
|
|
12
|
+
|
|
13
|
+
- Reject final symlinks, FIFOs, non-directories, canonical-path drift, and
|
|
14
|
+
descriptor/path identity replacement before or after directory sync.
|
|
15
|
+
- Propagate POSIX directory synchronization failures while classifying known
|
|
16
|
+
unsupported Windows directory flushing only after revalidating the target;
|
|
17
|
+
directory-open access failures remain strict.
|
|
18
|
+
- Route sibling-temp, root, and pinned-write best-effort parent synchronization
|
|
19
|
+
through the shared guarded primitive instead of maintaining divergent implementations.
|
|
20
|
+
|
|
21
|
+
### Docs and Tooling
|
|
22
|
+
|
|
23
|
+
- Document directory receipts, pin lifecycle, platform outcomes, custom
|
|
24
|
+
creation callbacks, and the boundary between filesystem durability and
|
|
25
|
+
application commit protocols.
|
|
26
|
+
|
|
3
27
|
## 0.4.6 - 2026-07-24
|
|
4
28
|
|
|
5
29
|
### Highlights
|
package/README.md
CHANGED
|
@@ -26,7 +26,7 @@ Full docs and reference at **[fs-safe.io](https://fs-safe.io)**.
|
|
|
26
26
|
|
|
27
27
|
## Contents
|
|
28
28
|
|
|
29
|
-
[Why this exists](#why-this-exists) · [Not a sandbox](#not-a-sandbox) · [Install](#install) · [Quick start](#quick-start) · [Reading](#reading) · [Subpaths](#subpaths) · [Failure semantics](#failure-semantics-in-the-name) · [Atomic writes](#atomic-writes) · [External outputs](#external-outputs) · [Stores](#stores) · [Secure absolute reads](#secure-absolute-file-reads) · [Walking](#directory-walking) · [Archive extraction](#archive-extraction) · [Path scopes](#advanced-path-scopes) · [Errors](#errors) · [Safety model](#safety-model) · [Limitations](#limitations)
|
|
29
|
+
[Why this exists](#why-this-exists) · [Not a sandbox](#not-a-sandbox) · [Install](#install) · [Quick start](#quick-start) · [Reading](#reading) · [Subpaths](#subpaths) · [Failure semantics](#failure-semantics-in-the-name) · [Directory durability](#directory-durability) · [Atomic writes](#atomic-writes) · [External outputs](#external-outputs) · [Stores](#stores) · [Secure absolute reads](#secure-absolute-file-reads) · [Walking](#directory-walking) · [Archive extraction](#archive-extraction) · [Path scopes](#advanced-path-scopes) · [Errors](#errors) · [Safety model](#safety-model) · [Limitations](#limitations)
|
|
30
30
|
|
|
31
31
|
## Why this exists
|
|
32
32
|
|
|
@@ -189,6 +189,7 @@ that OpenClaw needs to compose higher-level APIs are grouped under
|
|
|
189
189
|
| `@openclaw/fs-safe/store` | `fileStore`, `fileStoreSync`, and `jsonStore` |
|
|
190
190
|
| `@openclaw/fs-safe/secret` | strict and try-style secret file read/write helpers |
|
|
191
191
|
| `@openclaw/fs-safe/atomic` | `replaceFileAtomic`, `replaceFileAtomicSync`, `replaceDirectoryAtomic`, `movePathWithCopyFallback` |
|
|
192
|
+
| `@openclaw/fs-safe/durability` | pinned directory identities, strict and best-effort directory sync, durable nested-directory creation |
|
|
192
193
|
| `@openclaw/fs-safe/temp` | `tempWorkspace`, `tempWorkspaceSync`, `withTempWorkspace`, `resolveSecureTempRoot` |
|
|
193
194
|
| `@openclaw/fs-safe/secure-file` | fd-pinned absolute file reads with owner, mode, ACL, trusted-dir, size, and timeout checks |
|
|
194
195
|
| `@openclaw/fs-safe/file-lock` | `acquireFileLock`, `withFileLock`, `createFileLockManager`, and related lock types |
|
|
@@ -216,6 +217,35 @@ performs the root-bounded open and JSON object validation in one step. Use
|
|
|
216
217
|
`readRootStructuredFileSync()` when the parser lives outside fs-safe, such as
|
|
217
218
|
JSON5-backed plugin manifests.
|
|
218
219
|
|
|
220
|
+
## Directory durability
|
|
221
|
+
|
|
222
|
+
```ts
|
|
223
|
+
import { ensureDurableDirectory, pinDirectory } from "@openclaw/fs-safe/durability";
|
|
224
|
+
|
|
225
|
+
const receipt = await ensureDurableDirectory({
|
|
226
|
+
directoryPath: "/srv/backups/sqlite",
|
|
227
|
+
mode: 0o700,
|
|
228
|
+
});
|
|
229
|
+
const pinned = await pinDirectory(receipt);
|
|
230
|
+
try {
|
|
231
|
+
await publishSnapshot();
|
|
232
|
+
const outcome = await pinned.sync();
|
|
233
|
+
// `unsupported` is explicit on platforms without directory flushing.
|
|
234
|
+
console.log(outcome.status);
|
|
235
|
+
} finally {
|
|
236
|
+
await pinned.close();
|
|
237
|
+
}
|
|
238
|
+
```
|
|
239
|
+
|
|
240
|
+
The durability subpath pins a directory descriptor to its pathname identity,
|
|
241
|
+
detects symlink/FIFO/replacement races, and synchronizes every new parent edge
|
|
242
|
+
when creating a nested directory. Strict sync propagates real I/O failures and
|
|
243
|
+
reports known Windows directory-flush limitations explicitly. Separate
|
|
244
|
+
best-effort helpers preserve operations that do not promise crash durability.
|
|
245
|
+
|
|
246
|
+
See [Directory durability](docs/durability.md) for the receipt, pin lifecycle,
|
|
247
|
+
creation callback, and platform contract.
|
|
248
|
+
|
|
219
249
|
## Atomic writes
|
|
220
250
|
|
|
221
251
|
`replaceFileAtomic()` writes a sibling temp file, optionally fsyncs it, and renames it over the destination. Mode preservation, rename retry / copy fallback on `EPERM`, parent-directory fsync, and a `beforeRename` hook for backup or observer flows are all opt-in. `movePathWithCopyFallback()` stages cross-device moves before commit and removes only the copied source entries, so concurrent source additions or replacements are preserved.
|
|
@@ -0,0 +1,44 @@
|
|
|
1
|
+
import { type Stats } from "node:fs";
|
|
2
|
+
import { type FileIdentityStat } from "./file-identity.js";
|
|
3
|
+
export type DirectorySyncOutcome = {
|
|
4
|
+
status: "synced";
|
|
5
|
+
} | {
|
|
6
|
+
status: "unsupported";
|
|
7
|
+
code?: string;
|
|
8
|
+
};
|
|
9
|
+
export type DirectoryReceipt = {
|
|
10
|
+
path: string;
|
|
11
|
+
realPath: string;
|
|
12
|
+
identity: Stats;
|
|
13
|
+
};
|
|
14
|
+
export type DurableDirectoryReceipt = DirectoryReceipt & {
|
|
15
|
+
parentSync: DirectorySyncOutcome | {
|
|
16
|
+
status: "not-needed";
|
|
17
|
+
};
|
|
18
|
+
};
|
|
19
|
+
export type PinnedDirectory = {
|
|
20
|
+
readonly receipt: DirectoryReceipt;
|
|
21
|
+
assertCurrent(): Promise<void>;
|
|
22
|
+
sync(): Promise<DirectorySyncOutcome>;
|
|
23
|
+
close(): Promise<void>;
|
|
24
|
+
};
|
|
25
|
+
export type EnsureDurableDirectoryOptions = {
|
|
26
|
+
directoryPath: string;
|
|
27
|
+
label?: string;
|
|
28
|
+
mode?: number;
|
|
29
|
+
expectedExistingIdentity?: FileIdentityStat;
|
|
30
|
+
create?: (directoryPath: string) => Promise<void>;
|
|
31
|
+
};
|
|
32
|
+
export declare function pinDirectory(directory: string | DirectoryReceipt, options?: {
|
|
33
|
+
label?: string;
|
|
34
|
+
}): Promise<PinnedDirectory>;
|
|
35
|
+
export declare function syncDirectory(directory: string | DirectoryReceipt, options?: {
|
|
36
|
+
label?: string;
|
|
37
|
+
}): Promise<DirectorySyncOutcome>;
|
|
38
|
+
export declare function syncDirectorySync(directory: string | DirectoryReceipt, options?: {
|
|
39
|
+
label?: string;
|
|
40
|
+
}): DirectorySyncOutcome;
|
|
41
|
+
export declare function syncDirectoryBestEffort(directoryPath: string): Promise<void>;
|
|
42
|
+
export declare function syncDirectoryBestEffortSync(directoryPath: string): void;
|
|
43
|
+
export declare function ensureDurableDirectory(options: EnsureDurableDirectoryOptions): Promise<DurableDirectoryReceipt>;
|
|
44
|
+
//# sourceMappingURL=directory-durability.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"directory-durability.d.ts","sourceRoot":"","sources":["../src/directory-durability.ts"],"names":[],"mappings":"AAAA,OAAe,EAAE,KAAK,KAAK,EAAE,MAAM,SAAS,CAAC;AAM7C,OAAO,EAAoB,KAAK,gBAAgB,EAAE,MAAM,oBAAoB,CAAC;AAE7E,MAAM,MAAM,oBAAoB,GAC5B;IAAE,MAAM,EAAE,QAAQ,CAAA;CAAE,GACpB;IAAE,MAAM,EAAE,aAAa,CAAC;IAAC,IAAI,CAAC,EAAE,MAAM,CAAA;CAAE,CAAC;AAE7C,MAAM,MAAM,gBAAgB,GAAG;IAC7B,IAAI,EAAE,MAAM,CAAC;IACb,QAAQ,EAAE,MAAM,CAAC;IACjB,QAAQ,EAAE,KAAK,CAAC;CACjB,CAAC;AAEF,MAAM,MAAM,uBAAuB,GAAG,gBAAgB,GAAG;IACvD,UAAU,EAAE,oBAAoB,GAAG;QAAE,MAAM,EAAE,YAAY,CAAA;KAAE,CAAC;CAC7D,CAAC;AAEF,MAAM,MAAM,eAAe,GAAG;IAC5B,QAAQ,CAAC,OAAO,EAAE,gBAAgB,CAAC;IACnC,aAAa,IAAI,OAAO,CAAC,IAAI,CAAC,CAAC;IAC/B,IAAI,IAAI,OAAO,CAAC,oBAAoB,CAAC,CAAC;IACtC,KAAK,IAAI,OAAO,CAAC,IAAI,CAAC,CAAC;CACxB,CAAC;AAEF,MAAM,MAAM,6BAA6B,GAAG;IAC1C,aAAa,EAAE,MAAM,CAAC;IACtB,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,wBAAwB,CAAC,EAAE,gBAAgB,CAAC;IAC5C,MAAM,CAAC,EAAE,CAAC,aAAa,EAAE,MAAM,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC;CACnD,CAAC;AAgKF,wBAAsB,YAAY,CAChC,SAAS,EAAE,MAAM,GAAG,gBAAgB,EACpC,OAAO,GAAE;IAAE,KAAK,CAAC,EAAE,MAAM,CAAA;CAAO,GAC/B,OAAO,CAAC,eAAe,CAAC,CAa1B;AAED,wBAAsB,aAAa,CACjC,SAAS,EAAE,MAAM,GAAG,gBAAgB,EACpC,OAAO,GAAE;IAAE,KAAK,CAAC,EAAE,MAAM,CAAA;CAAO,GAC/B,OAAO,CAAC,oBAAoB,CAAC,CAmB/B;AAED,wBAAgB,iBAAiB,CAC/B,SAAS,EAAE,MAAM,GAAG,gBAAgB,EACpC,OAAO,GAAE;IAAE,KAAK,CAAC,EAAE,MAAM,CAAA;CAAO,GAC/B,oBAAoB,CAuCtB;AAED,wBAAsB,uBAAuB,CAAC,aAAa,EAAE,MAAM,GAAG,OAAO,CAAC,IAAI,CAAC,CAElF;AAED,wBAAgB,2BAA2B,CAAC,aAAa,EAAE,MAAM,GAAG,IAAI,CAMvE;AAuBD,wBAAsB,sBAAsB,CAC1C,OAAO,EAAE,6BAA6B,GACrC,OAAO,CAAC,uBAAuB,CAAC,CA8ElC"}
|
|
@@ -0,0 +1,299 @@
|
|
|
1
|
+
import fsSync, {} from "node:fs";
|
|
2
|
+
import fs from "node:fs/promises";
|
|
3
|
+
import path from "node:path";
|
|
4
|
+
import { ensureAbsoluteDirectory } from "./absolute-path.js";
|
|
5
|
+
import { FsSafeError } from "./errors.js";
|
|
6
|
+
import { sameFileIdentity } from "./file-identity.js";
|
|
7
|
+
function directoryOpenFlags() {
|
|
8
|
+
if (process.platform === "win32") {
|
|
9
|
+
return "r";
|
|
10
|
+
}
|
|
11
|
+
return (fsSync.constants.O_RDONLY |
|
|
12
|
+
fsSync.constants.O_DIRECTORY |
|
|
13
|
+
fsSync.constants.O_NOFOLLOW |
|
|
14
|
+
fsSync.constants.O_NONBLOCK);
|
|
15
|
+
}
|
|
16
|
+
function isWindowsDirectorySyncUnsupported(error) {
|
|
17
|
+
if (process.platform !== "win32") {
|
|
18
|
+
return false;
|
|
19
|
+
}
|
|
20
|
+
const code = error.code;
|
|
21
|
+
return (code === "EACCES" ||
|
|
22
|
+
code === "EINVAL" ||
|
|
23
|
+
code === "EISDIR" ||
|
|
24
|
+
code === "ENOSYS" ||
|
|
25
|
+
code === "ENOTSUP" ||
|
|
26
|
+
code === "EPERM");
|
|
27
|
+
}
|
|
28
|
+
function isWindowsDirectoryOpenUnsupported(error) {
|
|
29
|
+
if (process.platform !== "win32") {
|
|
30
|
+
return false;
|
|
31
|
+
}
|
|
32
|
+
const code = error.code;
|
|
33
|
+
return code === "EINVAL" || code === "EISDIR" || code === "ENOSYS" || code === "ENOTSUP";
|
|
34
|
+
}
|
|
35
|
+
function unsupportedOutcome(error) {
|
|
36
|
+
const code = error.code;
|
|
37
|
+
return code ? { status: "unsupported", code } : { status: "unsupported" };
|
|
38
|
+
}
|
|
39
|
+
function assertDirectory(identity, pathname, label) {
|
|
40
|
+
if (identity.isSymbolicLink() || !identity.isDirectory()) {
|
|
41
|
+
throw new FsSafeError("not-file", `${label} must be a real directory: ${pathname}`);
|
|
42
|
+
}
|
|
43
|
+
}
|
|
44
|
+
async function createDirectoryReceipt(directoryPath, label) {
|
|
45
|
+
const resolvedPath = path.resolve(directoryPath);
|
|
46
|
+
const identity = await fs.lstat(resolvedPath);
|
|
47
|
+
assertDirectory(identity, resolvedPath, label);
|
|
48
|
+
return {
|
|
49
|
+
path: resolvedPath,
|
|
50
|
+
realPath: await fs.realpath(resolvedPath),
|
|
51
|
+
identity,
|
|
52
|
+
};
|
|
53
|
+
}
|
|
54
|
+
function createDirectoryReceiptSync(directoryPath, label) {
|
|
55
|
+
const resolvedPath = path.resolve(directoryPath);
|
|
56
|
+
const identity = fsSync.lstatSync(resolvedPath);
|
|
57
|
+
assertDirectory(identity, resolvedPath, label);
|
|
58
|
+
return {
|
|
59
|
+
path: resolvedPath,
|
|
60
|
+
realPath: fsSync.realpathSync(resolvedPath),
|
|
61
|
+
identity,
|
|
62
|
+
};
|
|
63
|
+
}
|
|
64
|
+
async function assertDirectoryReceiptCurrent(receipt, label) {
|
|
65
|
+
const currentIdentity = await fs.lstat(receipt.path);
|
|
66
|
+
assertDirectory(currentIdentity, receipt.path, label);
|
|
67
|
+
if (!sameFileIdentity(receipt.identity, currentIdentity) ||
|
|
68
|
+
(await fs.realpath(receipt.path)) !== receipt.realPath) {
|
|
69
|
+
throw new FsSafeError("path-mismatch", `${label} changed during durable directory operation: ${receipt.path}`);
|
|
70
|
+
}
|
|
71
|
+
}
|
|
72
|
+
function assertDirectoryReceiptCurrentSync(receipt, label) {
|
|
73
|
+
const currentIdentity = fsSync.lstatSync(receipt.path);
|
|
74
|
+
assertDirectory(currentIdentity, receipt.path, label);
|
|
75
|
+
if (!sameFileIdentity(receipt.identity, currentIdentity) ||
|
|
76
|
+
fsSync.realpathSync(receipt.path) !== receipt.realPath) {
|
|
77
|
+
throw new FsSafeError("path-mismatch", `${label} changed during durable directory operation: ${receipt.path}`);
|
|
78
|
+
}
|
|
79
|
+
}
|
|
80
|
+
async function assertOpenDirectoryCurrent(handle, receipt, label) {
|
|
81
|
+
const openedIdentity = await handle.stat();
|
|
82
|
+
assertDirectory(openedIdentity, receipt.path, label);
|
|
83
|
+
if (!sameFileIdentity(receipt.identity, openedIdentity)) {
|
|
84
|
+
throw new FsSafeError("path-mismatch", `${label} handle changed during directory sync: ${receipt.path}`);
|
|
85
|
+
}
|
|
86
|
+
await assertDirectoryReceiptCurrent(receipt, label);
|
|
87
|
+
}
|
|
88
|
+
class PinnedDirectoryImpl {
|
|
89
|
+
receipt;
|
|
90
|
+
#handle;
|
|
91
|
+
#label;
|
|
92
|
+
#closed = false;
|
|
93
|
+
constructor(handle, receipt, label) {
|
|
94
|
+
this.#handle = handle;
|
|
95
|
+
this.receipt = receipt;
|
|
96
|
+
this.#label = label;
|
|
97
|
+
}
|
|
98
|
+
async assertCurrent() {
|
|
99
|
+
if (this.#closed) {
|
|
100
|
+
throw new FsSafeError("helper-failed", `${this.#label} pin is already closed`);
|
|
101
|
+
}
|
|
102
|
+
await assertOpenDirectoryCurrent(this.#handle, this.receipt, this.#label);
|
|
103
|
+
}
|
|
104
|
+
async sync() {
|
|
105
|
+
await this.assertCurrent();
|
|
106
|
+
try {
|
|
107
|
+
await this.#handle.sync();
|
|
108
|
+
}
|
|
109
|
+
catch (error) {
|
|
110
|
+
if (!isWindowsDirectorySyncUnsupported(error)) {
|
|
111
|
+
throw error;
|
|
112
|
+
}
|
|
113
|
+
await this.assertCurrent();
|
|
114
|
+
return unsupportedOutcome(error);
|
|
115
|
+
}
|
|
116
|
+
await this.assertCurrent();
|
|
117
|
+
return { status: "synced" };
|
|
118
|
+
}
|
|
119
|
+
async close() {
|
|
120
|
+
if (this.#closed) {
|
|
121
|
+
return;
|
|
122
|
+
}
|
|
123
|
+
this.#closed = true;
|
|
124
|
+
await this.#handle.close();
|
|
125
|
+
}
|
|
126
|
+
}
|
|
127
|
+
export async function pinDirectory(directory, options = {}) {
|
|
128
|
+
const label = options.label ?? "directory";
|
|
129
|
+
const receipt = typeof directory === "string" ? await createDirectoryReceipt(directory, label) : directory;
|
|
130
|
+
await assertDirectoryReceiptCurrent(receipt, label);
|
|
131
|
+
const handle = await fs.open(receipt.path, directoryOpenFlags());
|
|
132
|
+
try {
|
|
133
|
+
await assertOpenDirectoryCurrent(handle, receipt, label);
|
|
134
|
+
return new PinnedDirectoryImpl(handle, receipt, label);
|
|
135
|
+
}
|
|
136
|
+
catch (error) {
|
|
137
|
+
await handle.close().catch(() => undefined);
|
|
138
|
+
throw error;
|
|
139
|
+
}
|
|
140
|
+
}
|
|
141
|
+
export async function syncDirectory(directory, options = {}) {
|
|
142
|
+
const label = options.label ?? "directory";
|
|
143
|
+
const receipt = typeof directory === "string" ? await createDirectoryReceipt(directory, label) : directory;
|
|
144
|
+
let pinned;
|
|
145
|
+
try {
|
|
146
|
+
pinned = await pinDirectory(receipt, { label });
|
|
147
|
+
}
|
|
148
|
+
catch (error) {
|
|
149
|
+
if (!isWindowsDirectoryOpenUnsupported(error)) {
|
|
150
|
+
throw error;
|
|
151
|
+
}
|
|
152
|
+
await assertDirectoryReceiptCurrent(receipt, label);
|
|
153
|
+
return unsupportedOutcome(error);
|
|
154
|
+
}
|
|
155
|
+
try {
|
|
156
|
+
return await pinned.sync();
|
|
157
|
+
}
|
|
158
|
+
finally {
|
|
159
|
+
await pinned.close();
|
|
160
|
+
}
|
|
161
|
+
}
|
|
162
|
+
export function syncDirectorySync(directory, options = {}) {
|
|
163
|
+
const label = options.label ?? "directory";
|
|
164
|
+
const receipt = typeof directory === "string" ? createDirectoryReceiptSync(directory, label) : directory;
|
|
165
|
+
assertDirectoryReceiptCurrentSync(receipt, label);
|
|
166
|
+
let descriptor;
|
|
167
|
+
try {
|
|
168
|
+
descriptor = fsSync.openSync(receipt.path, directoryOpenFlags());
|
|
169
|
+
}
|
|
170
|
+
catch (error) {
|
|
171
|
+
if (!isWindowsDirectoryOpenUnsupported(error)) {
|
|
172
|
+
throw error;
|
|
173
|
+
}
|
|
174
|
+
assertDirectoryReceiptCurrentSync(receipt, label);
|
|
175
|
+
return unsupportedOutcome(error);
|
|
176
|
+
}
|
|
177
|
+
try {
|
|
178
|
+
const openedIdentity = fsSync.fstatSync(descriptor);
|
|
179
|
+
assertDirectory(openedIdentity, receipt.path, label);
|
|
180
|
+
if (!sameFileIdentity(receipt.identity, openedIdentity)) {
|
|
181
|
+
throw new FsSafeError("path-mismatch", `${label} handle changed during directory sync: ${receipt.path}`);
|
|
182
|
+
}
|
|
183
|
+
assertDirectoryReceiptCurrentSync(receipt, label);
|
|
184
|
+
try {
|
|
185
|
+
fsSync.fsyncSync(descriptor);
|
|
186
|
+
}
|
|
187
|
+
catch (error) {
|
|
188
|
+
if (!isWindowsDirectorySyncUnsupported(error)) {
|
|
189
|
+
throw error;
|
|
190
|
+
}
|
|
191
|
+
assertDirectoryReceiptCurrentSync(receipt, label);
|
|
192
|
+
return unsupportedOutcome(error);
|
|
193
|
+
}
|
|
194
|
+
assertDirectoryReceiptCurrentSync(receipt, label);
|
|
195
|
+
return { status: "synced" };
|
|
196
|
+
}
|
|
197
|
+
finally {
|
|
198
|
+
fsSync.closeSync(descriptor);
|
|
199
|
+
}
|
|
200
|
+
}
|
|
201
|
+
export async function syncDirectoryBestEffort(directoryPath) {
|
|
202
|
+
await syncDirectory(directoryPath).catch(() => undefined);
|
|
203
|
+
}
|
|
204
|
+
export function syncDirectoryBestEffortSync(directoryPath) {
|
|
205
|
+
try {
|
|
206
|
+
syncDirectorySync(directoryPath);
|
|
207
|
+
}
|
|
208
|
+
catch {
|
|
209
|
+
// Compatibility helper for operations whose primary write may remain usable.
|
|
210
|
+
}
|
|
211
|
+
}
|
|
212
|
+
async function findExistingAncestorReceipt(targetPath, label) {
|
|
213
|
+
let currentPath = path.resolve(targetPath);
|
|
214
|
+
while (true) {
|
|
215
|
+
try {
|
|
216
|
+
return await createDirectoryReceipt(currentPath, label);
|
|
217
|
+
}
|
|
218
|
+
catch (error) {
|
|
219
|
+
if (error.code !== "ENOENT") {
|
|
220
|
+
throw error;
|
|
221
|
+
}
|
|
222
|
+
}
|
|
223
|
+
const parentPath = path.dirname(currentPath);
|
|
224
|
+
if (parentPath === currentPath) {
|
|
225
|
+
throw new FsSafeError("not-found", `${label} has no existing directory ancestor`);
|
|
226
|
+
}
|
|
227
|
+
currentPath = parentPath;
|
|
228
|
+
}
|
|
229
|
+
}
|
|
230
|
+
export async function ensureDurableDirectory(options) {
|
|
231
|
+
const directoryPath = path.resolve(options.directoryPath);
|
|
232
|
+
const label = options.label ?? "directory";
|
|
233
|
+
const ancestorReceipt = await findExistingAncestorReceipt(directoryPath, label);
|
|
234
|
+
const targetExists = ancestorReceipt.path === directoryPath;
|
|
235
|
+
if (options.expectedExistingIdentity &&
|
|
236
|
+
(!targetExists || !sameFileIdentity(options.expectedExistingIdentity, ancestorReceipt.identity))) {
|
|
237
|
+
throw new FsSafeError("path-mismatch", `${label} changed before durable directory pinning: ${directoryPath}`);
|
|
238
|
+
}
|
|
239
|
+
const ancestor = await pinDirectory(ancestorReceipt, { label });
|
|
240
|
+
const pinnedDirectories = [ancestor];
|
|
241
|
+
try {
|
|
242
|
+
await ancestor.assertCurrent();
|
|
243
|
+
if (!targetExists) {
|
|
244
|
+
if (options.create) {
|
|
245
|
+
await options.create(directoryPath);
|
|
246
|
+
}
|
|
247
|
+
else {
|
|
248
|
+
const created = await ensureAbsoluteDirectory(directoryPath, {
|
|
249
|
+
mode: options.mode,
|
|
250
|
+
scopeLabel: label,
|
|
251
|
+
});
|
|
252
|
+
if (!created.ok) {
|
|
253
|
+
throw created.error;
|
|
254
|
+
}
|
|
255
|
+
}
|
|
256
|
+
}
|
|
257
|
+
await ancestor.assertCurrent();
|
|
258
|
+
let currentPath = ancestor.receipt.path;
|
|
259
|
+
for (const segment of path
|
|
260
|
+
.relative(ancestor.receipt.path, directoryPath)
|
|
261
|
+
.split(path.sep)
|
|
262
|
+
.filter(Boolean)) {
|
|
263
|
+
currentPath = path.join(currentPath, segment);
|
|
264
|
+
pinnedDirectories.push(await pinDirectory(currentPath, { label }));
|
|
265
|
+
}
|
|
266
|
+
let parentSync = { status: "not-needed" };
|
|
267
|
+
for (let index = pinnedDirectories.length - 1; index > 0; index -= 1) {
|
|
268
|
+
const parent = pinnedDirectories[index - 1];
|
|
269
|
+
const child = pinnedDirectories[index];
|
|
270
|
+
if (!parent || !child) {
|
|
271
|
+
throw new FsSafeError("helper-failed", `${label} directory pin chain is incomplete`);
|
|
272
|
+
}
|
|
273
|
+
await child.assertCurrent();
|
|
274
|
+
try {
|
|
275
|
+
const outcome = await parent.sync();
|
|
276
|
+
if (outcome.status === "unsupported") {
|
|
277
|
+
parentSync = outcome;
|
|
278
|
+
}
|
|
279
|
+
else if (parentSync.status === "not-needed") {
|
|
280
|
+
parentSync = outcome;
|
|
281
|
+
}
|
|
282
|
+
}
|
|
283
|
+
catch (error) {
|
|
284
|
+
throw new FsSafeError("helper-failed", `${label} could not sync created directory edge ${child.receipt.path} through ${parent.receipt.path}`, { cause: error });
|
|
285
|
+
}
|
|
286
|
+
await child.assertCurrent();
|
|
287
|
+
}
|
|
288
|
+
const finalReceipt = pinnedDirectories.at(-1)?.receipt;
|
|
289
|
+
if (!finalReceipt) {
|
|
290
|
+
throw new FsSafeError("helper-failed", `${label} directory receipt is missing`);
|
|
291
|
+
}
|
|
292
|
+
await ancestor.assertCurrent();
|
|
293
|
+
await assertDirectoryReceiptCurrent(finalReceipt, label);
|
|
294
|
+
return { ...finalReceipt, parentSync };
|
|
295
|
+
}
|
|
296
|
+
finally {
|
|
297
|
+
await Promise.all(pinnedDirectories.toReversed().map(async (directory) => directory.close()));
|
|
298
|
+
}
|
|
299
|
+
}
|
|
@@ -0,0 +1,2 @@
|
|
|
1
|
+
export { ensureDurableDirectory, pinDirectory, syncDirectory, syncDirectoryBestEffort, syncDirectoryBestEffortSync, syncDirectorySync, type DirectoryReceipt, type DirectorySyncOutcome, type DurableDirectoryReceipt, type EnsureDurableDirectoryOptions, type PinnedDirectory, } from "./directory-durability.js";
|
|
2
|
+
//# sourceMappingURL=durability.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"durability.d.ts","sourceRoot":"","sources":["../src/durability.ts"],"names":[],"mappings":"AAAA,OAAO,EACL,sBAAsB,EACtB,YAAY,EACZ,aAAa,EACb,uBAAuB,EACvB,2BAA2B,EAC3B,iBAAiB,EACjB,KAAK,gBAAgB,EACrB,KAAK,oBAAoB,EACzB,KAAK,uBAAuB,EAC5B,KAAK,6BAA6B,EAClC,KAAK,eAAe,GACrB,MAAM,2BAA2B,CAAC"}
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
export { ensureDurableDirectory, pinDirectory, syncDirectory, syncDirectoryBestEffort, syncDirectoryBestEffortSync, syncDirectorySync, } from "./directory-durability.js";
|
package/dist/fsync.d.ts
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
|
-
export
|
|
1
|
+
export { syncDirectoryBestEffort } from "./directory-durability.js";
|
|
2
2
|
//# sourceMappingURL=fsync.d.ts.map
|
package/dist/fsync.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"fsync.d.ts","sourceRoot":"","sources":["../src/fsync.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"fsync.d.ts","sourceRoot":"","sources":["../src/fsync.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,uBAAuB,EAAE,MAAM,2BAA2B,CAAC"}
|
package/dist/fsync.js
CHANGED
|
@@ -1,21 +1 @@
|
|
|
1
|
-
|
|
2
|
-
import fs from "node:fs/promises";
|
|
3
|
-
export async function syncDirectoryBestEffort(dirPath) {
|
|
4
|
-
if (process.platform === "win32") {
|
|
5
|
-
return;
|
|
6
|
-
}
|
|
7
|
-
let handle;
|
|
8
|
-
try {
|
|
9
|
-
const flags = fsSync.constants.O_RDONLY |
|
|
10
|
-
("O_DIRECTORY" in fsSync.constants ? fsSync.constants.O_DIRECTORY : 0) |
|
|
11
|
-
("O_NOFOLLOW" in fsSync.constants ? fsSync.constants.O_NOFOLLOW : 0);
|
|
12
|
-
handle = await fs.open(dirPath, flags);
|
|
13
|
-
await handle.sync();
|
|
14
|
-
}
|
|
15
|
-
catch {
|
|
16
|
-
// Some filesystems reject directory handles; keep the write usable there.
|
|
17
|
-
}
|
|
18
|
-
finally {
|
|
19
|
-
await handle?.close().catch(() => undefined);
|
|
20
|
-
}
|
|
21
|
-
}
|
|
1
|
+
export { syncDirectoryBestEffort } from "./directory-durability.js";
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"sibling-temp.d.ts","sourceRoot":"","sources":["../src/sibling-temp.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"sibling-temp.d.ts","sourceRoot":"","sources":["../src/sibling-temp.ts"],"names":[],"mappings":"AAcA,MAAM,MAAM,2BAA2B,CAAC,CAAC,IAAI;IAC3C,GAAG,EAAE,MAAM,CAAC;IACZ,SAAS,EAAE,CAAC,QAAQ,EAAE,MAAM,KAAK,OAAO,CAAC,CAAC,CAAC,CAAC;IAC5C,gBAAgB,EAAE,CAAC,MAAM,EAAE,CAAC,KAAK,MAAM,CAAC;IACxC,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,YAAY,CAAC,EAAE,OAAO,CAAC;IACvB,aAAa,CAAC,EAAE,OAAO,CAAC;CACzB,CAAC;AAEF,MAAM,MAAM,0BAA0B,CAAC,CAAC,IAAI;IAC1C,QAAQ,EAAE,MAAM,CAAC;IACjB,MAAM,EAAE,CAAC,CAAC;CACX,CAAC;AA8BF,wBAAsB,oBAAoB,CAAC,CAAC,EAC1C,OAAO,EAAE,2BAA2B,CAAC,CAAC,CAAC,GACtC,OAAO,CAAC,0BAA0B,CAAC,CAAC,CAAC,CAAC,CAyCxC;AAkBD,wBAAsB,uBAAuB,CAAC,MAAM,EAAE;IACpD,OAAO,EAAE,MAAM,CAAC;IAChB,UAAU,EAAE,MAAM,CAAC;IACnB,SAAS,EAAE,CAAC,QAAQ,EAAE,MAAM,KAAK,OAAO,CAAC,IAAI,CAAC,CAAC;IAC/C,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,UAAU,CAAC,EAAE,MAAM,CAAC;CACrB,GAAG,OAAO,CAAC,IAAI,CAAC,CA8ChB"}
|
package/dist/sibling-temp.js
CHANGED
|
@@ -2,6 +2,7 @@ import crypto, { randomUUID } from "node:crypto";
|
|
|
2
2
|
import fs from "node:fs/promises";
|
|
3
3
|
import path from "node:path";
|
|
4
4
|
import { assertAsyncDirectoryGuard, createAsyncDirectoryGuard } from "./directory-guard.js";
|
|
5
|
+
import { syncDirectoryBestEffort } from "./directory-durability.js";
|
|
5
6
|
import { withAsyncDirectoryGuards } from "./guarded-mutation.js";
|
|
6
7
|
import { sanitizeUntrustedFileName } from "./filename.js";
|
|
7
8
|
import { root } from "./root.js";
|
|
@@ -30,19 +31,6 @@ async function syncFileBestEffort(filePath) {
|
|
|
30
31
|
await handle.close();
|
|
31
32
|
}
|
|
32
33
|
}
|
|
33
|
-
async function syncDirectoryBestEffort(dirPath) {
|
|
34
|
-
let handle;
|
|
35
|
-
try {
|
|
36
|
-
handle = await fs.open(dirPath, "r");
|
|
37
|
-
await handle.sync();
|
|
38
|
-
}
|
|
39
|
-
catch {
|
|
40
|
-
// Best-effort on platforms/filesystems that do not support directory fsync.
|
|
41
|
-
}
|
|
42
|
-
finally {
|
|
43
|
-
await handle?.close().catch(() => undefined);
|
|
44
|
-
}
|
|
45
|
-
}
|
|
46
34
|
function assertFinalPathIsSibling(dir, filePath) {
|
|
47
35
|
const resolvedDir = path.resolve(dir);
|
|
48
36
|
const resolvedFile = path.resolve(filePath);
|
package/docs/advanced.md
CHANGED
|
@@ -5,7 +5,7 @@ description: "Lower-level composition helpers under @openclaw/fs-safe/advanced.
|
|
|
5
5
|
|
|
6
6
|
# `@openclaw/fs-safe/advanced`
|
|
7
7
|
|
|
8
|
-
Composition primitives that OpenClaw uses to build higher-level APIs. They are public — semver applies — but treated as a less stable surface than the focused subpaths (`root`, `json`, `store`, `temp`, `archive`, `errors`). Reach for them only when you are building a primitive of your own and the focused subpaths do not cover it.
|
|
8
|
+
Composition primitives that OpenClaw uses to build higher-level APIs. They are public — semver applies — but treated as a less stable surface than the focused subpaths (`root`, `json`, `store`, `temp`, `archive`, `durability`, `errors`). Reach for them only when you are building a primitive of your own and the focused subpaths do not cover it.
|
|
9
9
|
|
|
10
10
|
```ts
|
|
11
11
|
import {
|
|
@@ -0,0 +1,89 @@
|
|
|
1
|
+
---
|
|
2
|
+
title: Directory durability
|
|
3
|
+
description: "Pin directory identities, fsync publication metadata, and durably create nested directory paths."
|
|
4
|
+
---
|
|
5
|
+
|
|
6
|
+
# Directory durability
|
|
7
|
+
|
|
8
|
+
`@openclaw/fs-safe/durability` provides the directory side of crash-safe file
|
|
9
|
+
publication. Flushing a file does not guarantee that its containing directory
|
|
10
|
+
entry reached storage; callers that promise durable create, link, rename, or
|
|
11
|
+
unlink operations must also synchronize the affected directory.
|
|
12
|
+
|
|
13
|
+
```ts
|
|
14
|
+
import {
|
|
15
|
+
ensureDurableDirectory,
|
|
16
|
+
pinDirectory,
|
|
17
|
+
} from "@openclaw/fs-safe/durability";
|
|
18
|
+
|
|
19
|
+
const repository = await ensureDurableDirectory({
|
|
20
|
+
directoryPath: "/srv/backups/sqlite",
|
|
21
|
+
mode: 0o700,
|
|
22
|
+
});
|
|
23
|
+
|
|
24
|
+
const pinned = await pinDirectory(repository, { label: "backup repository" });
|
|
25
|
+
try {
|
|
26
|
+
await publishSnapshot();
|
|
27
|
+
const outcome = await pinned.sync();
|
|
28
|
+
if (outcome.status === "unsupported") {
|
|
29
|
+
// Decide at the product boundary whether this platform can weaken the promise.
|
|
30
|
+
}
|
|
31
|
+
} finally {
|
|
32
|
+
await pinned.close();
|
|
33
|
+
}
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
## Outcomes and failure semantics
|
|
37
|
+
|
|
38
|
+
`syncDirectory()` and `PinnedDirectory.sync()` return:
|
|
39
|
+
|
|
40
|
+
```ts
|
|
41
|
+
type DirectorySyncOutcome =
|
|
42
|
+
| { status: "synced" }
|
|
43
|
+
| { status: "unsupported"; code?: string };
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
POSIX synchronization failures propagate. Windows directory handles do not
|
|
47
|
+
portably support `FlushFileBuffers`; the known unsupported error family is
|
|
48
|
+
reported as `unsupported` after the pathname and pinned identity are checked
|
|
49
|
+
again. Directory-open access failures and other Windows I/O failures still
|
|
50
|
+
propagate.
|
|
51
|
+
|
|
52
|
+
`syncDirectoryBestEffort()` and `syncDirectoryBestEffortSync()` intentionally
|
|
53
|
+
discard both unsupported outcomes and failures. Use them only when the primary
|
|
54
|
+
write remains useful without a crash-durability promise.
|
|
55
|
+
|
|
56
|
+
## Pinned directories
|
|
57
|
+
|
|
58
|
+
`pinDirectory()` rejects final symlinks and non-directories. On POSIX it opens
|
|
59
|
+
with `O_DIRECTORY`, `O_NOFOLLOW`, and `O_NONBLOCK`, then compares the open
|
|
60
|
+
descriptor, pathname identity, and canonical path. `assertCurrent()` repeats
|
|
61
|
+
those checks. This prevents a pathname replacement from turning a later sync
|
|
62
|
+
into proof for a different directory.
|
|
63
|
+
|
|
64
|
+
Call `close()` in `finally`. Closing is idempotent; using a closed pin fails.
|
|
65
|
+
|
|
66
|
+
## Durable directory creation
|
|
67
|
+
|
|
68
|
+
`ensureDurableDirectory()` finds and pins the nearest existing ancestor,
|
|
69
|
+
creates the requested path, opens every new directory segment, and synchronizes
|
|
70
|
+
each new parent-to-child edge from the leaf upward. It returns the final
|
|
71
|
+
directory receipt plus the aggregate parent-sync outcome.
|
|
72
|
+
|
|
73
|
+
By default it uses fs-safe's guarded one-segment-at-a-time absolute-directory
|
|
74
|
+
creator. Advanced callers can pass `create` when directory creation needs
|
|
75
|
+
platform-specific ACLs or another product-owned policy. The callback owns the
|
|
76
|
+
safety of its mutations and must create exactly `directoryPath`; fs-safe
|
|
77
|
+
validates and pins every resulting segment before any synchronization is
|
|
78
|
+
accepted.
|
|
79
|
+
|
|
80
|
+
`expectedExistingIdentity` binds an existing target to an identity observed by
|
|
81
|
+
the caller before a separate permission or policy check. A missing or replaced
|
|
82
|
+
target fails with `FsSafeError("path-mismatch")`.
|
|
83
|
+
|
|
84
|
+
## Scope
|
|
85
|
+
|
|
86
|
+
These primitives establish path identity and filesystem synchronization. They
|
|
87
|
+
do not decide application commit protocols, marker formats, permission policy,
|
|
88
|
+
or whether an unsupported platform is acceptable. Keep those decisions at the
|
|
89
|
+
owning product boundary.
|
package/docs/index.md
CHANGED
|
@@ -52,6 +52,7 @@ await fs.remove("notes/archive/today.txt");
|
|
|
52
52
|
| [`@openclaw/fs-safe/config`](config.md) | Process-global Python helper and lock-option defaults. |
|
|
53
53
|
| [Python helper policy](python-helper.md) | Choose `auto`, `off`, or `require` for POSIX fd-relative hardening. |
|
|
54
54
|
| [`replaceFileAtomic`](atomic.md) | Sibling-temp + rename, fsync hooks, mode preservation, copy fallback. |
|
|
55
|
+
| [Directory durability](durability.md) | Pinned directory identities, explicit sync outcomes, and durable nested-directory creation. |
|
|
55
56
|
| [`writeExternalFileWithinRoot`](output.md) | Stage external-library file output in private temp storage, then finalize under a root. |
|
|
56
57
|
| [`writeJson` / `readJson*`](json.md) | JSON state files with strict and lenient read variants. |
|
|
57
58
|
| [`@openclaw/fs-safe/store`](store.md) | Overview of `fileStore`, `fileStoreSync`, and `jsonStore`. |
|
package/docs/security-model.md
CHANGED
|
@@ -61,6 +61,19 @@ When `hardlinks: "reject"` is set, reads stat the target and refuse if `nlink >
|
|
|
61
61
|
|
|
62
62
|
Within one process, async writes to the same target are queued so their temp-write/rename phases do not overlap. Cross-process writers still need an external protocol such as the sidecar lock helpers.
|
|
63
63
|
|
|
64
|
+
### Directory durability
|
|
65
|
+
|
|
66
|
+
`pinDirectory()` opens a directory without following its final component on
|
|
67
|
+
POSIX, verifies the descriptor against the pathname identity and canonical
|
|
68
|
+
path, and repeats those checks around synchronization. `ensureDurableDirectory()`
|
|
69
|
+
pins the nearest existing ancestor and each newly created segment before
|
|
70
|
+
synchronizing every new directory edge from the leaf upward.
|
|
71
|
+
|
|
72
|
+
Known Windows directory-flush limitations are returned as an explicit
|
|
73
|
+
`unsupported` outcome. POSIX and other I/O failures propagate from the strict
|
|
74
|
+
API. The separately named best-effort helpers intentionally provide no crash
|
|
75
|
+
durability guarantee.
|
|
76
|
+
|
|
64
77
|
### Archive extraction
|
|
65
78
|
|
|
66
79
|
`extractArchive` first stages into a private temp directory (mode 0700) outside the destination, validates each entry path against `..` and absolute prefixes, refuses link-type entries by default, enforces entry count and byte budgets, and only then merges the staged tree into the destination through the same boundary checks used by direct writes.
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@openclaw/fs-safe",
|
|
3
|
-
"version": "0.4.
|
|
3
|
+
"version": "0.4.7",
|
|
4
4
|
"description": "Capability-style filesystem roots for Node.js apps that handle untrusted relative paths.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"filesystem",
|
|
@@ -96,6 +96,10 @@
|
|
|
96
96
|
"types": "./dist/atomic.d.ts",
|
|
97
97
|
"default": "./dist/atomic.js"
|
|
98
98
|
},
|
|
99
|
+
"./durability": {
|
|
100
|
+
"types": "./dist/durability.d.ts",
|
|
101
|
+
"default": "./dist/durability.js"
|
|
102
|
+
},
|
|
99
103
|
"./archive": {
|
|
100
104
|
"types": "./dist/archive.d.ts",
|
|
101
105
|
"default": "./dist/archive.js"
|