@openclaw/fs-safe 0.1.1 → 0.2.0

package/docs/temp.md

@@ -0,0 +1,284 @@
+# Temp workspaces
+
+`@openclaw/fs-safe/temp` is the stable temp surface: private temp **workspaces** with auto-cleanup plus the secure per-user temp root the helpers default to.
+
+```ts
+import {
+  tempWorkspace,
+  withTempWorkspace,
+  tempWorkspaceSync,
+  withTempWorkspaceSync,
+  resolveSecureTempRoot,
+} from "@openclaw/fs-safe/temp";
+```
+
+## Private temp workspaces
+
+A private workspace is a directory created at mode `0o700` under a caller-provided temp root. It is unique per call (random suffix) and cleaned up when you call `cleanup()` or leave an `await using` scope.
+
+### `tempWorkspace`
+
+The compact factory. Returns:
+
+```ts
+type TempWorkspace = {
+  dir: string;
+  store: FileStore;
+  path(fileName: string): string;
+  write(fileName: string, data: string | Uint8Array): Promise<string>;
+  writeText(fileName: string, data: string): Promise<string>;
+  writeJson(fileName: string, data: unknown, options?: { trailingNewline?: boolean }): Promise<string>;
+  copyIn(fileName: string, sourcePath: string): Promise<string>;
+  read(fileName: string): Promise<Buffer>;
+  cleanup(): Promise<void>;
+  [Symbol.asyncDispose](): Promise<void>;
+};
+```
+
+```ts
+import { tempWorkspace } from "@openclaw/fs-safe/temp";
+
+await using workspace = await tempWorkspace({ rootDir: "/tmp/my-app", prefix: "build-" });
+const inputPath = await workspace.write("input.txt", "data");
+await runBuild(workspace.dir, inputPath);
+```
+
+`write` writes at `mode` (default `0o600`); `writeText` and `writeJson` are convenience wrappers for the common scratch-file shapes; `copyIn` ingests an absolute source path through the same atomic-rename machinery as `Root.copyIn`. `read` is a small accessor that reads back any file you wrote into the workspace.
+
+`store` is a `fileStore({ rootDir: workspace.dir, private: true })` handle. Use
+it when you want the richer store surface, including `writeStream`, `exists`,
+`remove`, `readJsonIfExists`, or `store.json<T>(rel)`:
+
+```ts
+await using workspace = await tempWorkspace({ rootDir: "/tmp/my-app", prefix: "build-" });
+const state = workspace.store.json<State>("state.json");
+await state.write({ ready: true });
+```
+
+The workspace owns cleanup; the store is only a view over the workspace
+directory.
+
+The sync variant `tempWorkspaceSync` exposes the same surface with sync return
+types and a `FileStoreSync` at `workspace.store`.
+
+### `withTempWorkspace`
+
+The recommended shape. Auto-cleanup on every exit path:
+
+```ts
+import { withTempWorkspace } from "@openclaw/fs-safe/temp";
+
+const result = await withTempWorkspace({ rootDir: "/tmp/my-app", prefix: "build-" }, async (workspace) => {
+  await workspace.write("input.txt", "data");
+  return await runBuild(workspace.dir);
+});
+```
+
+The callback receives the same workspace shape as `tempWorkspace()`. Cleanup is wired to run after the callback resolves or rejects.
+
+### Manual lifetime
+
+Lower-level. You manage the lifetime:
+
+```ts
+const workspace = await tempWorkspace({ rootDir: "/tmp/my-app", prefix: "scan-" });
+try {
+  // …work in workspace.dir…
+} finally {
+  await workspace.cleanup();
+}
+```
+
+### Sync variants
+
+`tempWorkspaceSync` and `withTempWorkspaceSync` are the synchronous siblings. Useful for setup code in tests or boot paths that have not entered async land yet.
+
+### Options
+
+```ts
+type TempWorkspaceOptions = {
+  rootDir: string;          // parent directory for workspaces
+  prefix: string;           // dir prefix (sanitized)
+  dirMode?: number;         // dir mode; default 0o700
+  mode?: number;            // file write mode; default 0o600
+};
+```
+
+## Advanced temp primitives
+
+When you don't need the stable workspace abstraction, the lower-level temp-file
+and sibling-temp helpers live behind `@openclaw/fs-safe/advanced`. They are
+composition primitives for stores and atomic writers, not the primary API.
+`tempWorkspace()` carries the stable lifetime contract for application code;
+`tempFile()` is a one-shot building block whose options may move as store and
+archive internals evolve.
+
+### `tempFile`
+
+```ts
+import { tempFile } from "@openclaw/fs-safe/advanced";
+
+const target = await tempFile({ fileName: "report.pdf", prefix: "render-" });
+try {
+  await render(target.path);
+  await fs.copyFile(target.path, "/srv/workspace/reports/today.pdf");
+} finally {
+  await target.cleanup();
+}
+```
+
+Returns:
+
+```ts
+type TempFile = {
+  path: string;                            // absolute path; safe to write to
+  dir: string;                             // the enclosing private workspace dir
+  file(fileName?: string): string;          // resolve another file in the same dir
+  cleanup(): Promise<void>;                 // removes the private workspace dir
+  [Symbol.asyncDispose](): Promise<void>;   // alias of cleanup()
+};
+```
+
+### `withTempFile`
+
+Same shape with auto-cleanup:
+
+```ts
+import { withTempFile } from "@openclaw/fs-safe/advanced";
+
+await withTempFile({ fileName: "out.zip", prefix: "pack-" }, async (filePath) => {
+  await pack(filePath);
+  await uploadAndForget(filePath);
+});
+```
+
+## Sibling temp writes
+
+When you want to write to a temp file in **the same directory** as a future destination — useful when you need atomic placement but don't want to use `replaceFileAtomic`'s full machinery.
+
+### `writeSiblingTempFile`
+
+```ts
+import { writeSiblingTempFile } from "@openclaw/fs-safe/advanced";
+
+const result = await writeSiblingTempFile<string>({
+  dir: "/srv/workspace",
+  mode: 0o600,
+  writeTemp: async (tempPath) => {
+    await fs.writeFile(tempPath, JSON.stringify(state));
+    return "state.json";
+  },
+  resolveFinalPath: (fileName) => path.join("/srv/workspace", fileName),
+});
+// result.filePath, result.result (returned by writeTemp)
+```
+
+`writeSiblingTempFile` chooses a random sibling name in `dir`, calls your `writeTemp()` callback, validates that `resolveFinalPath(result)` is still inside that same directory, and renames the temp file there.
+
+By default it preserves the historical private-helper behavior of chmodding
+`dir` to `dirMode` (default `0o700`). Pass `chmodDir: false` when the directory
+is a public staging/output path whose existing mode must be preserved.
+
+### `writeViaSiblingTempPath`
+
+A higher-level convenience for callback-based producers. The callback writes to
+a private temp path, then the helper copies the result into `targetPath` through
+the root boundary:
+
+```ts
+import { writeViaSiblingTempPath } from "@openclaw/fs-safe/advanced";
+
+await writeViaSiblingTempPath({
+  rootDir: "/srv/workspace",
+  targetPath: "/srv/workspace/state.json",
+  writeTemp: async (tempPath) => {
+    await fs.writeFile(tempPath, JSON.stringify(state));
+  },
+});
+```
+
+If `replaceFileAtomic` does what you need, prefer that. Use
+`writeViaSiblingTempPath` when the producer needs a concrete temp pathname but
+the final destination still needs root-boundary checks.
+
+## Secure temp root
+
+The `resolveSecureTempRoot()` helper picks a per-user directory under the system temp dir, creates it at mode `0o700` if missing, and returns the absolute path. The other helpers in this module call it by default; you can call it directly if you need to materialize the root yourself.
+
+```ts
+import { resolveSecureTempRoot } from "@openclaw/fs-safe/temp";
+
+const tempRoot = resolveSecureTempRoot({ fallbackPrefix: "my-app" });
+// e.g. /tmp/my-app-501
+```
+
+### Options
+
+```ts
+type ResolveSecureTempRootOptions = {
+  fallbackPrefix: string;   // base name for the per-user fallback dir
+  preferredDir?: string;    // optional preferred secure temp root
+  tmpdir?: () => string;    // override os.tmpdir()
+};
+```
+
+The directory name embeds the user's UID (POSIX) or username so multi-user systems don't collide. On unsupported platforms, falls back to `os.tmpdir()` directly with a `helper-unavailable` error code surfaced to callers that explicitly required the secure root.
+
+## Common patterns
+
+### Build something, atomically place it
+
+```ts
+import { replaceDirectoryAtomic } from "@openclaw/fs-safe/atomic";
+
+await withTempWorkspace({ rootDir: "/srv/site/tmp", prefix: "build-" }, async (ws) => {
+  await runCompiler({ outDir: ws.dir });
+  await replaceDirectoryAtomic({
+    stagedDir: ws.dir,
+    targetDir: "/srv/site/public",
+  });
+});
+```
+
+### Stream a download to a sibling temp, then commit
+
+```ts
+import { writeSiblingTempFile } from "@openclaw/fs-safe/advanced";
+import fs from "node:fs/promises";
+
+const r = await writeSiblingTempFile({
+  dir: "/srv/cache",
+  writeTemp: async (tempPath) => {
+    const handle = await fs.open(tempPath, "w");
+    try {
+      await pipeline(downloadStream, handle.createWriteStream());
+    } finally {
+      await handle.close();
+    }
+    return "blob.bin";
+  },
+  resolveFinalPath: (fileName) => path.join("/srv/cache", fileName),
+});
+
+console.log(`downloaded ${r.filePath}`);
+```
+
+### Per-call private scratch in a test
+
+```ts
+import { withTempWorkspace } from "@openclaw/fs-safe/temp";
+
+it("processes a fixture", async () => {
+  await withTempWorkspace({ rootDir: "/tmp/my-tests", prefix: "test-" }, async (ws) => {
+    await fs.writeFile(path.join(ws.dir, "input.txt"), fixture);
+    const out = await processFile(path.join(ws.dir, "input.txt"));
+    expect(out).toEqual(expected);
+  });
+});
+```
+
+## See also
+
+- [Atomic writes](atomic.md) — `replaceDirectoryAtomic` for whole-directory swaps.
+- [`root()`](root.md) — `fs.copyIn(rel, sourceAbs)` for moving files from a temp into a `Root`.
+- [File lock](sidecar-lock.md) — when many processes share a temp tree.

package/docs/test-hooks.md

@@ -0,0 +1,74 @@
+---
+title: Test hooks
+description: "Internal-only injection hooks used by the fs-safe test suite. Active only when NODE_ENV=test or VITEST=true."
+---
+
+# `@openclaw/fs-safe/test-hooks`
+
+Internal injection points the `fs-safe` test suite uses to deterministically reproduce open/lstat races. They are exposed as a public subpath so downstream test suites can reuse the same harness, but they are **not** part of the supported runtime API.
+
+```ts
+import {
+  getFsSafeTestHooks,
+  __setFsSafeTestHooksForTest,
+  type FsSafeTestHooks,
+} from "@openclaw/fs-safe/test-hooks";
+```
+
+## When the hooks are active
+
+Hooks are only honored when one of the following is true:
+
+- `process.env.NODE_ENV === "test"`
+- `process.env.VITEST === "true"`
+
+Calling `__setFsSafeTestHooksForTest(hooks)` outside of those environments throws. `getFsSafeTestHooks()` returns `undefined` when no hooks are registered, regardless of the environment.
+
+## Shape
+
+```ts
+type FsSafeTestHooks = {
+  afterPreOpenLstat?: (filePath: string) => Promise<void> | void;
+  beforeOpen?: (filePath: string, flags: number) => Promise<void> | void;
+  afterOpen?: (filePath: string, handle: FileHandle) => Promise<void> | void;
+};
+```
+
+| Hook | Fires when |
+|---|---|
+| `afterPreOpenLstat` | A pre-open `lstat` has just resolved. Use this to swap a path between validation and open. |
+| `beforeOpen` | The library is about to call `open(path, flags)`. Use this to inject a TOCTOU window. |
+| `afterOpen` | An open just succeeded. Use this to mutate state before the post-open identity check runs. |
+
+Each hook may be sync or async; async hooks are awaited.
+
+## Usage
+
+```ts
+import { afterEach, beforeEach } from "vitest";
+import { __setFsSafeTestHooksForTest } from "@openclaw/fs-safe/test-hooks";
+
+beforeEach(() => {
+  __setFsSafeTestHooksForTest({
+    beforeOpen: async (filePath) => {
+      // swap a victim file with a symlink right before fs-safe opens it
+      await replaceWithSymlink(filePath);
+    },
+  });
+});
+
+afterEach(() => {
+  __setFsSafeTestHooksForTest(undefined);
+});
+```
+
+Always clear the hooks in `afterEach` so a stuck hook does not leak across tests.
+
+## Stability
+
+The shape can grow new optional fields between minor versions. Treat the surface as test-only and do not rely on it from production code.
+
+## Related pages
+
+- [Testing](testing.md) — broader notes on testing against `fs-safe`.
+- [Security model](security-model.md) — the races these hooks help reproduce.

package/docs/testing.md

@@ -0,0 +1,178 @@
+# Testing
+
+`@openclaw/fs-safe/test-hooks` exposes a small set of test-only injection points. They are inert in production: the hooks only activate when `process.env.NODE_ENV === "test"`. Outside test mode, calls to set hooks are no-ops, so leaking a test setup line into production is safe but ineffective.
+
+```ts
+import {
+  __setFsSafeTestHooksForTest,
+  type FsSafeTestHooks,
+} from "@openclaw/fs-safe/test-hooks";
+```
+
+The double-underscore prefix is a deliberate "hands off" signal: production code should never import this module. ESLint or your equivalent linter should flag it.
+
+## When to reach for hooks
+
+- Reproduce a TOCTOU race deterministically: simulate a symlink swap between resolve and open, or between write and rename.
+- Force Node-only behavior without uninstalling Python from your runners.
+- Inject latency to test cancellation/timeout paths.
+
+If you don't need to inject a race, you don't need hooks — most tests should drive the library through normal calls and assert on observable behavior.
+
+## Hooks API
+
+```ts
+type FsSafeTestHooks = {
+  afterPreOpenLstat?: (filePath: string) => Promise<void> | void;
+  beforeOpen?: (filePath: string, flags: number) => Promise<void> | void;
+  afterOpen?: (filePath: string, handle: import("node:fs/promises").FileHandle) => Promise<void> | void;
+};
+
+function __setFsSafeTestHooksForTest(hooks?: FsSafeTestHooks): void;
+function getFsSafeTestHooks(): FsSafeTestHooks | undefined;
+```
+
+Hooks are called at well-defined points in the library's hot paths:
+
+- **`afterPreOpenLstat`** — runs after the pre-open `lstat`. A common use is to swap the path's target via `fs.symlink`/`fs.unlink` to drive a TOCTOU race.
+- **`beforeOpen`** — runs before `fs.open` with the exact flags the root read path will use.
+- **`afterOpen`** — runs after the file handle is opened. Useful to wrap handle methods or inject a size race before a stream is consumed.
+
+`__setFsSafeTestHooksForTest(undefined)` clears all hooks. Always clean up between tests.
+
+## Example: simulate a TOCTOU swap
+
+```ts
+import { describe, it, beforeEach, afterEach, expect } from "vitest";
+import { mkdtemp, mkdir, rm, symlink, unlink, writeFile } from "node:fs/promises";
+import os from "node:os";
+import path from "node:path";
+import { root, FsSafeError } from "@openclaw/fs-safe";
+import { __setFsSafeTestHooksForTest } from "@openclaw/fs-safe/test-hooks";
+
+let dir: string;
+beforeEach(async () => {
+  dir = await mkdtemp(path.join(os.tmpdir(), "fs-safe-toctou-"));
+  await writeFile(path.join(dir, "real.txt"), "secret");
+  await writeFile(path.join(dir, "decoy.txt"), "decoy");
+});
+afterEach(async () => {
+  __setFsSafeTestHooksForTest(undefined);
+  await rm(dir, { recursive: true, force: true });
+});
+
+it("rejects a swap between resolve and open", async () => {
+  const fs = await root(dir, { symlinks: "reject" });
+
+  __setFsSafeTestHooksForTest({
+    afterPreOpenLstat: async (absPath) => {
+      // swap real.txt for a symlink to decoy.txt right before the open
+      await unlink(absPath);
+      await symlink(path.join(dir, "decoy.txt"), absPath);
+    },
+  });
+
+  await expect(fs.read("real.txt")).rejects.toMatchObject({
+    name: "FsSafeError",
+    code: expect.stringMatching(/symlink|path-mismatch/),
+  });
+});
+```
+
+The `code` may be `symlink` (caught at open by `O_NOFOLLOW`) or `path-mismatch` (caught by the post-open identity check) depending on platform — both are correct refusals.
+
+## Example: force Node-only fallback behavior
+
+```ts
+import { configureFsSafePython } from "@openclaw/fs-safe/config";
+
+beforeEach(() => {
+  configureFsSafePython({ mode: "off" });
+});
+
+afterEach(() => {
+  configureFsSafePython({ mode: "auto", pythonPath: undefined });
+});
+
+it("runs without the Python helper", async () => {
+  const fs = await root(dir);
+  await fs.write("file.txt", "ok");
+  await expect(fs.readText("file.txt")).resolves.toBe("ok");
+});
+```
+
+## Cleanup is mandatory
+
+Hooks set by `__setFsSafeTestHooksForTest` persist across tests until explicitly cleared. Always clear in `afterEach` (or your test framework's equivalent) — leaked hooks will silently change behavior in unrelated tests and cause maddening intermittent failures.
+
+```ts
+import { afterEach } from "vitest";
+import { __setFsSafeTestHooksForTest } from "@openclaw/fs-safe/test-hooks";
+
+afterEach(() => {
+  __setFsSafeTestHooksForTest(undefined);
+});
+```
+
+A global hook clear in your test setup file is a good safety net.
+
+## Patterns for testing fs-safe-using code
+
+You usually don't need hooks. Most tests follow this shape:
+
+```ts
+import { mkdtemp, rm } from "node:fs/promises";
+import os from "node:os";
+import path from "node:path";
+import { root } from "@openclaw/fs-safe";
+
+let dir: string;
+let fs: Awaited<ReturnType<typeof root>>;
+
+beforeEach(async () => {
+  dir = await mkdtemp(path.join(os.tmpdir(), "my-feature-"));
+  fs = await root(dir, { symlinks: "reject", hardlinks: "reject", mkdir: true });
+});
+afterEach(async () => {
+  await rm(dir, { recursive: true, force: true });
+});
+
+it("writes and reads through the boundary", async () => {
+  await fs.write("notes/today.txt", "hello");
+  expect(await fs.readText("notes/today.txt")).toBe("hello");
+});
+```
+
+For tests that need a private temp workspace, [`withTempWorkspace`](temp.md) makes the setup-and-teardown story trivial.
+
+## Repo test shards
+
+Run the full local gate before handoff:
+
+```sh
+pnpm check
+```
+
+Run only the security boundary corpus while iterating on root/path/archive/temp hardening:
+
+```sh
+pnpm test:security
+```
+
+Run the static primitive guard after changing low-level filesystem helpers:
+
+```sh
+pnpm lint:fs-boundary
+```
+
+It catches the specific raw fallback patterns that previously led to
+check-then-use bugs, such as direct copy-to-destination fallback and sync temp
+workspace reads that bypass pinned file descriptors.
+
+`pnpm check` also runs `pnpm lint:file-size`. New source and test files should stay under 500 lines. Existing larger files have explicit budgets in `scripts/check-file-size.mjs`; do not increase those budgets as part of unrelated work.
+
+## See also
+
+- [Security model](security-model.md) — what the boundary is supposed to defend; design tests around the same threats.
+- [`root()`](root.md) — the surface most tests will exercise.
+- [Temp workspaces](temp.md) — `withTempWorkspace` for cleanup-on-exit test directories.

package/docs/timing.md

@@ -0,0 +1,121 @@
+# Timing
+
+`withTimeout(promise, timeoutMs, labelOrOptions?)` is a small helper for putting a wall-clock ceiling on an async operation. It rejects with a synthetic timeout error after `timeoutMs` and clears its internal timer when the wrapped promise settles first.
+
+```ts
+import { withTimeout } from "@openclaw/fs-safe/advanced";
+```
+
+## Signature
+
+```ts
+function withTimeout<T>(
+  promise: Promise<T>,
+  timeoutMs: number,
+  labelOrOptions?: string | {
+    label?: string;
+    message?: string;
+    createError?: () => Error;
+  },
+): Promise<T>;
+```
+
+If `timeoutMs` is `0`, negative, `Infinity`, or `NaN`, the helper is a no-op and simply awaits the original promise.
+
+## Examples
+
+### Simple ceiling
+
+```ts
+import { withTimeout } from "@openclaw/fs-safe/advanced";
+
+const buf = await withTimeout(
+  fs.readFile("/srv/big.bin"),
+  5_000,
+  "read big.bin",
+);
+```
+
+If the read doesn't resolve within 5 seconds, the returned promise rejects with `Error: read big.bin timed out after 5000ms`. The underlying `fs.readFile` continues until Node finishes it — `withTimeout` does not cancel the wrapped work, only the wait.
+
+### Custom message
+
+```ts
+await withTimeout(work(), 5_000, {
+  message: "build did not finish in time",
+});
+```
+
+### Custom error factory
+
+```ts
+class BuildTimeout extends Error {}
+
+await withTimeout(work(), 5_000, {
+  createError: () => new BuildTimeout("build timeout (5s)"),
+});
+```
+
+`createError` is called when the timer fires; the returned error is what the promise rejects with. Use this when callers branch on `instanceof` or want a custom `cause`.
+
+## Cancellation
+
+`withTimeout` does **not** abort the wrapped operation when the timer fires — it just stops waiting. If you need real cancellation, the wrapped operation must opt into an `AbortSignal` itself:
+
+```ts
+const controller = new AbortController();
+const timer = setTimeout(() => controller.abort(), 5_000);
+
+try {
+  const res = await fetch(url, { signal: controller.signal });
+  // ...
+} finally {
+  clearTimeout(timer);
+}
+```
+
+For the common "I want a deadline AND cancellation" shape, use `AbortSignal.timeout(ms)` directly — it's the standard library's answer and handles both at once.
+
+## Patterns
+
+### Bound a credential refresh
+
+```ts
+const fresh = await withTimeout(
+  refreshToken(currentRefresh),
+  5_000,
+  "refresh oauth token",
+);
+await writeSecretFileAtomic({ rootDir, filePath, content: JSON.stringify(fresh) });
+```
+
+### Compose with archive extraction
+
+```ts
+import { extractArchive } from "@openclaw/fs-safe/archive";
+
+await extractArchive({
+  archivePath,
+  destDir,
+  kind: "zip",
+  timeoutMs: 30_000,
+});
+```
+
+`extractArchive` already takes `timeoutMs` and uses `withTimeout` internally — you don't need to wrap it. Reach for `withTimeout` for operations that don't carry their own timeout knob.
+
+### Disable in tests
+
+When unit-testing flaky code, you might want to disable the timeout. Pass `0`:
+
+```ts
+await withTimeout(work(), process.env.NODE_ENV === "test" ? 0 : 5_000, "work");
+```
+
+Better, gate it from the caller — `withTimeout(p, 0, ...)` returns the promise as-is.
+
+## See also
+
+- [Archive extraction](archive.md) — `extractArchive` already takes `timeoutMs`.
+- [File lock](sidecar-lock.md) — retry policy is a different form of bounded waiting.
+- [`AbortSignal.timeout`](https://developer.mozilla.org/en-US/docs/Web/API/AbortSignal/timeout_static) — standard-library cancellation when you need to *abort*, not just *give up*.