@openclaw/fs-safe 0.1.0 → 0.1.2

package/docs/sidecar-lock.md

@@ -0,0 +1,229 @@
+# File lock
+
+`acquireFileLock()` and `withFileLock()` provide a cross-process file lock with retry, stale-lock reclaim, and process-exit cleanup. The lock is implemented as a sidecar file (e.g. `state.json` ↔ `state.json.lock`) — only one acquirer can create the sidecar with `O_CREAT | O_EXCL` at a time.
+
+```ts
+import { acquireFileLock } from "@openclaw/fs-safe/file-lock";
+
+const handle = await acquireFileLock("/var/lib/app/state.json", {
+  managerKey: "snapshot",
+  staleMs: 5 * 60_000,
+  payload: async () => ({ pid: process.pid, host: os.hostname() }),
+});
+try {
+  // ...exclusive work on /var/lib/app/state.json...
+} finally {
+  await handle.release();
+}
+```
+
+## Why sidecar?
+
+The lock file sits next to the protected resource. If a process crashes mid-lock, the next acquirer notices the held entry, inspects its payload (PID, host, acquired-at timestamp), and decides — via `shouldReclaim` (defaulting to "is the lock older than `staleMs`?") — whether to take it over.
+
+The library installs a `process.on("exit")` handler that releases all currently-held locks synchronously, so well-behaved exits leave no stale sidecars. Crashes still need the reclaim path.
+
+## API
+
+```ts
+function acquireFileLock<TPayload>(
+  targetPath: string,
+  options: FileLockAcquireOptions<TPayload>,
+): Promise<FileLockHandle>;
+
+function withFileLock<T, TPayload>(
+  targetPath: string,
+  options: FileLockAcquireOptions<TPayload>,
+  fn: () => Promise<T>,
+): Promise<T>;
+
+function createFileLockManager(key: string): FileLockManager;
+```
+
+`managerKey` is an optional identifier used to keep state isolated across multiple lock domains in the same process. Use distinct keys for distinct domains (`"snapshot"`, `"compact"`, `"build"`). If omitted, fs-safe derives one from the target path.
+
+## Acquire options
+
+```ts
+type FileLockAcquireOptions<TPayload extends Record<string, unknown>> = {
+  managerKey?: string;                   // optional in-process manager namespace
+  lockPath?: string;                     // override; defaults to `${targetPath}.lock`
+  staleMs: number;                       // how long until a held lock is considered stale
+  timeoutMs?: number;                    // overall acquire deadline; default unbounded
+  retry?: FileLockRetryOptions;
+  allowReentrant?: boolean;              // if this process already holds it, increment a count instead of failing
+  payload: () => TPayload | Promise<TPayload>;
+  shouldReclaim?: (params: {
+    lockPath: string;
+    normalizedTargetPath: string;
+    payload: Record<string, unknown> | null;
+    staleMs: number;
+    nowMs: number;
+    heldByThisProcess: boolean;
+  }) => boolean | Promise<boolean>;
+  metadata?: Record<string, unknown>;    // attached to heldEntries() output for diagnostics
+};
+
+type FileLockRetryOptions = {
+  retries?: number;       // number of retry attempts after the first failure
+  factor?: number;        // exponential backoff factor (default 2)
+  minTimeout?: number;    // initial delay (ms)
+  maxTimeout?: number;    // delay cap (ms)
+  randomize?: boolean;    // jitter
+};
+```
+
+`payload` is a function so you can re-evaluate it on each retry (e.g. timestamp, PID).
+
+## Release handle
+
+```ts
+type FileLockHandle = {
+  lockPath: string;
+  normalizedTargetPath: string;
+  release: () => Promise<void>;
+  [Symbol.asyncDispose](): Promise<void>;
+};
+```
+
+Always release in a `finally`:
+
+```ts
+const handle = await acquireFileLock(targetPath, {
+  staleMs: 60_000,
+  payload: () => ({ pid: process.pid }),
+});
+try {
+  await doExclusiveWork();
+} finally {
+  await handle.release();
+}
+```
+
+If your process dies before `release()` runs and skips the exit handler, the next acquirer reclaims the lock once `staleMs` elapses (or your `shouldReclaim` returns true).
+
+## `withFileLock` — common shape made one-liner
+
+```ts
+const result = await withFileLock(
+  "/var/lib/app/state.json",
+  {
+    managerKey: "compact",
+    staleMs: 30_000,
+    payload: () => ({ pid: process.pid, what: "compact" }),
+  },
+  async () => {
+    return await runCompaction();
+  },
+);
+```
+
+Acquires, runs `fn`, releases regardless of success/failure. Returns the result of `fn`.
+
+## Long-lived managers
+
+Most callers should use `acquireFileLock()` or `withFileLock()`. Use `createFileLockManager(key)` only when a long-lived service needs diagnostics or lifecycle control over locks it currently holds:
+
+```ts
+const locks = createFileLockManager("session-writes");
+const handle = await locks.acquire(sessionPath, {
+  staleMs: 60_000,
+  payload: () => ({ pid: process.pid }),
+});
+
+for (const held of locks.heldEntries()) {
+  console.log(held.lockPath, held.acquiredAt);
+}
+
+await handle.release();
+await locks.drain();
+```
+
+## Reclaim policy: `shouldReclaim`
+
+The default policy reclaims locks whose `acquiredAt` is older than `staleMs`. Pass a custom callback when you want a richer notion of "is the holder still alive":
+
+```ts
+import { kill } from "node:process";
+
+const handle = await acquireFileLock(targetPath, {
+  staleMs: 60_000,
+  payload: () => ({ pid: process.pid }),
+  shouldReclaim: ({ payload, nowMs, staleMs }) => {
+    if (!payload) return true;
+    const pid = Number(payload.pid);
+    if (!Number.isFinite(pid)) return true;
+    try {
+      kill(pid, 0);
+      return false;                     // process still alive — don't reclaim
+    } catch {
+      return true;                      // process gone — reclaim
+    }
+  },
+});
+```
+
+`heldByThisProcess` is true when this manager already holds the lock (relevant for the reentrant case).
+
+## What sidecar locks defend against
+
+- **Two processes writing the same file at once.** `acquire` serializes the critical section.
+- **A crashed holder leaving a stale lock.** `staleMs` plus optional `shouldReclaim` recovers it.
+- **Race between simultaneous acquire attempts.** `O_CREAT | O_EXCL` ensures one wins.
+
+## What they do **not** defend against
+
+- **Misbehaving holders that ignore the lock.** Locks are advisory — only callers that go through `acquire` are bound.
+- **Holders that never call `release` and have no liveness check.** Without a real `shouldReclaim`, the lock relies on `staleMs` alone — pick a deadline that is comfortably longer than your real work but short enough to recover from crashes.
+- **Multi-host coordination over network filesystems.** Behavior depends on the underlying filesystem's `O_EXCL` semantics; treat as best-effort.
+
+## Common patterns
+
+### Compact under lock
+
+```ts
+await withFileLock(
+  "/var/lib/app/db.sqlite",
+  {
+    staleMs: 30_000,
+    payload: () => ({ pid: process.pid, what: "compact" }),
+  },
+  async () => {
+    await runCompaction();
+  },
+);
+```
+
+### Try once, give up if held
+
+```ts
+try {
+  await withFileLock(
+    targetPath,
+    { staleMs: 30_000, retry: { retries: 0 }, payload: () => ({ pid: process.pid }) },
+    async () => await work(),
+  );
+} catch (err) {
+  console.log("another process is doing this; skipping");
+}
+```
+
+### Wait politely with backoff
+
+```ts
+await withFileLock(
+  targetPath,
+  {
+    staleMs: 60_000,
+    timeoutMs: 30_000,
+    retry: { retries: 30, minTimeout: 100, maxTimeout: 5_000, factor: 1.7, randomize: true },
+    payload: () => ({ pid: process.pid }),
+  },
+  async () => await work(),
+);
+```
+
+## See also
+
+- [Atomic writes](atomic.md) — single-writer atomicity that often replaces the need for a lock entirely.
+- `createAsyncLock` from `@openclaw/fs-safe/advanced` — in-process serialization for a single Node process.

package/docs/store.md

@@ -0,0 +1,44 @@
+---
+title: Store
+description: "Overview of @openclaw/fs-safe/store: fileStore, fileStoreSync, and jsonStore."
+---
+
+# `@openclaw/fs-safe/store`
+
+The `store` subpath bundles two managed wrappers around the same safe-write primitives `root()` uses:
+
+```ts
+import {
+  fileStore,
+  fileStoreSync,
+  jsonStore,
+  type FileStore,
+  type FileStoreOptions,
+  type FileStoreSync,
+  type JsonStore,
+  type JsonStoreOptions,
+} from "@openclaw/fs-safe/store";
+```
+
+| Helper | Use it for |
+|---|---|
+| [`fileStore()`](file-store.md) | Multi-file directories with safe relative paths, size limits, atomic replacement, stream writes, copy-in, and TTL-based pruning. |
+| `fileStoreSync()` | Synchronous variant of `fileStore()` for places that genuinely cannot await. |
+| [`jsonStore()`](json-store.md) | A single keyed JSON state file with explicit fallback, atomic writes, and optional sidecar locking around read-modify-write updates. |
+| [Private file-store mode](private-file-store.md) | `fileStore({ private: true })` for credentials, tokens, and per-agent state at `0600` files under `0700` directories. |
+
+`fileStore().json("rel.json")` and `jsonStore({ filePath })` are intentionally separate primitives. Use `fileStore().json(...)` when JSON state lives alongside other files in the same managed directory; use `jsonStore({ filePath })` when you have a single absolute path and want the keyed JSON shape directly.
+
+## Picking a shape
+
+- **Multi-file directory under one root** — reach for `fileStore()`. It exposes `write`, `writeJson`, `writeText`, `writeStream`, `read*`, `open`, `copyIn`, `remove`, and `pruneExpired` against safe relative paths.
+- **One JSON state file** — reach for `jsonStore({ filePath })`. Its `update()` and `updateOr()` methods cover the merge-into-defaults and read-modify-write cases.
+- **Credentials or tokens** — pass `private: true` to `fileStore()`. Same store shape; writes route through the secret-file atomic path with `0600`/`0700` permissions.
+
+## Related pages
+
+- [`fileStore`](file-store.md) — full API for the multi-file store.
+- [`jsonStore`](json-store.md) — single-file JSON store with locking.
+- [Private file-store mode](private-file-store.md) — credential-shaped variant.
+- [JSON files](json.md) — lower-level `readJson` / `writeJson` helpers.
+- [Atomic writes](atomic.md) — what `fileStore` and `jsonStore` use under the hood.

package/docs/temp.md

@@ -0,0 +1,276 @@
+# Temp workspaces
+
+`@openclaw/fs-safe/temp` is the stable temp surface: private temp **workspaces** with auto-cleanup plus the secure per-user temp root the helpers default to.
+
+```ts
+import {
+  tempWorkspace,
+  withTempWorkspace,
+  tempWorkspaceSync,
+  withTempWorkspaceSync,
+  resolveSecureTempRoot,
+} from "@openclaw/fs-safe/temp";
+```
+
+## Private temp workspaces
+
+A private workspace is a directory created at mode `0o700` under a caller-provided temp root. It is unique per call (random suffix) and cleaned up when you call `cleanup()` or leave an `await using` scope.
+
+### `tempWorkspace`
+
+The compact factory. Returns:
+
+```ts
+type TempWorkspace = {
+  dir: string;
+  store: FileStore;
+  path(fileName: string): string;
+  write(fileName: string, data: string | Uint8Array): Promise<string>;
+  writeText(fileName: string, data: string): Promise<string>;
+  writeJson(fileName: string, data: unknown, options?: { trailingNewline?: boolean }): Promise<string>;
+  copyIn(fileName: string, sourcePath: string): Promise<string>;
+  read(fileName: string): Promise<Buffer>;
+  cleanup(): Promise<void>;
+  [Symbol.asyncDispose](): Promise<void>;
+};
+```
+
+```ts
+import { tempWorkspace } from "@openclaw/fs-safe/temp";
+
+await using workspace = await tempWorkspace({ rootDir: "/tmp/my-app", prefix: "build-" });
+const inputPath = await workspace.write("input.txt", "data");
+await runBuild(workspace.dir, inputPath);
+```
+
+`write` writes at `mode` (default `0o600`); `writeText` and `writeJson` are convenience wrappers for the common scratch-file shapes; `copyIn` ingests an absolute source path through the same atomic-rename machinery as `Root.copyIn`. `read` is a small accessor that reads back any file you wrote into the workspace.
+
+`store` is a `fileStore({ rootDir: workspace.dir, private: true })` handle. Use
+it when you want the richer store surface, including `writeStream`, `exists`,
+`remove`, `readJsonIfExists`, or `store.json<T>(rel)`:
+
+```ts
+await using workspace = await tempWorkspace({ rootDir: "/tmp/my-app", prefix: "build-" });
+const state = workspace.store.json<State>("state.json");
+await state.write({ ready: true });
+```
+
+The workspace owns cleanup; the store is only a view over the workspace
+directory.
+
+The sync variant `tempWorkspaceSync` exposes the same surface with sync return
+types and a `FileStoreSync` at `workspace.store`.
+
+### `withTempWorkspace`
+
+The recommended shape. Auto-cleanup on every exit path:
+
+```ts
+import { withTempWorkspace } from "@openclaw/fs-safe/temp";
+
+const result = await withTempWorkspace({ rootDir: "/tmp/my-app", prefix: "build-" }, async (workspace) => {
+  await workspace.write("input.txt", "data");
+  return await runBuild(workspace.dir);
+});
+```
+
+The callback receives the same workspace shape as `tempWorkspace()`. Cleanup is wired to run after the callback resolves or rejects.
+
+### Manual lifetime
+
+Lower-level. You manage the lifetime:
+
+```ts
+const workspace = await tempWorkspace({ rootDir: "/tmp/my-app", prefix: "scan-" });
+try {
+  // …work in workspace.dir…
+} finally {
+  await workspace.cleanup();
+}
+```
+
+### Sync variants
+
+`tempWorkspaceSync` and `withTempWorkspaceSync` are the synchronous siblings. Useful for setup code in tests or boot paths that have not entered async land yet.
+
+### Options
+
+```ts
+type TempWorkspaceOptions = {
+  rootDir: string;          // parent directory for workspaces
+  prefix: string;           // dir prefix (sanitized)
+  dirMode?: number;         // dir mode; default 0o700
+  mode?: number;            // file write mode; default 0o600
+};
+```
+
+## Advanced temp primitives
+
+When you don't need the stable workspace abstraction, the lower-level temp-file
+and sibling-temp helpers live behind `@openclaw/fs-safe/advanced`. They are
+composition primitives for stores and atomic writers, not the primary API.
+`tempWorkspace()` carries the stable lifetime contract for application code;
+`tempFile()` is a one-shot building block whose options may move as store and
+archive internals evolve.
+
+### `tempFile`
+
+```ts
+import { tempFile } from "@openclaw/fs-safe/advanced";
+
+const target = await tempFile({ fileName: "report.pdf", prefix: "render-" });
+try {
+  await render(target.path);
+  await fs.copyFile(target.path, "/srv/workspace/reports/today.pdf");
+} finally {
+  await target.cleanup();
+}
+```
+
+Returns:
+
+```ts
+type TempFile = {
+  path: string;                            // absolute path; safe to write to
+  dir: string;                             // the enclosing private workspace dir
+  file(fileName?: string): string;          // resolve another file in the same dir
+  cleanup(): Promise<void>;                 // removes the private workspace dir
+  [Symbol.asyncDispose](): Promise<void>;   // alias of cleanup()
+};
+```
+
+### `withTempFile`
+
+Same shape with auto-cleanup:
+
+```ts
+import { withTempFile } from "@openclaw/fs-safe/advanced";
+
+await withTempFile({ fileName: "out.zip", prefix: "pack-" }, async (filePath) => {
+  await pack(filePath);
+  await uploadAndForget(filePath);
+});
+```
+
+## Sibling temp writes
+
+When you want to write to a temp file in **the same directory** as a future destination — useful when you need atomic placement but don't want to use `replaceFileAtomic`'s full machinery.
+
+### `writeSiblingTempFile`
+
+```ts
+import { writeSiblingTempFile } from "@openclaw/fs-safe/advanced";
+
+const result = await writeSiblingTempFile<string>({
+  dir: "/srv/workspace",
+  mode: 0o600,
+  writeTemp: async (tempPath) => {
+    await fs.writeFile(tempPath, JSON.stringify(state));
+    return "state.json";
+  },
+  resolveFinalPath: (fileName) => path.join("/srv/workspace", fileName),
+});
+// result.filePath, result.result (returned by writeTemp)
+```
+
+`writeSiblingTempFile` chooses a random sibling name in `dir`, calls your `writeTemp()` callback, validates that `resolveFinalPath(result)` is still inside that same directory, and renames the temp file there.
+
+### `writeViaSiblingTempPath`
+
+A higher-level convenience — write content + rename in one call:
+
+```ts
+import { writeViaSiblingTempPath } from "@openclaw/fs-safe/advanced";
+
+await writeViaSiblingTempPath({
+  rootDir: "/srv/workspace",
+  targetPath: "/srv/workspace/state.json",
+  writeTemp: async (tempPath) => {
+    await fs.writeFile(tempPath, JSON.stringify(state));
+  },
+});
+```
+
+If `replaceFileAtomic` does what you need, prefer that — `writeViaSiblingTempPath` is the lower-level building block.
+
+## Secure temp root
+
+The `resolveSecureTempRoot()` helper picks a per-user directory under the system temp dir, creates it at mode `0o700` if missing, and returns the absolute path. The other helpers in this module call it by default; you can call it directly if you need to materialize the root yourself.
+
+```ts
+import { resolveSecureTempRoot } from "@openclaw/fs-safe/temp";
+
+const tempRoot = resolveSecureTempRoot({ fallbackPrefix: "my-app" });
+// e.g. /tmp/my-app-501
+```
+
+### Options
+
+```ts
+type ResolveSecureTempRootOptions = {
+  fallbackPrefix: string;   // base name for the per-user fallback dir
+  preferredDir?: string;    // optional preferred secure temp root
+  tmpdir?: () => string;    // override os.tmpdir()
+};
+```
+
+The directory name embeds the user's UID (POSIX) or username so multi-user systems don't collide. On unsupported platforms, falls back to `os.tmpdir()` directly with a `helper-unavailable` error code surfaced to callers that explicitly required the secure root.
+
+## Common patterns
+
+### Build something, atomically place it
+
+```ts
+import { replaceDirectoryAtomic } from "@openclaw/fs-safe/atomic";
+
+await withTempWorkspace({ rootDir: "/srv/site/tmp", prefix: "build-" }, async (ws) => {
+  await runCompiler({ outDir: ws.dir });
+  await replaceDirectoryAtomic({
+    stagedDir: ws.dir,
+    targetDir: "/srv/site/public",
+  });
+});
+```
+
+### Stream a download to a sibling temp, then commit
+
+```ts
+import { writeSiblingTempFile } from "@openclaw/fs-safe/advanced";
+import fs from "node:fs/promises";
+
+const r = await writeSiblingTempFile({
+  dir: "/srv/cache",
+  writeTemp: async (tempPath) => {
+    const handle = await fs.open(tempPath, "w");
+    try {
+      await pipeline(downloadStream, handle.createWriteStream());
+    } finally {
+      await handle.close();
+    }
+    return "blob.bin";
+  },
+  resolveFinalPath: (fileName) => path.join("/srv/cache", fileName),
+});
+
+console.log(`downloaded ${r.filePath}`);
+```
+
+### Per-call private scratch in a test
+
+```ts
+import { withTempWorkspace } from "@openclaw/fs-safe/temp";
+
+it("processes a fixture", async () => {
+  await withTempWorkspace({ rootDir: "/tmp/my-tests", prefix: "test-" }, async (ws) => {
+    await fs.writeFile(path.join(ws.dir, "input.txt"), fixture);
+    const out = await processFile(path.join(ws.dir, "input.txt"));
+    expect(out).toEqual(expected);
+  });
+});
+```
+
+## See also
+
+- [Atomic writes](atomic.md) — `replaceDirectoryAtomic` for whole-directory swaps.
+- [`root()`](root.md) — `fs.copyIn(rel, sourceAbs)` for moving files from a temp into a `Root`.
+- [File lock](sidecar-lock.md) — when many processes share a temp tree.

package/docs/test-hooks.md

@@ -0,0 +1,74 @@
+---
+title: Test hooks
+description: "Internal-only injection hooks used by the fs-safe test suite. Active only when NODE_ENV=test or VITEST=true."
+---
+
+# `@openclaw/fs-safe/test-hooks`
+
+Internal injection points the `fs-safe` test suite uses to deterministically reproduce open/lstat races. They are exposed as a public subpath so downstream test suites can reuse the same harness, but they are **not** part of the supported runtime API.
+
+```ts
+import {
+  getFsSafeTestHooks,
+  __setFsSafeTestHooksForTest,
+  type FsSafeTestHooks,
+} from "@openclaw/fs-safe/test-hooks";
+```
+
+## When the hooks are active
+
+Hooks are only honored when one of the following is true:
+
+- `process.env.NODE_ENV === "test"`
+- `process.env.VITEST === "true"`
+
+Calling `__setFsSafeTestHooksForTest(hooks)` outside of those environments throws. `getFsSafeTestHooks()` returns `undefined` when no hooks are registered, regardless of the environment.
+
+## Shape
+
+```ts
+type FsSafeTestHooks = {
+  afterPreOpenLstat?: (filePath: string) => Promise<void> | void;
+  beforeOpen?: (filePath: string, flags: number) => Promise<void> | void;
+  afterOpen?: (filePath: string, handle: FileHandle) => Promise<void> | void;
+};
+```
+
+| Hook | Fires when |
+|---|---|
+| `afterPreOpenLstat` | A pre-open `lstat` has just resolved. Use this to swap a path between validation and open. |
+| `beforeOpen` | The library is about to call `open(path, flags)`. Use this to inject a TOCTOU window. |
+| `afterOpen` | An open just succeeded. Use this to mutate state before the post-open identity check runs. |
+
+Each hook may be sync or async; async hooks are awaited.
+
+## Usage
+
+```ts
+import { afterEach, beforeEach } from "vitest";
+import { __setFsSafeTestHooksForTest } from "@openclaw/fs-safe/test-hooks";
+
+beforeEach(() => {
+  __setFsSafeTestHooksForTest({
+    beforeOpen: async (filePath) => {
+      // swap a victim file with a symlink right before fs-safe opens it
+      await replaceWithSymlink(filePath);
+    },
+  });
+});
+
+afterEach(() => {
+  __setFsSafeTestHooksForTest(undefined);
+});
+```
+
+Always clear the hooks in `afterEach` so a stuck hook does not leak across tests.
+
+## Stability
+
+The shape can grow new optional fields between minor versions. Treat the surface as test-only and do not rely on it from production code.
+
+## Related pages
+
+- [Testing](testing.md) — broader notes on testing against `fs-safe`.
+- [Security model](security-model.md) — the races these hooks help reproduce.