@openclaw/fs-safe 0.1.2 → 0.2.0

package/docs/sidecar-lock.md

@@ -1,6 +1,6 @@
 # File lock
 
-`acquireFileLock()` and `withFileLock()` provide a cross-process file lock with retry, stale-lock reclaim, and process-exit cleanup. The lock is implemented as a sidecar file (e.g. `state.json` ↔ `state.json.lock`) — only one acquirer can create the sidecar with `O_CREAT | O_EXCL` at a time.
+`acquireFileLock()` and `withFileLock()` provide a cross-process file lock with retry and process-exit cleanup. The lock is implemented as a sidecar file (e.g. `state.json` ↔ `state.json.lock`) — only one acquirer can create the sidecar with `O_CREAT | O_EXCL` at a time.
 
 ```ts
 import { acquireFileLock } from "@openclaw/fs-safe/file-lock";
@@ -19,9 +19,9 @@ try {
 
 ## Why sidecar?
 
-The lock file sits next to the protected resource. If a process crashes mid-lock, the next acquirer notices the held entry, inspects its payload (PID, host, acquired-at timestamp), and decides — via `shouldReclaim` (defaulting to "is the lock older than `staleMs`?") — whether to take it over.
+The lock file sits next to the protected resource. If a process crashes mid-lock, the next acquirer notices the held entry, inspects its payload (PID, host, acquired-at timestamp), and decides — via `shouldReclaim` (defaulting to "is the lock older than `staleMs`?") — whether it should keep waiting or fail.
 
-The library installs a `process.on("exit")` handler that releases all currently-held locks synchronously, so well-behaved exits leave no stale sidecars. Crashes still need the reclaim path.
+The library installs a `process.on("exit")` handler that releases all currently-held locks synchronously, so well-behaved exits leave no stale sidecars. Crashed holders leave their sidecar behind; remove those through an application-owned recovery path after you have proved the holder cannot still be writing.
 
 ## API
 
@@ -48,9 +48,10 @@ function createFileLockManager(key: string): FileLockManager;
 type FileLockAcquireOptions<TPayload extends Record<string, unknown>> = {
   managerKey?: string;                   // optional in-process manager namespace
   lockPath?: string;                     // override; defaults to `${targetPath}.lock`
-  staleMs: number;                       // how long until a held lock is considered stale
+  staleMs?: number;                      // default 30_000
   timeoutMs?: number;                    // overall acquire deadline; default unbounded
   retry?: FileLockRetryOptions;
+  staleRecovery?: "fail-closed";         // default
   allowReentrant?: boolean;              // if this process already holds it, increment a count instead of failing
   payload: () => TPayload | Promise<TPayload>;
   shouldReclaim?: (params: {
@@ -100,7 +101,7 @@ try {
 }
 ```
 
-If your process dies before `release()` runs and skips the exit handler, the next acquirer reclaims the lock once `staleMs` elapses (or your `shouldReclaim` returns true).
+If your process dies before `release()` runs and skips the exit handler, the sidecar remains. Once `staleMs` elapses (or your `shouldReclaim` returns true), acquisition fails closed instead of deleting by path, because Node cannot atomically bind that deletion to the file that was inspected.
 
 ## `withFileLock` — common shape made one-liner
 
@@ -139,9 +140,9 @@ await handle.release();
 await locks.drain();
 ```
 
-## Reclaim policy: `shouldReclaim`
+## Stale policy: `shouldReclaim`
 
-The default policy reclaims locks whose `acquiredAt` is older than `staleMs`. Pass a custom callback when you want a richer notion of "is the holder still alive":
+The default policy treats locks whose `createdAt` is older than `staleMs` as stale. Pass a custom callback when you want a richer notion of "is the holder still alive":
 
 ```ts
 import { kill } from "node:process";
@@ -155,26 +156,26 @@ const handle = await acquireFileLock(targetPath, {
     if (!Number.isFinite(pid)) return true;
     try {
       kill(pid, 0);
-      return false;                     // process still alive — don't reclaim
+      return false;                     // process still alive — keep waiting
     } catch {
-      return true;                      // process gone — reclaim
+      return true;                      // process gone — fail closed for recovery
     }
   },
 });
 ```
 
-`heldByThisProcess` is true when this manager already holds the lock (relevant for the reentrant case).
+`heldByThisProcess` is true when this manager already holds the lock (relevant for the reentrant case). A `true` result does not delete the sidecar; it lets the acquire loop stop waiting once the retry/timeout policy says to give up.
 
 ## What sidecar locks defend against
 
 - **Two processes writing the same file at once.** `acquire` serializes the critical section.
-- **A crashed holder leaving a stale lock.** `staleMs` plus optional `shouldReclaim` recovers it.
+- **Accidentally deleting a fresh lock during stale recovery.** Stale third-party locks fail closed because safe compare-and-unlink is not available through Node's path APIs.
 - **Race between simultaneous acquire attempts.** `O_CREAT | O_EXCL` ensures one wins.
 
 ## What they do **not** defend against
 
 - **Misbehaving holders that ignore the lock.** Locks are advisory — only callers that go through `acquire` are bound.
-- **Holders that never call `release` and have no liveness check.** Without a real `shouldReclaim`, the lock relies on `staleMs` alone — pick a deadline that is comfortably longer than your real work but short enough to recover from crashes.
+- **Automatic stale lock deletion.** If a process crashes, use the payload and your own supervisor/process table to decide when to remove the sidecar.
 - **Multi-host coordination over network filesystems.** Behavior depends on the underlying filesystem's `O_EXCL` semantics; treat as best-effort.
 
 ## Common patterns

package/docs/store.md

@@ -9,9 +9,13 @@ The `store` subpath bundles two managed wrappers around the same safe-write prim
 
 ```ts
 import {
+  ensureJsonDurableQueueDirs,
   fileStore,
   fileStoreSync,
   jsonStore,
+  loadPendingJsonDurableQueueEntries,
+  resolveJsonDurableQueueEntryPaths,
+  writeJsonDurableQueueEntry,
   type FileStore,
   type FileStoreOptions,
   type FileStoreSync,
@@ -25,6 +29,7 @@ import {
 | [`fileStore()`](file-store.md) | Multi-file directories with safe relative paths, size limits, atomic replacement, stream writes, copy-in, and TTL-based pruning. |
 | `fileStoreSync()` | Synchronous variant of `fileStore()` for places that genuinely cannot await. |
 | [`jsonStore()`](json-store.md) | A single keyed JSON state file with explicit fallback, atomic writes, and optional sidecar locking around read-modify-write updates. |
+| Durable JSON queue helpers | Append/load/ack JSON entry files using atomic writes and delivered markers. |
 | [Private file-store mode](private-file-store.md) | `fileStore({ private: true })` for credentials, tokens, and per-agent state at `0600` files under `0700` directories. |
 
 `fileStore().json("rel.json")` and `jsonStore({ filePath })` are intentionally separate primitives. Use `fileStore().json(...)` when JSON state lives alongside other files in the same managed directory; use `jsonStore({ filePath })` when you have a single absolute path and want the keyed JSON shape directly.
@@ -34,6 +39,34 @@ import {
 - **Multi-file directory under one root** — reach for `fileStore()`. It exposes `write`, `writeJson`, `writeText`, `writeStream`, `read*`, `open`, `copyIn`, `remove`, and `pruneExpired` against safe relative paths.
 - **One JSON state file** — reach for `jsonStore({ filePath })`. Its `update()` and `updateOr()` methods cover the merge-into-defaults and read-modify-write cases.
 - **Credentials or tokens** — pass `private: true` to `fileStore()`. Same store shape; writes route through the secret-file atomic path with `0600`/`0700` permissions.
+- **Durable work queues** — use the durable JSON queue helpers when each work item is a standalone JSON file and acknowledgement is represented by moving it through a short-lived `.delivered` marker.
+
+## Durable JSON queues
+
+The durable queue helpers are intentionally low-level. They do not decide retry,
+dedupe, or recovery policy; they just provide the filesystem mechanics that
+several queue implementations otherwise rewrite by hand.
+
+```ts
+await ensureJsonDurableQueueDirs({ queueDir, failedDir });
+
+const paths = resolveJsonDurableQueueEntryPaths(queueDir, id);
+await writeJsonDurableQueueEntry({
+  filePath: paths.jsonPath,
+  entry,
+  tempPrefix: "queue",
+});
+
+const pending = await loadPendingJsonDurableQueueEntries({ queueDir, tempPrefix: "queue" });
+```
+
+`id` must be a single safe path segment: non-empty, not dot-prefixed, and made
+from letters, numbers, `_`, `-`, and `.`. Slashes, backslashes, NUL bytes, `.`,
+and `..` are rejected.
+
+Use `ackJsonDurableQueueEntry()` after durable processing succeeds and
+`moveJsonDurableQueueEntryToFailed()` when the caller wants to quarantine an
+entry for inspection.
 
 ## Related pages
 

package/docs/temp.md

@@ -175,9 +175,15 @@ const result = await writeSiblingTempFile<string>({
 
 `writeSiblingTempFile` chooses a random sibling name in `dir`, calls your `writeTemp()` callback, validates that `resolveFinalPath(result)` is still inside that same directory, and renames the temp file there.
 
+By default it preserves the historical private-helper behavior of chmodding
+`dir` to `dirMode` (default `0o700`). Pass `chmodDir: false` when the directory
+is a public staging/output path whose existing mode must be preserved.
+
 ### `writeViaSiblingTempPath`
 
-A higher-level convenience — write content + rename in one call:
+A higher-level convenience for callback-based producers. The callback writes to
+a private temp path, then the helper copies the result into `targetPath` through
+the root boundary:
 
 ```ts
 import { writeViaSiblingTempPath } from "@openclaw/fs-safe/advanced";
@@ -191,7 +197,9 @@ await writeViaSiblingTempPath({
 });
 ```
 
-If `replaceFileAtomic` does what you need, prefer that — `writeViaSiblingTempPath` is the lower-level building block.
+If `replaceFileAtomic` does what you need, prefer that. Use
+`writeViaSiblingTempPath` when the producer needs a concrete temp pathname but
+the final destination still needs root-boundary checks.
 
 ## Secure temp root
 

package/docs/testing.md

@@ -159,6 +159,16 @@ Run only the security boundary corpus while iterating on root/path/archive/temp
 pnpm test:security
 ```
 
+Run the static primitive guard after changing low-level filesystem helpers:
+
+```sh
+pnpm lint:fs-boundary
+```
+
+It catches the specific raw fallback patterns that previously led to
+check-then-use bugs, such as direct copy-to-destination fallback and sync temp
+workspace reads that bypass pinned file descriptors.
+
 `pnpm check` also runs `pnpm lint:file-size`. New source and test files should stay under 500 lines. Existing larger files have explicit budgets in `scripts/check-file-size.mjs`; do not increase those budgets as part of unrelated work.
 
 ## See also

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@openclaw/fs-safe",
-  "version": "0.1.2",
+  "version": "0.2.0",
   "description": "Capability-style filesystem roots for Node.js apps that handle untrusted relative paths.",
   "license": "MIT",
   "repository": {
@@ -36,6 +36,10 @@
       "types": "./dist/path.d.ts",
       "default": "./dist/path.js"
     },
+    "./output": {
+      "types": "./dist/output.d.ts",
+      "default": "./dist/output.js"
+    },
     "./advanced": {
       "types": "./dist/advanced.d.ts",
       "default": "./dist/advanced.js"
@@ -97,11 +101,12 @@
     "benchmark": "node scripts/benchmark.mjs",
     "build": "tsc -p tsconfig.json",
     "lint:file-size": "node scripts/check-file-size.mjs",
+    "lint:fs-boundary": "node scripts/check-fs-boundary-primitives.mjs",
     "prepack": "node scripts/prepack-build.mjs",
     "test": "vitest run",
     "test:coverage": "vitest run --coverage",
     "test:security": "vitest run test/fs-safe.test.ts test/openclaw-read-bypass-parity.test.ts test/openclaw-write-bypass-parity.test.ts test/additional-bypass-parity.test.ts test/adversarial-boundary-payloads.test.ts",
-    "check": "pnpm lint:file-size && pnpm build && pnpm test",
+    "check": "pnpm lint:file-size && pnpm lint:fs-boundary && pnpm build && pnpm test",
     "docs:site": "node scripts/build-docs-site.mjs"
   },
   "optionalDependencies": {