@openclaw/fs-safe 0.1.1 → 0.2.0

package/docs/json.md

@@ -0,0 +1,192 @@
+# JSON files
+
+`@openclaw/fs-safe/json` is the standalone JSON surface: strict and lenient read variants plus atomic JSON writes.
+
+```ts
+import {
+  tryReadJson,
+  readJson,
+  readJsonIfExists,
+  readJsonSync,
+  tryReadJsonSync,
+  readRootJsonSync,
+  readRootJsonObjectSync,
+  readRootStructuredFileSync,
+  writeJson,
+  writeJsonSync,
+  JsonFileReadError,
+} from "@openclaw/fs-safe/json";
+```
+
+## Three reads, three failure shapes
+
+Same input, three distinct contracts — pick the one whose error story matches your call site:
+
+```ts
+await readJson<T>("./manifest.json");      // throws JsonFileReadError on missing or invalid
+await readJsonIfExists<T>("./cache.json"); // returns null on missing; throws on invalid
+await tryReadJson<T>("./optional.json");   // returns null on missing or invalid
+```
+
+| Helper | Missing file | Invalid JSON |
+|---|---|---|
+| `readJson` | throws | throws |
+| `readJsonIfExists` | `null` | throws |
+| `tryReadJson` | `null` | `null` |
+
+Use `readJson` when missing-or-malformed is a programmer error you want to surface immediately. Use `readJsonIfExists` when "file not there" is normal but malformed bytes should still page someone. Use `tryReadJson` when neither outcome should crash the caller.
+
+`JsonFileReadError` carries `cause` so you can inspect whether the underlying failure was an `ENOENT`, a `SyntaxError`, or something else.
+
+## Reading
+
+### `readJson<T>(filePath)`
+
+Async strict reader. Throws `JsonFileReadError` on missing or invalid input. The cast is unchecked — validate the shape with your own schema (zod, valibot, …) if it came from an untrusted source.
+
+```ts
+const manifest = await readJson<Manifest>("./manifest.json");
+```
+
+### `readJsonIfExists<T>(filePath)`
+
+Async semi-lenient reader. Returns `null` if the file is missing; throws `JsonFileReadError` if the file exists but cannot be parsed.
+
+```ts
+const cache = (await readJsonIfExists<Cache>("./cache.json")) ?? freshCache();
+```
+
+### `tryReadJson<T>(filePath)`
+
+Async lenient reader. Returns `null` for any failure (missing, unreadable, invalid). The "no fuss" sibling.
+
+```ts
+const optional = (await tryReadJson<Settings>("./settings.json")) ?? defaults;
+```
+
+### `readJsonSync<T>(filePath)`
+
+Synchronous strict reader. Throws `JsonFileReadError` on missing or invalid input, matching the async `readJson` contract.
+
+### `tryReadJsonSync<T>(pathname)`
+
+Synchronous, generic, lenient. Returns `T | null`. Useful in boot paths where you want a typed result without async.
+
+## Root-bounded structured reads
+
+Use the root-bounded readers when you already have a trusted root directory and
+a caller-controlled relative path, but you only need one synchronous structured
+read instead of a full `root()` handle.
+
+```ts
+const result = readRootJsonObjectSync({
+  rootDir: "/safe/workspace",
+  relativePath: "plugin/openclaw.plugin.json",
+  boundaryLabel: "plugin manifest",
+});
+
+if (!result.ok) {
+  // reason is "open", "parse", or "invalid"
+  throw new Error(result.reason);
+}
+
+console.log(result.value);
+```
+
+`readRootJsonSync()` parses any JSON value. `readRootJsonObjectSync()` only
+accepts objects. `readRootStructuredFileSync()` accepts a custom parser and
+validator so callers can layer JSON5, TOML, YAML, or domain-specific validation
+without making `fs-safe` depend on those formats.
+
+## Writing
+
+### `writeJson(filePath, value, options?)`
+
+Async atomic JSON write. `JSON.stringify(value, null, 2)` + sibling-temp + rename. Defaults to file mode `0o600`.
+
+```ts
+await writeJson("./state.json", state, { trailingNewline: true });
+```
+
+Options:
+
+```ts
+type WriteJsonOptions = {
+  mode?: number;             // file mode (default 0o600)
+  dirMode?: number;          // mode for parent dirs created on demand
+  trailingNewline?: boolean; // append "\n" if missing (default false)
+  durable?: boolean;         // default true; false skips temp/parent fsync
+};
+```
+
+`durable: false` preserves atomic temp-file replacement but skips the temp-file
+and parent-directory `fsync` calls. Use it only for reconstructible JSON state
+where lower latency matters more than crash-durability.
+
+### `writeJsonSync(pathname, data)`
+
+Synchronous variant. Convenience wrapper that uses the sync atomic-write path with sensible defaults.
+
+```ts
+writeJsonSync("./prefs.json", { theme: "dark" });
+```
+
+For atomic text writes, use [`writeTextAtomic`](atomic.md) from `@openclaw/fs-safe/atomic`. For in-process serialization, use `createAsyncLock` from the advanced surface, or prefer [`jsonStore`](json-store.md) when you want a JSON-specific read-modify-write helper.
+
+## Common patterns
+
+### Read-modify-write
+
+```ts
+const state = (await readJsonIfExists<State>("./state.json")) ?? initialState();
+state.lastRun = Date.now();
+await writeJson("./state.json", state, { mode: 0o600, dirMode: 0o700 });
+```
+
+### Atomic with secure mode
+
+For credentials or other sensitive JSON, write at mode `0o600`:
+
+```ts
+await writeJson("./auth.json", token, { mode: 0o600, dirMode: 0o700 });
+```
+
+For higher-assurance secrets, prefer the dedicated [secret-file helpers](secret-file.md) — they create the parent directory at `0o700` if missing.
+
+### Strict load on boot
+
+```ts
+let manifest: Manifest;
+try {
+  manifest = await readJson<Manifest>("./manifest.json");
+} catch (err) {
+  if (err instanceof JsonFileReadError) {
+    console.error("manifest unreadable:", err.cause);
+    process.exit(1);
+  }
+  throw err;
+}
+```
+
+### Concurrent readers, single writer
+
+```ts
+const state = await readJsonIfExists<State>("./state.json");
+// missing returns null; malformed JSON still throws
+```
+
+## Error reference
+
+| Throw / return | When |
+|---|---|
+| `null` (lenient reads) | File missing or contents are not valid JSON. |
+| `JsonFileReadError` | `readJson` or `readJsonIfExists` saw unreadable or invalid input. Inspect `cause`. |
+| Native `NodeJS.ErrnoException` | Lower-level fs errors not wrapped. |
+
+## See also
+
+- [JSON store](json-store.md) — a single-file state wrapper with explicit per-call fallback (`readOr` / `updateOr`) and optional sidecar locking.
+- [Atomic writes](atomic.md) — lower-level sibling-temp replacement helpers.
+- [Secret files](secret-file.md) — JSON-or-text writes with mode 0600 in mode 0700 dirs.
+- [Private file-store mode](private-file-store.md) — root-bounded JSON+text state stores.
+- [File lock](sidecar-lock.md) — cross-process coordination.

package/docs/local-roots.md

@@ -0,0 +1,164 @@
+# Local roots
+
+`local-roots` is a small set of helpers for code that holds a list of trusted base directories ("roots") and wants to look up an absolute path or a relative-to-some-root reference against any of them.
+
+The shape covers two needs:
+
+- "Resolve this path string to an absolute path that lives inside one of the configured roots, or refuse it."
+- "Read this path, where it could be `/abs/path`, `~/relative`, `file://…`, or simply the basename of a file in one of my roots."
+
+```ts
+import {
+  readLocalFileFromRoots,
+  resolveLocalPathFromRootsSync,
+} from "@openclaw/fs-safe/advanced";
+```
+
+## Shape of a "roots input"
+
+Both helpers take roots as either an array of strings or a `LocalRootsInputOptions` record. Each root is an absolute path the caller already trusts:
+
+```ts
+type LocalRootsInputOptions = {
+  roots: string[];                    // absolute paths
+  allowAbsolute?: boolean;            // accept absolute inputs (default true)
+  allowFileUrls?: boolean;            // accept file:// URLs (default true)
+  expandHome?: boolean;               // expand ~ in inputs (default true)
+};
+```
+
+If a root is a symlink, it is canonicalized at lookup time. The helpers work in the order roots are listed: the first root that contains the resolved path wins.
+
+## `resolveLocalPathFromRootsSync(input, options)`
+
+Synchronous resolution. Returns:
+
+```ts
+type LocalRootsPathResult =
+  | { ok: true; absolutePath: string; rootDir: string; relativePath: string }
+  | { ok: false; reason: "outside-roots" | "invalid-input" };
+```
+
+```ts
+import { resolveLocalPathFromRootsSync } from "@openclaw/fs-safe/advanced";
+
+const r = resolveLocalPathFromRootsSync("photo.jpg", {
+  roots: ["/srv/uploads", "/srv/cache"],
+});
+
+if (!r.ok) return reply(400, r.reason);
+console.log(r.absolutePath);  // /srv/uploads/photo.jpg (assuming it's there)
+console.log(r.rootDir);       // /srv/uploads
+console.log(r.relativePath);  // photo.jpg
+```
+
+### Resolution order
+
+For each candidate input:
+
+1. If the input is a `file://` URL and `allowFileUrls` is true, decode to an absolute path.
+2. If the input begins with `~/` and `expandHome` is true, expand to the user's home dir.
+3. If the input is absolute (`/...` or Windows drive) and `allowAbsolute` is true, accept it as-is and check it falls under one of the roots.
+4. Otherwise, treat the input as relative and resolve it against each root in order until one contains the resulting path.
+
+If no root contains the result, returns `{ ok: false, reason: "outside-roots" }`.
+
+`"invalid-input"` covers empty strings, embedded NULs, encoded `..` traversal, Windows network paths (`\\server\share`), and other constructs that should not be resolved at all.
+
+## `readLocalFileFromRoots(input, options)`
+
+Async. Resolves through the same logic, then reads the file via [`Root`](root.md) so the read benefits from boundary checks, `O_NOFOLLOW`, and fd identity verification.
+
+```ts
+type LocalRootsReadResult = ReadResult & {
+  rootDir: string;
+  relativePath: string;
+};
+
+const r = await readLocalFileFromRoots("photo.jpg", {
+  roots: ["/srv/uploads", "/srv/cache"],
+  maxBytes: 8 * 1024 * 1024,
+});
+if (!r) return reply(404);
+process.stdout.write(r.buffer);
+```
+
+The result extends `ReadResult` (`{ buffer, realPath, stat }`) with the matched `rootDir` and the path relative to it. Returns `null` if the input doesn't resolve into any root or the file is missing.
+
+### Read options
+
+```ts
+type ReadLocalFileFromRootsOptions = LocalRootsInputOptions & {
+  hardlinks?: "reject" | "allow";
+  maxBytes?: number;
+  symlinks?: "reject" | "follow-within-root";
+};
+```
+
+The read-side options are forwarded to `Root` for the actual read.
+
+## `local-file-access` companions
+
+The `local-file-access` module (re-exported from `@openclaw/fs-safe/advanced`) supplies a few small helpers for input normalization that the roots helpers use under the hood. They are also useful on their own:
+
+```ts
+import {
+  assertNoWindowsNetworkPath,
+  basenameFromMediaSource,
+  hasEncodedFileUrlSeparator,
+  isWindowsDriveLetterPath,
+  isWindowsNetworkPath,
+  safeFileURLToPath,
+  trySafeFileURLToPath,
+} from "@openclaw/fs-safe/advanced";
+```
+
+- `safeFileURLToPath(fileUrl)` — `url.fileURLToPath` with explicit error throwing. Refuses URLs that decode to network paths.
+- `trySafeFileURLToPath(fileUrl)` — same, returns `undefined` instead of throwing.
+- `isWindowsDriveLetterPath(p, platform?)` — true for `C:\...` style absolute paths when the platform is Windows.
+- `isWindowsNetworkPath(p, platform?)` — true for `\\server\share` and `//server/share` style paths when the platform is Windows.
+- `assertNoWindowsNetworkPath(p, label?)` — throws if it is.
+- `basenameFromMediaSource(source?)` — best-effort filename extraction from URLs / data URIs / paths, for naming downloaded media.
+- `hasEncodedFileUrlSeparator(pathname)` — true for paths containing percent-encoded `/` (`%2F` / `%5C`), which often indicate traversal attempts.
+
+## Common patterns
+
+### Multi-root config: search project, then user, then system
+
+```ts
+const text = await readLocalFileFromRoots(name, {
+  roots: [path.join(projectDir, "templates"), path.join(homedir(), ".app/templates"), "/etc/app/templates"],
+  allowAbsolute: false,  // only resolve names, never absolute paths
+  maxBytes: 256 * 1024,
+});
+```
+
+### Validate a file:// URL at the API boundary
+
+```ts
+import { safeFileURLToPath, isWindowsNetworkPath } from "@openclaw/fs-safe/advanced";
+
+let abs: string;
+try {
+  abs = safeFileURLToPath(req.body.fileUrl);
+} catch {
+  return reply(400, "invalid file URL");
+}
+if (isWindowsNetworkPath(abs)) return reply(400, "network paths not allowed");
+```
+
+### Deny absolute, allow relative-only
+
+```ts
+const r = resolveLocalPathFromRootsSync(input, {
+  roots: ["/srv/workspace"],
+  allowAbsolute: false,
+  allowFileUrls: false,
+});
+```
+
+## See also
+
+- [`root()`](root.md) — single-root variant of this multi-root setup.
+- [Path helpers](path.md) — `isPathInside`, `safeRealpathSync` for ad-hoc checks.
+- [`pathScope()`](path-scope.md) — single-root with `Result`-style returns.

package/docs/output.md

@@ -0,0 +1,92 @@
+# External outputs
+
+`@openclaw/fs-safe/output` covers the case where another library insists on
+writing to an absolute path you give it. Browser downloads, renderers, media
+tools, and native libraries often have this shape:
+
+```ts
+import { writeExternalFileWithinRoot } from "@openclaw/fs-safe/output";
+
+await writeExternalFileWithinRoot({
+  rootDir: "/srv/workspace/downloads",
+  path: "reports/today.pdf",
+  write: async (filePath) => {
+    await download.saveAs(filePath);
+  },
+});
+```
+
+The external writer never receives the final destination path. It receives a
+private temp file path instead. After the callback returns, fs-safe copies that
+staged file into the requested target through the same root boundary used by
+`Root.copyIn()`.
+
+## Signature
+
+```ts
+function writeExternalFileWithinRoot<T = void>(
+  options: ExternalFileWriteOptions<T>,
+): Promise<ExternalFileWriteResult<T>>;
+
+type ExternalFileWriteOptions<T = void> = {
+  rootDir: string;
+  path: string; // relative or absolute, but must stay under rootDir
+  write: (filePath: string) => Promise<T>;
+  maxBytes?: number;
+  mode?: number;
+};
+
+type ExternalFileWriteResult<T = void> = {
+  path: string; // final absolute path under the canonical root
+  result: T;    // value returned by write()
+};
+```
+
+The requested `path` must name a file. Missing destination parents are created
+by the helper because the operation is "produce this output file under the
+root"; callers should choose the filename before calling this API.
+
+Use `maxBytes` when the external producer can create arbitrarily large files.
+Use `mode` when the finalized file needs a specific POSIX mode. Both are
+enforced during the `Root.copyIn()` finalization step, after the external writer
+has produced the staged file and before the final target is committed.
+
+## Why not pass the final path to the library?
+
+If a target parent can be swapped after validation, handing an external library
+the final path can make the library write outside the intended root before
+fs-safe has a chance to finalize or reject the operation. This helper stages in
+a private temp workspace first, then finalizes with `Root.copyIn()`. That keeps
+the trust-boundary write inside fs-safe's root-aware copy/atomic-write path.
+
+## Browser download example
+
+```ts
+const outputPath = requestedOutputPath || sanitizeBrowserSuggestedName(suggestedFilename);
+
+await writeExternalFileWithinRoot({
+  rootDir: downloadsRoot,
+  path: outputPath,
+  maxBytes: 512 * 1024 * 1024,
+  write: async (filePath) => {
+    await download.saveAs(filePath);
+  },
+});
+```
+
+The chosen path may be absolute if it is already inside `downloadsRoot`, or
+relative to `downloadsRoot`. Traversal, symlink parent escapes, hardlinked final
+targets, over-large staged files, and missing temp files surface as
+`FsSafeError`s.
+
+This helper is not the right fit when the final filename depends on inspecting
+the produced bytes. In that case, write to a private temp workspace, sniff or
+validate the file, choose the final name, then copy or write into the root with
+the normal root APIs.
+
+## See also
+
+- [Root writes](writing.md) — `write`, `copyIn`, `move`, and `mkdir`.
+- [Temp workspaces](temp.md) — private scratch directories for longer workflows.
+- [`pathScope()`](path-scope.md) — validation-only helper when you must pass an
+  absolute path directly to another library.

package/docs/path-scope.md

@@ -0,0 +1,130 @@
+# pathScope()
+
+`pathScope()` is an advanced helper with the same boundary semantics as `root()`, but it operates on **absolute paths** the caller already trusts and returns plain `{ ok, path }` results instead of throwing. Use it when you want the boundary check up front before handing an absolute path to another library.
+
+```ts
+import { pathScope } from "@openclaw/fs-safe/advanced";
+
+const uploads = pathScope("/srv/uploads", { label: "uploads directory" });
+
+const photo = uploads.resolve("user/photo.jpg");
+if (!photo.ok) throw new Error(photo.error);
+
+await sharp(photo.path).resize(800).toFile(/* … */);
+```
+
+## When to reach for it
+
+`root()` is the right answer when you want method-style I/O (`fs.write`, `fs.readText`). `pathScope()` is the right answer when:
+
+- You're handing absolute paths to **other libraries** that take absolute path strings (Sharp, FFmpeg, `tar`, native modules) and just want the boundary check up front.
+- You want to validate user-supplied paths in bulk and return a typed `Result` rather than catching exceptions.
+- You want the scope's `label` to appear in the error message: `"Invalid path: must stay within uploads directory"`.
+
+## Signature
+
+```ts
+function pathScope(rootDir: string, options: PathScopeOptions): PathScope;
+
+type PathScopeOptions = {
+  label: string;          // appears in error messages
+};
+
+type PathScopeResolveOptions = {
+  defaultName?: string;   // used when the requested path is empty
+};
+
+type PathResult = { ok: true; path: string } | { ok: false; error: string };
+type PathsResult = { ok: true; paths: string[] } | { ok: false; error: string };
+
+type PathScope = {
+  rootDir: string;
+  label: string;
+  resolve(requestedPath: string, options?: PathScopeResolveOptions): PathResult;
+  resolveAll(requestedPaths: string[]): PathsResult;
+  existing(requestedPaths: string[]): Promise<PathsResult>;
+  files(requestedPaths: string[]): Promise<PathsResult>;
+  writable(requestedPath: string, options?: PathScopeResolveOptions): Promise<PathResult>;
+  ensureDir(requestedPath: string, options?: PathScopeResolveOptions & { mode?: number }): Promise<PathResult>;
+};
+```
+
+## Methods
+
+### `resolve(rel, options?)`
+
+Synchronous, lexical only — no filesystem touch. Resolves the relative path against `rootDir` and verifies the result stays inside the scope. Use when you only need the path string:
+
+```ts
+const r = uploads.resolve("user/photo.jpg");
+if (!r.ok) return reply(400, r.error);
+console.log(r.path); // /srv/uploads/user/photo.jpg
+```
+
+If the requested path is empty, `defaultName` (when set) is used.
+
+### `resolveAll(rels)`
+
+Synchronous bulk resolve. Returns `{ ok: true, paths }` if every input is in scope, or `{ ok: false, error }` on the first failure.
+
+### `existing(rels)`
+
+Async. Like `resolveAll`, but additionally allows trusted absolute paths whose `realpath` is inside the scope. Useful when callers pass either a relative path or an absolute path you've already validated.
+
+### `files(rels)`
+
+Async. Strict variant: every input must already exist as a regular file inside the scope, with `nlink === 1`. Symlinked entries are rejected.
+
+### `writable(rel, options?)`
+
+Async. Resolves a writable target: ensures the parent directory exists inside the scope, refuses targets whose existing inode is a symlink or hardlinked, and returns the absolute path to use for the write.
+
+```ts
+const t = await uploads.writable("reports/2026/05/report.pdf");
+if (!t.ok) return reply(400, t.error);
+await fs.writeFile(t.path, body);
+```
+
+### `ensureDir(rel, options?)`
+
+Async. `mkdir -p` inside the scope. Walks each segment, refuses any symlink in the path, creates missing directories. Optional `mode` sets the directory mode.
+
+```ts
+const dir = await uploads.ensureDir("inbox", { mode: 0o755 });
+if (!dir.ok) return reply(500, dir.error);
+```
+
+## Result type vs throwing
+
+`pathScope` returns `{ ok: true, path }` / `{ ok: false, error }` instead of throwing. This makes it pleasant to use at validation boundaries:
+
+```ts
+function handle(req: Req, res: Res) {
+  const r = uploads.resolve(req.body.path);
+  if (!r.ok) return res.status(400).json({ error: r.error });
+  return doWork(r.path);
+}
+```
+
+For exception-flavored APIs, use `root()` instead — it throws `FsSafeError` with a typed `code`.
+
+## Combine with `root()`
+
+A common pattern at the edge of an HTTP handler: validate the input with `pathScope`, do the I/O with `root()`:
+
+```ts
+const uploads = pathScope("/srv/uploads", { label: "uploads" });
+const fs = await root("/srv/uploads", { hardlinks: "reject" });
+
+const r = uploads.resolve(req.body.path);
+if (!r.ok) return reply(400, r.error);
+const rel = path.relative(fs.rootReal, r.path);
+
+const text = await fs.readText(rel);
+```
+
+## See also
+
+- [`root()`](root.md) — method-style boundary that throws on failure.
+- [Path helpers](path.md) — `isPathInside`, `safeRealpathSync`, `isWithinDir` for ad-hoc checks.
+- [Errors](errors.md) — error semantics (note: `pathScope` returns errors, doesn't throw).