@openclaw/fs-safe 0.1.0 → 0.1.2

package/docs/quickstart.md

@@ -0,0 +1,147 @@
+# Quickstart
+
+Five minutes. By the end you will have a working `root()` and know how to read, write, atomically replace, and unpack an archive — without your code being able to escape the workspace.
+
+If you have used Go's `os.Root` / `OpenInRoot` or Rust's [`cap-std`](https://github.com/bytecodealliance/cap-std), this is the same shape: a capability-style handle that carries the boundary across every operation. The first thing to internalize is that you stop reasoning about *paths* and start reasoning about *the handle*.
+
+## 1. Build a root
+
+```ts
+import { root } from "@openclaw/fs-safe";
+
+const fs = await root("/srv/jobs/incoming", {
+  hardlinks: "reject",  // refuse files that are hardlinks of out-of-tree inodes
+  symlinks: "reject",   // refuse to traverse a symlink during open
+  mkdir: true,          // create missing parent dirs on write
+});
+```
+
+`root()` resolves the directory through the real filesystem (so symlinked roots become canonical) and verifies it exists. The defaults you pass apply to every call below; per-call options override them.
+
+If the root directory itself does not exist yet, `root()` throws `FsSafeError` with code `not-found`. Either create the directory before calling `root()`, or call `await fs.ensureRoot()` after a successful `root()` to create empty subpaths.
+
+## 2. Read and write text
+
+```ts
+await fs.write("notes/today.txt", "hello\n");
+const text = await fs.readText("notes/today.txt");
+```
+
+Writes use a sibling temp file plus `rename`, so a partial write never appears at the destination. Reads open with `O_NOFOLLOW` where available and verify the opened fd matches the path identity before returning the buffer.
+
+`create()` is the don't-clobber variant of `write()` and throws `already-exists` when the target is already there:
+
+```ts
+await fs.create("notes/README.md", "seed\n"); // throws if it already exists
+```
+
+## 3. JSON, with parsing
+
+```ts
+type Config = { tokens: string[]; updatedAt: string };
+
+await fs.writeJson("state/config.json", { tokens: [], updatedAt: new Date().toISOString() }, {
+  space: 2,
+});
+
+const config = await fs.readJson<Config>("state/config.json");
+```
+
+`writeJson` stringifies and writes atomically. `readJson` reads through the same boundary and parses; validate the shape at your application boundary if it came from a less-trusted source.
+
+## 4. Move and remove
+
+```ts
+await fs.move("notes/today.txt", "notes/archive/today.txt", { overwrite: true });
+await fs.remove("notes/archive/today.txt");
+```
+
+`move()` defaults to no clobber. Pass `{ overwrite: true }` when replacing the target is intentional. `remove()` works on files and empty directories. For non-empty directories, list and remove children first or use [`replaceDirectoryAtomic`](atomic.md#replacedirectoryatomic).
+
+## 5. Inspect
+
+```ts
+const here = await fs.exists("state/config.json"); // boolean
+const stat = await fs.stat("state/config.json");   // { kind, size, mtimeMs, ... }
+const names = await fs.list("state");              // string[]
+const entries = await fs.list("state", { withFileTypes: true }); // DirEntry[]
+```
+
+`exists`, `stat`, and `list` are boundary-checked but **do not pin a later operation** to the same filesystem object. For race-resistant reads or writes, use `read()`, `open()`, `write()`, `create()`, `copyIn()`, `move()`, or `remove()` — they pin the path identity at the point of use.
+
+## 6. Catch escapes
+
+```ts
+import { FsSafeError } from "@openclaw/fs-safe";
+
+try {
+  await fs.write("../escape.txt", "x");
+} catch (err) {
+  if (err instanceof FsSafeError && err.code === "outside-workspace") {
+    // log, count, drop the request
+  } else {
+    throw err;
+  }
+}
+```
+
+Error codes are a closed union — branch on `err.code` instead of matching message text. The full list lives in the [Errors](errors.md) reference.
+
+## 7. Replace a config file atomically
+
+```ts
+import { replaceFileAtomic } from "@openclaw/fs-safe/atomic";
+
+await replaceFileAtomic({
+  filePath: "/srv/jobs/incoming/state/config.json",
+  content: JSON.stringify(state, null, 2),
+  mode: 0o600,
+  syncTempFile: true,
+  syncParentDir: true,
+});
+```
+
+Use `replaceFileAtomic` directly when you have an absolute path you trust and want sibling-temp + rename without going through `root()`. See [Atomic writes](atomic.md).
+
+## 8. Unpack a ZIP
+
+```ts
+import { extractArchive, resolveArchiveKind } from "@openclaw/fs-safe/archive";
+
+const kind = resolveArchiveKind("upload.zip");
+if (!kind) throw new Error("unsupported archive");
+
+await extractArchive({
+  archivePath: "/srv/jobs/incoming/uploads/upload.zip",
+  destDir: "/srv/jobs/incoming/extracted",
+  kind,
+  timeoutMs: 15_000,
+  limits: {
+    maxArchiveBytes: 256 * 1024 * 1024,
+    maxEntries: 50_000,
+    maxExtractedBytes: 512 * 1024 * 1024,
+    maxEntryBytes: 256 * 1024 * 1024,
+  },
+});
+```
+
+Extraction stages into a private dir and merges through the same boundary used by direct writes, so a symlinked entry can't trick the merge into following an out-of-tree path. See [Archive extraction](archive.md).
+
+## 9. Get a private scratch directory
+
+```ts
+import { withTempWorkspace } from "@openclaw/fs-safe/temp";
+
+await withTempWorkspace({ rootDir: "/srv/jobs/tmp", prefix: "build-" }, async (workspace) => {
+  await fs.copyIn("input.bin", "/tmp/source.bin");
+  // ...do work in workspace.dir; auto-cleaned on exit
+});
+```
+
+The directory is mode `0700`, sits under a per-user secure temp root, and is removed when the callback returns or throws. See [Temp workspaces](temp.md).
+
+## Where to next
+
+- [Security model](security-model.md) — exactly what the boundary defends against, what it does not.
+- [Root API](root.md) — every method on the `Root` handle, including streaming and reader callbacks.
+- [Errors](errors.md) — the full code union and what each one means.

package/docs/reading.md

@@ -0,0 +1,174 @@
+# Reading
+
+The `Root` handle exposes five read shapes. Pick the narrowest one that gives you what you need — narrower shapes do less work and surface fewer footguns.
+
+```ts
+const result = await fs.read("notes/today.txt");        // { buffer, realPath, stat }
+const text   = await fs.readText("notes/today.txt");    // string
+const bytes  = await fs.readBytes("image.png");         // Buffer
+const json   = await fs.readJson<Config>("config.json"); // T
+const opened = await fs.open("large.log");               // FileHandle for streaming
+```
+
+## What every read does
+
+Regardless of shape, every read goes through the same boundary checks:
+
+1. Resolve the relative path against the canonical real root.
+2. Reject anything that escapes the root (`outside-workspace`).
+3. Reject `..` segments and absolute inputs (unless via `readAbsolute` with an in-root absolute path).
+4. Open with `O_NOFOLLOW` where available. A symlink in the path triggers `symlink` unless the call's `symlinks` policy is `follow-within-root`.
+5. Stat the open fd and compare to the resolved path's identity (`sameFileIdentity`). A swap mid-call triggers `path-mismatch`.
+6. If `hardlinks: "reject"`, refuse files with `nlink > 1` (`hardlink`).
+7. If `maxBytes` is set, refuse reads larger than the cap (`too-large`).
+
+## Read shapes
+
+### `fs.read(rel, options?)`
+
+The full result. Use it when you need both the bytes and the verified `realPath` or `stat`:
+
+```ts
+const { buffer, realPath, stat } = await fs.read("notes/today.txt");
+console.log(`${stat.size} bytes at ${realPath}`);
+```
+
+### `fs.readText(rel, options?)`
+
+`buffer.toString(encoding)`. Defaults to `defaults.encoding ?? "utf8"`. Pass `encoding` per call to override:
+
+```ts
+const utf16 = await fs.readText("doc.txt", { encoding: "utf16le" });
+```
+
+### `fs.readBytes(rel, options?)`
+
+The buffer alone. Useful when you don't care about the realPath or stat:
+
+```ts
+const png = await fs.readBytes("image.png");
+```
+
+### `fs.readJson<T>(rel, options?)`
+
+`readText` + `JSON.parse`. The generic is a *cast*, not a validator — validate the parsed value at your application boundary if it came from a less-trusted source.
+
+```ts
+type Config = { tokens: string[] };
+const config = await fs.readJson<Config>("config.json");
+```
+
+For tighter control over malformed-or-missing JSON, use the standalone helpers in [`@openclaw/fs-safe/json`](json.md): `tryReadJson` (returns `null` on missing/invalid) vs `readJson` (throws).
+
+### `fs.open(rel, options?)`
+
+Returns a `FileHandle` plus the verified `realPath` and `stat`. Use this for streaming or partial reads, and **always close the handle**:
+
+```ts
+const opened = await fs.open("large.log");
+try {
+  const stream = opened.handle.createReadStream();
+  for await (const chunk of stream) {
+    process.stdout.write(chunk);
+  }
+} finally {
+  await opened.handle.close();
+}
+```
+
+## Read options
+
+```ts
+type RootReadOptions = {
+  hardlinks?: "reject" | "allow";   // override defaults.hardlinks
+  maxBytes?: number;                // refuse reads larger than this many bytes
+  nonBlockingRead?: boolean;        // schedule the read off the main loop
+  symlinks?: "reject" | "follow-within-root"; // override defaults.symlinks
+};
+```
+
+`maxBytes` is enforced eagerly: the library reads up to `maxBytes + 1` and throws `too-large` if there is more, so a hostile target cannot silently exhaust memory.
+
+`nonBlockingRead` is a scheduling hint. It does not affect safety — it lets you keep the event loop responsive when reading large files.
+
+## `readAbsolute()` and `reader()`
+
+Some APIs hand you an absolute path that the caller has already produced. Going back to a relative form just to call `read()` is awkward, so the library exposes:
+
+```ts
+fs.readAbsolute(absPath, options?)   // ReadResult, abs path must be inside the root
+fs.reader(options?)              // (path) => Promise<Buffer>
+```
+
+`readAbsolute` accepts absolute paths. Anything outside the root throws `outside-workspace`.
+
+`reader()` returns a closure that takes either a relative or an absolute path and returns a Buffer. Useful for plugging `fs-safe` into framework loader hooks:
+
+```ts
+const load = fs.reader({ maxBytes: 4 * 1024 * 1024 });
+await someLibrary.parseTemplate({ load });
+```
+
+## Inspection vs reading
+
+`fs.exists`, `fs.stat`, and `fs.list` are advisory. They are safe to call to drive UI or decisions, but they do **not** pin the file:
+
+```ts
+if (await fs.exists("notes/today.txt")) {
+  // the file existed when stat() ran — it may not now
+  const text = await fs.readText("notes/today.txt"); // this is the call that pins
+}
+```
+
+A symlink swap between `exists` and `readText` is caught by the read; the boundary is per-call.
+
+## Streaming patterns
+
+### Read into a writable stream
+
+```ts
+import { pipeline } from "node:stream/promises";
+
+const opened = await fs.open("large.log");
+try {
+  await pipeline(opened.handle.createReadStream(), process.stdout);
+} finally {
+  await opened.handle.close();
+}
+```
+
+### Read in chunks
+
+```ts
+const opened = await fs.open("large.bin");
+try {
+  const buf = Buffer.alloc(64 * 1024);
+  let off = 0;
+  while (true) {
+    const { bytesRead } = await opened.handle.read(buf, 0, buf.length, off);
+    if (bytesRead === 0) break;
+    consume(buf.subarray(0, bytesRead));
+    off += bytesRead;
+  }
+} finally {
+  await opened.handle.close();
+}
+```
+
+## Common errors
+
+- **`outside-workspace`** — relative path escaped the root, or `readAbsolute` got an absolute path outside.
+- **`not-found`** — the file is gone.
+- **`not-file`** — you read a directory or a non-regular file (FIFO, socket, …).
+- **`symlink`** — a path component is a symlink and the policy is `reject`.
+- **`path-mismatch`** — opened fd identity did not match the resolved path. Almost always a TOCTOU swap by something else.
+- **`hardlink`** — `hardlinks: "reject"` saw `nlink > 1`.
+- **`too-large`** — read exceeded `maxBytes`.
+
+See [Errors](errors.md) for the full list.
+
+## See also
+
+- [Writing](writing.md) — companion verbs for produce-side I/O.
+- [JSON files](json.md) — standalone strict/lenient JSON helpers.
+- [Secure file reads](secure-file.md) — pinned absolute file reads with permission checks.

package/docs/regular-file.md

@@ -0,0 +1,175 @@
+# Regular file helpers
+
+The advanced `regular-file` helpers provide direct read/append/stat helpers for absolute file paths, with an explicit "regular file or nothing" contract. Useful when you have a trusted absolute path and want a thin layer on top of `fs` that:
+
+- refuses non-regular files (directories, FIFOs, sockets, symlinks)
+- enforces a `maxBytes` read cap
+- separates "missing" from "io-error" in the result type
+
+```ts
+import {
+  readRegularFile,
+  readRegularFileSync,
+  appendRegularFile,
+  appendRegularFileSync,
+  resolveRegularFileAppendFlags,
+  statRegularFile,
+  statRegularFileSync,
+  type AppendRegularFileOptions,
+  type RegularFileStatResult,
+} from "@openclaw/fs-safe/advanced";
+```
+
+## Stat
+
+### `statRegularFile(filePath)`
+
+Async. Returns:
+
+```ts
+type RegularFileStatResult =
+  | { missing: true }
+  | { missing: false; stat: Stats };
+```
+
+A non-regular file (directory, FIFO, …) returns `{ missing: false }` with a `stat` whose `isFile()` is false — the helper does not throw, you decide what to do.
+
+```ts
+import { statRegularFile } from "@openclaw/fs-safe/advanced";
+
+const r = await statRegularFile("/var/log/app.log");
+if (r.missing) return;
+if (!r.stat.isFile()) throw new Error("expected a regular file");
+console.log(`size=${r.stat.size}`);
+```
+
+### `statRegularFileSync(filePath)`
+
+Synchronous variant. Same shape.
+
+## Read
+
+### `readRegularFile(params)`
+
+Async. Reads the entire file into a Buffer if it is a regular file, with `maxBytes` enforcement.
+
+```ts
+import { readRegularFile } from "@openclaw/fs-safe/advanced";
+
+const result = await readRegularFile({
+  filePath: "/var/log/app.log",
+  maxBytes: 4 * 1024 * 1024,
+});
+if (result.missing) return null;
+if (!result.regular) throw new Error("not a regular file");
+processLog(result.buffer);
+```
+
+Result shape:
+
+```ts
+type Result =
+  | { missing: true }
+  | { missing: false; regular: false; stat: Stats }
+  | { missing: false; regular: true; stat: Stats; buffer: Buffer };
+```
+
+Throws `FsSafeError` with code `too-large` if the file exceeds `maxBytes`. Other I/O errors propagate as `NodeJS.ErrnoException`.
+
+### `readRegularFileSync(params)`
+
+Synchronous variant. Same shape; the only required field is `filePath`. `maxBytes` is optional.
+
+## Append
+
+### `appendRegularFile(options)`
+
+Async. Opens the file in append mode, writes data, closes. Refuses non-regular targets:
+
+```ts
+import { appendRegularFile } from "@openclaw/fs-safe/advanced";
+
+await appendRegularFile({
+  filePath: "/var/log/app.log",
+  data: `[${new Date().toISOString()}] ${line}\n`,
+  encoding: "utf8",
+  prependNewlineIfNeeded: true,
+});
+```
+
+### Options
+
+```ts
+type AppendRegularFileOptions = {
+  filePath: string;
+  data: string | Buffer;
+  encoding?: BufferEncoding;             // default utf8 when data is string
+  prependNewlineIfNeeded?: boolean;      // insert "\n" if file does not end with one
+  flags?: number;                         // raw open flags; default O_WRONLY | O_APPEND
+  mode?: number;                          // default 0o644 if file is created
+};
+```
+
+`prependNewlineIfNeeded` reads the trailing byte of the existing file and prepends a `\n` to your data if it isn't already present. Useful for log appenders that want to preserve line boundaries even when callers forget the newline.
+
+### `appendRegularFileSync(options)`
+
+Synchronous. Same options.
+
+### `resolveRegularFileAppendFlags(append, truncateExisting)`
+
+Helper that returns the right open-flag bitmask for combinations of "append" / "truncate". Use it when you're building your own open path and want to match the append helpers' behavior:
+
+```ts
+import { resolveRegularFileAppendFlags } from "@openclaw/fs-safe/advanced";
+
+const flags = resolveRegularFileAppendFlags(true, false); // O_WRONLY | O_APPEND | O_CREAT
+```
+
+## Difference from `Root` methods
+
+| `regular-file` | `Root` |
+|---|---|
+| Absolute paths only. | Relative to the root. |
+| No identity check post-open. | Identity check on every read/write. |
+| Caller must be confident the path is trusted. | Boundary check is automatic. |
+| Returns explicit `{missing, regular}` shape. | Throws `FsSafeError` with `code`. |
+
+If your call site already trusts the path (it came from your own config, not a caller), `regular-file` is a thinner, faster surface. If the path is caller-influenced, prefer `root()` or wrap in [`pathScope()`](path-scope.md).
+
+## Common patterns
+
+### Read a config file if it's there, else seed
+
+```ts
+const r = await readRegularFile({ filePath: "/etc/app/config.json", maxBytes: 64 * 1024 });
+if (r.missing) {
+  await writeJson("/etc/app/config.json", defaultConfig);
+} else if (r.regular) {
+  applyConfig(JSON.parse(r.buffer.toString("utf8")));
+} else {
+  throw new Error("/etc/app/config.json is not a regular file");
+}
+```
+
+### Cheap "exists and is a file" check
+
+```ts
+const r = await statRegularFile(p);
+if (r.missing || !r.stat.isFile()) return false;
+return true;
+```
+
+### Bounded log tail
+
+```ts
+const r = await readRegularFile({ filePath: logPath, maxBytes: 1 * 1024 * 1024 });
+if (r.missing || !r.regular) return [];
+return r.buffer.toString("utf8").split("\n").slice(-100);
+```
+
+## See also
+
+- [Reading](reading.md) — `Root` reads with boundary checks.
+- [Atomic writes](atomic.md) — for atomic write semantics, prefer `replaceFileAtomic`.
+- [`fs.appendFile`](https://nodejs.org/api/fs.html#fsappendfilepath-data-options-callback) — Node's stock append, without regular-file gating.