@warlock.js/ai-workspace 5.29.0 → 5.31.0

package/CHANGELOG.md

@@ -4,6 +4,73 @@ All notable changes to `@warlock.js/ai-workspace` are documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). `@warlock.js/*` packages are released in lockstep — every package shares the same version number, so a version below may list only the changes that affected this package.
 
+## 5.31.0 - 2026-10-18
+
+### Upgrading
+
+- **BREAKING** `ws.tools.all()` now also returns `multi_edit`, `list_dir`, `todo_write` and `todo_read` (plus `shell_output` / `shell_kill` when `shell.background` is set), and `readonly().tools.all()` adds `list_dir` and the todo tools. To keep the previous set, use `ws.tools.pick("readFile", "editFile", "writeFile", "runShell", "runTests", "grep", "glob")`.
+- **BREAKING** `WorkspaceOps` gains `multiEdit`, `listDir`, `startShell`, `shellOutput`, `shellKill`, `listShells` and `dispose`, and `Workspace` gains those plus `readTodos` / `writeTodos`. A custom implementation or test double must add them.
+- **BREAKING** With deny rules in force (`shell.deny`, `shell.denyPatterns` or `shell.allowCommand`) and no allowlist entry for it, a command that runs another command out of the policy's sight — `sh`/`bash -c`, `eval`, `source`, `xargs`, `sudo`, `setsid`, … — is now refused (`reason: "nested-command"`). List the runner in `shell.allow` to permit it deliberately. `shell.deny` now also matches case-insensitively.
+- Shell refusal messages changed: they now name the cause and say how to fix the command. Branch on the new `WorkspacePolicyError.reason` instead of parsing `message`.
+- `makeRunShellTool(ops, { background: true })` returns `ToolContract<RunShellInput, RunShellResult | RunShellBackgroundResult>`; without `background` the type is unchanged.
+
+### Added
+
+- `list_dir` tool and `ws.listDir(path, { depth, ignore })`: a bounded, jailed directory listing (breadth-first, types and file sizes, `denyPaths` and `list.ignore` left out, symlinks reported but never followed), configured by the new `list` policy (`ignore`, `maxDepth`, `maxEntries`).
+- `multi_edit` tool and `ws.multiEdit({ path, edits, expectHash })`: several exact-string replacements in one file, applied in order and written once — all of them or none. `WorkspaceEditError.editIndex` names the failing edit.
+- Background commands: with `shell.background` (`{ maxJobs?, timeoutMs? }`), `run_shell({ background: true })` / `ws.startShell()` start a long-running command owned by the workspace; `shell_output` / `ws.shellOutput(id, { since, waitMs })` read its new output (newest `shell.maxOutputBytes` kept, with a `cursor`), and `shell_kill` / `ws.shellKill(id)` kill its whole process tree. `ws.listShells()` lists them; `WorkspaceJobError` reports an unknown id. Backends implement the new optional `start()`.
+- `ws.dispose()` kills every background command (whole process trees) and makes the workspace refuse new commands; trees still running when the Node process exits are killed too.
+- `todo_write` / `todo_read` tools and `ws.writeTodos()` / `ws.readTodos()`: an in-memory todo list per workspace, shared with its `readonly()` / `scope()` projections. `TodoStore` is exported.
+- `shell.denyPatterns` (string prefix patterns or RegExps) and `shell.allowCommand(argv, ctx)` (return `true`, `false`, or a refusal explanation) check every command's arguments, so a policy can refuse `git push --force` or `rm -rf /` while allowing `git` and `rm`.
+- `shell.interpreter: "sh"` — an explicit opt-in to shell syntax. The line runs through `/bin/sh -c` (`cmd.exe /d /v:off /s /c` on Windows) after it is parsed and every simple command in it — inside `;`, `&&`, `||`, pipes, subshells, `$( )`, backticks, here-documents and `NAME=value` prefixes — is checked against `deny` / `allow` / `denyPatterns` / `allowCommand`. Redirection targets and `cd` directories must stay in the jail (re-checked with symlinks resolved). Syntax the parser cannot read with confidence is refused. The default stays `"none"` (argv, no shell).
+- `WorkspacePolicyError.reason` (`WorkspaceShellRefusalReason`: `"metacharacter"`, `"unbalanced-quotes"`, `"denied-executable"`, `"not-allowed"`, `"denied-pattern"`, `"nested-command"`, `"unsupported-syntax"`, `"path-escape"`) and `executable`, so hosts can phrase refusals themselves. In argv mode the message suggests the split, e.g. "`&&` is not supported in argv mode: run `git add -A`, then `git commit -m …`, as separate commands."
+- `checkShellCommand(policy, command)` returns the shell policy's verdict (reason, message, every checked argv, paths) without running anything; `analyzeArgvCommand()` and `tokenizeCommand()` are exported.
+- Backends may implement an optional `stat()`; the local and mock backends do. The mock backend's scripted commands accept `running: true` for background starts.
+- The workspace skill and docs cover the new tools, shell mode, argument rules, background commands and `dispose()`.
+
+### Changed
+
+- The docs, the skill and the path resolver's comments explain how the workspace jail differs from `@warlock.js/fs`'s new confined reader (`fs.confined`) and why the workspace keeps its own resolver: it accepts absolute paths inside the jail, `allowPaths` roots, `denyPaths`, links that stay inside and paths that do not exist yet. Behaviour is unchanged.
+
+### Fixed
+
+- The package README no longer ends with a stray empty code block.
+- The workspace tools now publish a JSON Schema of their input, so providers send the model the real parameter names instead of an empty object schema.
+- `run_tests` quotes its `pattern` for the shell in use: single quotes under `interpreter: "sh"`, so `$` and backticks in a test-name filter stay literal.
+
+### Security
+
+- Wrapper commands no longer hide the command they run: `env`, `time`, `timeout`, `nice`, `nohup`, `command`, `builtin`, `exec` and `find -exec` are unwrapped and the inner command is checked too, and a wrapper used with options the policy cannot read is refused. Before, `env rm -rf /` passed a `deny: ["rm"]` policy.
+- `shell.deny` matches executable names case-insensitively, so `RM` on a case-insensitive filesystem no longer slips past `deny: ["rm"]`.
+- A model-supplied `timeoutMs` on `run_shell` can now only lower `shell.timeoutMs`, as documented, never raise it.
+- Shell mode removes `ENV`, `BASH_ENV`, `SHELLOPTS`, `BASHOPTS`, `CDPATH`, `IFS`, `PS4`, `GLOBIGNORE` and `BASH_FUNC_*` from the spawned environment, refuses assignments to such variables, refuses `&` (use background commands), and refuses arithmetic-evaluating constructs (`$(( ))`, array subscripts in builtins) that bash can use to run hidden commands.
+- The docs, README and skill now state plainly that shell mode is a policy layer checked before `/bin/sh` runs the line (defence in depth), not an OS sandbox, and recommend running untrusted agents in a container, microVM or as a separate unprivileged user.
+- Killing a background command, a timeout and `dispose()` signal only the process tree the workspace spawned (its own process group on POSIX, `taskkill /T` on its PID on Windows).
+
+## 5.30.0 - 2026-10-16
+
+### Upgrading
+
+- `RunShellResult.cancelled` is now a required field: a custom `WorkspaceOps`/`Workspace` implementation or a test double that builds a `RunShellResult` must set it (`false` unless the command was cancelled).
+- `Workspace.readFile()` and `WorkspaceOps.readFile()` now return `ReadFileResult` (adds `startLine`, `endLine`, `truncated`, `bytesCapped`, `totalBytes`); custom implementations must return those fields.
+- A read window that starts past the end of the file now returns empty `content` instead of one numbered empty line.
+- `read.maxBytes` now actually caps the returned `content`; a read that used to return a whole long file under a small cap now comes back cut, with `bytesCapped: true`.
+
+### Added
+
+- `exec()` takes a per-call `signal` (`ExecOptions`); `run_shell` and `run_tests` pass the agent run's `ctx.signal`. An already-aborted signal never starts the command; an abort mid-run kills the process tree. Both resolve with the new `cancelled: true` on `RunShellResult`, never together with `timedOut`.
+- `WorkspaceBackendExecOptions.signal` and `WorkspaceBackendExecResult.cancelled?` for custom backends; the local backend removes its abort listener when the command settles, and the mock backend honours a pre-aborted signal.
+- Read results carry `startLine`, `endLine`, `truncated`, `bytesCapped` and `totalBytes`; `readFile()` and the `read_file` tool now return the same `ReadFileResult` for the same arguments.
+
+### Fixed
+
+- `read.maxBytes` is enforced: the returned `content` (line-number prefixes and newlines included) never exceeds it, so a single long line no longer bypasses the cap. Lines are cut on a character boundary, never mid-character; `hash` remains the SHA-256 of the whole file.
+- Command output clipped at `shell.maxOutputBytes` is cut on a character boundary instead of ending in a replacement character.
+- A read window past the end of the file returns empty `content` instead of one numbered empty line.
+- Writes, edits and removes of the same path are serialized, so two concurrent `edit_file` calls (the agent now dispatches a trip's tool calls concurrently) can no longer both pass the stale-hash guard and silently drop one edit; the second re-reads and is rejected as stale, or applies on top when it pinned no hash.
+- Aborting a command after it has already exited (while its stdio is still draining) no longer reports it as `cancelled`; the tree is still reaped so the result settles.
+- On Windows, a `taskkill` that cannot be started no longer crashes the process with an unhandled `'error'` event; the direct child is killed instead.
+
 ## 5.29.0 - 2026-10-01
 
 ### Changed
@@ -75,6 +142,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
 ### Changed
 
 - Lockstep release maintenance and dependency refresh.
+
 ## 5.20.0 - 2026-09-24
 
 ### Changed

package/README.md

@@ -16,11 +16,14 @@ OS sandbox. Every path is `realpath`-resolved and must sit under `cwd` (or an
 `allowPaths` root); `denyPaths` globs are blocked even inside `cwd`; shell commands
 are gated by an executable allow/deny list; and `process.env` is **never** inherited
 wholesale (a command can't even find `node`/`npm` unless the policy opts in via
-`shell.inheritEnv: ["PATH"]`). What it does **not** do is contain what an *allowed*
+`shell.inheritEnv: ["PATH"]`). What it does **not** do is contain what an _allowed_
 process does once it runs — an allowed `node` can still open a socket or read a file
-its own user can reach. For untrusted code you want real isolation (a container /
-microVM backend), which is a later phase. Treat this as least-privilege guardrails
-for a **trusted** agent, not a security boundary around a hostile one.
+its own user can reach. Shell mode (`interpreter: "sh"`) is likewise a policy layer
+checked before `/bin/sh` runs the line — defence in depth, not a sandbox. For an
+untrusted agent, run the workspace inside a container or microVM with only the
+project mounted, or at least as a separate unprivileged user, and keep this policy
+as a second layer. Treat it as least-privilege guardrails for a **trusted** agent,
+not a security boundary around a hostile one.
 
 ## Install
 
@@ -41,7 +44,7 @@ import "@warlock.js/ai-workspace";
 ### 1. Minimal build-loop agent
 
 Hand the whole tool set to an agent and let it read, edit, and run tests until green.
-The failure comes back as tool *data*, so the agent self-corrects.
+The failure comes back as tool _data_, so the agent self-corrects.
 
 ```ts
 import "@warlock.js/ai-workspace";
@@ -66,7 +69,7 @@ await dev.execute("`npm test` fails on the cart-total suite. Make it green.");
 
 A reviewer needs to read and search — never to write or run a shell. Pick exactly
 those three tools, or take a `readonly()` projection (which omits every mutating tool
-*and* rejects the mutating direct methods).
+_and_ rejects the mutating direct methods).
 
 ```ts
 const ws = ai.workspace({ cwd: "/srv/acme-api" });
@@ -80,7 +83,7 @@ await reviewer.execute("Review src/cart for off-by-one bugs and summarize.");
 
 // Equivalent, as a projection of the same jail:
 const ro = ws.readonly();
-ai.agent({ model, tools: ro.tools.all() }); // read_file + grep + glob only
+ai.agent({ model, tools: ro.tools.all() }); // read_file, list_dir, grep, glob, todo tools
 ```
 
 ### 3. Direct programmatic use (no LLM)
@@ -114,29 +117,46 @@ for (const file of await ws.glob("src/models/**/*.ts")) {
 }
 ```
 
-## What's in this release (W1)
+## What's in the package
 
 - **Policy jail** — `cwd` path jail (`realpath`-resolved), `allowPaths` / `denyPaths`,
-  a fail-closed shell allow/deny list (deny wins), per-command timeout + output cap,
-  and opt-in `inheritEnv` (no blanket `process.env`).
+  a fail-closed shell allow/deny list (deny wins), argument rules (`denyPatterns`,
+  `allowCommand`), per-command timeout + output cap, and opt-in `inheritEnv` (no
+  blanket `process.env`).
+- **argv by default, shell on request** — commands run as one argv without a shell;
+  a refusal names the operator and suggests the split. `shell.interpreter: "sh"`
+  opts in to pipes, `&&` and redirections, with every command in the line checked
+  and redirections kept inside the jail.
 - **Backends** — `local` (default; `@warlock.js/fs` + `node:child_process`) and
   `mock` (in-memory, for hermetic disk-free tests).
-- **Seven tools** under `ws.tools.*` — `read_file`, `edit_file`, `write_file`,
-  `run_shell`, `run_tests`, `grep`, `glob` — each a `ToolContract` ready for
-  `ai.agent({ tools })`.
-- **Direct methods** — `readFile` / `writeFile` / `editFile` / `exec` / `grep` /
-  `glob` / `exists` / `mkdir` / `remove`, sharing the exact same policy seam.
+- **Tools** under `ws.tools.*` — `read_file`, `edit_file`, `multi_edit`, `write_file`,
+  `list_dir`, `run_shell` (optionally in the background), `shell_output`,
+  `shell_kill`, `run_tests`, `grep`, `glob`, `todo_write`, `todo_read` — each a
+  `ToolContract` (with a JSON Schema of its input) ready for `ai.agent({ tools })`.
+- **Direct methods** — `readFile` / `writeFile` / `editFile` / `multiEdit` /
+  `listDir` / `exec` / `startShell` / `shellOutput` / `shellKill` / `listShells` /
+  `grep` / `glob` / `exists` / `mkdir` / `remove` / `readTodos` / `writeTodos`,
+  sharing the exact same policy seam, and `dispose()`.
+- **Background commands** — with `shell.background`, dev servers and watchers run
+  as jobs owned by the workspace, read incrementally, and are killed (whole process
+  tree) by `shell_kill` or `dispose()`.
 - **`tools.all()` / `tools.pick(...)`** — every tool, or a least-privilege subset.
-- **`readonly()`** — a projection that vends only read/grep/glob and rejects every
-  mutating direct method.
+- **`readonly()`** — a projection that vends only the read-side tools (read, list,
+  grep, glob, todos) and rejects every mutating direct method.
 - **`scope(subdir)`** — a sub-jailed view rooted at `subdir` (tighter blast radius).
 - **Read-before-edit guard** — `read_file` returns a SHA-256 `hash`; `edit_file`
   requires an exact, unique `oldString` (or `replaceAll`) and rejects on an
   `expectHash` mismatch.
+- **Bounded reads** — `read.maxBytes` caps the UTF-8 bytes a read returns (line
+  numbers included), cutting on a character boundary and flagging `bytesCapped`;
+  the `hash` still covers the whole file.
+- **Cancellable commands** — `exec(cmd, { signal })`, and the agent run's signal
+  in `run_shell` / `run_tests`, stop the command (process tree killed, or never
+  started when already aborted) and report `cancelled: true`.
 
-**Deferred to later phases** (intentionally absent here): the transactional
-lifecycle (`snapshot` / `restore` / `diff` / `changes`), `on()` hooks, `dispose`,
-`usage()`, and the worktree / container backends plus provisioning helpers.
+**Not in the package yet**: the transactional lifecycle (`snapshot` / `restore` /
+`diff` / `changes`), `on()` hooks, `usage()`, and the worktree / container backends
+plus provisioning helpers.
 
 ## Skills
 
@@ -146,4 +166,3 @@ Progressive-disclosure, per-task docs live under [`skills/`](./skills/README.md)
   and operate it (policy, tools, direct methods, `readonly` / `scope`).
 - [`build-loop-agent/`](./skills/build-loop-agent/SKILL.md) — wire `ws.tools.all()`
   into a coding agent that reads → edits → runs tests until green.
-```