unbound-cli 1.4.0 → 1.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/PLAN-web-4887.md +38 -0
- package/PLAN.md +79 -90
- package/README.md +4 -2
- package/package.json +1 -1
- package/src/commands/policy.js +14 -7
- package/src/lib/no-ai-guard.js +77 -0
- package/test/no-ai-guard.test.js +363 -0
package/PLAN-web-4887.md
CHANGED
|
@@ -320,6 +320,8 @@ forward-compatible: if the backend later adds `explanation`, the existing
|
|
|
320
320
|
|
|
321
321
|
## 8. Claude Code skill — distribution and content
|
|
322
322
|
|
|
323
|
+
> ~~DEPRECATED — replaced by CLI-side guards in §14~~. The external Claude Code skill described below was never shipped; setup-repo PR #163 was closed unmerged. CLI-side deterministic enforcement (layers 1+2+3) supersedes this approach. See §14 for the shipped design.
|
|
324
|
+
|
|
323
325
|
The CLI cannot edit Claude's system prompt, but it can drop a skill file. Skills are the closest analog to a CLAUDE.md hook that we can install programmatically.
|
|
324
326
|
|
|
325
327
|
### 8.1 Where it lives
|
|
@@ -475,3 +477,39 @@ These are intentionally out of scope for WEB-4887 to keep the surface narrow. Sp
|
|
|
475
477
|
Each `unbound setup claude-code` and `unbound onboard` run **overwrites** `~/.claude/skills/unbound-tool-policy/SKILL.md` with the version fetched from the `setup` repo's main branch at install time. No checksum gating, no diff-and-prompt. Rationale: the skill is a derived artifact of whatever's in the setup repo's main; treating it as an install-time output (not user state) keeps the mental model simple and ensures Claude is always steered with the current rules.
|
|
476
478
|
|
|
477
479
|
(The skill source-of-truth was originally bundled in unbound-cli during initial implementation; relocated to the setup repo to follow the existing per-tool-artifact convention. See companion PR https://github.com/websentry-ai/setup/pull/163.)
|
|
480
|
+
|
|
481
|
+
---
|
|
482
|
+
|
|
483
|
+
## 14. Steering enforcement (post-skill-kill)
|
|
484
|
+
|
|
485
|
+
§8 (external Claude Code skill) is deprecated. Setup-repo PR #163 was closed unmerged, and the install-side artifact path was abandoned in favor of CLI-side deterministic enforcement that ships in the same binary as the commands it protects. Three layers, evaluated in order, implemented in `src/lib/no-ai-guard.js` and wired into the `.action()` handlers of `create-terminal` (line 1430) and `create-mcp` (line 1602) immediately after `requireLogin()`.
|
|
486
|
+
|
|
487
|
+
### 14.1 Layer 1 — hard guard at the action boundary
|
|
488
|
+
|
|
489
|
+
`unbound policy tool create-terminal` and `create-mcp` reject invocations that pass neither `--prompt` nor an explicit `--no-ai` opt-out. The error message names the AI-assist form, instructs the caller to retry with `--prompt "<text>"`, and surfaces `--no-ai` as the documented opt-out for raw classification flags. Exit code is `2` (invalid usage), routed through `err.exitCode = 2` and propagated by the `process.exitCode = err.exitCode || 1` catch block.
|
|
490
|
+
|
|
491
|
+
`--prompt` and `--no-ai` are also declared mutex: passing both exits 2 with a "not both" message. This avoids ambiguity about which path the caller intended.
|
|
492
|
+
|
|
493
|
+
Why an action-time guard and not a commander.js parse-time validator: commander has no first-class mutex API for "exactly one of A, B" that also accommodates a default-to-A steering nudge, and threading the AI-assist routing through commander's option parsing would complicate the existing `--prompt` branch in `policy.js`. Throwing from `.action()` keeps the guard local to the two subcommands it protects.
|
|
494
|
+
|
|
495
|
+
### 14.2 Layer 2 — Claude Code detection + env-gated escape hatch
|
|
496
|
+
|
|
497
|
+
When `process.env.CLAUDECODE === '1'`, `--no-ai` is additionally rejected unless `process.env.UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE === '1'` is also set. The rejection message explicitly says "intended for interactive humans, not for agents." This is steering, not security: the env var is in plain sight in the error message, and an adversarial agent that reads the message could set it itself. Accepted residual risk per the original principal-architect review. Server-side detection (deferred ticket) if pick-rate telemetry later shows leakage.
|
|
498
|
+
|
|
499
|
+
`CLAUDECODE=1` is set by current Claude Code variants. If future variants do not set it, layer 1 still steers to `--prompt`, and the layer-2 nudge silently no-ops — an acceptable degradation. Future Claude Code variants can match `CLAUDECODE` to remain steered by this layer.
|
|
500
|
+
|
|
501
|
+
### 14.3 Layer 3 — banner-first `--help`
|
|
502
|
+
|
|
503
|
+
`unbound policy tool create-terminal --help` (and `create-mcp --help`) opens with a short banner that names the AI-assisted form before commander's auto-generated `Usage:` line. This is the first text a reader (human or agent) sees on `--help`, so it leads the eye to `--prompt`. Implementation: `.addHelpText('before', helpBannerFor(subcommandName))`. The existing `addHelpText('after', ...)` examples are updated so every manual-flag example prepends `--no-ai`, making the documented opt-out form unambiguous.
|
|
504
|
+
|
|
505
|
+
### 14.4 Why the external skill was killed
|
|
506
|
+
|
|
507
|
+
The original §8 plan installed `~/.claude/skills/unbound-tool-policy/SKILL.md` via `unbound setup claude-code` and `unbound onboard`, with per-user MDM rollout in setup-repo PR #163. Three problems killed it:
|
|
508
|
+
|
|
509
|
+
1. **Install-time artifact, runtime enforcement.** A skill file in `~/.claude/` is a soft nudge — Claude reads it, but nothing in the CLI enforces what the skill recommends. CLI-side guards close the loop: the binary that creates the policy is also the binary that decides which path to route through.
|
|
510
|
+
2. **Distribution sprawl.** Skill content lived in the setup repo to follow the per-tool-artifact convention. Updating steering rules meant a setup-repo PR, an MDM rollout, and a CLI release on different cadences. Folding the rules into the CLI collapses that to one release surface.
|
|
511
|
+
3. **MDM rollout complexity.** Per-user install across `get_all_user_homes()` under `_run_as_user(username, ...)` was correct but added scope to a security-critical primitive for a feature that the CLI can enforce on its own.
|
|
512
|
+
|
|
513
|
+
### 14.5 Versioning
|
|
514
|
+
|
|
515
|
+
Minor bump: `1.4.0` → `1.5.0`. The breaking surface is two subcommands and the migration is a one-flag (`--no-ai`) prepend. Major bump would over-signal. README ships a `BREAKING CHANGE in 1.5.0` callout at the top of the Tool policy section; PR description names both subcommands and shows the migration.
|
package/PLAN.md
CHANGED
|
@@ -1,117 +1,106 @@
|
|
|
1
|
-
# Implementation Plan: WEB-4887
|
|
1
|
+
# Implementation Plan: WEB-4887 CLI `--no-ai` steering guard (layers 1+2+3)
|
|
2
2
|
|
|
3
|
-
> Generated by /implementation-plan on 2026-06-
|
|
3
|
+
> Generated by /implementation-plan on 2026-06-19. Source: principal-architect.
|
|
4
4
|
>
|
|
5
|
-
>
|
|
6
|
-
> (`web-4887-cli-ai-assist-policy`, commits `a90713e`..`f4508dc`, PR #54).
|
|
7
|
-
> Phase 1 acceptance criteria still hold — see git history. This document
|
|
8
|
-
> contracts Phase 2 (MCP tool AI-assist) only; downstream engineers should
|
|
9
|
-
> read Phase 1's exports in `src/lib/policy-ai-assist.js` to understand the
|
|
10
|
-
> infrastructure being extended (preflight, admin probe, error matrix,
|
|
11
|
-
> confirmation, BLOCK/WARN guard, flag-precedence merge).
|
|
5
|
+
> Successor to PR #54 (`web-4887-cli-ai-assist-policy`) which shipped `--prompt`-based AI-assist for `create-terminal` / `create-mcp`. This plan replaces the killed external Claude Code skill (`unbound-tool-policy`, setup repo PR #163 closed unmerged) with CLI-side deterministic enforcement.
|
|
12
6
|
|
|
13
7
|
## Goal
|
|
14
|
-
|
|
8
|
+
`unbound policy tool create-terminal` and `unbound policy tool create-mcp` deterministically route Claude Code agents through the AI-assist `--prompt` path: invocations without `--prompt` and without an explicit `--no-ai` opt-out exit with code 2 and a copy-paste-ready remediation message; under `CLAUDECODE=1` even `--no-ai` is rejected unless an env-gated escape hatch is set; `--help` leads with the AI-assisted form.
|
|
15
9
|
|
|
16
10
|
## Established Context (do not override)
|
|
17
|
-
-
|
|
18
|
-
-
|
|
19
|
-
-
|
|
20
|
-
-
|
|
21
|
-
-
|
|
22
|
-
-
|
|
23
|
-
-
|
|
24
|
-
-
|
|
25
|
-
-
|
|
26
|
-
-
|
|
11
|
+
- Repo `/Users/dinesh/Code/unbound-cli` — Node.js CLI on commander.js `^12.1.0`, tests via `node --test`.
|
|
12
|
+
- Active branch `web-4887-cli-noai-guard`, stacked on `web-4887-cli-ai-assist-policy` (PR #54). The eventual PR base = `web-4887-cli-ai-assist-policy`; GitHub auto-retargets to `main` after #54 merges.
|
|
13
|
+
- `src/commands/policy.js` — `create-terminal` at line 1430, `create-mcp` at line 1602. Both have an existing `opts.prompt`-presence branch (lines 1473 / 1646) with an empty-`--prompt` guard that exits 2 (lines 1475 / 1648). Flag-path required-field errors at lines 1539–1547 (terminal) / 1708–1715 (MCP), thrown into the handler `catch` which sets `process.exitCode = 1` (lines 1595–1598 / 1767–1770).
|
|
14
|
+
- `src/lib/policy-ai-assist.js` owns the AI-assist `--prompt` path and **MUST NOT be modified** by this work — the new layers route BEFORE that module is reached.
|
|
15
|
+
- Test convention: integration tests at the `commander.parseAsync` layer. Fresh `new Command()` per test, `src/api.js` `.get`/`.post` and `src/config.js` `isLoggedIn`/`getApiKey`/`getBaseUrl` stubbed, `require.cache` invalidated to reset module-level caches. Pattern established in `test/policy-ai-assist.test.js` (`loadFreshModules`, `buildHarness`, `runCreate`).
|
|
16
|
+
- Exit-code convention: `output.error(msg)` + `process.exitCode = N` (no `process.exit`). Exit 2 is already used for invalid-flag failures (line 1476). Exit 1 is reserved for runtime/operational failures.
|
|
17
|
+
- This is a **breaking change** for any caller of `create-terminal` / `create-mcp` with raw flags (no `--prompt`). Clean break recommended over deprecation window; breaking surface is exactly two subcommands.
|
|
18
|
+
- Version bump: `package.json` `1.4.0` → `1.5.0` (minor; team versions liberally; breaking surface is narrow).
|
|
19
|
+
- Do NOT bundle a Claude Code skill or anything that writes to `~/.claude/` — the whole point of this work is to eliminate that external artifact.
|
|
20
|
+
- The original spec doc `PLAN-web-4887.md` is the source of truth for §1–§13 (Phase 1 and Phase 2 AI-assist scope) and must not be overwritten — only extended with a new §14 and a deprecation marker on §8.
|
|
27
21
|
|
|
28
22
|
## Out of Scope
|
|
29
|
-
-
|
|
30
|
-
-
|
|
31
|
-
- AI-assist
|
|
32
|
-
-
|
|
33
|
-
-
|
|
34
|
-
-
|
|
35
|
-
-
|
|
36
|
-
-
|
|
37
|
-
-
|
|
23
|
+
- Modifying `src/lib/policy-ai-assist.js`. Routing decisions in this work happen BEFORE the AI-assist module is reached.
|
|
24
|
+
- Any Claude Code skill, hook, or `~/.claude/` artifact. The point of this work is to remove that external steering layer.
|
|
25
|
+
- Changes to `create-terminal`/`create-mcp` AI-assist behavior, prompt handling, preview rendering, merge precedence, BLOCK/WARN custom-message guard, or response routing.
|
|
26
|
+
- Changes to `update`, `delete`, `get`, `list` tool-policy subcommands. The guard does NOT extend to those (no AI-assist exists for them).
|
|
27
|
+
- Changes to non-tool policy subcommands (cost, model, security).
|
|
28
|
+
- Eval harness updates (`test/eval/`).
|
|
29
|
+
- Telemetry on guard fires (deferred until adoption data demands it).
|
|
30
|
+
- Deprecation-window plumbing (one-release-with-warning option (b)). Clean break (option (a)) is committed.
|
|
31
|
+
- Server-side detection of agents bypassing the guard (deferred — separate ticket if pick-rate telemetry shows leakage).
|
|
32
|
+
- Major-version bump to `2.0.0` (architect endorses minor `1.5.0` per team versioning practice).
|
|
38
33
|
|
|
39
34
|
## Files to Change
|
|
40
35
|
| Path | Change | Acceptance criterion |
|
|
41
36
|
|------|--------|----------------------|
|
|
42
|
-
| `src/commands/policy.js` | (a)
|
|
43
|
-
| `
|
|
44
|
-
| `README.md` |
|
|
45
|
-
| `PLAN-web-4887.md` | §
|
|
37
|
+
| `src/commands/policy.js` | (a) On both `create-terminal` (line 1430) and `create-mcp` (line 1602), add `.option('--no-ai', 'Opt out of the AI-assist guard and use raw classification flags. Mutually exclusive with --prompt.')`. (b) Insert `.addHelpText('before', helpBannerFor('create-terminal'))` (resp. `'create-mcp'`) above the existing `.action(...)`. (c) Update the existing `.addHelpText('after', ...)` examples (lines 1444–1468 and 1617–1641) so every manual-flag example prepends `--no-ai`; AI-assisted (`--prompt`) examples unchanged. (d) At the top of each `.action()` handler — inside the existing `try`, immediately after `requireLogin()` (lines 1471 / 1644) — call `assertSteering(opts, { subcommandName: 'create-terminal' })` (resp. `'create-mcp'`). (e) Change the outer handler `catch` (lines 1596–1598 and 1768–1770) from `process.exitCode = 1` to `process.exitCode = err.exitCode || 1` so the guard's `err.exitCode = 2` propagates (mirrors the already-existing pattern on the AI-assist branch at line 1512). | #1, #2, #3, #4, #5, #6 |
|
|
38
|
+
| `package.json` | Bump `"version"` from `"1.4.0"` to `"1.5.0"`. | #7 |
|
|
39
|
+
| `README.md` | In the "Tool policy" examples section, update every manual-flag example for `create-terminal` and `create-mcp` to prepend `--no-ai`. AI-assisted (`--prompt`) examples unchanged. Add a one-paragraph note under the section header documenting (i) the guard, (ii) the `--no-ai` opt-out, (iii) the `UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE=1` env-var escape hatch for Claude Code interactive humans. Add a top-of-section `BREAKING CHANGE in 1.5.0:` callout. | #9 |
|
|
40
|
+
| `PLAN-web-4887.md` | Append §14 "Steering enforcement (post-skill-kill)" documenting (i) layer 1 hard guard, (ii) layer 2 `CLAUDECODE=1` detection + `UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE=1` escape hatch, (iii) layer 3 banner-first `--help`, (iv) why the external skill (PR #163) was killed. Mark §8 "Claude Code skill — distribution and content" as `~~DEPRECATED — replaced by CLI-side guards in §14~~` with a forward pointer to §14. Do NOT alter §1–§7 or §9–§13. | #10 |
|
|
46
41
|
|
|
47
42
|
## Files to Create
|
|
48
43
|
| Path | Purpose | Acceptance criterion |
|
|
49
44
|
|------|---------|----------------------|
|
|
50
|
-
| `
|
|
45
|
+
| `src/lib/no-ai-guard.js` | New module, ~80 LOC, no external dependencies. Exports two pure functions: `assertSteering(opts, { subcommandName, env = process.env })` (runs layers 1 + 2; throws an `Error` with `err.exitCode = 2` and the appropriate stderr message — see AD-8 wording in the source plan) and `helpBannerFor(subcommandName)` (returns the multi-line banner string for `.addHelpText('before', ...)`). The `env` parameter is dependency-injected (defaults to `process.env`) so tests can exercise the production codepath without touching global env. | #1, #3, #4 |
|
|
46
|
+
| `test/no-ai-guard.test.js` | New integration test file at `commander.parseAsync` layer. Uses `node:test` + `node:assert/strict`. Reuses the harness pattern from `test/policy-ai-assist.test.js` (`loadFreshModules`, fresh `new Command()`, stubbed `api.get`/`api.post`, captured `output.error`/`output.success`/`output.warn`, `program.exitOverride()`). Adds a local `withEnv(overrides, fn)` helper that saves+restores `process.env.CLAUDECODE` and `process.env.UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE` per test. Defines 11 `test(...)` cases (T1–T11) covering every acceptance criterion below; `beforeEach`/`after` hooks reset `process.exitCode = 0` (same convention as `test/policy-ai-assist.test.js` lines 97–106). | #1, #2, #3, #4, #5, #6, #8 |
|
|
51
47
|
|
|
52
48
|
## Test Surface
|
|
53
|
-
- **Layer:** CLI command
|
|
49
|
+
- **Layer:** CLI command — `commander.parseAsync(['node','unbound','policy','tool','create-terminal'|'create-mcp', ...argv])` against a fresh `new Command()` with `src/commands/policy.js`'s `register` invoked per test. `src/api.js` `.get`/`.post` and `src/config.js` `isLoggedIn`/`getApiKey`/`getBaseUrl` are stubbed. `process.env.CLAUDECODE` and `process.env.UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE` are set per-test via a save+restore helper. NO unit tests on `assertSteering` or `helpBannerFor` directly — coverage is end-to-end through `parseAsync`.
|
|
54
50
|
- **Tests to add or extend:**
|
|
55
|
-
- `test/
|
|
56
|
-
1.
|
|
57
|
-
2.
|
|
58
|
-
3.
|
|
59
|
-
4.
|
|
60
|
-
5.
|
|
61
|
-
6.
|
|
62
|
-
7.
|
|
63
|
-
8.
|
|
64
|
-
9.
|
|
65
|
-
10.
|
|
66
|
-
11.
|
|
67
|
-
12. User-attributed `--action BLOCK` without `--custom-message` → M11.
|
|
68
|
-
13. 7 HTTP routing cases parametrized (200 success, 200 success_false length, 200 success_false generic, 401, 422, 5xx, network/timeout) → exit codes 0/2/2/3/2/4/4 → M5.
|
|
69
|
-
14. `--yes` prints preview + skips confirm + issues create POST; assert "Resolved MCP policy" header on stdout, Name/Tools/Action rows present, create POST fires → M6, M7.
|
|
70
|
-
15. `--json` suppresses preview but still POSTs create → M6.
|
|
71
|
-
16. No `--yes`, readline-stubbed `'n'` → preview prints, no create POST → M6.
|
|
72
|
-
17. Out-of-scope keyword `staging` + `--yes` → warning logged AND assist-mcp call fires → M9.
|
|
73
|
-
18. Non-admin probe → exit 3 before assist-mcp call (defensive smoke test against `_privilegesCache` bleed across tests).
|
|
74
|
-
- M1, M10, M12, M13 are verified by inspection / build smoke (M1: `require('../src/lib/policy-ai-assist')` exposes new symbols AND all Phase 1 symbols; M10: existing `test/policy-conditions.test.js` + flag-path tests if any continue to pass — running `npm test` validates this; M12: `README.md` diff; M13: `PLAN-web-4887.md` diff).
|
|
51
|
+
- `test/no-ai-guard.test.js` — covers acceptance criteria #1, #2, #3, #4, #5, #6, #8. Test cases:
|
|
52
|
+
1. **T1** — `create-terminal` with no `--prompt` and no `--no-ai`, no `CLAUDECODE` → `process.exitCode === 2` AND captured `output.error` contains `"AI-assisted"`, `"Retry with"`, AND `"--prompt"`. No `api.post` calls observed. (#1)
|
|
53
|
+
2. **T2** — `create-mcp` with no `--prompt` and no `--no-ai`, no `CLAUDECODE` → same shape as T1, scoped to `create-mcp`. (#1)
|
|
54
|
+
3. **T3** — `create-terminal --no-ai --name X --command-family git --field command='git push*' --action AUDIT` with no `CLAUDECODE` → `api.post('/api/v1/command-policies/', ...)` fires (observed in stub). `process.exitCode === 0`. (#2)
|
|
55
|
+
4. **T4** — `create-mcp --no-ai --name X --mcp-server linear --mcp-action-type read --action AUDIT` with no `CLAUDECODE` → `api.post('/api/v1/command-policies/', ...)` fires. `process.exitCode === 0`. (#2)
|
|
56
|
+
5. **T5** — `create-terminal --prompt "block rm -rf" --no-ai` → `process.exitCode === 2` AND captured `output.error` contains `"not both"`. No `api.post` calls. (#3)
|
|
57
|
+
6. **T6** — `create-terminal --no-ai` under `CLAUDECODE=1`, `UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE` unset → `process.exitCode === 2` AND captured `output.error` contains `"CLAUDECODE=1"` AND `"intended for interactive humans"`. No `api.post` calls. (#4)
|
|
58
|
+
7. **T7** — `create-terminal --no-ai` under `CLAUDECODE=1` AND `UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE=1` → layer 2 bypassed, falls into existing flag-path, which then exits 1 with `"--name is required"` error (asserting the env-escape hatch worked end-to-end). (#4)
|
|
59
|
+
8. **T8** — `create-terminal --prompt "block rm -rf"` under `CLAUDECODE=1` → assist endpoint stub observes `api.post('/api/v1/command-policies/assist/', ...)`. Layer 2 does not interfere. (#5)
|
|
60
|
+
9. **T9** — `create-terminal --help` (collected via `program.outputHelp()` redirected stdout or `--help` triggering commander's help output capture) → stdout contains the AI-assisted banner block AND `idx(banner) < idx("Usage:")` (banner appears before commander's auto-generated Usage line). (#6)
|
|
61
|
+
10. **T10** — same as T9 for `create-mcp --help`. (#6)
|
|
62
|
+
11. **T11** (regression smoke) — `create-terminal --prompt "audit npm installs"` with no `CLAUDECODE`, no `--no-ai` → assist endpoint stub observes the assist POST. Confirms the existing Phase-1 happy-path is unaffected by the new guard. (#8)
|
|
75
63
|
|
|
76
64
|
## Acceptance Criteria
|
|
77
|
-
1.
|
|
78
|
-
2.
|
|
79
|
-
3.
|
|
80
|
-
4.
|
|
81
|
-
5.
|
|
82
|
-
6.
|
|
83
|
-
7.
|
|
84
|
-
8.
|
|
85
|
-
9.
|
|
86
|
-
10.
|
|
87
|
-
11. **M11.** Given the merged body has `action ∈ {BLOCK, WARN}` and no `custom_message`, when the command runs with `--yes`, then `process.exitCode === 2` AND no create POST fires AND `output.error` wording is user-attributed (`--action BLOCK requires --custom-message. ...`) when `opts.action` was passed by the user, or AI-attributed (`AI assist set --action to BLOCK ...`) when only the AI returned the action.
|
|
88
|
-
12. **M12.** Given `README.md`, when read, the "Tool policy examples" MCP section's first example uses `unbound policy tool create-mcp --prompt "..."` AND a flag-based example appears below it as a documented fallback.
|
|
89
|
-
13. **M13.** Given `PLAN-web-4887.md`, when read, §7 is marked "Phase 2 DELIVERED" with a cross-link to the Phase 2 commit AND §7.4 includes a note that assist-mcp has no `explanation` field AND §11.1, §12.2, §12.3 remain in the deferred list.
|
|
65
|
+
1. Given `unbound policy tool create-terminal` (resp. `create-mcp`) invoked with neither `--prompt` nor `--no-ai`, when the command runs, then `process.exitCode === 2`, `output.error` is called with a message containing the substrings `"AI-assisted"`, `"Retry with"`, and `"--prompt"`, and no network call to `api.post` is made.
|
|
66
|
+
2. Given `unbound policy tool create-terminal --no-ai --name X --command-family git --field command='git push*' --action AUDIT` (resp. `create-mcp --no-ai --name X --mcp-server linear --mcp-action-type read --action AUDIT`) with `process.env.CLAUDECODE` unset, when the command runs, then `api.post` is called with path `/api/v1/command-policies/` and `process.exitCode === 0`.
|
|
67
|
+
3. Given `unbound policy tool create-terminal --prompt "X" --no-ai`, when the command runs, then `process.exitCode === 2`, `output.error` is called with a message containing the substring `"not both"`, and no network call is made.
|
|
68
|
+
4. Given `unbound policy tool create-terminal --no-ai` under `process.env.CLAUDECODE === '1'` and `process.env.UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE` unset, when the command runs, then `process.exitCode === 2` AND `output.error` is called with a message containing `"CLAUDECODE=1"` AND `"intended for interactive humans"`. Given the same invocation but with `process.env.UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE === '1'`, layer 2 is bypassed and the existing flag-path errors out on `"--name is required"` (with `process.exitCode === 1`).
|
|
69
|
+
5. Given `unbound policy tool create-terminal --prompt "block rm -rf"` under `process.env.CLAUDECODE === '1'`, when the command runs, then `api.post('/api/v1/command-policies/assist/', ...)` is invoked (assist endpoint is reached; layer 2 does not interfere).
|
|
70
|
+
6. Given `unbound policy tool create-terminal --help` (resp. `create-mcp --help`), when the command runs, then the help output (captured stdout) contains the AI-assisted banner block (substring `"AI-ASSISTED (preferred):"`) AND the index of that banner is strictly less than the index of `"Usage:"`.
|
|
71
|
+
7. `package.json` `version` is `"1.5.0"`.
|
|
72
|
+
8. `npm test` exits 0 with all existing tests in `test/*.test.js` (including `test/policy-ai-assist.test.js` and `test/policy-ai-assist-mcp.test.js`) passing unmodified.
|
|
73
|
+
9. `README.md` Tool policy section contains: (i) every manual-flag example using `--no-ai`, (ii) a paragraph documenting the guard + `--no-ai` + `UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE` env var, (iii) a `BREAKING CHANGE in 1.5.0:` callout at the top of the section.
|
|
74
|
+
10. `PLAN-web-4887.md` has a new §14 "Steering enforcement (post-skill-kill)" documenting the three layers and the env var, AND §8 begins with `~~DEPRECATED — replaced by CLI-side guards in §14~~` and a forward pointer to §14.
|
|
90
75
|
|
|
91
76
|
## Risks & Mitigations
|
|
92
|
-
- **Risk
|
|
93
|
-
- **Mitigation:**
|
|
94
|
-
- **Risk
|
|
95
|
-
- **Mitigation:**
|
|
96
|
-
- **Risk
|
|
97
|
-
- **Mitigation:**
|
|
98
|
-
- **Risk
|
|
99
|
-
- **Mitigation:**
|
|
100
|
-
- **Risk
|
|
101
|
-
- **Mitigation:**
|
|
102
|
-
- **Risk
|
|
103
|
-
- **Mitigation:**
|
|
104
|
-
- **Risk
|
|
105
|
-
- **Mitigation:**
|
|
77
|
+
- **Risk R1: BREAKING CHANGE breaks existing scripts.** Any CI / shell script today running `unbound policy tool create-terminal --name X --command-family git --field K=V --action AUDIT` starts exiting 2 immediately on upgrade.
|
|
78
|
+
- **Mitigation:** PR description includes a `BREAKING CHANGE:` callout naming both subcommands and showing the one-flag (`--no-ai`) migration. README breaking-change banner. Team automation audited in PR review. Breaking surface is narrow (two subcommands).
|
|
79
|
+
- **Risk R2: Claude reads layer-2 stderr and sets `UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE=1` itself.** The env var is in plain sight in the error message; an adversarial reading would lead the agent to set the var and bypass.
|
|
80
|
+
- **Mitigation:** Message wording explicitly says `"This is intended for interactive humans, not for agents."` Accepted residual risk — this is steering, not security. Server-side detection (deferred ticket) if pick-rate telemetry later shows leakage.
|
|
81
|
+
- **Risk R3: `addHelpText('before', ...)` placement varies across commander.js versions.** Could cause the banner to render in the wrong slot.
|
|
82
|
+
- **Mitigation:** Project pins `commander ^12.1.0`. T9/T10 assert `idx(banner) < idx("Usage:")` and fail loudly on drift. Smoke-verify locally before commit.
|
|
83
|
+
- **Risk R4: `process.env.CLAUDECODE` may not be `'1'` in every Claude Code variant.** Worst case: layer 2 silently no-ops.
|
|
84
|
+
- **Mitigation:** Layer 1 still steers in the absence of layer 2. Acceptable degradation; document the env var the CLI checks for in `PLAN-web-4887.md` §14 so future Claude Code variants can match.
|
|
85
|
+
- **Risk R5: Test brittleness around `process.exitCode`.** The runner conflates a leftover non-zero `process.exitCode` with a test-file failure.
|
|
86
|
+
- **Mitigation:** Reuse `beforeEach(() => { process.exitCode = 0; })` and `after(() => { process.exitCode = 0; })` pattern verbatim from `test/policy-ai-assist.test.js` lines 97–106. Do not invent a new convention.
|
|
87
|
+
- **Risk R6: `catch`-block change (`process.exitCode = err.exitCode || 1`) subtly alters exit codes for existing thrown `Error`s that lack `.exitCode`.**
|
|
88
|
+
- **Mitigation:** Today's behavior for such errors is `process.exitCode = 1`; the `|| 1` fallback preserves that exactly. Change is purely additive for errors that DO carry `.exitCode`. Existing `test/policy-ai-assist.test.js` and `test/policy-ai-assist-mcp.test.js` confirm regression-free behavior.
|
|
89
|
+
- **Risk R7: Test harness module-cache bleed between `test/no-ai-guard.test.js` and the existing assist tests.** Module-level caches (e.g. `_privilegesCache` in `src/lib/policy-ai-assist.js`) could leak state.
|
|
90
|
+
- **Mitigation:** Copy the `loadFreshModules` pattern from `test/policy-ai-assist.test.js` lines 9–25 verbatim — it already invalidates `require.cache` for the relevant modules.
|
|
106
91
|
|
|
107
92
|
## Sequencing
|
|
108
|
-
1.
|
|
109
|
-
2.
|
|
110
|
-
3.
|
|
111
|
-
4.
|
|
112
|
-
5.
|
|
113
|
-
6.
|
|
114
|
-
7.
|
|
93
|
+
1. Create `src/lib/no-ai-guard.js` with `assertSteering` and `helpBannerFor`. No call sites yet. Verify the module loads cleanly: `node -e "require('./src/lib/no-ai-guard')"`.
|
|
94
|
+
2. In `src/commands/policy.js`, add `.option('--no-ai', ...)` to both `create-terminal` (line 1430 block) and `create-mcp` (line 1602 block).
|
|
95
|
+
3. In `src/commands/policy.js`, wire `assertSteering(opts, { subcommandName: ... })` into both `.action()` handlers immediately after `requireLogin()` (lines 1471 / 1644).
|
|
96
|
+
4. In `src/commands/policy.js`, change the outer handler `catch` blocks (lines 1596–1598 and 1768–1770) to use `process.exitCode = err.exitCode || 1`.
|
|
97
|
+
5. In `src/commands/policy.js`, add `.addHelpText('before', helpBannerFor('create-terminal'))` / `'create-mcp'` and update the manual examples in the existing `.addHelpText('after', ...)` blocks (lines 1444–1468 and 1617–1641) to prepend `--no-ai`.
|
|
98
|
+
6. Bump `package.json` `version` to `1.5.0`.
|
|
99
|
+
7. Write `test/no-ai-guard.test.js` (T1–T11). Run `npm test`. Iterate stderr wording in `src/lib/no-ai-guard.js` until tests pass and copy remains human-readable.
|
|
100
|
+
8. Update `README.md` Tool policy section per the table above.
|
|
101
|
+
9. Update `PLAN-web-4887.md` — append §14, mark §8 deprecated.
|
|
102
|
+
10. Run `npm test` once more for full regression confirmation.
|
|
103
|
+
11. Self-review the diff. Commit. Open PR with base `web-4887-cli-ai-assist-policy`; PR description includes the `BREAKING CHANGE:` callout and the `--no-ai` migration example.
|
|
115
104
|
|
|
116
105
|
## Open Questions
|
|
117
|
-
None
|
|
106
|
+
- None.
|
package/README.md
CHANGED
|
@@ -195,6 +195,8 @@ unbound policy security create --name "429 Fallback" --sub-type error-code-routi
|
|
|
195
195
|
|
|
196
196
|
Tool policy examples:
|
|
197
197
|
|
|
198
|
+
> **BREAKING CHANGE in 1.5.0:** `create-terminal` and `create-mcp` now require either `--prompt` (AI-assisted creation) or an explicit `--no-ai` opt-out. Invocations with raw classification flags but no `--no-ai` exit with code 2 and a remediation message. Under Claude Code (`CLAUDECODE=1`), even `--no-ai` is rejected unless you also set `UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE=1` — this is intended for interactive humans, not agents.
|
|
199
|
+
|
|
198
200
|
```bash
|
|
199
201
|
# AI-assisted (preferred): describe the policy in natural language.
|
|
200
202
|
unbound policy tool create-terminal --prompt "block rm -rf"
|
|
@@ -207,11 +209,11 @@ unbound policy tool families
|
|
|
207
209
|
unbound policy tool mcp-servers
|
|
208
210
|
|
|
209
211
|
# Flag-based fallback: block destructive shell commands explicitly
|
|
210
|
-
unbound policy tool create-terminal --name "Block rm -rf" --command-family filesystem \
|
|
212
|
+
unbound policy tool create-terminal --no-ai --name "Block rm -rf" --command-family filesystem \
|
|
211
213
|
--field command='rm -rf*' --action BLOCK --custom-message "Destructive command blocked."
|
|
212
214
|
|
|
213
215
|
# Flag-based MCP fallback: audit Linear write operations
|
|
214
|
-
unbound policy tool create-mcp --name "Audit Linear writes" --mcp-server Linear \
|
|
216
|
+
unbound policy tool create-mcp --no-ai --name "Audit Linear writes" --mcp-server Linear \
|
|
215
217
|
--mcp-action-type write --action AUDIT
|
|
216
218
|
|
|
217
219
|
# List, get, delete
|
package/package.json
CHANGED
package/src/commands/policy.js
CHANGED
|
@@ -2,6 +2,7 @@ const config = require('../config');
|
|
|
2
2
|
const api = require('../api');
|
|
3
3
|
const output = require('../output');
|
|
4
4
|
const { formatDate, confirm, parseCommaSeparated } = require('../utils');
|
|
5
|
+
const { assertSteering, helpBannerFor } = require('../lib/no-ai-guard');
|
|
5
6
|
|
|
6
7
|
// ============================================================================
|
|
7
8
|
// Constants and docs URLs
|
|
@@ -1430,6 +1431,7 @@ Subcommands:
|
|
|
1430
1431
|
.command('create-terminal')
|
|
1431
1432
|
.description('Create a TERMINAL_COMMAND policy: monitor or block shell commands run by AI coding tools.')
|
|
1432
1433
|
.option('--prompt <text>', 'Natural-language description; routes through Unbound AI assist. Mutually exclusive with --command-family/--field/--config. Compatible with --name/--description/--action overrides.')
|
|
1434
|
+
.option('--no-ai', 'Opt out of the AI-assist guard and use raw classification flags. Mutually exclusive with --prompt.')
|
|
1433
1435
|
.option('--name <name>', 'Policy name (required unless --prompt is used)')
|
|
1434
1436
|
.option('--command-family <family>', 'Command family (e.g. git, filesystem, network). See `policy tool families`.')
|
|
1435
1437
|
.option('--field <key=pattern>', 'Match field: <key>=<pattern>. Repeatable — multiple --field are ANDed (all must match), one per field, for a more specific policy. At least one required unless --config is used.', (val, prev = []) => [...prev, val])
|
|
@@ -1441,6 +1443,7 @@ Subcommands:
|
|
|
1441
1443
|
.option('--config <json>', 'Advanced: raw config JSON (skips --field builder)')
|
|
1442
1444
|
.option('--yes', 'Skip the confirmation prompt for --prompt flows (preview is still printed)')
|
|
1443
1445
|
.option('--json', 'Output raw JSON of the created policy')
|
|
1446
|
+
.addHelpText('before', helpBannerFor('create-terminal'))
|
|
1444
1447
|
.addHelpText('after', `
|
|
1445
1448
|
Examples:
|
|
1446
1449
|
# AI-assisted (preferred): describe the policy in natural language.
|
|
@@ -1450,15 +1453,15 @@ Examples:
|
|
|
1450
1453
|
$ unbound policy tool create-terminal --prompt "block rm -rf" \\
|
|
1451
1454
|
--name "Block recursive deletes" --action WARN
|
|
1452
1455
|
|
|
1453
|
-
$ unbound policy tool create-terminal --name "Block rm -rf" \\
|
|
1456
|
+
$ unbound policy tool create-terminal --no-ai --name "Block rm -rf" \\
|
|
1454
1457
|
--command-family filesystem --field command='rm -rf*' --action BLOCK \\
|
|
1455
1458
|
--custom-message "Destructive commands are blocked."
|
|
1456
1459
|
|
|
1457
|
-
$ unbound policy tool create-terminal --name "Audit git push" \\
|
|
1460
|
+
$ unbound policy tool create-terminal --no-ai --name "Audit git push" \\
|
|
1458
1461
|
--command-family git --field command='git push*' --action AUDIT
|
|
1459
1462
|
|
|
1460
1463
|
# Multiple --field are ANDed — this fires only on a push to main on origin:
|
|
1461
|
-
$ unbound policy tool create-terminal --name "Block push to main on origin" \\
|
|
1464
|
+
$ unbound policy tool create-terminal --no-ai --name "Block push to main on origin" \\
|
|
1462
1465
|
--command-family git_action \\
|
|
1463
1466
|
--field operation=push --field branch=main --field remote='*origin*' \\
|
|
1464
1467
|
--action BLOCK --custom-message "Pushing to main on origin is blocked."
|
|
@@ -1469,6 +1472,7 @@ Learn more: ${DOCS_TOOL}
|
|
|
1469
1472
|
.action(async (opts) => {
|
|
1470
1473
|
try {
|
|
1471
1474
|
requireLogin();
|
|
1475
|
+
assertSteering(opts, { subcommandName: 'create-terminal' });
|
|
1472
1476
|
|
|
1473
1477
|
if (opts.prompt !== undefined) {
|
|
1474
1478
|
if (typeof opts.prompt !== 'string' || opts.prompt.trim() === '') {
|
|
@@ -1594,7 +1598,7 @@ Learn more: ${DOCS_TOOL}
|
|
|
1594
1598
|
displayToolPolicy(unwrapToolPolicy(data));
|
|
1595
1599
|
} catch (err) {
|
|
1596
1600
|
output.error(err.message);
|
|
1597
|
-
process.exitCode = 1;
|
|
1601
|
+
process.exitCode = err.exitCode || 1;
|
|
1598
1602
|
}
|
|
1599
1603
|
});
|
|
1600
1604
|
|
|
@@ -1602,6 +1606,7 @@ Learn more: ${DOCS_TOOL}
|
|
|
1602
1606
|
.command('create-mcp')
|
|
1603
1607
|
.description('Create an MCP_TOOL policy: monitor or block MCP server tool calls.')
|
|
1604
1608
|
.option('--prompt <text>', 'Natural-language description; routes through Unbound AI assist. Mutually exclusive with --mcp-server/--mcp-tool/--mcp-action-type/--config. Compatible with --name/--description/--action overrides.')
|
|
1609
|
+
.option('--no-ai', 'Opt out of the AI-assist guard and use raw classification flags. Mutually exclusive with --prompt.')
|
|
1605
1610
|
.option('--name <name>', 'Policy name (required unless --prompt is used). Override AI suggestion when used with --prompt.')
|
|
1606
1611
|
.option('--mcp-server <server>', 'MCP server name as shown by `policy tool mcp-servers` (e.g. linear, github). Resolved to its canonical group automatically.')
|
|
1607
1612
|
.option('--mcp-tool <tool>', 'Specific tool name on the MCP server (e.g. create_issue). Mutually exclusive with --mcp-action-type.')
|
|
@@ -1614,6 +1619,7 @@ Learn more: ${DOCS_TOOL}
|
|
|
1614
1619
|
.option('--config <json>', 'Advanced: raw config JSON')
|
|
1615
1620
|
.option('--yes', 'Skip the confirmation prompt for --prompt flows (preview is still printed)')
|
|
1616
1621
|
.option('--json', 'Output raw JSON of the created policy')
|
|
1622
|
+
.addHelpText('before', helpBannerFor('create-mcp'))
|
|
1617
1623
|
.addHelpText('after', `
|
|
1618
1624
|
Examples:
|
|
1619
1625
|
# AI-assisted (preferred): describe the policy in natural language.
|
|
@@ -1625,12 +1631,12 @@ Examples:
|
|
|
1625
1631
|
--custom-message "Linear writes are monitored."
|
|
1626
1632
|
|
|
1627
1633
|
# Match a specific tool on a server (flag-based fallback):
|
|
1628
|
-
$ unbound policy tool create-mcp --name "Block Linear writes" \\
|
|
1634
|
+
$ unbound policy tool create-mcp --no-ai --name "Block Linear writes" \\
|
|
1629
1635
|
--mcp-server linear --mcp-tool create_issue --action BLOCK \\
|
|
1630
1636
|
--custom-message "Issue creation is blocked. Contact admin."
|
|
1631
1637
|
|
|
1632
1638
|
# Match all destructive tools on a server:
|
|
1633
|
-
$ unbound policy tool create-mcp --name "Audit all destructive Slack" \\
|
|
1639
|
+
$ unbound policy tool create-mcp --no-ai --name "Audit all destructive Slack" \\
|
|
1634
1640
|
--mcp-server slack --mcp-action-type destructive --action AUDIT
|
|
1635
1641
|
|
|
1636
1642
|
The server name is resolved to its canonical group automatically, so pass the
|
|
@@ -1642,6 +1648,7 @@ Learn more: ${DOCS_TOOL}
|
|
|
1642
1648
|
.action(async (opts) => {
|
|
1643
1649
|
try {
|
|
1644
1650
|
requireLogin();
|
|
1651
|
+
assertSteering(opts, { subcommandName: 'create-mcp' });
|
|
1645
1652
|
|
|
1646
1653
|
if (opts.prompt !== undefined) {
|
|
1647
1654
|
if (typeof opts.prompt !== 'string' || opts.prompt.trim() === '') {
|
|
@@ -1766,7 +1773,7 @@ Learn more: ${DOCS_TOOL}
|
|
|
1766
1773
|
displayToolPolicy(unwrapToolPolicy(data));
|
|
1767
1774
|
} catch (err) {
|
|
1768
1775
|
output.error(err.message);
|
|
1769
|
-
process.exitCode = 1;
|
|
1776
|
+
process.exitCode = err.exitCode || 1;
|
|
1770
1777
|
}
|
|
1771
1778
|
});
|
|
1772
1779
|
|
|
@@ -0,0 +1,77 @@
|
|
|
1
|
+
// Steering guard for `policy tool create-terminal` / `create-mcp`.
|
|
2
|
+
//
|
|
3
|
+
// Three layers, evaluated in order. Routing happens BEFORE src/lib/policy-ai-assist.js
|
|
4
|
+
// is reached — this module never touches the network and has no external deps.
|
|
5
|
+
//
|
|
6
|
+
// Layer 1 (assertSteering): require either --prompt or an explicit --no-ai opt-out.
|
|
7
|
+
// Reject --prompt + --no-ai (mutex).
|
|
8
|
+
// Layer 2 (assertSteering): under CLAUDECODE=1, reject --no-ai unless the env-gated
|
|
9
|
+
// escape hatch UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE=1 is set.
|
|
10
|
+
// Layer 3 (helpBannerFor): banner-first --help so the AI-assisted form is the
|
|
11
|
+
// one a reader sees first.
|
|
12
|
+
//
|
|
13
|
+
// Errors carry `err.exitCode = 2` so the .action() catch block in src/commands/policy.js
|
|
14
|
+
// can propagate via `process.exitCode = err.exitCode || 1`.
|
|
15
|
+
|
|
16
|
+
function makeGuardError(message) {
|
|
17
|
+
const err = new Error(message);
|
|
18
|
+
err.exitCode = 2;
|
|
19
|
+
return err;
|
|
20
|
+
}
|
|
21
|
+
|
|
22
|
+
// Layer 1 + Layer 2. Throws a guard error (exitCode 2) on rejection.
|
|
23
|
+
// `opts` is the parsed commander options object; `--no-ai` arrives as `opts.ai === false`.
|
|
24
|
+
// "prompt provided" means `--prompt` appeared on the command line at all (even
|
|
25
|
+
// empty-string) — the empty-prompt case is validated by the existing
|
|
26
|
+
// `opts.prompt.trim() === ''` check in the prompt branch so that callers see the
|
|
27
|
+
// dedicated "--prompt cannot be empty." error rather than the steering message.
|
|
28
|
+
function assertSteering(opts, { subcommandName, env = process.env } = {}) {
|
|
29
|
+
const hasPrompt = opts.prompt !== undefined;
|
|
30
|
+
const noAi = opts.ai === false;
|
|
31
|
+
|
|
32
|
+
// Mutex (layer 1): --prompt + --no-ai is incoherent — pick one.
|
|
33
|
+
if (hasPrompt && noAi) {
|
|
34
|
+
throw makeGuardError(
|
|
35
|
+
'Pass --prompt for AI-assisted creation or --no-ai for raw flags, not both.'
|
|
36
|
+
);
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
// Layer 1: neither --prompt nor --no-ai → steer to the AI-assisted form.
|
|
40
|
+
if (!hasPrompt && !noAi) {
|
|
41
|
+
throw makeGuardError(
|
|
42
|
+
`${subcommandName} requires AI-assisted creation. Retry with --prompt "<natural language description>". To use raw classification flags instead, opt out explicitly with --no-ai.`
|
|
43
|
+
);
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
// Layer 2: under Claude Code, even --no-ai is rejected unless the human-only
|
|
47
|
+
// escape hatch is set. This is steering, not security — the env var is in
|
|
48
|
+
// plain sight in the error message. See PLAN risk R2.
|
|
49
|
+
if (noAi && env.CLAUDECODE === '1' && env.UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE !== '1') {
|
|
50
|
+
throw makeGuardError(
|
|
51
|
+
`--no-ai is disabled under CLAUDECODE=1. This is intended for interactive humans, not for agents. Retry with --prompt "<natural language description>", or set UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE=1 if you are an interactive human.`
|
|
52
|
+
);
|
|
53
|
+
}
|
|
54
|
+
}
|
|
55
|
+
|
|
56
|
+
// Layer 3: banner rendered above commander's auto-generated `Usage:` line by
|
|
57
|
+
// `.addHelpText('before', helpBannerFor(...))`. Keep it short so the help screen
|
|
58
|
+
// is still scannable.
|
|
59
|
+
function helpBannerFor(subcommandName) {
|
|
60
|
+
const examples = {
|
|
61
|
+
'create-terminal': 'unbound policy tool create-terminal --prompt "block rm -rf"',
|
|
62
|
+
'create-mcp': 'unbound policy tool create-mcp --prompt "audit all Linear writes"',
|
|
63
|
+
};
|
|
64
|
+
if (!Object.prototype.hasOwnProperty.call(examples, subcommandName)) {
|
|
65
|
+
throw new Error(`helpBannerFor: unknown subcommand "${subcommandName}"`);
|
|
66
|
+
}
|
|
67
|
+
const promptExample = examples[subcommandName];
|
|
68
|
+
return [
|
|
69
|
+
'AI-ASSISTED (preferred):',
|
|
70
|
+
` $ ${promptExample}`,
|
|
71
|
+
'',
|
|
72
|
+
'To use raw classification flags instead, opt out explicitly with --no-ai.',
|
|
73
|
+
'',
|
|
74
|
+
].join('\n');
|
|
75
|
+
}
|
|
76
|
+
|
|
77
|
+
module.exports = { assertSteering, helpBannerFor };
|
|
@@ -0,0 +1,363 @@
|
|
|
1
|
+
const { test, beforeEach, after } = require('node:test');
|
|
2
|
+
const assert = require('node:assert/strict');
|
|
3
|
+
const { Command } = require('commander');
|
|
4
|
+
|
|
5
|
+
// Integration tests for the --no-ai steering guard (WEB-4887, layers 1+2+3).
|
|
6
|
+
// Invokes `unbound policy tool create-terminal` / `create-mcp` against a fresh
|
|
7
|
+
// Command instance with src/api.js and src/config.js stubbed. We never hit the
|
|
8
|
+
// network and never read/write user config.
|
|
9
|
+
//
|
|
10
|
+
// Harness mirrors test/policy-ai-assist.test.js (loadFreshModules, buildHarness,
|
|
11
|
+
// per-test process.exitCode reset). See PLAN risk R7.
|
|
12
|
+
|
|
13
|
+
function loadFreshModules() {
|
|
14
|
+
// Drop the module cache so each test gets a clean module-level
|
|
15
|
+
// `_privilegesCache` in policy-ai-assist and a clean stub surface.
|
|
16
|
+
for (const m of [
|
|
17
|
+
'../src/commands/policy',
|
|
18
|
+
'../src/lib/no-ai-guard',
|
|
19
|
+
'../src/lib/policy-ai-assist',
|
|
20
|
+
'../src/api',
|
|
21
|
+
'../src/config',
|
|
22
|
+
'../src/output',
|
|
23
|
+
]) {
|
|
24
|
+
delete require.cache[require.resolve(m)];
|
|
25
|
+
}
|
|
26
|
+
const api = require('../src/api');
|
|
27
|
+
const config = require('../src/config');
|
|
28
|
+
const output = require('../src/output');
|
|
29
|
+
return { api, config, output };
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
function buildHarness({
|
|
33
|
+
privilegesResponse = { is_admin: true, is_manager: false, is_member: false },
|
|
34
|
+
assistResponse = null,
|
|
35
|
+
assistMcpResponse = null,
|
|
36
|
+
createResponse = { id: 1, name: 'ok' },
|
|
37
|
+
} = {}) {
|
|
38
|
+
const { api, config, output } = loadFreshModules();
|
|
39
|
+
|
|
40
|
+
config.isLoggedIn = () => true;
|
|
41
|
+
config.getApiKey = () => 'fake-key';
|
|
42
|
+
config.getBaseUrl = () => 'https://b.acme';
|
|
43
|
+
|
|
44
|
+
const calls = { posts: [], gets: [] };
|
|
45
|
+
api.get = async (path, opts) => {
|
|
46
|
+
calls.gets.push({ path, opts });
|
|
47
|
+
if (path === '/api/v1/users/privileges/') return privilegesResponse;
|
|
48
|
+
throw new Error(`unexpected GET ${path}`);
|
|
49
|
+
};
|
|
50
|
+
api.post = async (path, opts) => {
|
|
51
|
+
calls.posts.push({ path, body: opts && opts.body });
|
|
52
|
+
if (path === '/api/v1/command-policies/assist/') return assistResponse;
|
|
53
|
+
if (path === '/api/v1/command-policies/assist-mcp/') return assistMcpResponse;
|
|
54
|
+
if (path === '/api/v1/command-policies/') return createResponse;
|
|
55
|
+
throw new Error(`unexpected POST ${path}`);
|
|
56
|
+
};
|
|
57
|
+
|
|
58
|
+
const captured = { success: [], error: [], warn: [], info: [], stdout: [] };
|
|
59
|
+
output.success = (m) => captured.success.push(m);
|
|
60
|
+
output.error = (m) => captured.error.push(m);
|
|
61
|
+
output.warn = (m) => captured.warn.push(m);
|
|
62
|
+
output.info = (m) => captured.info.push(m);
|
|
63
|
+
const origLog = console.log;
|
|
64
|
+
console.log = (m) => captured.stdout.push(String(m || ''));
|
|
65
|
+
const restoreLog = () => { console.log = origLog; };
|
|
66
|
+
|
|
67
|
+
return { api, config, output, calls, captured, restoreLog };
|
|
68
|
+
}
|
|
69
|
+
|
|
70
|
+
async function runArgv(argv) {
|
|
71
|
+
const { register } = require('../src/commands/policy');
|
|
72
|
+
const program = new Command();
|
|
73
|
+
program.exitOverride();
|
|
74
|
+
register(program);
|
|
75
|
+
await program.parseAsync(['node', 'unbound', ...argv]);
|
|
76
|
+
}
|
|
77
|
+
|
|
78
|
+
// Save+restore the two env vars the guard reads. Always set them to a known
|
|
79
|
+
// state on entry (undefined unless explicitly overridden) so a stale value from
|
|
80
|
+
// a previous test or the ambient shell cannot leak into this test.
|
|
81
|
+
function withEnv(overrides, fn) {
|
|
82
|
+
const keys = ['CLAUDECODE', 'UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE'];
|
|
83
|
+
const saved = {};
|
|
84
|
+
for (const k of keys) {
|
|
85
|
+
saved[k] = process.env[k];
|
|
86
|
+
if (Object.prototype.hasOwnProperty.call(overrides, k)) {
|
|
87
|
+
if (overrides[k] === undefined) delete process.env[k];
|
|
88
|
+
else process.env[k] = overrides[k];
|
|
89
|
+
} else {
|
|
90
|
+
delete process.env[k];
|
|
91
|
+
}
|
|
92
|
+
}
|
|
93
|
+
return Promise.resolve()
|
|
94
|
+
.then(fn)
|
|
95
|
+
.finally(() => {
|
|
96
|
+
for (const k of keys) {
|
|
97
|
+
if (saved[k] === undefined) delete process.env[k];
|
|
98
|
+
else process.env[k] = saved[k];
|
|
99
|
+
}
|
|
100
|
+
});
|
|
101
|
+
}
|
|
102
|
+
|
|
103
|
+
beforeEach(() => {
|
|
104
|
+
process.exitCode = 0;
|
|
105
|
+
});
|
|
106
|
+
|
|
107
|
+
// Several tests intentionally set process.exitCode to non-zero values to
|
|
108
|
+
// observe the CLI's exit-code routing. Reset it at the end so node:test does
|
|
109
|
+
// not interpret the file itself as having failed.
|
|
110
|
+
after(() => {
|
|
111
|
+
process.exitCode = 0;
|
|
112
|
+
});
|
|
113
|
+
|
|
114
|
+
// T1 — Layer 1, create-terminal: no --prompt + no --no-ai → exit 2, AI steering message.
|
|
115
|
+
test('T1: create-terminal without --prompt and without --no-ai exits 2 with AI-assist steering message', async () => {
|
|
116
|
+
const h = buildHarness();
|
|
117
|
+
await withEnv({}, async () => {
|
|
118
|
+
try {
|
|
119
|
+
await runArgv(['policy', 'tool', 'create-terminal']);
|
|
120
|
+
assert.equal(process.exitCode, 2);
|
|
121
|
+
const errs = h.captured.error.join('\n');
|
|
122
|
+
assert.ok(errs.includes('AI-assisted'), `expected "AI-assisted" in error: ${errs}`);
|
|
123
|
+
assert.ok(errs.includes('Retry with'), `expected "Retry with" in error: ${errs}`);
|
|
124
|
+
assert.ok(errs.includes('--prompt'), `expected "--prompt" in error: ${errs}`);
|
|
125
|
+
assert.equal(h.calls.posts.length, 0, 'no network call should have fired');
|
|
126
|
+
} finally {
|
|
127
|
+
h.restoreLog();
|
|
128
|
+
}
|
|
129
|
+
});
|
|
130
|
+
});
|
|
131
|
+
|
|
132
|
+
// T2 — Layer 1, create-mcp: same shape as T1, scoped to create-mcp.
|
|
133
|
+
test('T2: create-mcp without --prompt and without --no-ai exits 2 with AI-assist steering message', async () => {
|
|
134
|
+
const h = buildHarness();
|
|
135
|
+
await withEnv({}, async () => {
|
|
136
|
+
try {
|
|
137
|
+
await runArgv(['policy', 'tool', 'create-mcp']);
|
|
138
|
+
assert.equal(process.exitCode, 2);
|
|
139
|
+
const errs = h.captured.error.join('\n');
|
|
140
|
+
assert.ok(errs.includes('AI-assisted'), `expected "AI-assisted" in error: ${errs}`);
|
|
141
|
+
assert.ok(errs.includes('Retry with'), `expected "Retry with" in error: ${errs}`);
|
|
142
|
+
assert.ok(errs.includes('--prompt'), `expected "--prompt" in error: ${errs}`);
|
|
143
|
+
assert.equal(h.calls.posts.length, 0, 'no network call should have fired');
|
|
144
|
+
} finally {
|
|
145
|
+
h.restoreLog();
|
|
146
|
+
}
|
|
147
|
+
});
|
|
148
|
+
});
|
|
149
|
+
|
|
150
|
+
// T3 — Layer 1 happy path, create-terminal: --no-ai + raw flags → create POST fires.
|
|
151
|
+
test('T3: create-terminal --no-ai with required raw flags POSTs to /command-policies/ with exit 0', async () => {
|
|
152
|
+
const h = buildHarness();
|
|
153
|
+
await withEnv({}, async () => {
|
|
154
|
+
try {
|
|
155
|
+
await runArgv([
|
|
156
|
+
'policy', 'tool', 'create-terminal',
|
|
157
|
+
'--no-ai',
|
|
158
|
+
'--name', 'X',
|
|
159
|
+
'--command-family', 'git',
|
|
160
|
+
'--field', 'command=git push*',
|
|
161
|
+
'--action', 'AUDIT',
|
|
162
|
+
]);
|
|
163
|
+
assert.equal(process.exitCode || 0, 0);
|
|
164
|
+
assert.ok(
|
|
165
|
+
h.calls.posts.some((c) => c.path === '/api/v1/command-policies/'),
|
|
166
|
+
`expected create POST, got: ${JSON.stringify(h.calls.posts)}`
|
|
167
|
+
);
|
|
168
|
+
} finally {
|
|
169
|
+
h.restoreLog();
|
|
170
|
+
}
|
|
171
|
+
});
|
|
172
|
+
});
|
|
173
|
+
|
|
174
|
+
// T4 — Layer 1 happy path, create-mcp.
|
|
175
|
+
test('T4: create-mcp --no-ai with required raw flags POSTs to /command-policies/ with exit 0', async () => {
|
|
176
|
+
const h = buildHarness();
|
|
177
|
+
await withEnv({}, async () => {
|
|
178
|
+
try {
|
|
179
|
+
await runArgv([
|
|
180
|
+
'policy', 'tool', 'create-mcp',
|
|
181
|
+
'--no-ai',
|
|
182
|
+
'--name', 'X',
|
|
183
|
+
'--mcp-server', 'linear',
|
|
184
|
+
'--mcp-action-type', 'read',
|
|
185
|
+
'--action', 'AUDIT',
|
|
186
|
+
]);
|
|
187
|
+
assert.equal(process.exitCode || 0, 0);
|
|
188
|
+
assert.ok(
|
|
189
|
+
h.calls.posts.some((c) => c.path === '/api/v1/command-policies/'),
|
|
190
|
+
`expected create POST, got: ${JSON.stringify(h.calls.posts)}`
|
|
191
|
+
);
|
|
192
|
+
} finally {
|
|
193
|
+
h.restoreLog();
|
|
194
|
+
}
|
|
195
|
+
});
|
|
196
|
+
});
|
|
197
|
+
|
|
198
|
+
// T5 — Mutex (layer 1): --prompt + --no-ai exits 2 with "not both" wording.
|
|
199
|
+
test('T5: create-terminal --prompt + --no-ai exits 2 with "not both" wording and no network call', async () => {
|
|
200
|
+
const h = buildHarness();
|
|
201
|
+
await withEnv({}, async () => {
|
|
202
|
+
try {
|
|
203
|
+
await runArgv(['policy', 'tool', 'create-terminal', '--prompt', 'block rm -rf', '--no-ai']);
|
|
204
|
+
assert.equal(process.exitCode, 2);
|
|
205
|
+
const errs = h.captured.error.join('\n');
|
|
206
|
+
assert.ok(errs.includes('not both'), `expected "not both" in error: ${errs}`);
|
|
207
|
+
assert.equal(h.calls.posts.length, 0, 'no network call should have fired');
|
|
208
|
+
} finally {
|
|
209
|
+
h.restoreLog();
|
|
210
|
+
}
|
|
211
|
+
});
|
|
212
|
+
});
|
|
213
|
+
|
|
214
|
+
// T6 — Layer 2: --no-ai under CLAUDECODE=1 without escape hatch → exit 2.
|
|
215
|
+
test('T6: create-terminal --no-ai under CLAUDECODE=1 exits 2 with human-only wording and no network call', async () => {
|
|
216
|
+
const h = buildHarness();
|
|
217
|
+
await withEnv({ CLAUDECODE: '1' }, async () => {
|
|
218
|
+
try {
|
|
219
|
+
await runArgv([
|
|
220
|
+
'policy', 'tool', 'create-terminal',
|
|
221
|
+
'--no-ai',
|
|
222
|
+
'--name', 'X',
|
|
223
|
+
'--command-family', 'git',
|
|
224
|
+
'--field', 'command=git push*',
|
|
225
|
+
'--action', 'AUDIT',
|
|
226
|
+
]);
|
|
227
|
+
assert.equal(process.exitCode, 2);
|
|
228
|
+
const errs = h.captured.error.join('\n');
|
|
229
|
+
assert.ok(errs.includes('CLAUDECODE=1'), `expected "CLAUDECODE=1" in error: ${errs}`);
|
|
230
|
+
assert.ok(
|
|
231
|
+
errs.includes('intended for interactive humans'),
|
|
232
|
+
`expected "intended for interactive humans" in error: ${errs}`
|
|
233
|
+
);
|
|
234
|
+
assert.equal(h.calls.posts.length, 0, 'no network call should have fired');
|
|
235
|
+
} finally {
|
|
236
|
+
h.restoreLog();
|
|
237
|
+
}
|
|
238
|
+
});
|
|
239
|
+
});
|
|
240
|
+
|
|
241
|
+
// T7 — Layer 2 escape hatch: with UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE=1, layer 2 is
|
|
242
|
+
// bypassed and the existing flag-path then errors out on missing --name.
|
|
243
|
+
test('T7: --no-ai under CLAUDECODE=1 + UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE=1 bypasses layer 2 and falls through to flag-path required-field error', async () => {
|
|
244
|
+
const h = buildHarness();
|
|
245
|
+
await withEnv({ CLAUDECODE: '1', UNBOUND_ALLOW_NO_AI_UNDER_CLAUDE: '1' }, async () => {
|
|
246
|
+
try {
|
|
247
|
+
await runArgv(['policy', 'tool', 'create-terminal', '--no-ai']);
|
|
248
|
+
assert.equal(process.exitCode, 1, 'flag-path required-field error must use exit 1, not the guard exit 2');
|
|
249
|
+
const errs = h.captured.error.join('\n');
|
|
250
|
+
assert.ok(errs.includes('--name is required'), `expected "--name is required" in error: ${errs}`);
|
|
251
|
+
assert.equal(h.calls.posts.length, 0, 'no network call should have fired');
|
|
252
|
+
} finally {
|
|
253
|
+
h.restoreLog();
|
|
254
|
+
}
|
|
255
|
+
});
|
|
256
|
+
});
|
|
257
|
+
|
|
258
|
+
// T8 — Layer 2 does not interfere with --prompt under CLAUDECODE=1.
|
|
259
|
+
test('T8: create-terminal --prompt under CLAUDECODE=1 routes to the assist endpoint', async () => {
|
|
260
|
+
const h = buildHarness({
|
|
261
|
+
assistResponse: {
|
|
262
|
+
success: true,
|
|
263
|
+
form_updates: {
|
|
264
|
+
command_family: 'filesystem',
|
|
265
|
+
selected_field: 'command',
|
|
266
|
+
field_value: 'rm -rf*',
|
|
267
|
+
action: 'AUDIT',
|
|
268
|
+
name: 'X',
|
|
269
|
+
},
|
|
270
|
+
explanation: 'ok',
|
|
271
|
+
},
|
|
272
|
+
});
|
|
273
|
+
await withEnv({ CLAUDECODE: '1' }, async () => {
|
|
274
|
+
try {
|
|
275
|
+
await runArgv(['policy', 'tool', 'create-terminal', '--prompt', 'block rm -rf', '--yes']);
|
|
276
|
+
assert.ok(
|
|
277
|
+
h.calls.posts.some((c) => c.path === '/api/v1/command-policies/assist/'),
|
|
278
|
+
`expected assist POST, got: ${JSON.stringify(h.calls.posts.map((c) => c.path))}`
|
|
279
|
+
);
|
|
280
|
+
} finally {
|
|
281
|
+
h.restoreLog();
|
|
282
|
+
}
|
|
283
|
+
});
|
|
284
|
+
});
|
|
285
|
+
|
|
286
|
+
// T9 — Layer 3: create-terminal --help banner appears before "Usage:".
|
|
287
|
+
test('T9: create-terminal --help renders AI-ASSISTED banner before "Usage:"', async () => {
|
|
288
|
+
const h = buildHarness();
|
|
289
|
+
let captured = '';
|
|
290
|
+
const origWrite = process.stdout.write.bind(process.stdout);
|
|
291
|
+
process.stdout.write = (s) => { captured += s; return true; };
|
|
292
|
+
try {
|
|
293
|
+
await withEnv({}, async () => {
|
|
294
|
+
try {
|
|
295
|
+
await runArgv(['policy', 'tool', 'create-terminal', '--help']);
|
|
296
|
+
} catch (err) {
|
|
297
|
+
// commander.exitOverride() throws on --help with code 'commander.helpDisplayed'.
|
|
298
|
+
if (err && err.code && err.code !== 'commander.helpDisplayed') throw err;
|
|
299
|
+
}
|
|
300
|
+
});
|
|
301
|
+
} finally {
|
|
302
|
+
process.stdout.write = origWrite;
|
|
303
|
+
h.restoreLog();
|
|
304
|
+
}
|
|
305
|
+
const bannerIdx = captured.indexOf('AI-ASSISTED (preferred):');
|
|
306
|
+
const usageIdx = captured.indexOf('Usage:');
|
|
307
|
+
assert.notEqual(bannerIdx, -1, `expected "AI-ASSISTED (preferred):" banner in help output: ${captured}`);
|
|
308
|
+
assert.notEqual(usageIdx, -1, `expected "Usage:" in help output: ${captured}`);
|
|
309
|
+
assert.ok(bannerIdx < usageIdx, `banner (idx ${bannerIdx}) should precede Usage (idx ${usageIdx})`);
|
|
310
|
+
});
|
|
311
|
+
|
|
312
|
+
// T10 — Layer 3: create-mcp --help banner appears before "Usage:".
|
|
313
|
+
test('T10: create-mcp --help renders AI-ASSISTED banner before "Usage:"', async () => {
|
|
314
|
+
const h = buildHarness();
|
|
315
|
+
let captured = '';
|
|
316
|
+
const origWrite = process.stdout.write.bind(process.stdout);
|
|
317
|
+
process.stdout.write = (s) => { captured += s; return true; };
|
|
318
|
+
try {
|
|
319
|
+
await withEnv({}, async () => {
|
|
320
|
+
try {
|
|
321
|
+
await runArgv(['policy', 'tool', 'create-mcp', '--help']);
|
|
322
|
+
} catch (err) {
|
|
323
|
+
if (err && err.code && err.code !== 'commander.helpDisplayed') throw err;
|
|
324
|
+
}
|
|
325
|
+
});
|
|
326
|
+
} finally {
|
|
327
|
+
process.stdout.write = origWrite;
|
|
328
|
+
h.restoreLog();
|
|
329
|
+
}
|
|
330
|
+
const bannerIdx = captured.indexOf('AI-ASSISTED (preferred):');
|
|
331
|
+
const usageIdx = captured.indexOf('Usage:');
|
|
332
|
+
assert.notEqual(bannerIdx, -1, `expected "AI-ASSISTED (preferred):" banner in help output: ${captured}`);
|
|
333
|
+
assert.notEqual(usageIdx, -1, `expected "Usage:" in help output: ${captured}`);
|
|
334
|
+
assert.ok(bannerIdx < usageIdx, `banner (idx ${bannerIdx}) should precede Usage (idx ${usageIdx})`);
|
|
335
|
+
});
|
|
336
|
+
|
|
337
|
+
// T11 — Regression smoke: existing --prompt happy path still reaches assist endpoint.
|
|
338
|
+
test('T11: create-terminal --prompt without CLAUDECODE still routes to the assist endpoint (Phase-1 regression)', async () => {
|
|
339
|
+
const h = buildHarness({
|
|
340
|
+
assistResponse: {
|
|
341
|
+
success: true,
|
|
342
|
+
form_updates: {
|
|
343
|
+
command_family: 'git',
|
|
344
|
+
selected_field: 'command',
|
|
345
|
+
field_value: 'npm install*',
|
|
346
|
+
action: 'AUDIT',
|
|
347
|
+
name: 'X',
|
|
348
|
+
},
|
|
349
|
+
explanation: 'ok',
|
|
350
|
+
},
|
|
351
|
+
});
|
|
352
|
+
await withEnv({}, async () => {
|
|
353
|
+
try {
|
|
354
|
+
await runArgv(['policy', 'tool', 'create-terminal', '--prompt', 'audit npm installs', '--yes']);
|
|
355
|
+
assert.ok(
|
|
356
|
+
h.calls.posts.some((c) => c.path === '/api/v1/command-policies/assist/'),
|
|
357
|
+
`expected assist POST, got: ${JSON.stringify(h.calls.posts.map((c) => c.path))}`
|
|
358
|
+
);
|
|
359
|
+
} finally {
|
|
360
|
+
h.restoreLog();
|
|
361
|
+
}
|
|
362
|
+
});
|
|
363
|
+
});
|