aztrx-cli 0.1.0 → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +130 -24
- package/dist/cli.js +26 -5
- package/dist/core/badge.js +50 -0
- package/dist/core/fuzzer.js +1 -1
- package/dist/core/heal/boot.js +136 -0
- package/dist/core/heal/childEnv.js +60 -0
- package/dist/core/heal/index.js +47 -6
- package/dist/core/heal/redact.js +31 -1
- package/dist/core/heal/sandbox.js +53 -1
- package/dist/core/heal/verify.js +30 -1
- package/dist/core/httpFuzzer.js +258 -0
- package/dist/core/init.js +2 -2
- package/dist/core/interceptor.js +10 -1
- package/dist/core/orchestrator.js +41 -3
- package/dist/core/pr.js +44 -20
- package/dist/core/replay.js +35 -3
- package/dist/core/report.js +19 -6
- package/dist/core/resolver.js +158 -5
- package/dist/core/specCompiler.js +17 -2
- package/dist/core/studio.js +3 -4
- package/dist/core/ui.js +2 -0
- package/dist/ui/app.js +3 -2
- package/media/demo.gif +0 -0
- package/media/logo.svg +9 -0
- package/package.json +3 -3
package/README.md
CHANGED
|
@@ -1,21 +1,21 @@
|
|
|
1
|
-
#
|
|
1
|
+
# <img src="media/logo.svg" width="28" height="32" alt="Aztrx logo" align="absmiddle" /> Aztrx AI
|
|
2
2
|
|
|
3
3
|
> **Autonomous runtime stress-tester, deterministic bug minimizer, and self-healing engine for web applications.**
|
|
4
4
|
|
|
5
5
|
[](https://nodejs.org)
|
|
6
6
|
[](LICENSE)
|
|
7
7
|
|
|
8
|
-
Aztrx drives your web app like a hostile user — clicking, entering boundary data, and racing asynchronous UI states. When a runtime crash occurs (even one swallowed by a React Error Boundary), Aztrx intercepts it via the Chrome DevTools Protocol, maps it back to the exact source line, shrinks the interaction trace to the bare minimum with **ddmin**, and emits an executable, standalone **Playwright test** that proves the bug — not a log line.
|
|
8
|
+
Aztrx AI drives your web app like a hostile user — clicking, entering boundary data, and racing asynchronous UI states. When a runtime crash occurs (even one swallowed by a React Error Boundary), Aztrx AI intercepts it via the Chrome DevTools Protocol, maps it back to the exact source line, shrinks the interaction trace to the bare minimum with **ddmin**, and emits an executable, standalone **Playwright test** that proves the bug — not a log line.
|
|
9
9
|
|
|
10
10
|

|
|
11
11
|
|
|
12
12
|
---
|
|
13
13
|
|
|
14
|
-
## Why Aztrx
|
|
14
|
+
## Why Aztrx AI
|
|
15
15
|
|
|
16
|
-
- **Sees swallowed errors.** Error Boundaries and `window.onerror` miss the errors your app *catches*. Aztrx reads the real throw-site stack off the `Error` object, so a crash you've never seen in your logs becomes a finding you can't ignore.
|
|
16
|
+
- **Sees swallowed errors.** Error Boundaries and `window.onerror` miss the errors your app *catches*. Aztrx AI reads the real throw-site stack off the `Error` object, so a crash you've never seen in your logs becomes a finding you can't ignore.
|
|
17
17
|
- **Proves, not reports.** Every crash/error finding ships with an executable `.spec.ts` and a flake-rate verdict — `[deterministic 5/5]`, `[flaky 3/5]`, or `[unreliable]`.
|
|
18
|
-
- **Heals, not just finds.** `--heal` generates a patch through an LLM, gates it (redaction + AST safety), compiles it, and replays it against the repro inside an isolated git worktree — the patch is verified before a human ever sees it.
|
|
18
|
+
- **Heals, not just finds.** `--heal` generates a patch through an LLM, gates it (redaction + AST safety), compiles it, runs your test suite, and replays it against the repro inside an isolated git worktree — the patch is verified before a human ever sees it.
|
|
19
19
|
- **Safe by default.** A deny-by-default network guard blocks off-origin calls, and a destructive-action deny-list refuses to click "delete", "pay", or "logout".
|
|
20
20
|
|
|
21
21
|
---
|
|
@@ -28,25 +28,27 @@ Run against any running local dev server — no install, no repo clone:
|
|
|
28
28
|
npx aztrx-cli run http://localhost:3000 # deterministic walk
|
|
29
29
|
npx aztrx-cli run http://localhost:3000 --fuzz # seeded chaos (replayable)
|
|
30
30
|
npx aztrx-cli run http://localhost:3000 --fuzz --repro # + minimize → compile → validate
|
|
31
|
+
npx aztrx-cli run http://localhost:3000 --http-fuzz --repro # server-side 5xx hunt → proof
|
|
31
32
|
```
|
|
32
33
|
|
|
33
|
-
Install it globally
|
|
34
|
+
Install it globally:
|
|
34
35
|
|
|
35
36
|
```bash
|
|
36
37
|
npm i -g aztrx-cli
|
|
37
38
|
aztrx-cli run http://localhost:3000 --repro
|
|
38
39
|
```
|
|
39
40
|
|
|
40
|
-
|
|
41
|
-
> ```bash
|
|
42
|
-
> git clone https://github.com/DanisChaparov/aztrx
|
|
43
|
-
> cd aztrx
|
|
44
|
-
> npm install
|
|
45
|
-
> npm run build
|
|
46
|
-
> npm link # puts `aztrx-cli` on your PATH
|
|
47
|
-
> ```
|
|
41
|
+
Or install from source (contributors):
|
|
48
42
|
|
|
49
|
-
|
|
43
|
+
```bash
|
|
44
|
+
git clone https://github.com/DanisChaparov/aztrx
|
|
45
|
+
cd aztrx
|
|
46
|
+
npm install
|
|
47
|
+
npm run build
|
|
48
|
+
npm link # puts `aztrx-cli` on your PATH
|
|
49
|
+
```
|
|
50
|
+
|
|
51
|
+
Aztrx AI drives Chromium through Playwright — the first run downloads the browser
|
|
50
52
|
automatically (`npx playwright install chromium` to force it).
|
|
51
53
|
|
|
52
54
|
---
|
|
@@ -91,6 +93,14 @@ npx aztrx-cli run http://localhost:3000 --upload --api-key <YOUR_API_KEY> --clou
|
|
|
91
93
|
|
|
92
94
|
Ship a runtime gate on every PR — see [Continuous Integration](#continuous-integration-github-action) below.
|
|
93
95
|
|
|
96
|
+
### 6. HTTP mutation fuzzing (`--http-fuzz`)
|
|
97
|
+
|
|
98
|
+
The DOM fuzzer (`--fuzz`) breaks the *client*. `--http-fuzz` attacks the *server*: it harvests the endpoints your app actually calls, then throws hostile requests at them — query overflow, JSON type-confusion, header injection, method confusion — and turns every `5xx` into an executable repro.
|
|
99
|
+
|
|
100
|
+
Every `5xx` finding also captures the server's *own* error response — its message and body — and, when the body leaks a stack trace, the server-side source line. So a finding reads "`HTTP 500 /api/cart` … `server: Cannot read properties of undefined` at `app/api/cart/route.ts:14`", not just "something 500'd".
|
|
101
|
+
|
|
102
|
+
When the 500 body leaks that server stack, `--heal` can also *fix* it: it boots the patched app inside the worktree (via `--start-command`, or auto-detected `scripts.dev`/`scripts.start`), waits for an HTTP readiness signal, and replays the repro against the booted server — so a server patch is verified against a *running* app, not a static file. Server findings whose body does not leak a stack are still reported and repro'd but not healed (there is no file to patch).
|
|
103
|
+
|
|
94
104
|
---
|
|
95
105
|
|
|
96
106
|
## CLI reference
|
|
@@ -98,6 +108,7 @@ Ship a runtime gate on every PR — see [Continuous Integration](#continuous-int
|
|
|
98
108
|
| Flag | Description | Default |
|
|
99
109
|
| --- | --- | --- |
|
|
100
110
|
| `--fuzz` | Seeded chaos fuzzing instead of the deterministic walk | — |
|
|
111
|
+
| `--http-fuzz` | Server-side mutation fuzzing — hostile requests against the target origin | — |
|
|
101
112
|
| `--repro` | Minimize (ddmin) → emit Playwright spec → validate flake rate | — |
|
|
102
113
|
| `--heal` | Generate + verify an LLM patch (implies `--repro`) | — |
|
|
103
114
|
| `--upload` | Stream run findings to the cloud ingest backend | — |
|
|
@@ -108,7 +119,12 @@ Ship a runtime gate on every PR — see [Continuous Integration](#continuous-int
|
|
|
108
119
|
| `--repro-runs <n>` | Flake-rate replay iterations | `3` |
|
|
109
120
|
| `--heal-model <model>` | Fallback LLM tier | `claude-sonnet-5` |
|
|
110
121
|
| `--heal-fast-model <model>` | Fast/cheap first tier | `claude-haiku-4-5` |
|
|
122
|
+
| `--test-command <cmd>` | Test command run against a healed patch | `npm test` (auto-detected) |
|
|
123
|
+
| `--test-timeout <ms>` | Timeout for the heal test gate | `300000` |
|
|
124
|
+
| `--no-test` | Skip the test gate during healing | — |
|
|
125
|
+
| `--start-command <cmd>` | Command to boot the app for server healing | `scripts.dev` → `scripts.start` (auto-detected) |
|
|
111
126
|
| `--pr-comment [path]` | Write a GitHub PR markdown comment | `.aztrx/pr-comment.md` |
|
|
127
|
+
| `--badge [path]` | Write a self-contained SVG status badge | `.aztrx/badge.svg` |
|
|
112
128
|
| `--telemetry` | Collect anonymized tuples locally (opt-in) | — |
|
|
113
129
|
| `--share-data` | Also upload the sanitized tuples (opt-in) | — |
|
|
114
130
|
| `--repo <path>` | Root path for sourcemap → source resolution | cwd |
|
|
@@ -132,16 +148,17 @@ Every run writes self-contained artifacts inside `.aztrx/` (gitignored):
|
|
|
132
148
|
│ └── 458f6bf71977.spec.ts # minimal, executable Playwright repro
|
|
133
149
|
├── heal/
|
|
134
150
|
│ └── fix.patch # gated, compiler-checked fix
|
|
135
|
-
├── events.jsonl # run log, streamed by `aztrx studio`
|
|
151
|
+
├── events.jsonl # run log, streamed by `aztrx-cli studio`
|
|
136
152
|
├── telemetry/dataset.jsonl # opt-in anonymized tuple dataset
|
|
137
|
-
|
|
153
|
+
├── pr-comment.md # GitHub PR markdown (with --pr-comment)
|
|
154
|
+
└── badge.svg # status badge (with --badge)
|
|
138
155
|
```
|
|
139
156
|
|
|
140
157
|
---
|
|
141
158
|
|
|
142
159
|
## Architecture
|
|
143
160
|
|
|
144
|
-
Aztrx is a decoupled, event-driven pipeline — modules talk only through an
|
|
161
|
+
Aztrx AI is a decoupled, event-driven pipeline — modules talk only through an
|
|
145
162
|
`EventBus`; the orchestrator wires them together.
|
|
146
163
|
|
|
147
164
|
```
|
|
@@ -184,7 +201,7 @@ jobs:
|
|
|
184
201
|
permissions: { contents: read, pull-requests: write }
|
|
185
202
|
steps:
|
|
186
203
|
- uses: actions/checkout@v4
|
|
187
|
-
- uses: DanisChaparov/aztrx@
|
|
204
|
+
- uses: DanisChaparov/aztrx@e5b89247cb0fba7f5c0febe8fa27f5e1c74cd898
|
|
188
205
|
with:
|
|
189
206
|
url: http://localhost:3000
|
|
190
207
|
start-command: npm run dev # optional — boot the app in the background
|
|
@@ -198,7 +215,7 @@ Or as a reusable workflow:
|
|
|
198
215
|
on: pull_request
|
|
199
216
|
jobs:
|
|
200
217
|
aztrx:
|
|
201
|
-
uses: DanisChaparov/aztrx/.github/workflows/aztrx-pr.yml@
|
|
218
|
+
uses: DanisChaparov/aztrx/.github/workflows/aztrx-pr.yml@e5b89247cb0fba7f5c0febe8fa27f5e1c74cd898
|
|
202
219
|
with:
|
|
203
220
|
url: http://localhost:3000
|
|
204
221
|
start-command: npm run dev
|
|
@@ -208,6 +225,63 @@ jobs:
|
|
|
208
225
|
|
|
209
226
|
---
|
|
210
227
|
|
|
228
|
+
## Status badge
|
|
229
|
+
|
|
230
|
+
Hang a live badge in your README that reflects your *actual* crash/error state —
|
|
231
|
+
not a static "protected" sticker.
|
|
232
|
+
|
|
233
|
+
```bash
|
|
234
|
+
npx aztrx-cli run http://localhost:3000 --badge badge.svg
|
|
235
|
+
```
|
|
236
|
+
|
|
237
|
+
```markdown
|
|
238
|
+
[](https://github.com/DanisChaparov/aztrx)
|
|
239
|
+
```
|
|
240
|
+
|
|
241
|
+
The badge is a self-contained SVG generated from the run's findings — green
|
|
242
|
+
`crash-free` or red `N findings`. It's honest because it's *earned*: regenerate it
|
|
243
|
+
in CI on every push to `main` and commit it back.
|
|
244
|
+
|
|
245
|
+
```yaml
|
|
246
|
+
# .github/workflows/badge.yml — keep the badge honest on every push to main
|
|
247
|
+
on:
|
|
248
|
+
push:
|
|
249
|
+
branches: [main]
|
|
250
|
+
jobs:
|
|
251
|
+
badge:
|
|
252
|
+
runs-on: ubuntu-latest
|
|
253
|
+
permissions:
|
|
254
|
+
contents: write
|
|
255
|
+
steps:
|
|
256
|
+
- uses: actions/checkout@v4
|
|
257
|
+
- uses: actions/setup-node@v4
|
|
258
|
+
with:
|
|
259
|
+
node-version: 20
|
|
260
|
+
- run: npm ci
|
|
261
|
+
- name: Boot dev server
|
|
262
|
+
run: |
|
|
263
|
+
nohup npm run dev > /tmp/dev.log 2>&1 &
|
|
264
|
+
for i in $(seq 1 60); do
|
|
265
|
+
curl -sS -o /dev/null http://localhost:3000 && break
|
|
266
|
+
sleep 2
|
|
267
|
+
done
|
|
268
|
+
- name: Generate badge
|
|
269
|
+
run: npx --yes aztrx-cli@0.1.1 run http://localhost:3000 --badge badge.svg
|
|
270
|
+
- name: Commit badge
|
|
271
|
+
run: |
|
|
272
|
+
git config user.name "github-actions[bot]"
|
|
273
|
+
git config user.email "github-actions[bot]@users.noreply.github.com"
|
|
274
|
+
git add badge.svg
|
|
275
|
+
git commit -m "chore: update aztrx badge" || echo "no change"
|
|
276
|
+
git push
|
|
277
|
+
```
|
|
278
|
+
|
|
279
|
+
No `--fail-on` here on purpose: the badge reflects the findings whatever they
|
|
280
|
+
are, and the run still exits 0 so the commit step always runs. Swap `push` for a
|
|
281
|
+
`schedule` cron if you'd rather regenerate daily than on every push.
|
|
282
|
+
|
|
283
|
+
---
|
|
284
|
+
|
|
211
285
|
## Smart Cloud Router
|
|
212
286
|
|
|
213
287
|
`--heal` is backed by a two-tier router. The fast/cheap model
|
|
@@ -227,11 +301,42 @@ strips secrets, anonymizes URLs to `<host>`, and scrubs repo paths to `<repo>`.
|
|
|
227
301
|
Uploads are fire-and-forget, bounded by a 2s timeout, and never affect the exit
|
|
228
302
|
code.
|
|
229
303
|
|
|
230
|
-
## Security
|
|
304
|
+
## Security & data flow
|
|
231
305
|
|
|
232
|
-
-
|
|
233
|
-
|
|
234
|
-
|
|
306
|
+
**Local-first by default.** A run never phones home unless you pass an opt-in
|
|
307
|
+
flag. By default nothing leaves your machine — no telemetry, no cloud sync, no
|
|
308
|
+
LLM call.
|
|
309
|
+
|
|
310
|
+
| What | Leaves your machine | When |
|
|
311
|
+
| --- | --- | --- |
|
|
312
|
+
| `run` (default) | nothing | — |
|
|
313
|
+
| `--heal` | redacted file + redacted error/stack, to the LLM API | only with `--heal` + `ANTHROPIC_API_KEY` |
|
|
314
|
+
| `--share-data` | a sanitized crash→repro→patch tuple | explicit opt-in |
|
|
315
|
+
| `--upload` | sanitized findings + counts | explicit opt-in |
|
|
316
|
+
|
|
317
|
+
### Invariants
|
|
318
|
+
|
|
319
|
+
- **Sourcemap containment.** A hostile sourcemap or stack URL can't read outside
|
|
320
|
+
your repo: every path is resolved against the repo root and rejected if it
|
|
321
|
+
escapes it — including through symlinks. Secret-bearing files (`.env`, `.npmrc`,
|
|
322
|
+
private keys) are never read into a report or PR comment.
|
|
323
|
+
- **Redaction before the model.** `--heal` redacts common secret patterns (keys,
|
|
324
|
+
tokens, credentials) from the file, error, and stack before they're sent; only
|
|
325
|
+
the repo-relative path and line/column are visible. Redaction is heuristic — it
|
|
326
|
+
is not a substitute for not committing secrets.
|
|
327
|
+
- **Isolated sandbox, no commits.** Patches land in a detached `git worktree` in
|
|
328
|
+
the OS temp dir — never your working tree. Aztrx AI never commits. A patch must
|
|
329
|
+
parse, add no new imports / `eval` / `child_process`, typecheck, *and* pass your
|
|
330
|
+
own test suite before it's offered as a `.patch` for you to review.
|
|
331
|
+
- **Deny-by-default network.** Only the target origin (plus explicit
|
|
332
|
+
`--allow-host`) is reachable; off-origin calls are blocked.
|
|
333
|
+
- **Destructive-action deny-list.** Never clicks delete / pay / logout.
|
|
334
|
+
- **Studio is localhost-only.** The dashboard binds `127.0.0.1` with no wildcard
|
|
335
|
+
CORS.
|
|
336
|
+
- **`.aztrx/` is gitignored** on `init` — repro specs, reports, and patches stay
|
|
337
|
+
out of history.
|
|
338
|
+
- **Pinned supply chain.** The GitHub Action pins `aztrx-cli@0.1.1` (never
|
|
339
|
+
`@latest`).
|
|
235
340
|
|
|
236
341
|
---
|
|
237
342
|
|
|
@@ -271,6 +376,7 @@ Full per-case table and scoring notes live in
|
|
|
271
376
|
- [x] B2B ($29/mo) — Smart Cloud Router (haiku fast-tier → verify → Sonnet fallback)
|
|
272
377
|
- [x] B2B ($29/mo) — Cloud dashboard (api.aztrx.app)
|
|
273
378
|
- [x] Data flywheel — opt-in anonymized patch-tuple collection (F11)
|
|
379
|
+
- [x] Server-side healing — heal server `5xx` findings (verify a patch by booting the patched server; requires a leaked server stack + a resolvable start command)
|
|
274
380
|
|
|
275
381
|
## Contributing
|
|
276
382
|
|
package/dist/cli.js
CHANGED
|
@@ -8,6 +8,7 @@ import { renderTui } from "./ui/app.js";
|
|
|
8
8
|
import { initProject } from "./core/init.js";
|
|
9
9
|
import { startStudio } from "./core/studio.js";
|
|
10
10
|
import { writePrComment } from "./core/pr.js";
|
|
11
|
+
import { writeBadge } from "./core/badge.js";
|
|
11
12
|
import { flushTelemetry } from "./core/telemetry/index.js";
|
|
12
13
|
import { flushCloud } from "./core/cloud/index.js";
|
|
13
14
|
function collect(value, prev) {
|
|
@@ -15,7 +16,7 @@ function collect(value, prev) {
|
|
|
15
16
|
return prev;
|
|
16
17
|
}
|
|
17
18
|
program
|
|
18
|
-
.name("aztrx")
|
|
19
|
+
.name("aztrx-cli")
|
|
19
20
|
.description("Runtime stress-testing for web apps — detect bugs, prove them with a repro")
|
|
20
21
|
.option("--repo <path>", "project root to inspect/watch (default: cwd)", process.cwd());
|
|
21
22
|
program
|
|
@@ -33,7 +34,7 @@ program
|
|
|
33
34
|
if (res.gitignoreUpdated)
|
|
34
35
|
console.log(pc.green("✓") + " .aztrx/ added to .gitignore");
|
|
35
36
|
console.log("");
|
|
36
|
-
console.log(pc.dim(`Next: npx aztrx run ${res.url} --repo .`));
|
|
37
|
+
console.log(pc.dim(`Next: npx aztrx-cli run ${res.url} --repo .`));
|
|
37
38
|
});
|
|
38
39
|
program
|
|
39
40
|
.command("studio")
|
|
@@ -52,16 +53,23 @@ program
|
|
|
52
53
|
.option("--crash-test", "throw a deliberate error to verify capture")
|
|
53
54
|
.option("--fail-on", "exit 1 if any crash/error finding is present")
|
|
54
55
|
.option("--fuzz", "chaos fuzzing instead of the deterministic walk (F5)")
|
|
56
|
+
.option("--http-fuzz", "HTTP-layer mutation fuzzing — hostile requests against the target origin (F5-http)")
|
|
57
|
+
.option("--http-fuzz-mutations", "with --http-fuzz: also send POST/PUT body mutations (default: GET-only)")
|
|
55
58
|
.option("--seed <n>", "RNG seed for fuzz", "42")
|
|
56
59
|
.option("--repro", "minimize + compile + validate each finding (F7-F9)")
|
|
57
60
|
.option("--repro-runs <n>", "replay iterations for the flake-rate gate", "3")
|
|
58
61
|
.option("--heal", "closed-loop healing for crash/error findings (implies --repro)")
|
|
59
62
|
.option("--heal-model <model>", "LLM model for healing — the fallback tier (default claude-sonnet-5)")
|
|
60
63
|
.option("--heal-fast-model <model>", "fast/cheap first tier for the smart router (default claude-haiku-4-5)")
|
|
64
|
+
.option("--test-command <cmd>", "test command run against a healed patch (default: npm test, auto-detected)")
|
|
65
|
+
.option("--test-timeout <ms>", "timeout for the heal test gate, ms", "300000")
|
|
66
|
+
.option("--no-test", "skip the test gate during healing")
|
|
67
|
+
.option("--start-command <cmd>", "command to boot the app for server healing (default: auto-detect scripts.dev/scripts.start)")
|
|
61
68
|
.option("--pr-comment [path]", "write a GitHub PR markdown comment (default .aztrx/pr-comment.md)")
|
|
69
|
+
.option("--badge [path]", "write a self-contained SVG badge (default .aztrx/badge.svg)")
|
|
62
70
|
.option("--telemetry", "opt-in: collect anonymized crash→repro→patch tuples locally (.aztrx/telemetry)")
|
|
63
71
|
.option("--share-data", "opt-in: also upload the sanitized tuples to the telemetry endpoint")
|
|
64
|
-
.option("--upload", "opt-in: stream run results to the Aztrx cloud dashboard (needs --api-key)")
|
|
72
|
+
.option("--upload", "opt-in: stream run results to the Aztrx AI cloud dashboard (needs --api-key)")
|
|
65
73
|
.option("--api-key <key>", "API key for --upload / --share-data (defaults to $AZTRX_API_KEY)")
|
|
66
74
|
.option("--cloud-url <url>", "override the cloud ingest base URL (default https://api.aztrx.app)")
|
|
67
75
|
.option("--allow-host <host>", "add a host to the network allow-list (repeatable)", collect, [])
|
|
@@ -78,6 +86,8 @@ program
|
|
|
78
86
|
dryRun: opts.dryRun,
|
|
79
87
|
crashTest: opts.crashTest,
|
|
80
88
|
fuzz: opts.fuzz,
|
|
89
|
+
httpFuzz: opts.httpFuzz,
|
|
90
|
+
httpFuzzMutations: opts.httpFuzzMutations,
|
|
81
91
|
repro: opts.repro || opts.heal,
|
|
82
92
|
seed: parseInt(opts.seed, 10),
|
|
83
93
|
allowHosts: opts.allowHost ?? [],
|
|
@@ -85,6 +95,10 @@ program
|
|
|
85
95
|
heal: opts.heal,
|
|
86
96
|
healModel: opts.healModel,
|
|
87
97
|
healFastModel: opts.healFastModel,
|
|
98
|
+
testCommand: opts.testCommand,
|
|
99
|
+
testTimeoutMs: opts.testTimeoutMs ? parseInt(opts.testTimeoutMs, 10) : undefined,
|
|
100
|
+
skipTest: opts.test === false,
|
|
101
|
+
startCommand: opts.startCommand,
|
|
88
102
|
telemetry: opts.telemetry,
|
|
89
103
|
shareData: opts.shareData,
|
|
90
104
|
upload: opts.upload,
|
|
@@ -103,13 +117,13 @@ program
|
|
|
103
117
|
done: runPromise,
|
|
104
118
|
targetUrl: url,
|
|
105
119
|
repoRoot,
|
|
106
|
-
mode: opts.fuzz ? `fuzz (seed ${opts.seed})` : opts.heal ? "repro → heal" : opts.repro ? "repro" : "deterministic walk",
|
|
120
|
+
mode: opts.fuzz ? `fuzz (seed ${opts.seed})` : opts.httpFuzz ? "http fuzz" : opts.heal ? "repro → heal" : opts.repro ? "repro" : "deterministic walk",
|
|
107
121
|
});
|
|
108
122
|
try {
|
|
109
123
|
findings = await runPromise;
|
|
110
124
|
}
|
|
111
125
|
catch (e) {
|
|
112
|
-
console.error(pc.red("Aztrx run failed:"), e.message);
|
|
126
|
+
console.error(pc.red("Aztrx AI run failed:"), e.message);
|
|
113
127
|
process.exit(1);
|
|
114
128
|
}
|
|
115
129
|
}
|
|
@@ -123,6 +137,13 @@ program
|
|
|
123
137
|
writePrComment(repoRoot, url, findings, prPath);
|
|
124
138
|
console.log(pc.dim(`PR comment: ${path.relative(repoRoot, prPath)}`));
|
|
125
139
|
}
|
|
140
|
+
if (opts.badge) {
|
|
141
|
+
const badgePath = typeof opts.badge === "string"
|
|
142
|
+
? opts.badge
|
|
143
|
+
: path.join(repoRoot, ".aztrx", "badge.svg");
|
|
144
|
+
writeBadge(repoRoot, findings, badgePath);
|
|
145
|
+
console.log(pc.dim(`Badge: ${path.relative(repoRoot, badgePath)}`));
|
|
146
|
+
}
|
|
126
147
|
// Drain any in-flight telemetry uploads (each bounded) before exit, so a
|
|
127
148
|
// pending `--share-data` dispatch isn't killed mid-flight. Never affects
|
|
128
149
|
// the exit code.
|
|
@@ -0,0 +1,50 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Self-contained SVG status badge generated from a real run's findings. Unlike a
|
|
3
|
+
* static "Protected by …" sticker, this badge is *earned*: it reflects the
|
|
4
|
+
* crash/error count of the run that produced it, so a stale badge is a
|
|
5
|
+
* regeneration problem — not a lie baked into the image.
|
|
6
|
+
*/
|
|
7
|
+
import * as fs from "fs";
|
|
8
|
+
import * as path from "path";
|
|
9
|
+
const CHAR_W = 7.1; // Verdana 11px approximate advance width
|
|
10
|
+
const PAD_X = 11; // horizontal padding per text block
|
|
11
|
+
const LABEL = "#27272a"; // zinc-800 — brand monochrome
|
|
12
|
+
const GREEN = "#16a34a";
|
|
13
|
+
const RED = "#dc2626";
|
|
14
|
+
function criticalCount(findings) {
|
|
15
|
+
return findings.filter((f) => f.severity === "crash" || f.severity === "error").length;
|
|
16
|
+
}
|
|
17
|
+
function blockWidth(text) {
|
|
18
|
+
return Math.round(text.length * CHAR_W + 2 * PAD_X);
|
|
19
|
+
}
|
|
20
|
+
export function renderBadge(findings, label = "aztrx") {
|
|
21
|
+
const n = criticalCount(findings);
|
|
22
|
+
const message = n === 0 ? "crash-free" : `${n} finding${n === 1 ? "" : "s"}`;
|
|
23
|
+
const color = n === 0 ? GREEN : RED;
|
|
24
|
+
const labelW = blockWidth(label);
|
|
25
|
+
const msgW = blockWidth(message);
|
|
26
|
+
const totalW = labelW + msgW;
|
|
27
|
+
const labelX = labelW / 2;
|
|
28
|
+
const msgX = labelW + msgW / 2;
|
|
29
|
+
return [
|
|
30
|
+
`<svg xmlns="http://www.w3.org/2000/svg" width="${totalW}" height="20" role="img" aria-label="${label}: ${message}">`,
|
|
31
|
+
` <linearGradient id="g" x2="0" y2="100%"><stop offset="0" stop-color="#ffffff" stop-opacity="0.14"/><stop offset="1" stop-opacity="0"/></linearGradient>`,
|
|
32
|
+
` <clipPath id="r"><rect width="${totalW}" height="20" rx="3" fill="#fff"/></clipPath>`,
|
|
33
|
+
` <g clip-path="url(#r)">`,
|
|
34
|
+
` <rect width="${labelW}" height="20" fill="${LABEL}"/>`,
|
|
35
|
+
` <rect x="${labelW}" width="${msgW}" height="20" fill="${color}"/>`,
|
|
36
|
+
` <rect width="${totalW}" height="20" fill="url(#g)"/>`,
|
|
37
|
+
` </g>`,
|
|
38
|
+
` <g fill="#fff" text-anchor="middle" font-family="Verdana,Geneva,DejaVu Sans,sans-serif" font-size="11" font-weight="600">`,
|
|
39
|
+
` <text x="${labelX}" y="14">${label}</text>`,
|
|
40
|
+
` <text x="${msgX}" y="14">${message}</text>`,
|
|
41
|
+
` </g>`,
|
|
42
|
+
`</svg>`,
|
|
43
|
+
].join("\n") + "\n";
|
|
44
|
+
}
|
|
45
|
+
export function writeBadge(repoRoot, findings, filePath) {
|
|
46
|
+
const file = filePath ?? path.join(repoRoot, ".aztrx", "badge.svg");
|
|
47
|
+
fs.mkdirSync(path.dirname(file), { recursive: true });
|
|
48
|
+
fs.writeFileSync(file, renderBadge(findings), "utf-8");
|
|
49
|
+
return file;
|
|
50
|
+
}
|
package/dist/core/fuzzer.js
CHANGED
|
@@ -0,0 +1,136 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* F10 gate #4 — boot the *patched* app for server-side verification. A server
|
|
3
|
+
* finding (e.g. `HTTP 500 /api/cart`) can't be verified by statically serving
|
|
4
|
+
* the worktree — the 500 only reappears when the route actually runs. So before
|
|
5
|
+
* the replay, this boots the patched server inside the worktree on a free port,
|
|
6
|
+
* waits for an HTTP readiness signal, and returns a `close` hook that tree-kills
|
|
7
|
+
* the process (and its children) so nothing is left holding the port.
|
|
8
|
+
*/
|
|
9
|
+
import { spawn } from "child_process";
|
|
10
|
+
import * as fs from "fs";
|
|
11
|
+
import * as net from "net";
|
|
12
|
+
import * as path from "path";
|
|
13
|
+
import { buildChildEnv } from "./childEnv.js";
|
|
14
|
+
/** Auto-detect how the app starts, mirroring `runTests`'s `npm test` convention:
|
|
15
|
+
* prefer `scripts.dev` (no build step), then `scripts.start`. Null when neither
|
|
16
|
+
* exists — the caller then requires an explicit `--start-command`. */
|
|
17
|
+
export function detectStartCommand(repoRoot) {
|
|
18
|
+
try {
|
|
19
|
+
const pkg = JSON.parse(fs.readFileSync(path.join(repoRoot, "package.json"), "utf-8"));
|
|
20
|
+
const s = pkg.scripts;
|
|
21
|
+
if (s && typeof s.dev === "string")
|
|
22
|
+
return "npm run dev";
|
|
23
|
+
if (s && typeof s.start === "string")
|
|
24
|
+
return "npm run start";
|
|
25
|
+
}
|
|
26
|
+
catch {
|
|
27
|
+
// no package.json, or unparseable — fall through to null
|
|
28
|
+
}
|
|
29
|
+
return null;
|
|
30
|
+
}
|
|
31
|
+
/** Allocate a free loopback port. Best-effort: there is a small window between
|
|
32
|
+
* closing the probe and the app binding, so a collision is surfaced as a boot
|
|
33
|
+
* timeout rather than silently mis-directed. */
|
|
34
|
+
function freePort() {
|
|
35
|
+
return new Promise((resolve, reject) => {
|
|
36
|
+
const srv = net.createServer();
|
|
37
|
+
srv.once("error", reject);
|
|
38
|
+
srv.listen(0, "127.0.0.1", () => {
|
|
39
|
+
const addr = srv.address();
|
|
40
|
+
srv.close(() => resolve(addr.port));
|
|
41
|
+
});
|
|
42
|
+
});
|
|
43
|
+
}
|
|
44
|
+
export async function bootServer(opts) {
|
|
45
|
+
const { worktreeDir, repoRoot, startCommand } = opts;
|
|
46
|
+
const timeoutMs = opts.timeoutMs ?? 60_000;
|
|
47
|
+
// A fresh worktree has no node_modules — symlink the root's so the booted
|
|
48
|
+
// server resolves its dependencies (the same junction trick sandbox.ts uses).
|
|
49
|
+
const rootNodeModules = path.join(repoRoot, "node_modules");
|
|
50
|
+
const wtNodeModules = path.join(worktreeDir, "node_modules");
|
|
51
|
+
if (!fs.existsSync(wtNodeModules) && fs.existsSync(rootNodeModules)) {
|
|
52
|
+
try {
|
|
53
|
+
fs.symlinkSync(rootNodeModules, wtNodeModules, process.platform === "win32" ? "junction" : "dir");
|
|
54
|
+
}
|
|
55
|
+
catch {
|
|
56
|
+
/* resolution errors surface in the readiness timeout below */
|
|
57
|
+
}
|
|
58
|
+
}
|
|
59
|
+
const port = await freePort();
|
|
60
|
+
// Support scripts that hardcode a port via `-p {port}`; `PORT` is also set in
|
|
61
|
+
// the environment for the (more common) scripts that read `process.env.PORT`.
|
|
62
|
+
const command = startCommand.replace(/\{port\}/g, String(port));
|
|
63
|
+
// Ring buffer of the last ~40 log lines, so a boot timeout can tell the user
|
|
64
|
+
// *why* the server didn't come up rather than just "timeout".
|
|
65
|
+
const lines = [];
|
|
66
|
+
const push = (chunk) => {
|
|
67
|
+
for (const line of chunk.toString().split(/\r?\n/)) {
|
|
68
|
+
if (line) {
|
|
69
|
+
lines.push(line);
|
|
70
|
+
if (lines.length > 40)
|
|
71
|
+
lines.shift();
|
|
72
|
+
}
|
|
73
|
+
}
|
|
74
|
+
};
|
|
75
|
+
const child = spawn(command, {
|
|
76
|
+
shell: true,
|
|
77
|
+
cwd: worktreeDir,
|
|
78
|
+
// Minimal allow-list — the booted app is patched PR code; it must not see
|
|
79
|
+
// the caller's ANTHROPIC_API_KEY or other CI secrets.
|
|
80
|
+
env: buildChildEnv({ PORT: String(port), CI: "true" }),
|
|
81
|
+
// On POSIX, detach so the server + its children form their own process
|
|
82
|
+
// group — close() can then signal the whole group. Windows can't do group
|
|
83
|
+
// signaling; it relies on `taskkill /T` below instead.
|
|
84
|
+
detached: process.platform !== "win32",
|
|
85
|
+
stdio: ["ignore", "pipe", "pipe"],
|
|
86
|
+
});
|
|
87
|
+
child.stdout?.on("data", push);
|
|
88
|
+
child.stderr?.on("data", push);
|
|
89
|
+
let closed = false;
|
|
90
|
+
const close = async () => {
|
|
91
|
+
if (closed || !child.pid)
|
|
92
|
+
return;
|
|
93
|
+
closed = true;
|
|
94
|
+
if (process.platform === "win32") {
|
|
95
|
+
// `shell: true` spawns cmd.exe which spawns the real server as a child —
|
|
96
|
+
// a plain child.kill() would orphan that child and leave the port taken.
|
|
97
|
+
await new Promise((resolve) => {
|
|
98
|
+
const killer = spawn("taskkill", ["/PID", String(child.pid), "/T", "/F"], {
|
|
99
|
+
stdio: "ignore",
|
|
100
|
+
});
|
|
101
|
+
killer.on("exit", () => resolve());
|
|
102
|
+
killer.on("error", () => resolve());
|
|
103
|
+
});
|
|
104
|
+
}
|
|
105
|
+
else {
|
|
106
|
+
try {
|
|
107
|
+
process.kill(-child.pid, "SIGTERM");
|
|
108
|
+
}
|
|
109
|
+
catch {
|
|
110
|
+
// already exited
|
|
111
|
+
}
|
|
112
|
+
}
|
|
113
|
+
};
|
|
114
|
+
// Readiness: poll until the server answers with *any* HTTP response (2xx/4xx/
|
|
115
|
+
// 5xx all mean "the listener is up"). A still-compiling dev server (Next) may
|
|
116
|
+
// take a while on its first request — the loop keeps retrying until it's hot.
|
|
117
|
+
const url = `http://127.0.0.1:${port}`;
|
|
118
|
+
const deadline = Date.now() + timeoutMs;
|
|
119
|
+
let ready = false;
|
|
120
|
+
while (Date.now() < deadline) {
|
|
121
|
+
try {
|
|
122
|
+
const res = await fetch(url, { signal: AbortSignal.timeout(1000) });
|
|
123
|
+
await res.arrayBuffer().catch(() => { });
|
|
124
|
+
ready = true;
|
|
125
|
+
break;
|
|
126
|
+
}
|
|
127
|
+
catch {
|
|
128
|
+
await new Promise((r) => setTimeout(r, 400));
|
|
129
|
+
}
|
|
130
|
+
}
|
|
131
|
+
if (!ready) {
|
|
132
|
+
await close();
|
|
133
|
+
throw new Error(`server did not become ready in ${timeoutMs}ms: ${startCommand}\n${lines.join("\n").slice(-2000)}`);
|
|
134
|
+
}
|
|
135
|
+
return { url, close, logs: () => lines.join("\n") };
|
|
136
|
+
}
|
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* Child-process environment isolation. The heal sandbox runs the project's own
|
|
3
|
+
* test suite, `tsc`, and dev server — which means it executes untrusted PR code.
|
|
4
|
+
* Handing that code the full `process.env` would leak the caller's secrets
|
|
5
|
+
* (ANTHROPIC_API_KEY, GH_TOKEN, NPM_TOKEN, AWS_*, …) into a process the PR author
|
|
6
|
+
* controls. Children therefore get only a minimal allow-list of harmless,
|
|
7
|
+
* platform-essential variables.
|
|
8
|
+
*
|
|
9
|
+
* Escape hatch: a caller whose dev server legitimately needs app config (e.g.
|
|
10
|
+
* `DATABASE_URL`) can pass it through by naming it in `AZTRX_ENV_ALLOW`
|
|
11
|
+
* (comma-separated) — without re-opening the leak for every secret in the env.
|
|
12
|
+
*/
|
|
13
|
+
const ALLOWED = new Set([
|
|
14
|
+
"PATH",
|
|
15
|
+
"HOME",
|
|
16
|
+
"USER",
|
|
17
|
+
"USERNAME",
|
|
18
|
+
"LOGNAME",
|
|
19
|
+
"SHELL",
|
|
20
|
+
"TMP",
|
|
21
|
+
"TEMP",
|
|
22
|
+
"TMPDIR",
|
|
23
|
+
"SystemRoot",
|
|
24
|
+
"SystemDrive",
|
|
25
|
+
"windir",
|
|
26
|
+
"COMSPEC",
|
|
27
|
+
"PATHEXT",
|
|
28
|
+
"LANG",
|
|
29
|
+
"LC_ALL",
|
|
30
|
+
"LC_CTYPE",
|
|
31
|
+
"TERM",
|
|
32
|
+
"NODE_ENV",
|
|
33
|
+
"CI",
|
|
34
|
+
"PORT",
|
|
35
|
+
]);
|
|
36
|
+
/** Build a minimal child environment: harmless platform vars, anything named in
|
|
37
|
+
* `AZTRX_ENV_ALLOW`, and any caller-supplied `extra` (e.g. `{ PORT: "4317" }`). */
|
|
38
|
+
export function buildChildEnv(extra) {
|
|
39
|
+
const env = {};
|
|
40
|
+
for (const key of ALLOWED) {
|
|
41
|
+
const v = process.env[key];
|
|
42
|
+
if (v !== undefined)
|
|
43
|
+
env[key] = v;
|
|
44
|
+
}
|
|
45
|
+
const allow = process.env.AZTRX_ENV_ALLOW;
|
|
46
|
+
if (allow) {
|
|
47
|
+
for (const raw of allow.split(",")) {
|
|
48
|
+
const name = raw.trim();
|
|
49
|
+
if (!name)
|
|
50
|
+
continue;
|
|
51
|
+
const v = process.env[name];
|
|
52
|
+
if (v !== undefined)
|
|
53
|
+
env[name] = v;
|
|
54
|
+
}
|
|
55
|
+
}
|
|
56
|
+
if (extra)
|
|
57
|
+
for (const [k, v] of Object.entries(extra))
|
|
58
|
+
env[k] = v;
|
|
59
|
+
return env;
|
|
60
|
+
}
|