@post-print/agent-test 0.1.12 → 0.1.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +69 -7
- package/dist/cli-entry.d.ts.map +1 -1
- package/dist/cli-entry.js +1 -1
- package/dist/cli-entry.js.map +1 -1
- package/dist/cli.d.ts +24 -1
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +88 -9
- package/dist/cli.js.map +1 -1
- package/dist/debug-bundle.d.ts +66 -0
- package/dist/debug-bundle.d.ts.map +1 -0
- package/dist/debug-bundle.js +241 -0
- package/dist/debug-bundle.js.map +1 -0
- package/dist/doctor.d.ts +15 -0
- package/dist/doctor.d.ts.map +1 -0
- package/dist/doctor.js +49 -0
- package/dist/doctor.js.map +1 -0
- package/dist/expect.d.ts +3 -0
- package/dist/expect.d.ts.map +1 -1
- package/dist/expect.js +79 -46
- package/dist/expect.js.map +1 -1
- package/dist/failures.d.ts +4 -0
- package/dist/failures.d.ts.map +1 -0
- package/dist/failures.js +7 -0
- package/dist/failures.js.map +1 -0
- package/dist/html-report.d.ts +11 -0
- package/dist/html-report.d.ts.map +1 -0
- package/dist/html-report.js +462 -0
- package/dist/html-report.js.map +1 -0
- package/dist/index.d.ts +5 -2
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +4 -1
- package/dist/index.js.map +1 -1
- package/dist/live-isolation.d.ts +7 -9
- package/dist/live-isolation.d.ts.map +1 -1
- package/dist/live-isolation.js +44 -28
- package/dist/live-isolation.js.map +1 -1
- package/dist/progress.d.ts.map +1 -1
- package/dist/progress.js +2 -1
- package/dist/progress.js.map +1 -1
- package/dist/record-trace.d.ts +8 -0
- package/dist/record-trace.d.ts.map +1 -1
- package/dist/record-trace.js +23 -4
- package/dist/record-trace.js.map +1 -1
- package/dist/run-suite.d.ts +6 -0
- package/dist/run-suite.d.ts.map +1 -1
- package/dist/run-suite.js +210 -87
- package/dist/run-suite.js.map +1 -1
- package/dist/scenario-seed.d.ts.map +1 -1
- package/dist/scenario-seed.js +6 -2
- package/dist/scenario-seed.js.map +1 -1
- package/dist/theme.d.ts +9 -1
- package/dist/theme.d.ts.map +1 -1
- package/dist/theme.js +33 -15
- package/dist/theme.js.map +1 -1
- package/dist/types.d.ts +20 -1
- package/dist/types.d.ts.map +1 -1
- package/dist/warnings.d.ts.map +1 -1
- package/dist/warnings.js +1 -2
- package/dist/warnings.js.map +1 -1
- package/package.json +5 -2
package/README.md
CHANGED
|
@@ -2,29 +2,91 @@
|
|
|
2
2
|
|
|
3
3
|
**Source of truth for** agent-test package.
|
|
4
4
|
|
|
5
|
-
<!-- doc-meta: owner=eng | last-reviewed=2026-07-
|
|
5
|
+
<!-- doc-meta: owner=eng | last-reviewed=2026-07-16 -->
|
|
6
6
|
|
|
7
7
|
Jest-shaped agent scenario runner built on `@post-print/agent-harness`.
|
|
8
8
|
|
|
9
|
-
##
|
|
9
|
+
## In-repo smoke (this monorepo)
|
|
10
|
+
|
|
11
|
+
After `bun run build` at the repo root:
|
|
12
|
+
|
|
13
|
+
```bash
|
|
14
|
+
node packages/test/dist/cli.js --suites-dir packages/test/fixtures --suite smoke
|
|
15
|
+
node packages/test/dist/cli.js --doctor
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
There is no top-level `agent-suites/` here — consumer examples below assume a consuming repo. Live `--live` needs an **exported** `CURSOR_API_KEY` (copy repo-root `.env.example`; the CLI does not auto-load `.env`).
|
|
19
|
+
|
|
20
|
+
## CLI (consumers, Node >= 22)
|
|
10
21
|
|
|
11
22
|
Works under **Node >= 22** (the published `agent-test` bin):
|
|
12
23
|
|
|
13
24
|
```bash
|
|
14
25
|
npx agent-test --suites-dir agent-suites
|
|
15
|
-
npx agent-test --suite ambient-routing
|
|
16
|
-
npx agent-test --live --suite ambient-routing # CURSOR_API_KEY required
|
|
26
|
+
npx agent-test --suites-dir agent-suites --suite ambient-routing
|
|
27
|
+
npx agent-test --suites-dir agent-suites --live --suite ambient-routing # exported CURSOR_API_KEY required
|
|
28
|
+
npx agent-test --doctor
|
|
17
29
|
```
|
|
18
30
|
|
|
19
31
|
Bun is fine for local package development (`bun install` / `bun run build` in this monorepo), but consumers do not need Bun to run suites.
|
|
20
32
|
|
|
21
|
-
Default suites root: `agent-suites
|
|
33
|
+
Default suites root: `agent-suites/` (must exist, or pass `--suites-dir`).
|
|
34
|
+
|
|
35
|
+
Live output always uses ANSI color (including under Cursor agent shells that set `NO_COLOR`).
|
|
36
|
+
|
|
37
|
+
## Environment variables
|
|
38
|
+
|
|
39
|
+
See repo-root `.env.example`. Common knobs:
|
|
40
|
+
|
|
41
|
+
| Variable | Purpose |
|
|
42
|
+
| ------------------------------------------- | ------------------------------------------------ |
|
|
43
|
+
| `CURSOR_API_KEY` | Required for `--live` and judge classifiers |
|
|
44
|
+
| `AGENT_TEST_DEBUG` | Same as `--debug` when `1`/`true` |
|
|
45
|
+
| `AGENT_TEST_VERBOSE` | Extra tips (e.g. OOM isolation) when `1` |
|
|
46
|
+
| `AGENT_TEST_VERBOSE_PATHS` | Print full paths when `1` |
|
|
47
|
+
| `AGENT_TEST_QUIET` | Suppress progress when `1` |
|
|
48
|
+
| `AGENT_TEST_TIMEOUT_MS` | Live hard timeout (default 600000; `0` disables) |
|
|
49
|
+
| `AGENT_TEST_ALLOW_IN_PLACE` | Allow `--no-worktree` live runs when `1` |
|
|
50
|
+
| `AGENT_TEST_NO_WORKTREE` | Disable worktree isolation when `1`/`true` |
|
|
51
|
+
| `AGENT_TEST_NO_ISOLATE` | Disable isolated subprocesses when `1` |
|
|
52
|
+
| `AGENT_TEST_SCENARIO_SETTLE_MS` | Settle delay between live scenarios |
|
|
53
|
+
| `CURSOR_AGENT_MODEL` / `CURSOR_JUDGE_MODEL` | Optional model overrides |
|
|
54
|
+
| `CURSOR_JUDGE_TEMPERATURE` | Optional judge temperature |
|
|
55
|
+
|
|
56
|
+
## Debug mode
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
npx agent-test --suites-dir agent-suites --suite smoke --debug
|
|
60
|
+
npx agent-test --suites-dir agent-suites --live --debug --debug-dir ./agent-test-debug
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
`--debug` (or `AGENT_TEST_DEBUG=1`) implies `--keep-recordings`, verbose failure detail, and full paths. Failed scenarios write a bundle under the session root:
|
|
64
|
+
|
|
65
|
+
```
|
|
66
|
+
sessions/<id>/<suite>/<scenario>.debug/
|
|
67
|
+
transcript.md
|
|
68
|
+
trace.json
|
|
69
|
+
failures.json # includes category + evidence
|
|
70
|
+
judge-debug.json # when judge criteria ran (SDK status/error, sizes, attempt)
|
|
71
|
+
environment.json # versions/models/timeout/isolation; CURSOR_API_KEY only as boolean
|
|
72
|
+
rerun.sh # shell-quoted exact re-run command (export CURSOR_API_KEY yourself)
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
`--debug-dir <path>` replaces `$TMPDIR/agent-spec` as the sessions parent (`<path>/sessions/<id>/…`).
|
|
76
|
+
|
|
77
|
+
Failure categories printed on FAIL lines and in `failures.json`:
|
|
22
78
|
|
|
23
|
-
|
|
79
|
+
| Category | Meaning |
|
|
80
|
+
| --- | --- |
|
|
81
|
+
| `rubric_miss` | Assertion/judge criterion miss |
|
|
82
|
+
| `judge_infra` | Judge SDK/API failure (not a criterion miss) |
|
|
83
|
+
| `agent_runtime` | Agent session error, timeout, AskQuestion, subprocess exit |
|
|
84
|
+
| `worktree_leak` | Live agent mutated the caller working tree |
|
|
85
|
+
| `recording_error` | Failed to persist a staging/fixture trace |
|
|
24
86
|
|
|
25
87
|
## Live dogfood
|
|
26
88
|
|
|
27
|
-
Live runs need `CURSOR_API_KEY` and a suites directory. Preflight fails when `--suites-dir` is
|
|
89
|
+
Live runs need `CURSOR_API_KEY` and a suites directory that exists. Preflight fails when the resolved suites directory is missing (default `agent-suites/` if `--suites-dir` is omitted).
|
|
28
90
|
|
|
29
91
|
Passing live runs write staging traces under `$TMPDIR/agent-spec/sessions/<pid>-<timestamp>/` (removed on exit unless `--keep-recordings`). Use `--record-fixtures` to overwrite each scenario's committed `replayTrace` path. `--no-worktree` requires `AGENT_TEST_ALLOW_IN_PLACE=1`.
|
|
30
92
|
|
package/dist/cli-entry.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"cli-entry.d.ts","sourceRoot":"","sources":["../src/cli-entry.ts"],"names":[],"mappings":"AAGA,oFAAoF;AACpF,wBAAgB,eAAe,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,CAMpD;AAED;;;GAGG;AACH,wBAAgB,SAAS,
|
|
1
|
+
{"version":3,"file":"cli-entry.d.ts","sourceRoot":"","sources":["../src/cli-entry.ts"],"names":[],"mappings":"AAGA,oFAAoF;AACpF,wBAAgB,eAAe,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,CAMpD;AAED;;;GAGG;AACH,wBAAgB,SAAS,CAAC,KAAK,EAAE,MAAM,GAAG,SAAS,EAAE,SAAS,EAAE,MAAM,GAAG,OAAO,CAE/E"}
|
package/dist/cli-entry.js
CHANGED
|
@@ -14,6 +14,6 @@ export function resolveRealPath(path) {
|
|
|
14
14
|
* Compares real paths so `node_modules/.bin/agent-test` matches `dist/cli.js`.
|
|
15
15
|
*/
|
|
16
16
|
export function isCliMain(argv1, entryPath) {
|
|
17
|
-
return
|
|
17
|
+
return argv1 !== undefined && resolveRealPath(argv1) === resolveRealPath(entryPath);
|
|
18
18
|
}
|
|
19
19
|
//# sourceMappingURL=cli-entry.js.map
|
package/dist/cli-entry.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"cli-entry.js","sourceRoot":"","sources":["../src/cli-entry.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,MAAM,SAAS,CAAC;AACvC,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AAEpC,oFAAoF;AACpF,MAAM,UAAU,eAAe,CAAC,IAAY;IAC3C,IAAI,CAAC;QACJ,OAAO,YAAY,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC;IACpC,CAAC;IAAC,MAAM,CAAC;QACR,OAAO,OAAO,CAAC,IAAI,CAAC,CAAC;IACtB,CAAC;AACF,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,SAAS,
|
|
1
|
+
{"version":3,"file":"cli-entry.js","sourceRoot":"","sources":["../src/cli-entry.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,MAAM,SAAS,CAAC;AACvC,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AAEpC,oFAAoF;AACpF,MAAM,UAAU,eAAe,CAAC,IAAY;IAC3C,IAAI,CAAC;QACJ,OAAO,YAAY,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC;IACpC,CAAC;IAAC,MAAM,CAAC;QACR,OAAO,OAAO,CAAC,IAAI,CAAC,CAAC;IACtB,CAAC;AACF,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,SAAS,CAAC,KAAyB,EAAE,SAAiB;IACrE,OAAO,KAAK,KAAK,SAAS,IAAI,eAAe,CAAC,KAAK,CAAC,KAAK,eAAe,CAAC,SAAS,CAAC,CAAC;AACrF,CAAC"}
|
package/dist/cli.d.ts
CHANGED
|
@@ -1,3 +1,26 @@
|
|
|
1
1
|
#!/usr/bin/env -S node --disable-warning=ExperimentalWarning
|
|
2
|
-
|
|
2
|
+
import { type AgentHost } from "@post-print/agent-harness";
|
|
3
|
+
export interface ParsedCliArgs {
|
|
4
|
+
cwd: string;
|
|
5
|
+
suitesDir: string;
|
|
6
|
+
host?: AgentHost;
|
|
7
|
+
filter?: string;
|
|
8
|
+
scenarioFilter?: string;
|
|
9
|
+
stagingSessionId?: string;
|
|
10
|
+
record: boolean;
|
|
11
|
+
recordFixtures: boolean;
|
|
12
|
+
live: boolean;
|
|
13
|
+
judge?: boolean;
|
|
14
|
+
worktree?: boolean;
|
|
15
|
+
keepRecordings: boolean;
|
|
16
|
+
timeoutMs?: number;
|
|
17
|
+
noTimeout: boolean;
|
|
18
|
+
allowUserInput: boolean;
|
|
19
|
+
doctor: boolean;
|
|
20
|
+
htmlReport: boolean;
|
|
21
|
+
debug: boolean;
|
|
22
|
+
debugDir?: string;
|
|
23
|
+
}
|
|
24
|
+
/** Parse agent-test CLI argv (exported for unit tests). */
|
|
25
|
+
export declare function parseCliArgs(argv: string[]): ParsedCliArgs;
|
|
3
26
|
//# sourceMappingURL=cli.d.ts.map
|
package/dist/cli.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"cli.d.ts","sourceRoot":"","sources":["../src/cli.ts"],"names":[],"mappings":""}
|
|
1
|
+
{"version":3,"file":"cli.d.ts","sourceRoot":"","sources":["../src/cli.ts"],"names":[],"mappings":";AAIA,OAAO,EAAE,KAAK,SAAS,EAAiC,MAAM,2BAA2B,CAAC;AAqB1F,MAAM,WAAW,aAAa;IAC7B,GAAG,EAAE,MAAM,CAAC;IACZ,SAAS,EAAE,MAAM,CAAC;IAClB,IAAI,CAAC,EAAE,SAAS,CAAC;IACjB,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,MAAM,EAAE,OAAO,CAAC;IAChB,cAAc,EAAE,OAAO,CAAC;IACxB,IAAI,EAAE,OAAO,CAAC;IACd,KAAK,CAAC,EAAE,OAAO,CAAC;IAChB,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,cAAc,EAAE,OAAO,CAAC;IACxB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,SAAS,EAAE,OAAO,CAAC;IACnB,cAAc,EAAE,OAAO,CAAC;IACxB,MAAM,EAAE,OAAO,CAAC;IAChB,UAAU,EAAE,OAAO,CAAC;IACpB,KAAK,EAAE,OAAO,CAAC;IACf,QAAQ,CAAC,EAAE,MAAM,CAAC;CAClB;AAED,2DAA2D;AAC3D,wBAAgB,YAAY,CAAC,IAAI,EAAE,MAAM,EAAE,GAAG,aAAa,CA8G1D"}
|
package/dist/cli.js
CHANGED
|
@@ -1,16 +1,20 @@
|
|
|
1
1
|
#!/usr/bin/env -S node --disable-warning=ExperimentalWarning
|
|
2
|
+
import { resolve } from "node:path";
|
|
2
3
|
import { fileURLToPath } from "node:url";
|
|
3
|
-
import { cleanupStaleScenarioWorktrees
|
|
4
|
+
import { cleanupStaleScenarioWorktrees } from "@post-print/agent-harness";
|
|
4
5
|
import { isCliMain } from "./cli-entry.js";
|
|
6
|
+
import { runDoctor } from "./doctor.js";
|
|
7
|
+
import { writeHtmlReport } from "./html-report.js";
|
|
5
8
|
import { assertLiveDogfoodPreflight } from "./preflight.js";
|
|
6
9
|
import { logProgress } from "./progress.js";
|
|
7
|
-
import { cleanupLegacyRepoRecordings, cleanupStagingSession, createLiveStagingSessionId, getLiveStagingSessionRoot, } from "./record-trace.js";
|
|
10
|
+
import { cleanupLegacyRepoRecordings, cleanupStagingSession, createLiveStagingSessionId, getLiveStagingSessionRoot, setLiveStagingRootOverride, } from "./record-trace.js";
|
|
8
11
|
import { registerLiveRunHandlers, runAllSuites } from "./run-suite.js";
|
|
9
12
|
import { configureCliColor, theme } from "./theme.js";
|
|
10
13
|
import { suppressNoisyRuntimeWarnings } from "./warnings.js";
|
|
11
14
|
suppressNoisyRuntimeWarnings();
|
|
12
15
|
configureCliColor();
|
|
13
|
-
|
|
16
|
+
/** Parse agent-test CLI argv (exported for unit tests). */
|
|
17
|
+
export function parseCliArgs(argv) {
|
|
14
18
|
const cwd = process.cwd();
|
|
15
19
|
let suitesDir = "agent-suites";
|
|
16
20
|
let host;
|
|
@@ -26,6 +30,10 @@ function parseArgs(argv) {
|
|
|
26
30
|
let timeoutMs;
|
|
27
31
|
let noTimeout = false;
|
|
28
32
|
let allowUserInput = false;
|
|
33
|
+
let doctor = false;
|
|
34
|
+
let htmlReport = true;
|
|
35
|
+
let debug = process.env.AGENT_TEST_DEBUG === "1" || process.env.AGENT_TEST_DEBUG === "true";
|
|
36
|
+
let debugDir;
|
|
29
37
|
for (let i = 2; i < argv.length; i++) {
|
|
30
38
|
const token = argv[i];
|
|
31
39
|
if (token === "--host" && argv[i + 1]) {
|
|
@@ -77,6 +85,23 @@ function parseArgs(argv) {
|
|
|
77
85
|
else if (token === "--allow-user-input") {
|
|
78
86
|
allowUserInput = true;
|
|
79
87
|
}
|
|
88
|
+
else if (token === "--doctor") {
|
|
89
|
+
doctor = true;
|
|
90
|
+
}
|
|
91
|
+
else if (token === "--no-html-report") {
|
|
92
|
+
htmlReport = false;
|
|
93
|
+
}
|
|
94
|
+
else if (token === "--debug") {
|
|
95
|
+
debug = true;
|
|
96
|
+
}
|
|
97
|
+
else if (token === "--debug-dir") {
|
|
98
|
+
const value = argv[++i];
|
|
99
|
+
if (!value || value.startsWith("-")) {
|
|
100
|
+
throw new Error("--debug-dir requires a non-empty path argument");
|
|
101
|
+
}
|
|
102
|
+
debugDir = value;
|
|
103
|
+
debug = true;
|
|
104
|
+
}
|
|
80
105
|
else if (token && !token.startsWith("-")) {
|
|
81
106
|
filter = token;
|
|
82
107
|
}
|
|
@@ -87,6 +112,12 @@ function parseArgs(argv) {
|
|
|
87
112
|
judge = judge ?? true;
|
|
88
113
|
worktree = worktree ?? true;
|
|
89
114
|
}
|
|
115
|
+
if (debug) {
|
|
116
|
+
keepRecordings = true;
|
|
117
|
+
process.env.AGENT_TEST_DEBUG = "1";
|
|
118
|
+
process.env.AGENT_TEST_VERBOSE = process.env.AGENT_TEST_VERBOSE ?? "1";
|
|
119
|
+
process.env.AGENT_TEST_VERBOSE_PATHS = process.env.AGENT_TEST_VERBOSE_PATHS ?? "1";
|
|
120
|
+
}
|
|
90
121
|
return {
|
|
91
122
|
cwd,
|
|
92
123
|
suitesDir,
|
|
@@ -103,6 +134,10 @@ function parseArgs(argv) {
|
|
|
103
134
|
timeoutMs: noTimeout ? 0 : timeoutMs,
|
|
104
135
|
noTimeout,
|
|
105
136
|
allowUserInput,
|
|
137
|
+
doctor,
|
|
138
|
+
htmlReport,
|
|
139
|
+
debug,
|
|
140
|
+
debugDir: debugDir ? resolve(cwd, debugDir) : undefined,
|
|
106
141
|
};
|
|
107
142
|
}
|
|
108
143
|
async function cleanupLiveRunArtifacts(cwd, stagingSessionRoot, keepRecordings) {
|
|
@@ -123,12 +158,29 @@ async function cleanupLiveRunArtifacts(cwd, stagingSessionRoot, keepRecordings)
|
|
|
123
158
|
}
|
|
124
159
|
}
|
|
125
160
|
async function main() {
|
|
126
|
-
|
|
161
|
+
let args;
|
|
162
|
+
try {
|
|
163
|
+
args = parseCliArgs(process.argv);
|
|
164
|
+
}
|
|
165
|
+
catch (error) {
|
|
166
|
+
console.error(error instanceof Error ? error.message : error);
|
|
167
|
+
return 1;
|
|
168
|
+
}
|
|
169
|
+
if (args.doctor) {
|
|
170
|
+
const report = runDoctor();
|
|
171
|
+
for (const message of report.messages) {
|
|
172
|
+
console.log(message);
|
|
173
|
+
}
|
|
174
|
+
return report.ok ? 0 : 1;
|
|
175
|
+
}
|
|
176
|
+
if (args.debugDir) {
|
|
177
|
+
setLiveStagingRootOverride(args.debugDir);
|
|
178
|
+
}
|
|
127
179
|
const isChild = process.env.AGENT_TEST_CHILD === "1";
|
|
128
|
-
const verbose = process.env.AGENT_TEST_VERBOSE === "1";
|
|
180
|
+
const verbose = args.debug || process.env.AGENT_TEST_VERBOSE === "1" || process.env.AGENT_TEST_DEBUG === "1";
|
|
129
181
|
const stagingSessionId = args.stagingSessionId?.trim() ||
|
|
130
182
|
process.env.AGENT_TEST_STAGING_SESSION_ID?.trim() ||
|
|
131
|
-
(args.live || (args.record && !args.recordFixtures)
|
|
183
|
+
(args.live || args.debug || (args.record && !args.recordFixtures)
|
|
132
184
|
? createLiveStagingSessionId()
|
|
133
185
|
: undefined);
|
|
134
186
|
const stagingSessionRoot = stagingSessionId
|
|
@@ -161,17 +213,20 @@ async function main() {
|
|
|
161
213
|
if (removed.length > 0) {
|
|
162
214
|
console.log(theme.warn(`Cleaned ${removed.length} stale agent-test worktree(s) from a prior crash`));
|
|
163
215
|
}
|
|
164
|
-
console.log(theme.banner("live"));
|
|
216
|
+
console.log(theme.banner(args.debug ? "live debug" : "live"));
|
|
165
217
|
if (stagingSessionRoot) {
|
|
166
218
|
console.log(theme.bannerSession(stagingSessionRoot));
|
|
167
219
|
}
|
|
168
220
|
if (worktreeDisabled) {
|
|
169
221
|
console.warn(theme.warn("running in repo cwd — agent file edits will persist in your working tree"));
|
|
170
222
|
}
|
|
171
|
-
if (process.env.AGENT_TEST_VERBOSE === "1") {
|
|
223
|
+
if (args.debug || process.env.AGENT_TEST_VERBOSE === "1") {
|
|
172
224
|
if (!args.keepRecordings) {
|
|
173
225
|
console.log(` ${theme.tip("traces removed on exit unless --keep-recordings")}`);
|
|
174
226
|
}
|
|
227
|
+
else {
|
|
228
|
+
console.log(` ${theme.tip("debug: recordings kept")}`);
|
|
229
|
+
}
|
|
175
230
|
console.log(` ${theme.tip("exit 137 = macOS OOM — isolated subprocesses (AGENT_TEST_NO_ISOLATE=1 to disable)")}`);
|
|
176
231
|
}
|
|
177
232
|
else if (!args.keepRecordings) {
|
|
@@ -179,11 +234,17 @@ async function main() {
|
|
|
179
234
|
}
|
|
180
235
|
}
|
|
181
236
|
}
|
|
237
|
+
else if (args.debug && !isChild && stagingSessionRoot) {
|
|
238
|
+
console.log(theme.banner("debug"));
|
|
239
|
+
console.log(theme.bannerSession(stagingSessionRoot));
|
|
240
|
+
}
|
|
182
241
|
const reports = await runAllSuites({
|
|
183
242
|
...args,
|
|
184
243
|
stagingSessionId,
|
|
185
244
|
timeoutMs: args.timeoutMs,
|
|
186
245
|
allowUserInput: args.allowUserInput,
|
|
246
|
+
debug: args.debug,
|
|
247
|
+
debugDir: args.debugDir,
|
|
187
248
|
});
|
|
188
249
|
let exitCode = 0;
|
|
189
250
|
if (!isChild) {
|
|
@@ -196,13 +257,31 @@ async function main() {
|
|
|
196
257
|
console.log(theme.failedScenarioName(result.scenario));
|
|
197
258
|
if (verbose) {
|
|
198
259
|
for (const failure of result.failures) {
|
|
199
|
-
console.log(theme.verboseFailure(failure.matcher, failure.message));
|
|
260
|
+
console.log(theme.verboseFailure(failure.matcher, failure.message, failure.evidence, failure.category));
|
|
200
261
|
}
|
|
201
262
|
}
|
|
263
|
+
if (args.debug && result.debugBundleDir) {
|
|
264
|
+
console.log(` ${theme.tip(result.debugBundleDir)}`);
|
|
265
|
+
}
|
|
202
266
|
}
|
|
203
267
|
}
|
|
204
268
|
console.log(theme.summary(report.suite, report.passed, report.failed, report.skipped));
|
|
205
269
|
}
|
|
270
|
+
if (args.debug && stagingSessionRoot) {
|
|
271
|
+
console.log(`\n${theme.tip(`debug session: ${stagingSessionRoot}`)}`);
|
|
272
|
+
}
|
|
273
|
+
if (args.htmlReport && reports.length > 0) {
|
|
274
|
+
try {
|
|
275
|
+
const reportPath = await writeHtmlReport(reports, {
|
|
276
|
+
host: args.host,
|
|
277
|
+
suitesDir: args.suitesDir,
|
|
278
|
+
});
|
|
279
|
+
console.log(`\n${theme.tip(`HTML report: ${reportPath}`)}`);
|
|
280
|
+
}
|
|
281
|
+
catch (error) {
|
|
282
|
+
console.warn(theme.warn(`HTML report failed: ${error instanceof Error ? error.message : String(error)}`));
|
|
283
|
+
}
|
|
284
|
+
}
|
|
206
285
|
}
|
|
207
286
|
else {
|
|
208
287
|
for (const report of reports) {
|
package/dist/cli.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"cli.js","sourceRoot":"","sources":["../src/cli.ts"],"names":[],"mappings":";AACA,OAAO,EAAE,aAAa,EAAE,MAAM,UAAU,CAAC;AAEzC,OAAO,EAEN,6BAA6B,GAC7B,MAAM,2BAA2B,CAAC;AAEnC,OAAO,EAAE,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAC3C,OAAO,EAAE,0BAA0B,EAAE,MAAM,gBAAgB,CAAC;AAC5D,OAAO,EAAE,WAAW,EAAE,MAAM,eAAe,CAAC;AAC5C,OAAO,EACN,2BAA2B,EAC3B,qBAAqB,EACrB,0BAA0B,EAC1B,yBAAyB,GACzB,MAAM,mBAAmB,CAAC;AAC3B,OAAO,EAAE,uBAAuB,EAAE,YAAY,EAAE,MAAM,gBAAgB,CAAC;AACvE,OAAO,EAAE,iBAAiB,EAAE,KAAK,EAAE,MAAM,YAAY,CAAC;AACtD,OAAO,EAAE,4BAA4B,EAAE,MAAM,eAAe,CAAC;AAE7D,4BAA4B,EAAE,CAAC;AAC/B,iBAAiB,EAAE,CAAC;AAEpB,SAAS,SAAS,CAAC,IAAc;IAiBhC,MAAM,GAAG,GAAG,OAAO,CAAC,GAAG,EAAE,CAAC;IAC1B,IAAI,SAAS,GAAG,cAAc,CAAC;IAC/B,IAAI,IAA2B,CAAC;IAChC,IAAI,MAA0B,CAAC;IAC/B,IAAI,cAAkC,CAAC;IACvC,IAAI,gBAAoC,CAAC;IACzC,IAAI,MAAM,GAAG,KAAK,CAAC;IACnB,IAAI,cAAc,GAAG,KAAK,CAAC;IAC3B,IAAI,IAAI,GAAG,KAAK,CAAC;IACjB,IAAI,KAA0B,CAAC;IAC/B,IAAI,QAA6B,CAAC;IAClC,IAAI,cAAc,GAAG,KAAK,CAAC;IAC3B,IAAI,SAA6B,CAAC;IAClC,IAAI,SAAS,GAAG,KAAK,CAAC;IACtB,IAAI,cAAc,GAAG,KAAK,CAAC;IAE3B,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACtC,MAAM,KAAK,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;QACtB,IAAI,KAAK,KAAK,QAAQ,IAAI,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;YACvC,IAAI,GAAG,IAAI,CAAC,EAAE,CAAC,CAAc,CAAC;QAC/B,CAAC;aAAM,IAAI,KAAK,KAAK,cAAc,IAAI,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;YACpD,SAAS,GAAG,IAAI,CAAC,EAAE,CAAC,CAAW,CAAC;QACjC,CAAC;aAAM,IAAI,KAAK,KAAK,SAAS,IAAI,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;YAC/C,MAAM,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;QACpB,CAAC;aAAM,IAAI,KAAK,KAAK,YAAY,IAAI,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;YAClD,cAAc,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;QAC5B,CAAC;aAAM,IAAI,KAAK,KAAK,sBAAsB,IAAI,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;YAC5D,gBAAgB,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;QAC9B,CAAC;aAAM,IAAI,KAAK,KAAK,UAAU,EAAE,CAAC;YACjC,MAAM,GAAG,IAAI,CAAC;QACf,CAAC;aAAM,IAAI,KAAK,KAAK,mBAAmB,EAAE,CAAC;YAC1C,MAAM,GAAG,IAAI,CAAC;YACd,cAAc,GAAG,IAAI,CAAC;QACvB,CAAC;aAAM,IAAI,KAAK,KAAK,QAAQ,EAAE,CAAC;YAC/B,IAAI,GAAG,IAAI,CAAC;QACb,CAAC;aAAM,IAAI,KAAK,KAAK,mBAAmB,EAAE,CAAC;YAC1C,cAAc,GAAG,IAAI,CAAC;QACvB,CAAC;aAAM,IAAI,KAAK,KAAK,SAAS,EAAE,CAAC;YAChC,KAAK,GAAG,IAAI,CAAC;QACd,CAAC;aAAM,IAAI,KAAK,KAAK,YAAY,EAAE,CAAC;YACnC,KAAK,GAAG,KAAK,CAAC;QACf,CAAC;aAAM,IAAI,KAAK,KAAK,eAAe,EAAE,CAAC;YACtC,QAAQ,GAAG,KAAK,CAAC;QAClB,CAAC;aAAM,IAAI,KAAK,KAAK,cAAc,IAAI,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;YACpD,MAAM,MAAM,GAAG,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;YACjC,IAAI,MAAM,CAAC,QAAQ,CAAC,MAAM,CAAC,IAAI,MAAM,GAAG,CAAC,EAAE,CAAC;gBAC3C,SAAS,GAAG,MAAM,CAAC;YACpB,CAAC;QACF,CAAC;aAAM,IAAI,KAAK,KAAK,cAAc,EAAE,CAAC;YACrC,SAAS,GAAG,IAAI,CAAC;QAClB,CAAC;aAAM,IAAI,KAAK,KAAK,oBAAoB,EAAE,CAAC;YAC3C,cAAc,GAAG,IAAI,CAAC;QACvB,CAAC;aAAM,IAAI,KAAK,IAAI,CAAC,KAAK,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE,CAAC;YAC5C,MAAM,GAAG,KAAK,CAAC;QAChB,CAAC;IACF,CAAC;IAED,IAAI,IAAI,EAAE,CAAC;QACV,IAAI,GAAG,IAAI,IAAI,QAAQ,CAAC;QACxB,MAAM,GAAG,IAAI,CAAC;QACd,KAAK,GAAG,KAAK,IAAI,IAAI,CAAC;QACtB,QAAQ,GAAG,QAAQ,IAAI,IAAI,CAAC;IAC7B,CAAC;IAED,OAAO;QACN,GAAG;QACH,SAAS;QACT,IAAI;QACJ,MAAM;QACN,cAAc;QACd,gBAAgB;QAChB,MAAM;QACN,cAAc;QACd,IAAI;QACJ,KAAK;QACL,QAAQ;QACR,cAAc;QACd,SAAS,EAAE,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,SAAS;QACpC,SAAS;QACT,cAAc;KACd,CAAC;AACH,CAAC;AAED,KAAK,UAAU,uBAAuB,CACrC,GAAW,EACX,kBAAsC,EACtC,cAAuB;IAEvB,IAAI,cAAc,EAAE,CAAC;QACpB,OAAO;IACR,CAAC;IAED,IAAI,kBAAkB,EAAE,CAAC;QACxB,IAAI,CAAC;YACJ,MAAM,qBAAqB,CAAC,kBAAkB,CAAC,CAAC;QACjD,CAAC;QAAC,MAAM,CAAC;YACR,cAAc;QACf,CAAC;IACF,CAAC;IAED,MAAM,aAAa,GAAG,MAAM,2BAA2B,CAAC,GAAG,CAAC,CAAC;IAC7D,IAAI,aAAa,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC9B,OAAO,CAAC,GAAG,CACV,+CAA+C,aAAa,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAC3E,CAAC;IACH,CAAC;AACF,CAAC;AAED,KAAK,UAAU,IAAI;IAClB,MAAM,IAAI,GAAG,SAAS,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;IACrC,MAAM,OAAO,GAAG,OAAO,CAAC,GAAG,CAAC,gBAAgB,KAAK,GAAG,CAAC;IACrD,MAAM,OAAO,GAAG,OAAO,CAAC,GAAG,CAAC,kBAAkB,KAAK,GAAG,CAAC;IACvD,MAAM,gBAAgB,GACrB,IAAI,CAAC,gBAAgB,EAAE,IAAI,EAAE;QAC7B,OAAO,CAAC,GAAG,CAAC,6BAA6B,EAAE,IAAI,EAAE;QACjD,CAAC,IAAI,CAAC,IAAI,IAAI,CAAC,IAAI,CAAC,MAAM,IAAI,CAAC,IAAI,CAAC,cAAc,CAAC;YAClD,CAAC,CAAC,0BAA0B,EAAE;YAC9B,CAAC,CAAC,SAAS,CAAC,CAAC;IACf,MAAM,kBAAkB,GAAG,gBAAgB;QAC1C,CAAC,CAAC,yBAAyB,CAAC,gBAAgB,CAAC;QAC7C,CAAC,CAAC,SAAS,CAAC;IAEb,IAAI,CAAC;QACJ,IAAI,IAAI,CAAC,IAAI,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,cAAc,EAAE,CAAC;YAC9C,OAAO,CAAC,KAAK,CAAC,sDAAsD,CAAC,CAAC;YACtE,OAAO,CAAC,CAAC;QACV,CAAC;QAED,IAAI,IAAI,CAAC,IAAI,EAAE,CAAC;YACf,IAAI,CAAC;gBACJ,MAAM,0BAA0B,CAAC,IAAI,CAAC,GAAG,EAAE,IAAI,CAAC,SAAS,CAAC,CAAC;YAC5D,CAAC;YAAC,OAAO,KAAK,EAAE,CAAC;gBAChB,OAAO,CAAC,KAAK,CAAC,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC;gBAC9D,OAAO,CAAC,CAAC;YACV,CAAC;YAED,MAAM,cAAc,GAAG,OAAO,CAAC,GAAG,CAAC,yBAAyB,KAAK,GAAG,CAAC;YACrE,MAAM,gBAAgB,GACrB,IAAI,CAAC,QAAQ,KAAK,KAAK;gBACvB,OAAO,CAAC,GAAG,CAAC,sBAAsB,KAAK,GAAG;gBAC1C,OAAO,CAAC,GAAG,CAAC,sBAAsB,KAAK,MAAM,CAAC;YAC/C,IAAI,gBAAgB,IAAI,CAAC,cAAc,EAAE,CAAC;gBACzC,OAAO,CAAC,KAAK,CACZ,4JAA4J,CAC5J,CAAC;gBACF,OAAO,CAAC,CAAC;YACV,CAAC;YAED,uBAAuB,EAAE,CAAC;YAC1B,IAAI,CAAC,OAAO,EAAE,CAAC;gBACd,MAAM,OAAO,GAAG,MAAM,6BAA6B,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;gBAC9D,IAAI,OAAO,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;oBACxB,OAAO,CAAC,GAAG,CACV,KAAK,CAAC,IAAI,CACT,WAAW,OAAO,CAAC,MAAM,kDAAkD,CAC3E,CACD,CAAC;gBACH,CAAC;gBACD,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;gBAClC,IAAI,kBAAkB,EAAE,CAAC;oBACxB,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,aAAa,CAAC,kBAAkB,CAAC,CAAC,CAAC;gBACtD,CAAC;gBACD,IAAI,gBAAgB,EAAE,CAAC;oBACtB,OAAO,CAAC,IAAI,CACX,KAAK,CAAC,IAAI,CACT,0EAA0E,CAC1E,CACD,CAAC;gBACH,CAAC;gBACD,IAAI,OAAO,CAAC,GAAG,CAAC,kBAAkB,KAAK,GAAG,EAAE,CAAC;oBAC5C,IAAI,CAAC,IAAI,CAAC,cAAc,EAAE,CAAC;wBAC1B,OAAO,CAAC,GAAG,CACV,KAAK,KAAK,CAAC,GAAG,CAAC,iDAAiD,CAAC,EAAE,CACnE,CAAC;oBACH,CAAC;oBACD,OAAO,CAAC,GAAG,CACV,KAAK,KAAK,CAAC,GAAG,CAAC,mFAAmF,CAAC,EAAE,CACrG,CAAC;gBACH,CAAC;qBAAM,IAAI,CAAC,IAAI,CAAC,cAAc,EAAE,CAAC;oBACjC,OAAO,CAAC,GAAG,CACV,KAAK,KAAK,CAAC,GAAG,CAAC,iDAAiD,CAAC,EAAE,CACnE,CAAC;gBACH,CAAC;YACF,CAAC;QACF,CAAC;QAED,MAAM,OAAO,GAAG,MAAM,YAAY,CAAC;YAClC,GAAG,IAAI;YACP,gBAAgB;YAChB,SAAS,EAAE,IAAI,CAAC,SAAS;YACzB,cAAc,EAAE,IAAI,CAAC,cAAc;SACnC,CAAC,CAAC;QAEH,IAAI,QAAQ,GAAG,CAAC,CAAC;QACjB,IAAI,CAAC,OAAO,EAAE,CAAC;YACd,KAAK,MAAM,MAAM,IAAI,OAAO,EAAE,CAAC;gBAC9B,MAAM,MAAM,GAAG,MAAM,CAAC,OAAO,CAAC,MAAM,CACnC,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC,MAAM,CAAC,MAAM,IAAI,CAAC,MAAM,CAAC,OAAO,CAC7C,CAAC;gBACF,IAAI,MAAM,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;oBACvB,QAAQ,GAAG,CAAC,CAAC;oBACb,OAAO,CAAC,GAAG,CAAC,KAAK,KAAK,CAAC,qBAAqB,EAAE,EAAE,CAAC,CAAC;oBAClD,KAAK,MAAM,MAAM,IAAI,MAAM,EAAE,CAAC;wBAC7B,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,kBAAkB,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC;wBACvD,IAAI,OAAO,EAAE,CAAC;4BACb,KAAK,MAAM,OAAO,IAAI,MAAM,CAAC,QAAQ,EAAE,CAAC;gCACvC,OAAO,CAAC,GAAG,CACV,KAAK,CAAC,cAAc,CAAC,OAAO,CAAC,OAAO,EAAE,OAAO,CAAC,OAAO,CAAC,CACtD,CAAC;4BACH,CAAC;wBACF,CAAC;oBACF,CAAC;gBACF,CAAC;gBACD,OAAO,CAAC,GAAG,CACV,KAAK,CAAC,OAAO,CACZ,MAAM,CAAC,KAAK,EACZ,MAAM,CAAC,MAAM,EACb,MAAM,CAAC,MAAM,EACb,MAAM,CAAC,OAAO,CACd,CACD,CAAC;YACH,CAAC;QACF,CAAC;aAAM,CAAC;YACP,KAAK,MAAM,MAAM,IAAI,OAAO,EAAE,CAAC;gBAC9B,IAAI,MAAM,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;oBACvB,QAAQ,GAAG,CAAC,CAAC;gBACd,CAAC;YACF,CAAC;QACF,CAAC;QAED,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAC1B,WAAW,CAAC,yBAAyB,IAAI,CAAC,SAAS,EAAE,CAAC,CAAC;YACvD,OAAO,CAAC,CAAC;QACV,CAAC;QAED,OAAO,QAAQ,CAAC;IACjB,CAAC;YAAS,CAAC;QACV,IAAI,gBAAgB,IAAI,CAAC,OAAO,EAAE,CAAC;YAClC,MAAM,uBAAuB,CAC5B,IAAI,CAAC,GAAG,EACR,kBAAkB,EAClB,IAAI,CAAC,cAAc,CACnB,CAAC;QACH,CAAC;IACF,CAAC;AACF,CAAC;AAED,MAAM,KAAK,GAAG,aAAa,CAAC,OAAO,IAAI,CAAC,GAAG,CAAC,CAAC;AAE7C,IAAI,SAAS,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,KAAK,CAAC,EAAE,CAAC;IACvC,KAAK,IAAI,EAAE;SACT,IAAI,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;SAClC,KAAK,CAAC,CAAC,KAAc,EAAE,EAAE;QACzB,OAAO,CAAC,KAAK,CAAC,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC;QAC9D,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IACjB,CAAC,CAAC,CAAC;AACL,CAAC"}
|
|
1
|
+
{"version":3,"file":"cli.js","sourceRoot":"","sources":["../src/cli.ts"],"names":[],"mappings":";AACA,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AACpC,OAAO,EAAE,aAAa,EAAE,MAAM,UAAU,CAAC;AAEzC,OAAO,EAAkB,6BAA6B,EAAE,MAAM,2BAA2B,CAAC;AAE1F,OAAO,EAAE,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAC3C,OAAO,EAAE,SAAS,EAAE,MAAM,aAAa,CAAC;AACxC,OAAO,EAAE,eAAe,EAAE,MAAM,kBAAkB,CAAC;AACnD,OAAO,EAAE,0BAA0B,EAAE,MAAM,gBAAgB,CAAC;AAC5D,OAAO,EAAE,WAAW,EAAE,MAAM,eAAe,CAAC;AAC5C,OAAO,EACN,2BAA2B,EAC3B,qBAAqB,EACrB,0BAA0B,EAC1B,yBAAyB,EACzB,0BAA0B,GAC1B,MAAM,mBAAmB,CAAC;AAC3B,OAAO,EAAE,uBAAuB,EAAE,YAAY,EAAE,MAAM,gBAAgB,CAAC;AACvE,OAAO,EAAE,iBAAiB,EAAE,KAAK,EAAE,MAAM,YAAY,CAAC;AACtD,OAAO,EAAE,4BAA4B,EAAE,MAAM,eAAe,CAAC;AAE7D,4BAA4B,EAAE,CAAC;AAC/B,iBAAiB,EAAE,CAAC;AAwBpB,2DAA2D;AAC3D,MAAM,UAAU,YAAY,CAAC,IAAc;IAC1C,MAAM,GAAG,GAAG,OAAO,CAAC,GAAG,EAAE,CAAC;IAC1B,IAAI,SAAS,GAAG,cAAc,CAAC;IAC/B,IAAI,IAA2B,CAAC;IAChC,IAAI,MAA0B,CAAC;IAC/B,IAAI,cAAkC,CAAC;IACvC,IAAI,gBAAoC,CAAC;IACzC,IAAI,MAAM,GAAG,KAAK,CAAC;IACnB,IAAI,cAAc,GAAG,KAAK,CAAC;IAC3B,IAAI,IAAI,GAAG,KAAK,CAAC;IACjB,IAAI,KAA0B,CAAC;IAC/B,IAAI,QAA6B,CAAC;IAClC,IAAI,cAAc,GAAG,KAAK,CAAC;IAC3B,IAAI,SAA6B,CAAC;IAClC,IAAI,SAAS,GAAG,KAAK,CAAC;IACtB,IAAI,cAAc,GAAG,KAAK,CAAC;IAC3B,IAAI,MAAM,GAAG,KAAK,CAAC;IACnB,IAAI,UAAU,GAAG,IAAI,CAAC;IACtB,IAAI,KAAK,GAAG,OAAO,CAAC,GAAG,CAAC,gBAAgB,KAAK,GAAG,IAAI,OAAO,CAAC,GAAG,CAAC,gBAAgB,KAAK,MAAM,CAAC;IAC5F,IAAI,QAA4B,CAAC;IAEjC,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACtC,MAAM,KAAK,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;QACtB,IAAI,KAAK,KAAK,QAAQ,IAAI,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;YACvC,IAAI,GAAG,IAAI,CAAC,EAAE,CAAC,CAAc,CAAC;QAC/B,CAAC;aAAM,IAAI,KAAK,KAAK,cAAc,IAAI,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;YACpD,SAAS,GAAG,IAAI,CAAC,EAAE,CAAC,CAAW,CAAC;QACjC,CAAC;aAAM,IAAI,KAAK,KAAK,SAAS,IAAI,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;YAC/C,MAAM,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;QACpB,CAAC;aAAM,IAAI,KAAK,KAAK,YAAY,IAAI,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;YAClD,cAAc,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;QAC5B,CAAC;aAAM,IAAI,KAAK,KAAK,sBAAsB,IAAI,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;YAC5D,gBAAgB,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;QAC9B,CAAC;aAAM,IAAI,KAAK,KAAK,UAAU,EAAE,CAAC;YACjC,MAAM,GAAG,IAAI,CAAC;QACf,CAAC;aAAM,IAAI,KAAK,KAAK,mBAAmB,EAAE,CAAC;YAC1C,MAAM,GAAG,IAAI,CAAC;YACd,cAAc,GAAG,IAAI,CAAC;QACvB,CAAC;aAAM,IAAI,KAAK,KAAK,QAAQ,EAAE,CAAC;YAC/B,IAAI,GAAG,IAAI,CAAC;QACb,CAAC;aAAM,IAAI,KAAK,KAAK,mBAAmB,EAAE,CAAC;YAC1C,cAAc,GAAG,IAAI,CAAC;QACvB,CAAC;aAAM,IAAI,KAAK,KAAK,SAAS,EAAE,CAAC;YAChC,KAAK,GAAG,IAAI,CAAC;QACd,CAAC;aAAM,IAAI,KAAK,KAAK,YAAY,EAAE,CAAC;YACnC,KAAK,GAAG,KAAK,CAAC;QACf,CAAC;aAAM,IAAI,KAAK,KAAK,eAAe,EAAE,CAAC;YACtC,QAAQ,GAAG,KAAK,CAAC;QAClB,CAAC;aAAM,IAAI,KAAK,KAAK,cAAc,IAAI,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;YACpD,MAAM,MAAM,GAAG,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;YACjC,IAAI,MAAM,CAAC,QAAQ,CAAC,MAAM,CAAC,IAAI,MAAM,GAAG,CAAC,EAAE,CAAC;gBAC3C,SAAS,GAAG,MAAM,CAAC;YACpB,CAAC;QACF,CAAC;aAAM,IAAI,KAAK,KAAK,cAAc,EAAE,CAAC;YACrC,SAAS,GAAG,IAAI,CAAC;QAClB,CAAC;aAAM,IAAI,KAAK,KAAK,oBAAoB,EAAE,CAAC;YAC3C,cAAc,GAAG,IAAI,CAAC;QACvB,CAAC;aAAM,IAAI,KAAK,KAAK,UAAU,EAAE,CAAC;YACjC,MAAM,GAAG,IAAI,CAAC;QACf,CAAC;aAAM,IAAI,KAAK,KAAK,kBAAkB,EAAE,CAAC;YACzC,UAAU,GAAG,KAAK,CAAC;QACpB,CAAC;aAAM,IAAI,KAAK,KAAK,SAAS,EAAE,CAAC;YAChC,KAAK,GAAG,IAAI,CAAC;QACd,CAAC;aAAM,IAAI,KAAK,KAAK,aAAa,EAAE,CAAC;YACpC,MAAM,KAAK,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC;YACxB,IAAI,CAAC,KAAK,IAAI,KAAK,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE,CAAC;gBACrC,MAAM,IAAI,KAAK,CAAC,gDAAgD,CAAC,CAAC;YACnE,CAAC;YACD,QAAQ,GAAG,KAAK,CAAC;YACjB,KAAK,GAAG,IAAI,CAAC;QACd,CAAC;aAAM,IAAI,KAAK,IAAI,CAAC,KAAK,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE,CAAC;YAC5C,MAAM,GAAG,KAAK,CAAC;QAChB,CAAC;IACF,CAAC;IAED,IAAI,IAAI,EAAE,CAAC;QACV,IAAI,GAAG,IAAI,IAAI,QAAQ,CAAC;QACxB,MAAM,GAAG,IAAI,CAAC;QACd,KAAK,GAAG,KAAK,IAAI,IAAI,CAAC;QACtB,QAAQ,GAAG,QAAQ,IAAI,IAAI,CAAC;IAC7B,CAAC;IAED,IAAI,KAAK,EAAE,CAAC;QACX,cAAc,GAAG,IAAI,CAAC;QACtB,OAAO,CAAC,GAAG,CAAC,gBAAgB,GAAG,GAAG,CAAC;QACnC,OAAO,CAAC,GAAG,CAAC,kBAAkB,GAAG,OAAO,CAAC,GAAG,CAAC,kBAAkB,IAAI,GAAG,CAAC;QACvE,OAAO,CAAC,GAAG,CAAC,wBAAwB,GAAG,OAAO,CAAC,GAAG,CAAC,wBAAwB,IAAI,GAAG,CAAC;IACpF,CAAC;IAED,OAAO;QACN,GAAG;QACH,SAAS;QACT,IAAI;QACJ,MAAM;QACN,cAAc;QACd,gBAAgB;QAChB,MAAM;QACN,cAAc;QACd,IAAI;QACJ,KAAK;QACL,QAAQ;QACR,cAAc;QACd,SAAS,EAAE,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,SAAS;QACpC,SAAS;QACT,cAAc;QACd,MAAM;QACN,UAAU;QACV,KAAK;QACL,QAAQ,EAAE,QAAQ,CAAC,CAAC,CAAC,OAAO,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC,CAAC,CAAC,SAAS;KACvD,CAAC;AACH,CAAC;AAED,KAAK,UAAU,uBAAuB,CACrC,GAAW,EACX,kBAAsC,EACtC,cAAuB;IAEvB,IAAI,cAAc,EAAE,CAAC;QACpB,OAAO;IACR,CAAC;IAED,IAAI,kBAAkB,EAAE,CAAC;QACxB,IAAI,CAAC;YACJ,MAAM,qBAAqB,CAAC,kBAAkB,CAAC,CAAC;QACjD,CAAC;QAAC,MAAM,CAAC;YACR,cAAc;QACf,CAAC;IACF,CAAC;IAED,MAAM,aAAa,GAAG,MAAM,2BAA2B,CAAC,GAAG,CAAC,CAAC;IAC7D,IAAI,aAAa,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC9B,OAAO,CAAC,GAAG,CAAC,+CAA+C,aAAa,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC;IAC1F,CAAC;AACF,CAAC;AAED,KAAK,UAAU,IAAI;IAClB,IAAI,IAAmB,CAAC;IACxB,IAAI,CAAC;QACJ,IAAI,GAAG,YAAY,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC;IACnC,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QAChB,OAAO,CAAC,KAAK,CAAC,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC;QAC9D,OAAO,CAAC,CAAC;IACV,CAAC;IAED,IAAI,IAAI,CAAC,MAAM,EAAE,CAAC;QACjB,MAAM,MAAM,GAAG,SAAS,EAAE,CAAC;QAC3B,KAAK,MAAM,OAAO,IAAI,MAAM,CAAC,QAAQ,EAAE,CAAC;YACvC,OAAO,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;QACtB,CAAC;QACD,OAAO,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IAC1B,CAAC;IAED,IAAI,IAAI,CAAC,QAAQ,EAAE,CAAC;QACnB,0BAA0B,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAC;IAC3C,CAAC;IAED,MAAM,OAAO,GAAG,OAAO,CAAC,GAAG,CAAC,gBAAgB,KAAK,GAAG,CAAC;IACrD,MAAM,OAAO,GACZ,IAAI,CAAC,KAAK,IAAI,OAAO,CAAC,GAAG,CAAC,kBAAkB,KAAK,GAAG,IAAI,OAAO,CAAC,GAAG,CAAC,gBAAgB,KAAK,GAAG,CAAC;IAC9F,MAAM,gBAAgB,GACrB,IAAI,CAAC,gBAAgB,EAAE,IAAI,EAAE;QAC7B,OAAO,CAAC,GAAG,CAAC,6BAA6B,EAAE,IAAI,EAAE;QACjD,CAAC,IAAI,CAAC,IAAI,IAAI,IAAI,CAAC,KAAK,IAAI,CAAC,IAAI,CAAC,MAAM,IAAI,CAAC,IAAI,CAAC,cAAc,CAAC;YAChE,CAAC,CAAC,0BAA0B,EAAE;YAC9B,CAAC,CAAC,SAAS,CAAC,CAAC;IACf,MAAM,kBAAkB,GAAG,gBAAgB;QAC1C,CAAC,CAAC,yBAAyB,CAAC,gBAAgB,CAAC;QAC7C,CAAC,CAAC,SAAS,CAAC;IAEb,IAAI,CAAC;QACJ,IAAI,IAAI,CAAC,IAAI,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,cAAc,EAAE,CAAC;YAC9C,OAAO,CAAC,KAAK,CAAC,sDAAsD,CAAC,CAAC;YACtE,OAAO,CAAC,CAAC;QACV,CAAC;QAED,IAAI,IAAI,CAAC,IAAI,EAAE,CAAC;YACf,IAAI,CAAC;gBACJ,MAAM,0BAA0B,CAAC,IAAI,CAAC,GAAG,EAAE,IAAI,CAAC,SAAS,CAAC,CAAC;YAC5D,CAAC;YAAC,OAAO,KAAK,EAAE,CAAC;gBAChB,OAAO,CAAC,KAAK,CAAC,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC;gBAC9D,OAAO,CAAC,CAAC;YACV,CAAC;YAED,MAAM,cAAc,GAAG,OAAO,CAAC,GAAG,CAAC,yBAAyB,KAAK,GAAG,CAAC;YACrE,MAAM,gBAAgB,GACrB,IAAI,CAAC,QAAQ,KAAK,KAAK;gBACvB,OAAO,CAAC,GAAG,CAAC,sBAAsB,KAAK,GAAG;gBAC1C,OAAO,CAAC,GAAG,CAAC,sBAAsB,KAAK,MAAM,CAAC;YAC/C,IAAI,gBAAgB,IAAI,CAAC,cAAc,EAAE,CAAC;gBACzC,OAAO,CAAC,KAAK,CACZ,4JAA4J,CAC5J,CAAC;gBACF,OAAO,CAAC,CAAC;YACV,CAAC;YAED,uBAAuB,EAAE,CAAC;YAC1B,IAAI,CAAC,OAAO,EAAE,CAAC;gBACd,MAAM,OAAO,GAAG,MAAM,6BAA6B,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;gBAC9D,IAAI,OAAO,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;oBACxB,OAAO,CAAC,GAAG,CACV,KAAK,CAAC,IAAI,CAAC,WAAW,OAAO,CAAC,MAAM,kDAAkD,CAAC,CACvF,CAAC;gBACH,CAAC;gBACD,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;gBAC9D,IAAI,kBAAkB,EAAE,CAAC;oBACxB,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,aAAa,CAAC,kBAAkB,CAAC,CAAC,CAAC;gBACtD,CAAC;gBACD,IAAI,gBAAgB,EAAE,CAAC;oBACtB,OAAO,CAAC,IAAI,CACX,KAAK,CAAC,IAAI,CAAC,0EAA0E,CAAC,CACtF,CAAC;gBACH,CAAC;gBACD,IAAI,IAAI,CAAC,KAAK,IAAI,OAAO,CAAC,GAAG,CAAC,kBAAkB,KAAK,GAAG,EAAE,CAAC;oBAC1D,IAAI,CAAC,IAAI,CAAC,cAAc,EAAE,CAAC;wBAC1B,OAAO,CAAC,GAAG,CAAC,KAAK,KAAK,CAAC,GAAG,CAAC,iDAAiD,CAAC,EAAE,CAAC,CAAC;oBAClF,CAAC;yBAAM,CAAC;wBACP,OAAO,CAAC,GAAG,CAAC,KAAK,KAAK,CAAC,GAAG,CAAC,wBAAwB,CAAC,EAAE,CAAC,CAAC;oBACzD,CAAC;oBACD,OAAO,CAAC,GAAG,CACV,KAAK,KAAK,CAAC,GAAG,CAAC,mFAAmF,CAAC,EAAE,CACrG,CAAC;gBACH,CAAC;qBAAM,IAAI,CAAC,IAAI,CAAC,cAAc,EAAE,CAAC;oBACjC,OAAO,CAAC,GAAG,CAAC,KAAK,KAAK,CAAC,GAAG,CAAC,iDAAiD,CAAC,EAAE,CAAC,CAAC;gBAClF,CAAC;YACF,CAAC;QACF,CAAC;aAAM,IAAI,IAAI,CAAC,KAAK,IAAI,CAAC,OAAO,IAAI,kBAAkB,EAAE,CAAC;YACzD,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC;YACnC,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,aAAa,CAAC,kBAAkB,CAAC,CAAC,CAAC;QACtD,CAAC;QAED,MAAM,OAAO,GAAG,MAAM,YAAY,CAAC;YAClC,GAAG,IAAI;YACP,gBAAgB;YAChB,SAAS,EAAE,IAAI,CAAC,SAAS;YACzB,cAAc,EAAE,IAAI,CAAC,cAAc;YACnC,KAAK,EAAE,IAAI,CAAC,KAAK;YACjB,QAAQ,EAAE,IAAI,CAAC,QAAQ;SACvB,CAAC,CAAC;QAEH,IAAI,QAAQ,GAAG,CAAC,CAAC;QACjB,IAAI,CAAC,OAAO,EAAE,CAAC;YACd,KAAK,MAAM,MAAM,IAAI,OAAO,EAAE,CAAC;gBAC9B,MAAM,MAAM,GAAG,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC,MAAM,CAAC,MAAM,IAAI,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;gBACpF,IAAI,MAAM,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;oBACvB,QAAQ,GAAG,CAAC,CAAC;oBACb,OAAO,CAAC,GAAG,CAAC,KAAK,KAAK,CAAC,qBAAqB,EAAE,EAAE,CAAC,CAAC;oBAClD,KAAK,MAAM,MAAM,IAAI,MAAM,EAAE,CAAC;wBAC7B,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,kBAAkB,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC;wBACvD,IAAI,OAAO,EAAE,CAAC;4BACb,KAAK,MAAM,OAAO,IAAI,MAAM,CAAC,QAAQ,EAAE,CAAC;gCACvC,OAAO,CAAC,GAAG,CACV,KAAK,CAAC,cAAc,CACnB,OAAO,CAAC,OAAO,EACf,OAAO,CAAC,OAAO,EACf,OAAO,CAAC,QAAQ,EAChB,OAAO,CAAC,QAAQ,CAChB,CACD,CAAC;4BACH,CAAC;wBACF,CAAC;wBACD,IAAI,IAAI,CAAC,KAAK,IAAI,MAAM,CAAC,cAAc,EAAE,CAAC;4BACzC,OAAO,CAAC,GAAG,CAAC,SAAS,KAAK,CAAC,GAAG,CAAC,MAAM,CAAC,cAAc,CAAC,EAAE,CAAC,CAAC;wBAC1D,CAAC;oBACF,CAAC;gBACF,CAAC;gBACD,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC,KAAK,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC;YACxF,CAAC;YAED,IAAI,IAAI,CAAC,KAAK,IAAI,kBAAkB,EAAE,CAAC;gBACtC,OAAO,CAAC,GAAG,CAAC,KAAK,KAAK,CAAC,GAAG,CAAC,kBAAkB,kBAAkB,EAAE,CAAC,EAAE,CAAC,CAAC;YACvE,CAAC;YAED,IAAI,IAAI,CAAC,UAAU,IAAI,OAAO,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;gBAC3C,IAAI,CAAC;oBACJ,MAAM,UAAU,GAAG,MAAM,eAAe,CAAC,OAAO,EAAE;wBACjD,IAAI,EAAE,IAAI,CAAC,IAAI;wBACf,SAAS,EAAE,IAAI,CAAC,SAAS;qBACzB,CAAC,CAAC;oBACH,OAAO,CAAC,GAAG,CAAC,KAAK,KAAK,CAAC,GAAG,CAAC,gBAAgB,UAAU,EAAE,CAAC,EAAE,CAAC,CAAC;gBAC7D,CAAC;gBAAC,OAAO,KAAK,EAAE,CAAC;oBAChB,OAAO,CAAC,IAAI,CACX,KAAK,CAAC,IAAI,CACT,uBAAuB,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,CAC/E,CACD,CAAC;gBACH,CAAC;YACF,CAAC;QACF,CAAC;aAAM,CAAC;YACP,KAAK,MAAM,MAAM,IAAI,OAAO,EAAE,CAAC;gBAC9B,IAAI,MAAM,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;oBACvB,QAAQ,GAAG,CAAC,CAAC;gBACd,CAAC;YACF,CAAC;QACF,CAAC;QAED,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YAC1B,WAAW,CAAC,yBAAyB,IAAI,CAAC,SAAS,EAAE,CAAC,CAAC;YACvD,OAAO,CAAC,CAAC;QACV,CAAC;QAED,OAAO,QAAQ,CAAC;IACjB,CAAC;YAAS,CAAC;QACV,IAAI,gBAAgB,IAAI,CAAC,OAAO,EAAE,CAAC;YAClC,MAAM,uBAAuB,CAAC,IAAI,CAAC,GAAG,EAAE,kBAAkB,EAAE,IAAI,CAAC,cAAc,CAAC,CAAC;QAClF,CAAC;IACF,CAAC;AACF,CAAC;AAED,MAAM,KAAK,GAAG,aAAa,CAAC,OAAO,IAAI,CAAC,GAAG,CAAC,CAAC;AAE7C,IAAI,SAAS,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,KAAK,CAAC,EAAE,CAAC;IACvC,KAAK,IAAI,EAAE;SACT,IAAI,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;SAClC,KAAK,CAAC,CAAC,KAAc,EAAE,EAAE;QACzB,OAAO,CAAC,KAAK,CAAC,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC;QAC9D,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IACjB,CAAC,CAAC,CAAC;AACL,CAAC"}
|
|
@@ -0,0 +1,66 @@
|
|
|
1
|
+
import type { AgentTrace } from "@post-print/agent-harness";
|
|
2
|
+
import type { AgentScenario, AssertionFailure, JudgeVerdictResult, ScenarioResult } from "./types.js";
|
|
3
|
+
export interface DebugRerunOptions {
|
|
4
|
+
cliPath: string;
|
|
5
|
+
cwd: string;
|
|
6
|
+
suitesDir: string;
|
|
7
|
+
suite: string;
|
|
8
|
+
scenario: string;
|
|
9
|
+
live: boolean;
|
|
10
|
+
host?: string;
|
|
11
|
+
judge?: boolean;
|
|
12
|
+
worktree?: boolean;
|
|
13
|
+
timeoutMs?: number;
|
|
14
|
+
noTimeout?: boolean;
|
|
15
|
+
allowUserInput?: boolean;
|
|
16
|
+
debugDir?: string;
|
|
17
|
+
keepRecordings?: boolean;
|
|
18
|
+
}
|
|
19
|
+
export interface DebugEnvironmentSnapshot {
|
|
20
|
+
nodeVersion: string;
|
|
21
|
+
bunVersion?: string;
|
|
22
|
+
packageVersion: string;
|
|
23
|
+
host?: string;
|
|
24
|
+
suite: string;
|
|
25
|
+
scenario: string;
|
|
26
|
+
cursorApiKeySet: boolean;
|
|
27
|
+
cursorAgentModel?: string;
|
|
28
|
+
cursorJudgeModel?: string;
|
|
29
|
+
cursorJudgeTemperature?: string;
|
|
30
|
+
timeoutMs?: number;
|
|
31
|
+
worktree?: boolean;
|
|
32
|
+
isolateLive?: boolean;
|
|
33
|
+
agentTestEnv: Record<string, string | undefined>;
|
|
34
|
+
}
|
|
35
|
+
export interface WriteDebugBundleOptions {
|
|
36
|
+
dir: string;
|
|
37
|
+
result: ScenarioResult;
|
|
38
|
+
trace?: AgentTrace;
|
|
39
|
+
scenario: AgentScenario;
|
|
40
|
+
environment: DebugEnvironmentSnapshot;
|
|
41
|
+
rerun: DebugRerunOptions;
|
|
42
|
+
}
|
|
43
|
+
/** Shell-quote a single argument for POSIX sh. */
|
|
44
|
+
export declare function shellQuote(value: string): string;
|
|
45
|
+
/** Flatten text for a single `# …` shell comment (no newline breakout). */
|
|
46
|
+
export declare function shellCommentText(value: string): string;
|
|
47
|
+
export declare function buildRerunCommand(options: DebugRerunOptions): string;
|
|
48
|
+
export declare function collectDebugEnvironment(options: {
|
|
49
|
+
suite: string;
|
|
50
|
+
scenario: string;
|
|
51
|
+
packageVersion: string;
|
|
52
|
+
host?: string;
|
|
53
|
+
timeoutMs?: number;
|
|
54
|
+
worktree?: boolean;
|
|
55
|
+
isolateLive?: boolean;
|
|
56
|
+
}): DebugEnvironmentSnapshot;
|
|
57
|
+
export declare function formatTranscriptMarkdown(options: {
|
|
58
|
+
scenario: AgentScenario;
|
|
59
|
+
trace?: AgentTrace;
|
|
60
|
+
failures: AssertionFailure[];
|
|
61
|
+
judgeVerdicts?: JudgeVerdictResult[];
|
|
62
|
+
}): string;
|
|
63
|
+
export declare function getDebugBundleDir(stagingSessionId: string, suiteName: string, scenarioName: string, getSessionRoot: (sessionId: string) => string): string;
|
|
64
|
+
/** Persist a failed-scenario debug bundle (trace, failures, transcript, judge, env, rerun). */
|
|
65
|
+
export declare function writeDebugBundle(options: WriteDebugBundleOptions): Promise<string>;
|
|
66
|
+
//# sourceMappingURL=debug-bundle.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"debug-bundle.d.ts","sourceRoot":"","sources":["../src/debug-bundle.ts"],"names":[],"mappings":"AAGA,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,2BAA2B,CAAC;AAG5D,OAAO,KAAK,EACX,aAAa,EACb,gBAAgB,EAChB,kBAAkB,EAClB,cAAc,EACd,MAAM,YAAY,CAAC;AAEpB,MAAM,WAAW,iBAAiB;IACjC,OAAO,EAAE,MAAM,CAAC;IAChB,GAAG,EAAE,MAAM,CAAC;IACZ,SAAS,EAAE,MAAM,CAAC;IAClB,KAAK,EAAE,MAAM,CAAC;IACd,QAAQ,EAAE,MAAM,CAAC;IACjB,IAAI,EAAE,OAAO,CAAC;IACd,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,KAAK,CAAC,EAAE,OAAO,CAAC;IAChB,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,SAAS,CAAC,EAAE,OAAO,CAAC;IACpB,cAAc,CAAC,EAAE,OAAO,CAAC;IACzB,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,cAAc,CAAC,EAAE,OAAO,CAAC;CACzB;AAED,MAAM,WAAW,wBAAwB;IACxC,WAAW,EAAE,MAAM,CAAC;IACpB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,cAAc,EAAE,MAAM,CAAC;IACvB,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,KAAK,EAAE,MAAM,CAAC;IACd,QAAQ,EAAE,MAAM,CAAC;IACjB,eAAe,EAAE,OAAO,CAAC;IACzB,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,sBAAsB,CAAC,EAAE,MAAM,CAAC;IAChC,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,WAAW,CAAC,EAAE,OAAO,CAAC;IACtB,YAAY,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,GAAG,SAAS,CAAC,CAAC;CACjD;AAED,MAAM,WAAW,uBAAuB;IACvC,GAAG,EAAE,MAAM,CAAC;IACZ,MAAM,EAAE,cAAc,CAAC;IACvB,KAAK,CAAC,EAAE,UAAU,CAAC;IACnB,QAAQ,EAAE,aAAa,CAAC;IACxB,WAAW,EAAE,wBAAwB,CAAC;IACtC,KAAK,EAAE,iBAAiB,CAAC;CACzB;AAeD,kDAAkD;AAClD,wBAAgB,UAAU,CAAC,KAAK,EAAE,MAAM,GAAG,MAAM,CAQhD;AAED,2EAA2E;AAC3E,wBAAgB,gBAAgB,CAAC,KAAK,EAAE,MAAM,GAAG,MAAM,CAEtD;AAED,wBAAgB,iBAAiB,CAAC,OAAO,EAAE,iBAAiB,GAAG,MAAM,CAwCpE;AAED,wBAAgB,uBAAuB,CAAC,OAAO,EAAE;IAChD,KAAK,EAAE,MAAM,CAAC;IACd,QAAQ,EAAE,MAAM,CAAC;IACjB,cAAc,EAAE,MAAM,CAAC;IACvB,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,SAAS,CAAC,EAAE,MAAM,CAAC;IACnB,QAAQ,CAAC,EAAE,OAAO,CAAC;IACnB,WAAW,CAAC,EAAE,OAAO,CAAC;CACtB,GAAG,wBAAwB,CAwB3B;AAaD,wBAAgB,wBAAwB,CAAC,OAAO,EAAE;IACjD,QAAQ,EAAE,aAAa,CAAC;IACxB,KAAK,CAAC,EAAE,UAAU,CAAC;IACnB,QAAQ,EAAE,gBAAgB,EAAE,CAAC;IAC7B,aAAa,CAAC,EAAE,kBAAkB,EAAE,CAAC;CACrC,GAAG,MAAM,CAsFT;AAED,wBAAgB,iBAAiB,CAChC,gBAAgB,EAAE,MAAM,EACxB,SAAS,EAAE,MAAM,EACjB,YAAY,EAAE,MAAM,EACpB,cAAc,EAAE,CAAC,SAAS,EAAE,MAAM,KAAK,MAAM,GAC3C,MAAM,CAMR;AAED,+FAA+F;AAC/F,wBAAsB,gBAAgB,CAAC,OAAO,EAAE,uBAAuB,GAAG,OAAO,CAAC,MAAM,CAAC,CAgFxF"}
|