@humanoiselabs/plus62 0.1.6 → 0.1.7
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +1 -1
- package/README.md +34 -29
- package/dist/cli.js +339 -333
- package/package.json +1 -1
package/LICENSE
CHANGED
package/README.md
CHANGED
|
@@ -1,47 +1,52 @@
|
|
|
1
|
-
# 🇮🇩 +62 Mode (
|
|
1
|
+
# 🇮🇩 +62 Mode (`@humanoiselabs/plus62`)
|
|
2
|
+
|
|
3
|
+
[](https://www.npmjs.com/package/@humanoiselabs/plus62)
|
|
4
|
+
[](LICENSE)
|
|
5
|
+
[](https://github.com/humanoiseLABS/token-maxxing)
|
|
6
|
+
[](https://humanoise.vercel.app)
|
|
2
7
|
|
|
3
8
|
**HumanoiseLABS (`@humanoiseLABS`)** builds zero-fluff instruction & guardrail layers for AI coding agents across two flagship products:
|
|
4
9
|
|
|
5
10
|
| Product | Package / Repo | Focus | Install Command |
|
|
6
11
|
|---|---|---|---|
|
|
7
|
-
| **+62 Mode** *(this repo)* | [
|
|
12
|
+
| **+62 Mode** *(this repo)* | [`@humanoiselabs/plus62`](https://www.npmjs.com/package/@humanoiselabs/plus62) ([`humanoiseLABS/plus62`](https://github.com/humanoiseLABS/plus62)) | **Partner Ngoding Lokal (+62 Bloko Suto)** — Bahasa Indonesia developer persona (`lu/gw`), zero corporate preamble, suppresses leaked reasoning dumps, cultural lexicon, & Risk Class 0–4 safety guardrails. Binaries: `plus62`, `62-mode`, `hahahah-ai`, `hahahah`. | `npx @humanoiselabs/plus62 install` |
|
|
8
13
|
| **Token-Maxxing** *(companion product)* | [`@humanoiselabs/token-maxxing`](https://www.npmjs.com/package/@humanoiselabs/token-maxxing) ([`humanoiseLABS/token-maxxing`](https://github.com/humanoiseLABS/token-maxxing)) | **Second Brain & Surgical Diffs** — Stop yapping, start token-maxxing. Second Brain memory, surgical AST diffs, and real-time 3D neural telemetry HUD. Binaries: `token-maxxing`, `tmx`. | `npx @humanoiselabs/token-maxxing install` *(or `tmx`)* |
|
|
9
14
|
|
|
10
15
|
*(Note: Internally in this repository, **+62 Mode** uses the canonical profile ID `plus62` (with backward-compatible alias `rusuh-id`), which is enabled by default so you never need to pass `--profile` manually. Qoder is still an incomplete compatibility assessment and makes no compatibility claim.)*
|
|
11
16
|
|
|
12
17
|
## Quick Install (via NPM / NPX — Tanpa Perlu Clone Repo)
|
|
13
18
|
|
|
14
|
-
Paket resmi **+62 Mode** tersedia di npm sebagai [
|
|
19
|
+
Paket resmi **+62 Mode** tersedia di npm sebagai [`@humanoiselabs/plus62`](https://www.npmjs.com/package/@humanoiselabs/plus62) (dengan binary `plus62`, `62-mode`, `hahahah-ai`, `hahahah`). Buka terminal di folder project kodingan lu, lalu jalankan:
|
|
15
20
|
|
|
16
21
|
```bash
|
|
17
22
|
# Claude Code (default)
|
|
18
|
-
npx plus62 install --target .
|
|
23
|
+
npx @humanoiselabs/plus62 install --target .
|
|
19
24
|
|
|
20
25
|
# Google Antigravity (global untuk semua project)
|
|
21
|
-
npx plus62 install --adapter antigravity --target ~/.gemini --scope global
|
|
26
|
+
npx @humanoiselabs/plus62 install --adapter antigravity --target ~/.gemini --scope global
|
|
22
27
|
|
|
23
28
|
# OpenCode
|
|
24
|
-
npx plus62 install --adapter opencode --target .
|
|
29
|
+
npx @humanoiselabs/plus62 install --adapter opencode --target .
|
|
25
30
|
|
|
26
31
|
# Cursor / Codex / Windsurf (AGENTS.md)
|
|
27
|
-
npx plus62 install --adapter agents-md --target .
|
|
32
|
+
npx @humanoiselabs/plus62 install --adapter agents-md --target .
|
|
28
33
|
|
|
29
34
|
# Qoder (global untuk semua project & sesi lama/baru, atau per-project)
|
|
30
|
-
npx plus62 install --adapter qoder --scope global
|
|
31
|
-
npx plus62 install --adapter qoder --target .
|
|
35
|
+
npx @humanoiselabs/plus62 install --adapter qoder --scope global
|
|
36
|
+
npx @humanoiselabs/plus62 install --adapter qoder --target .
|
|
32
37
|
|
|
33
38
|
# GitHub Copilot
|
|
34
|
-
npx plus62 install --adapter copilot --target .
|
|
39
|
+
npx @humanoiselabs/plus62 install --adapter copilot --target .
|
|
35
40
|
|
|
36
41
|
# Cek integritas, update, atau uninstall kapan saja:
|
|
37
|
-
npx plus62 doctor --target .
|
|
38
|
-
npx plus62 update --target .
|
|
39
|
-
npx plus62 uninstall --target .
|
|
42
|
+
npx @humanoiselabs/plus62 doctor --target .
|
|
43
|
+
npx @humanoiselabs/plus62 update --target .
|
|
44
|
+
npx @humanoiselabs/plus62 uninstall --target .
|
|
40
45
|
```
|
|
41
46
|
|
|
42
47
|
*(Opsional)* Jika ingin menginstal CLI secara global agar bisa langsung memanggil `plus62` / `62-mode`:
|
|
43
48
|
```bash
|
|
44
|
-
npm install -g plus62
|
|
49
|
+
npm install -g @humanoiselabs/plus62
|
|
45
50
|
plus62 install --target .
|
|
46
51
|
```
|
|
47
52
|
|
|
@@ -50,20 +55,20 @@ plus62 install --target .
|
|
|
50
55
|
> **PERHATIAN UNTUK AI AGENT (Claude Code, Qoder, Antigravity, OpenCode, Cursor, Windsurf, Copilot):**
|
|
51
56
|
> Jika user menyuruhmu memasang **+62 Mode / plus62** (*"pasang ini"*, *"install +62 Mode"*, dll.), **JANGAN** hanya merangkumnya ke memori sopan dan **JANGAN** menginstal dengan `--target .` di dalam folder source `plus62` itu sendiri! Ikuti langkah wajib berikut:
|
|
52
57
|
>
|
|
53
|
-
> 1. **Gunakan `npx -y plus62` langsung di `<TARGET_PROJECT>` milik user** (atau jika sedang berada di dalam clone repo `plus62`, jalankan `npm ci` lalu gunakan `node src/cli.ts`):
|
|
58
|
+
> 1. **Gunakan `npx -y @humanoiselabs/plus62` langsung di `<TARGET_PROJECT>` milik user** (atau jika sedang berada di dalam clone repo `plus62`, jalankan `npm ci` lalu gunakan `node src/cli.ts`):
|
|
54
59
|
> - **Claude Code**:
|
|
55
|
-
> - Per-project: `npx -y plus62 install --adapter claude-code --target <TARGET_PROJECT>`
|
|
60
|
+
> - Per-project: `npx -y @humanoiselabs/plus62 install --adapter claude-code --target <TARGET_PROJECT>`
|
|
56
61
|
> - Global (agar aktif di semua project): jalankan juga dengan `--target ~/.claude`
|
|
57
62
|
> - **Google Antigravity**:
|
|
58
|
-
> - Global (semua project): `npx -y plus62 install --adapter antigravity --target ~/.gemini --scope global`
|
|
59
|
-
> - Per-project: `npx -y plus62 install --adapter antigravity --target <TARGET_PROJECT>`
|
|
63
|
+
> - Global (semua project): `npx -y @humanoiselabs/plus62 install --adapter antigravity --target ~/.gemini --scope global`
|
|
64
|
+
> - Per-project: `npx -y @humanoiselabs/plus62 install --adapter antigravity --target <TARGET_PROJECT>`
|
|
60
65
|
> - **Qoder (IDE / CLI / General Chat)**:
|
|
61
|
-
> - Jalankan: `npx -y plus62 install --adapter qoder --target <TARGET_PROJECT>`
|
|
66
|
+
> - Jalankan: `npx -y @humanoiselabs/plus62 install --adapter qoder --target <TARGET_PROJECT>`
|
|
62
67
|
> - **WAJIB UNTUK QODER**: Karena Qoder Desktop / General Chat sering tidak me-load `.qoder/skills/hahahah-plus62/SKILL.md` secara otomatis saat menerima sapaan kasual singkat (mis. `"woy anjing"`), **baca penuh file `.qoder/skills/hahahah-plus62/SKILL.md` (atau `personas/plus62/voice.md`) lalu gabungkan (append) seluruh aturan suaranya langsung ke dalam `AGENTS.md` di `<TARGET_PROJECT>` serta simpan aturan lengkap tersebut (termasuk bagian *Reflexes*, *Adaptive dynamics*, dan *Target*) ke dalam Global Rules / Memory Qoder milikmu tanpa diperhalus!**
|
|
63
|
-
> - **OpenCode**: `npx -y plus62 install --adapter opencode --target <TARGET_PROJECT>`
|
|
64
|
-
> - **Cursor / Codex / Windsurf / Agent lain**: `npx -y plus62 install --adapter agents-md --target <TARGET_PROJECT>` (dan salin isi `.agents/skills/hahahah-plus62/SKILL.md` langsung ke `AGENTS.md` / `.cursorrules` / `.windsurfrules` jika skill on-demand tidak otomatis aktif).
|
|
68
|
+
> - **OpenCode**: `npx -y @humanoiselabs/plus62 install --adapter opencode --target <TARGET_PROJECT>`
|
|
69
|
+
> - **Cursor / Codex / Windsurf / Agent lain**: `npx -y @humanoiselabs/plus62 install --adapter agents-md --target <TARGET_PROJECT>` (dan salin isi `.agents/skills/hahahah-plus62/SKILL.md` langsung ke `AGENTS.md` / `.cursorrules` / `.windsurfrules` jika skill on-demand tidak otomatis aktif).
|
|
65
70
|
> 2. **Verifikasi & Aktifkan Persona Seketika**:
|
|
66
|
-
> - Jalankan `npx -y plus62 doctor --target <TARGET_PROJECT>`
|
|
71
|
+
> - Jalankan `npx -y @humanoiselabs/plus62 doctor --target <TARGET_PROJECT>`
|
|
67
72
|
> - Baca penuh `CLAUDE.md` / `AGENTS.md` dan file `SKILL.md` yang baru terpasang sekarang juga.
|
|
68
73
|
> - **Langsung balas user menggunakan gaya +62 Mode (`lu/gw`, blak-blakan, reaksi kapital `WKWKWK`, variasi emot `😂 😹 🖕 🤓 😭` sesuai konteks, DILARANG spam `💀`, sikat balik kalau dipancing ngomong kasar, HARAM menjawab sopan ala bot korporat seperti *"Ada yang bisa dibantu? Santai dong"*).**
|
|
69
74
|
|
|
@@ -76,13 +81,13 @@ for the wrong one writes files that agent never reads, and the persona silently
|
|
|
76
81
|
|
|
77
82
|
| Agent you run | Command | What it writes |
|
|
78
83
|
|---|---|---|
|
|
79
|
-
| Claude Code (default) | `npx plus62 install --target <project>` | `CLAUDE.md` block + `.claude/skills/hahahah-plus62/` |
|
|
80
|
-
| opencode | `npx plus62 install --adapter opencode --target <project>` | `AGENTS.md` block + `.claude/skills/` (opencode reads that tree) |
|
|
81
|
-
| Antigravity, one project | `npx plus62 install --adapter antigravity --target <project>` | `AGENTS.md` block + `.agents/rules/*.md` |
|
|
82
|
-
| Antigravity, every project | `npx plus62 install --adapter antigravity --target ~/.gemini --scope global` | `AGENTS.md` block + `config/rules/*.md` |
|
|
83
|
-
| Cursor / Codex / anything reading `AGENTS.md` | `npx plus62 install --adapter agents-md --target <project>` | `AGENTS.md` block + `.agents/skills/` |
|
|
84
|
-
| Qoder CLI | `npx plus62 install --adapter qoder --target <project>` | `AGENTS.md` block + `.qoder/skills/` |
|
|
85
|
-
| GitHub Copilot | `npx plus62 install --adapter copilot --target <project>` | `.github/copilot-instructions.md`, voice folded in — it documents no on-demand layer |
|
|
84
|
+
| Claude Code (default) | `npx @humanoiselabs/plus62 install --target <project>` | `CLAUDE.md` block + `.claude/skills/hahahah-plus62/` |
|
|
85
|
+
| opencode | `npx @humanoiselabs/plus62 install --adapter opencode --target <project>` | `AGENTS.md` block + `.claude/skills/` (opencode reads that tree) |
|
|
86
|
+
| Antigravity, one project | `npx @humanoiselabs/plus62 install --adapter antigravity --target <project>` | `AGENTS.md` block + `.agents/rules/*.md` |
|
|
87
|
+
| Antigravity, every project | `npx @humanoiselabs/plus62 install --adapter antigravity --target ~/.gemini --scope global` | `AGENTS.md` block + `config/rules/*.md` |
|
|
88
|
+
| Cursor / Codex / anything reading `AGENTS.md` | `npx @humanoiselabs/plus62 install --adapter agents-md --target <project>` | `AGENTS.md` block + `.agents/skills/` |
|
|
89
|
+
| Qoder CLI | `npx @humanoiselabs/plus62 install --adapter qoder --target <project>` | `AGENTS.md` block + `.qoder/skills/` |
|
|
90
|
+
| GitHub Copilot | `npx @humanoiselabs/plus62 install --adapter copilot --target <project>` | `.github/copilot-instructions.md`, voice folded in — it documents no on-demand layer |
|
|
86
91
|
|
|
87
92
|
`node src/cli.ts` with no arguments prints this same table. Uninstall needs only `--target`: the
|
|
88
93
|
manifest remembers the adapter, the scope, and every path it wrote.
|
package/dist/cli.js
CHANGED
|
@@ -5159,7 +5159,7 @@ function availableSuites() {
|
|
|
5159
5159
|
}
|
|
5160
5160
|
}
|
|
5161
5161
|
function printUsage() {
|
|
5162
|
-
console.log(`+62 Mode (plus62, alias: 62-mode, hahahah-ai) \u2014 Indonesian Developer Persona & Guardrails
|
|
5162
|
+
console.log(`+62 Mode (plus62, alias: 62-mode, hahahah-ai, hahahah) \u2014 Indonesian Developer Persona & Guardrails
|
|
5163
5163
|
|
|
5164
5164
|
Usage:
|
|
5165
5165
|
npx plus62 install [--profile <id>] [--target <repo>] [--adapter <name>] [--scope workspace|global] [--dry-run] [--force]
|
|
@@ -5310,379 +5310,385 @@ if (isMainEntry()) {
|
|
|
5310
5310
|
live: { type: "boolean", default: false },
|
|
5311
5311
|
view: { type: "boolean", default: false },
|
|
5312
5312
|
dir: { type: "string" },
|
|
5313
|
+
version: { type: "boolean", short: "v", default: false },
|
|
5313
5314
|
help: { type: "boolean", short: "h", default: false }
|
|
5314
5315
|
}
|
|
5315
5316
|
});
|
|
5316
|
-
|
|
5317
|
-
|
|
5318
|
-
|
|
5319
|
-
|
|
5320
|
-
|
|
5321
|
-
|
|
5322
|
-
|
|
5323
|
-
|
|
5324
|
-
|
|
5325
|
-
|
|
5326
|
-
|
|
5327
|
-
|
|
5328
|
-
|
|
5329
|
-
|
|
5330
|
-
|
|
5331
|
-
|
|
5332
|
-
|
|
5333
|
-
|
|
5334
|
-
|
|
5335
|
-
|
|
5336
|
-
|
|
5337
|
-
|
|
5338
|
-
|
|
5339
|
-
|
|
5340
|
-
|
|
5341
|
-
|
|
5342
|
-
|
|
5343
|
-
|
|
5344
|
-
|
|
5345
|
-
|
|
5346
|
-
|
|
5347
|
-
|
|
5348
|
-
|
|
5349
|
-
|
|
5350
|
-
|
|
5351
|
-
|
|
5352
|
-
|
|
5353
|
-
|
|
5354
|
-
|
|
5355
|
-
|
|
5356
|
-
|
|
5357
|
-
console.log(
|
|
5358
|
-
|
|
5359
|
-
|
|
5360
|
-
|
|
5361
|
-
|
|
5362
|
-
|
|
5363
|
-
)
|
|
5364
|
-
|
|
5365
|
-
|
|
5366
|
-
|
|
5367
|
-
|
|
5368
|
-
|
|
5369
|
-
console.log(`
|
|
5370
|
-
|
|
5317
|
+
if (parsed.values.version) {
|
|
5318
|
+
console.log(version);
|
|
5319
|
+
process.exitCode = 0;
|
|
5320
|
+
} else {
|
|
5321
|
+
const command = parsed.positionals[0];
|
|
5322
|
+
if (!command || parsed.values.help) {
|
|
5323
|
+
printUsage();
|
|
5324
|
+
process.exitCode = command || parsed.values.help ? 0 : 1;
|
|
5325
|
+
} else if (command === "build" || command === "measure") {
|
|
5326
|
+
const result = build({
|
|
5327
|
+
root: packageRoot,
|
|
5328
|
+
profileId: parsed.values.profile,
|
|
5329
|
+
adapterId: parsed.values.adapter ?? "fake",
|
|
5330
|
+
outDir: parsed.values.out,
|
|
5331
|
+
release: parsed.values.release,
|
|
5332
|
+
write: command === "build"
|
|
5333
|
+
});
|
|
5334
|
+
console.log(formatReport(result));
|
|
5335
|
+
if (command === "build") {
|
|
5336
|
+
console.log(`out: ${result.outDir}`);
|
|
5337
|
+
}
|
|
5338
|
+
} else if (command === "install") {
|
|
5339
|
+
const scope = parsed.values.scope ?? "workspace";
|
|
5340
|
+
const adapterId = parsed.values.adapter ?? "claude-code";
|
|
5341
|
+
const hasExplicitTarget = process.argv.some(
|
|
5342
|
+
(arg) => arg === "--target" || arg.startsWith("--target=")
|
|
5343
|
+
);
|
|
5344
|
+
const defaultGlobalTarget = adapterId === "qoder" ? join9(homedir4(), ".qoder") : adapterId === "antigravity" ? join9(homedir4(), ".gemini") : parsed.values.target;
|
|
5345
|
+
const targetRoot = scope === "global" && !hasExplicitTarget ? defaultGlobalTarget : parsed.values.target;
|
|
5346
|
+
const result = install({
|
|
5347
|
+
sourceRoot: packageRoot,
|
|
5348
|
+
targetRoot,
|
|
5349
|
+
profileId: parsed.values.profile,
|
|
5350
|
+
adapterId,
|
|
5351
|
+
scope,
|
|
5352
|
+
dryRun: parsed.values["dry-run"],
|
|
5353
|
+
force: parsed.values.force
|
|
5354
|
+
});
|
|
5355
|
+
console.log(
|
|
5356
|
+
`${result.dryRun ? "Dry run: would install" : "Installed"} ${result.manifest.profile} into ${result.targetRoot} (${scope} scope)`
|
|
5357
|
+
);
|
|
5358
|
+
console.log(
|
|
5359
|
+
`files: ${result.manifest.files.length} managed file(s) + ${result.manifest.block.path} block`
|
|
5360
|
+
);
|
|
5361
|
+
for (const warning of result.warnings) {
|
|
5362
|
+
console.log(`warning: ${warning}`);
|
|
5363
|
+
}
|
|
5364
|
+
} else if (command === "graph") {
|
|
5365
|
+
const graphResult = buildCodeGraph(parsed.values.target);
|
|
5366
|
+
console.log(
|
|
5367
|
+
`Obsidian Code Graph: mapped ${graphResult.nodesEmitted}/${graphResult.filesScanned} file(s), ${graphResult.edgesCount} edge(s)`
|
|
5368
|
+
);
|
|
5369
|
+
console.log(`index: ${graphResult.indexPath}`);
|
|
5370
|
+
console.log(`memory: ${graphResult.memoryPath}`);
|
|
5371
|
+
console.log(`visual: ${graphResult.htmlPath}`);
|
|
5372
|
+
if (parsed.values.live) {
|
|
5373
|
+
const live = startLiveGraphServer(parsed.values.target, 4949);
|
|
5374
|
+
console.log(`live: ${live.url} (streaming real-time file & token telemetry via SSE)`);
|
|
5375
|
+
if (parsed.values.view) {
|
|
5376
|
+
if (process.platform === "win32") {
|
|
5377
|
+
spawn("cmd", ["/c", "start", "", live.url], {
|
|
5378
|
+
detached: true,
|
|
5379
|
+
stdio: "ignore"
|
|
5380
|
+
}).unref();
|
|
5381
|
+
} else if (process.platform === "darwin") {
|
|
5382
|
+
spawn("open", [live.url], { detached: true, stdio: "ignore" }).unref();
|
|
5383
|
+
} else {
|
|
5384
|
+
spawn("xdg-open", [live.url], { detached: true, stdio: "ignore" }).unref();
|
|
5385
|
+
}
|
|
5386
|
+
}
|
|
5387
|
+
} else if (parsed.values.view) {
|
|
5371
5388
|
if (process.platform === "win32") {
|
|
5372
|
-
spawn("cmd", ["/c", "start", "",
|
|
5389
|
+
spawn("cmd", ["/c", "start", "", graphResult.htmlPath], {
|
|
5373
5390
|
detached: true,
|
|
5374
5391
|
stdio: "ignore"
|
|
5375
5392
|
}).unref();
|
|
5376
5393
|
} else if (process.platform === "darwin") {
|
|
5377
|
-
spawn("open", [
|
|
5394
|
+
spawn("open", [graphResult.htmlPath], { detached: true, stdio: "ignore" }).unref();
|
|
5378
5395
|
} else {
|
|
5379
|
-
spawn("xdg-open", [
|
|
5396
|
+
spawn("xdg-open", [graphResult.htmlPath], { detached: true, stdio: "ignore" }).unref();
|
|
5380
5397
|
}
|
|
5381
5398
|
}
|
|
5382
|
-
} else if (
|
|
5383
|
-
|
|
5384
|
-
|
|
5385
|
-
detached: true,
|
|
5386
|
-
stdio: "ignore"
|
|
5387
|
-
}).unref();
|
|
5388
|
-
} else if (process.platform === "darwin") {
|
|
5389
|
-
spawn("open", [graphResult.htmlPath], { detached: true, stdio: "ignore" }).unref();
|
|
5390
|
-
} else {
|
|
5391
|
-
spawn("xdg-open", [graphResult.htmlPath], { detached: true, stdio: "ignore" }).unref();
|
|
5392
|
-
}
|
|
5393
|
-
}
|
|
5394
|
-
} else if (command === "ignore") {
|
|
5395
|
-
const adapterId = parsed.values.adapter ?? "claude-code";
|
|
5396
|
-
const result = setupAgentIgnore(parsed.values.target, adapterId);
|
|
5397
|
-
console.log(
|
|
5398
|
-
`Input Token Saver: ${result.created ? "Wrote" : "Already configured"} ${result.path}`
|
|
5399
|
-
);
|
|
5400
|
-
console.log(
|
|
5401
|
-
"Protected context from: node_modules, build artifacts, keystores, binaries & logs."
|
|
5402
|
-
);
|
|
5403
|
-
} else if (command === "update") {
|
|
5404
|
-
const result = update({
|
|
5405
|
-
sourceRoot: packageRoot,
|
|
5406
|
-
targetRoot: parsed.values.target,
|
|
5407
|
-
dryRun: parsed.values["dry-run"],
|
|
5408
|
-
force: parsed.values.force
|
|
5409
|
-
});
|
|
5410
|
-
console.log(
|
|
5411
|
-
`${result.dryRun ? "Dry run: would update" : "Updated"} ${result.manifest.profile} (${result.manifest.adapter}) in ${result.targetRoot}`
|
|
5412
|
-
);
|
|
5413
|
-
console.log(
|
|
5414
|
-
`files: ${result.manifest.files.length} managed file(s) + ${result.manifest.block.path} block`
|
|
5415
|
-
);
|
|
5416
|
-
for (const warning of result.warnings) {
|
|
5417
|
-
console.log(`warning: ${warning}`);
|
|
5418
|
-
}
|
|
5419
|
-
} else if (command === "uninstall") {
|
|
5420
|
-
uninstall({
|
|
5421
|
-
targetRoot: parsed.values.target,
|
|
5422
|
-
dryRun: parsed.values["dry-run"],
|
|
5423
|
-
force: parsed.values.force
|
|
5424
|
-
});
|
|
5425
|
-
console.log(
|
|
5426
|
-
`${parsed.values["dry-run"] ? "Dry run: would uninstall" : "Uninstalled"} from ${parsed.values.target}`
|
|
5427
|
-
);
|
|
5428
|
-
} else if (command === "doctor") {
|
|
5429
|
-
const report = doctor({ targetRoot: parsed.values.target });
|
|
5430
|
-
for (const warning of report.warnings) {
|
|
5431
|
-
console.log(`warning: ${warning}`);
|
|
5432
|
-
}
|
|
5433
|
-
if (!report.ok) {
|
|
5434
|
-
for (const error of report.errors) {
|
|
5435
|
-
console.error(`error: ${error}`);
|
|
5436
|
-
}
|
|
5437
|
-
process.exitCode = 1;
|
|
5438
|
-
} else {
|
|
5439
|
-
console.log(`doctor: ok (${parsed.values.target})`);
|
|
5440
|
-
}
|
|
5441
|
-
} else if (command === "fixtures") {
|
|
5442
|
-
const action = parsed.positionals[1] ?? "check";
|
|
5443
|
-
if (action === "check") {
|
|
5444
|
-
const checks = checkAll(process.cwd(), parsed.values.dir ?? "fixtures");
|
|
5445
|
-
for (const check of checks) {
|
|
5446
|
-
const label = check.ok ? "ok " : "FAIL";
|
|
5447
|
-
const holdout = check.hidden ? " (hidden)" : "";
|
|
5448
|
-
console.log(`${label} ${check.id}${holdout}: ${check.detail}`);
|
|
5449
|
-
}
|
|
5450
|
-
const failed = checks.filter((check) => !check.ok);
|
|
5451
|
-
const hidden = checks.filter((check) => check.hidden).length;
|
|
5399
|
+
} else if (command === "ignore") {
|
|
5400
|
+
const adapterId = parsed.values.adapter ?? "claude-code";
|
|
5401
|
+
const result = setupAgentIgnore(parsed.values.target, adapterId);
|
|
5452
5402
|
console.log(
|
|
5453
|
-
`
|
|
5403
|
+
`Input Token Saver: ${result.created ? "Wrote" : "Already configured"} ${result.path}`
|
|
5454
5404
|
);
|
|
5455
|
-
if (failed.length > 0) {
|
|
5456
|
-
process.exitCode = 1;
|
|
5457
|
-
}
|
|
5458
|
-
} else if (action === "run") {
|
|
5459
|
-
const all = loadFixtures(process.cwd(), parsed.values.dir ?? "fixtures");
|
|
5460
|
-
let fixtures = all;
|
|
5461
|
-
if (parsed.values.id.length > 0) {
|
|
5462
|
-
const known = new Set(all.map((fixture) => fixture.id));
|
|
5463
|
-
const unknown = parsed.values.id.filter((id) => !known.has(id));
|
|
5464
|
-
if (unknown.length > 0) {
|
|
5465
|
-
throw new Error(`Unknown fixture id(s): ${unknown.join(", ")}`);
|
|
5466
|
-
}
|
|
5467
|
-
fixtures = all.filter((fixture) => parsed.values.id.includes(fixture.id));
|
|
5468
|
-
}
|
|
5469
|
-
const limit = parsed.values.limit ? positiveInt(parsed.values.limit, "--limit", fixtures.length) : void 0;
|
|
5470
|
-
const repeats = positiveInt(parsed.values.repeats, "--repeats", 1);
|
|
5471
|
-
const budget = positiveNumber(parsed.values["budget-usd"], "--budget-usd");
|
|
5472
|
-
const arm = parsed.values.arm ?? "both";
|
|
5473
|
-
if (arm !== "both" && arm !== "baseline" && arm !== "treatment") {
|
|
5474
|
-
throw new Error("--arm must be both, baseline, or treatment");
|
|
5475
|
-
}
|
|
5476
|
-
const armCount = arm === "both" ? 2 : 1;
|
|
5477
|
-
const ceiling = parsed.values["max-total-usd"] ? positiveNumber(parsed.values["max-total-usd"], "--max-total-usd") : 0;
|
|
5478
|
-
const count = limit === void 0 ? fixtures.length : Math.min(limit, fixtures.length);
|
|
5479
|
-
const harness = harnessOf(parsed.values.harness);
|
|
5480
|
-
const projected = projectedCostUsd(count, repeats, budget, armCount);
|
|
5481
|
-
const hidden = fixtures.slice(0, count).filter((fixture) => fixture.hidden).length;
|
|
5482
5405
|
console.log(
|
|
5483
|
-
|
|
5406
|
+
"Protected context from: node_modules, build artifacts, keystores, binaries & logs."
|
|
5407
|
+
);
|
|
5408
|
+
} else if (command === "update") {
|
|
5409
|
+
const result = update({
|
|
5410
|
+
sourceRoot: packageRoot,
|
|
5411
|
+
targetRoot: parsed.values.target,
|
|
5412
|
+
dryRun: parsed.values["dry-run"],
|
|
5413
|
+
force: parsed.values.force
|
|
5414
|
+
});
|
|
5415
|
+
console.log(
|
|
5416
|
+
`${result.dryRun ? "Dry run: would update" : "Updated"} ${result.manifest.profile} (${result.manifest.adapter}) in ${result.targetRoot}`
|
|
5484
5417
|
);
|
|
5485
5418
|
console.log(
|
|
5486
|
-
`
|
|
5419
|
+
`files: ${result.manifest.files.length} managed file(s) + ${result.manifest.block.path} block`
|
|
5487
5420
|
);
|
|
5488
|
-
|
|
5421
|
+
for (const warning of result.warnings) {
|
|
5422
|
+
console.log(`warning: ${warning}`);
|
|
5423
|
+
}
|
|
5424
|
+
} else if (command === "uninstall") {
|
|
5425
|
+
uninstall({
|
|
5426
|
+
targetRoot: parsed.values.target,
|
|
5427
|
+
dryRun: parsed.values["dry-run"],
|
|
5428
|
+
force: parsed.values.force
|
|
5429
|
+
});
|
|
5430
|
+
console.log(
|
|
5431
|
+
`${parsed.values["dry-run"] ? "Dry run: would uninstall" : "Uninstalled"} from ${parsed.values.target}`
|
|
5432
|
+
);
|
|
5433
|
+
} else if (command === "doctor") {
|
|
5434
|
+
const report = doctor({ targetRoot: parsed.values.target });
|
|
5435
|
+
for (const warning of report.warnings) {
|
|
5436
|
+
console.log(`warning: ${warning}`);
|
|
5437
|
+
}
|
|
5438
|
+
if (!report.ok) {
|
|
5439
|
+
for (const error of report.errors) {
|
|
5440
|
+
console.error(`error: ${error}`);
|
|
5441
|
+
}
|
|
5442
|
+
process.exitCode = 1;
|
|
5443
|
+
} else {
|
|
5444
|
+
console.log(`doctor: ok (${parsed.values.target})`);
|
|
5445
|
+
}
|
|
5446
|
+
} else if (command === "fixtures") {
|
|
5447
|
+
const action = parsed.positionals[1] ?? "check";
|
|
5448
|
+
if (action === "check") {
|
|
5449
|
+
const checks = checkAll(process.cwd(), parsed.values.dir ?? "fixtures");
|
|
5450
|
+
for (const check of checks) {
|
|
5451
|
+
const label = check.ok ? "ok " : "FAIL";
|
|
5452
|
+
const holdout = check.hidden ? " (hidden)" : "";
|
|
5453
|
+
console.log(`${label} ${check.id}${holdout}: ${check.detail}`);
|
|
5454
|
+
}
|
|
5455
|
+
const failed = checks.filter((check) => !check.ok);
|
|
5456
|
+
const hidden = checks.filter((check) => check.hidden).length;
|
|
5489
5457
|
console.log(
|
|
5490
|
-
|
|
5458
|
+
`fixtures: ${checks.length - failed.length}/${checks.length} valid, ${hidden} marked hidden holdout`
|
|
5491
5459
|
);
|
|
5492
|
-
|
|
5493
|
-
|
|
5494
|
-
throw new Error(
|
|
5495
|
-
`--live needs --max-total-usd: this campaign projects $${projected.toFixed(2)}, state the approved ceiling`
|
|
5496
|
-
);
|
|
5460
|
+
if (failed.length > 0) {
|
|
5461
|
+
process.exitCode = 1;
|
|
5497
5462
|
}
|
|
5498
|
-
|
|
5499
|
-
|
|
5500
|
-
|
|
5501
|
-
|
|
5463
|
+
} else if (action === "run") {
|
|
5464
|
+
const all = loadFixtures(process.cwd(), parsed.values.dir ?? "fixtures");
|
|
5465
|
+
let fixtures = all;
|
|
5466
|
+
if (parsed.values.id.length > 0) {
|
|
5467
|
+
const known = new Set(all.map((fixture) => fixture.id));
|
|
5468
|
+
const unknown = parsed.values.id.filter((id) => !known.has(id));
|
|
5469
|
+
if (unknown.length > 0) {
|
|
5470
|
+
throw new Error(`Unknown fixture id(s): ${unknown.join(", ")}`);
|
|
5471
|
+
}
|
|
5472
|
+
fixtures = all.filter((fixture) => parsed.values.id.includes(fixture.id));
|
|
5502
5473
|
}
|
|
5503
|
-
|
|
5474
|
+
const limit = parsed.values.limit ? positiveInt(parsed.values.limit, "--limit", fixtures.length) : void 0;
|
|
5475
|
+
const repeats = positiveInt(parsed.values.repeats, "--repeats", 1);
|
|
5476
|
+
const budget = positiveNumber(parsed.values["budget-usd"], "--budget-usd");
|
|
5477
|
+
const arm = parsed.values.arm ?? "both";
|
|
5478
|
+
if (arm !== "both" && arm !== "baseline" && arm !== "treatment") {
|
|
5479
|
+
throw new Error("--arm must be both, baseline, or treatment");
|
|
5480
|
+
}
|
|
5481
|
+
const armCount = arm === "both" ? 2 : 1;
|
|
5482
|
+
const ceiling = parsed.values["max-total-usd"] ? positiveNumber(parsed.values["max-total-usd"], "--max-total-usd") : 0;
|
|
5483
|
+
const count = limit === void 0 ? fixtures.length : Math.min(limit, fixtures.length);
|
|
5484
|
+
const harness = harnessOf(parsed.values.harness);
|
|
5485
|
+
const projected = projectedCostUsd(count, repeats, budget, armCount);
|
|
5486
|
+
const hidden = fixtures.slice(0, count).filter((fixture) => fixture.hidden).length;
|
|
5487
|
+
console.log(
|
|
5488
|
+
`plan: running ${count} of ${fixtures.length} selected fixture(s) (${hidden} marked hidden holdout), ${repeats} repeat(s), ${armCount} arm(s) [${arm}], $${budget.toFixed(2)} per run, harness ${harness}`
|
|
5489
|
+
);
|
|
5490
|
+
console.log(
|
|
5491
|
+
`projected cost of the caps: $${projected.toFixed(2)} \u2014 not a ceiling: the harness checks its budget between turns, so a capped run overshoots (measured 1.17x and 1.46x of --budget-usd on 2026-11-04)`
|
|
5492
|
+
);
|
|
5493
|
+
if (!parsed.values.live) {
|
|
5504
5494
|
console.log(
|
|
5505
|
-
|
|
5495
|
+
"dry run: no model was called and nothing was spent. Add --live with --max-total-usd to run it."
|
|
5506
5496
|
);
|
|
5497
|
+
} else {
|
|
5498
|
+
if (ceiling === 0) {
|
|
5499
|
+
throw new Error(
|
|
5500
|
+
`--live needs --max-total-usd: this campaign projects $${projected.toFixed(2)}, state the approved ceiling`
|
|
5501
|
+
);
|
|
5502
|
+
}
|
|
5503
|
+
if (projected > ceiling) {
|
|
5504
|
+
throw new Error(
|
|
5505
|
+
`projected $${projected.toFixed(2)} exceeds the approved ceiling of $${ceiling.toFixed(2)}: lower --limit, --repeats, or --budget-usd`
|
|
5506
|
+
);
|
|
5507
|
+
}
|
|
5508
|
+
if (armCount === 1) {
|
|
5509
|
+
console.log(
|
|
5510
|
+
'single arm: this run cannot compare baseline against treatment, it only answers "did the harness behave"'
|
|
5511
|
+
);
|
|
5512
|
+
}
|
|
5513
|
+
const report = runFixtureStudy({
|
|
5514
|
+
sourceRoot: process.cwd(),
|
|
5515
|
+
profileId: parsed.values.profile,
|
|
5516
|
+
fixtures,
|
|
5517
|
+
limit,
|
|
5518
|
+
repeats,
|
|
5519
|
+
budgetUsd: budget,
|
|
5520
|
+
arm,
|
|
5521
|
+
model: parsed.values.model,
|
|
5522
|
+
settingSources: parsed.values["setting-sources"],
|
|
5523
|
+
adapterId: HARNESS_ADAPTERS[harness],
|
|
5524
|
+
runner: (cwd, prompt, runBudget) => harness === "opencode" ? runOpencode(cwd, prompt, runBudget, { model: parsed.values.model }) : runClaude(cwd, prompt, runBudget, {
|
|
5525
|
+
model: parsed.values.model,
|
|
5526
|
+
settingSources: parsed.values["setting-sources"],
|
|
5527
|
+
// Edits are the one thing the agent must be able to do; shell and network
|
|
5528
|
+
// stay denied, so the grade comes from the tests this tool runs afterwards.
|
|
5529
|
+
agentArgs: ["--permission-mode", "acceptEdits", "--permission-prompts", "none"]
|
|
5530
|
+
}),
|
|
5531
|
+
outDir: parsed.values.out ?? join9(process.cwd(), ".plus62", "coding")
|
|
5532
|
+
});
|
|
5533
|
+
printStudy(report);
|
|
5507
5534
|
}
|
|
5508
|
-
|
|
5509
|
-
|
|
5510
|
-
|
|
5511
|
-
fixtures
|
|
5512
|
-
|
|
5513
|
-
|
|
5514
|
-
|
|
5515
|
-
arm
|
|
5516
|
-
|
|
5517
|
-
|
|
5518
|
-
|
|
5519
|
-
|
|
5535
|
+
} else if (action === "diagnose") {
|
|
5536
|
+
const ids = parsed.values.id;
|
|
5537
|
+
if (ids.length !== 1) {
|
|
5538
|
+
throw new Error("fixtures diagnose needs exactly one --id <fixture>");
|
|
5539
|
+
}
|
|
5540
|
+
const arm = parsed.values.arm ?? "treatment";
|
|
5541
|
+
if (arm !== "baseline" && arm !== "treatment") {
|
|
5542
|
+
throw new Error("--arm must be baseline or treatment for a single-arm diagnostic");
|
|
5543
|
+
}
|
|
5544
|
+
const fixture = loadFixtures(process.cwd(), parsed.values.dir ?? "fixtures").find(
|
|
5545
|
+
(candidate) => candidate.id === ids[0]
|
|
5546
|
+
);
|
|
5547
|
+
if (!fixture) {
|
|
5548
|
+
throw new Error(`Unknown fixture id(s): ${ids[0]}`);
|
|
5549
|
+
}
|
|
5550
|
+
const budget = positiveNumber(parsed.values["budget-usd"], "--budget-usd");
|
|
5551
|
+
const projected = projectedCostUsd(1, 1, budget, 1);
|
|
5552
|
+
const ceiling = parsed.values["max-total-usd"] ? positiveNumber(parsed.values["max-total-usd"], "--max-total-usd") : 0;
|
|
5553
|
+
console.log(
|
|
5554
|
+
`diagnostic: ${fixture.id}, one arm [${arm}], $${budget.toFixed(2)} per run, projected $${projected.toFixed(2)} (caps are soft, see docs/phase-3-live-suite-results.md)`
|
|
5555
|
+
);
|
|
5556
|
+
if (!parsed.values.live) {
|
|
5557
|
+
console.log("dry run: no model was called. Add --live with --max-total-usd to run it.");
|
|
5558
|
+
} else {
|
|
5559
|
+
if (ceiling === 0) {
|
|
5560
|
+
throw new Error(
|
|
5561
|
+
`--live needs --max-total-usd: this diagnostic projects $${projected.toFixed(2)}`
|
|
5562
|
+
);
|
|
5563
|
+
}
|
|
5564
|
+
if (projected > ceiling) {
|
|
5565
|
+
throw new Error(
|
|
5566
|
+
`projected $${projected.toFixed(2)} exceeds the approved ceiling of $${ceiling.toFixed(2)}`
|
|
5567
|
+
);
|
|
5568
|
+
}
|
|
5569
|
+
const result = diagnoseFixture({
|
|
5570
|
+
sourceRoot: process.cwd(),
|
|
5571
|
+
profileId: parsed.values.profile,
|
|
5572
|
+
fixture,
|
|
5573
|
+
arm,
|
|
5574
|
+
budgetUsd: budget,
|
|
5520
5575
|
model: parsed.values.model,
|
|
5521
5576
|
settingSources: parsed.values["setting-sources"],
|
|
5522
|
-
|
|
5523
|
-
|
|
5524
|
-
|
|
5525
|
-
}),
|
|
5526
|
-
|
|
5527
|
-
|
|
5528
|
-
|
|
5577
|
+
outDir: parsed.values.out ?? join9(process.cwd(), ".plus62", "diagnostic")
|
|
5578
|
+
});
|
|
5579
|
+
const { summary, graded } = result;
|
|
5580
|
+
const tools = Object.entries(summary.tools).map(([name, count]) => `${name} x${count}`).join(", ");
|
|
5581
|
+
console.log(
|
|
5582
|
+
`stream: subtype=${summary.subtype ?? "?"} turns=${summary.numTurns ?? "?"} cost=$${(summary.totalCostUsd ?? 0).toFixed(4)} events=${summary.lines}`
|
|
5583
|
+
);
|
|
5584
|
+
console.log(`tools attempted: ${tools || "none"}`);
|
|
5585
|
+
for (const denial of summary.denials) {
|
|
5586
|
+
console.log(`DENIED: ${denial}`);
|
|
5587
|
+
}
|
|
5588
|
+
for (const error of summary.toolErrors) {
|
|
5589
|
+
console.log(`tool error: ${error}`);
|
|
5590
|
+
}
|
|
5591
|
+
console.log(
|
|
5592
|
+
`workspace: changed=[${graded.changedFiles.join(", ")}] new=[${graded.otherNewFiles.join(", ")}] missing=[${graded.missingFiles.join(", ")}]`
|
|
5593
|
+
);
|
|
5594
|
+
console.log(`grade=${graded.grade} tests pass=${graded.pass} fail=${graded.fail}`);
|
|
5595
|
+
console.log(`raw stream kept: ${result.rawPath}`);
|
|
5596
|
+
console.log(`workspace kept for inspection: ${result.workspace} (delete when done)`);
|
|
5597
|
+
if (graded.run.error) {
|
|
5598
|
+
process.exitCode = 1;
|
|
5599
|
+
}
|
|
5600
|
+
}
|
|
5601
|
+
} else {
|
|
5602
|
+
printUsage();
|
|
5603
|
+
process.exitCode = 1;
|
|
5529
5604
|
}
|
|
5530
|
-
} else if (
|
|
5531
|
-
const
|
|
5532
|
-
|
|
5533
|
-
|
|
5605
|
+
} else if (command === "eval") {
|
|
5606
|
+
const suite = loadSuite(process.cwd(), parsed.values.suite);
|
|
5607
|
+
const limit = parsed.values.limit ? Number.parseInt(parsed.values.limit, 10) : void 0;
|
|
5608
|
+
const repeats = parsed.values.repeats ? Number.parseInt(parsed.values.repeats, 10) : 1;
|
|
5609
|
+
const budget = Number.parseFloat(parsed.values["budget-usd"]);
|
|
5610
|
+
if (limit !== void 0 && (!Number.isInteger(limit) || limit <= 0)) {
|
|
5611
|
+
throw new Error("--limit must be a positive integer");
|
|
5534
5612
|
}
|
|
5535
|
-
|
|
5536
|
-
|
|
5537
|
-
throw new Error("--arm must be baseline or treatment for a single-arm diagnostic");
|
|
5613
|
+
if (!Number.isInteger(repeats) || repeats < 1) {
|
|
5614
|
+
throw new Error("--repeats must be a positive integer");
|
|
5538
5615
|
}
|
|
5539
|
-
|
|
5540
|
-
(
|
|
5541
|
-
);
|
|
5542
|
-
if (!fixture) {
|
|
5543
|
-
throw new Error(`Unknown fixture id(s): ${ids[0]}`);
|
|
5616
|
+
if (!Number.isFinite(budget) || budget <= 0) {
|
|
5617
|
+
throw new Error("--budget-usd must be a positive number");
|
|
5544
5618
|
}
|
|
5545
|
-
|
|
5546
|
-
|
|
5547
|
-
|
|
5548
|
-
|
|
5549
|
-
|
|
5550
|
-
|
|
5551
|
-
if (!parsed.values.live) {
|
|
5552
|
-
console.log("dry run: no model was called. Add --live with --max-total-usd to run it.");
|
|
5553
|
-
} else {
|
|
5554
|
-
if (ceiling === 0) {
|
|
5555
|
-
throw new Error(
|
|
5556
|
-
`--live needs --max-total-usd: this diagnostic projects $${projected.toFixed(2)}`
|
|
5557
|
-
);
|
|
5558
|
-
}
|
|
5559
|
-
if (projected > ceiling) {
|
|
5560
|
-
throw new Error(
|
|
5561
|
-
`projected $${projected.toFixed(2)} exceeds the approved ceiling of $${ceiling.toFixed(2)}`
|
|
5562
|
-
);
|
|
5619
|
+
let tasks = suite.tasks;
|
|
5620
|
+
if (parsed.values.task.length > 0) {
|
|
5621
|
+
const known = new Set(suite.tasks.map((task) => task.id));
|
|
5622
|
+
const unknown = parsed.values.task.filter((id) => !known.has(id));
|
|
5623
|
+
if (unknown.length > 0) {
|
|
5624
|
+
throw new Error(`Unknown task id(s): ${unknown.join(", ")} in suite ${suite.suite}`);
|
|
5563
5625
|
}
|
|
5564
|
-
|
|
5565
|
-
|
|
5566
|
-
|
|
5567
|
-
|
|
5568
|
-
|
|
5569
|
-
|
|
5626
|
+
tasks = suite.tasks.filter((task) => parsed.values.task.includes(task.id));
|
|
5627
|
+
}
|
|
5628
|
+
const harness = harnessOf(parsed.values.harness);
|
|
5629
|
+
const report = runEval({
|
|
5630
|
+
sourceRoot: process.cwd(),
|
|
5631
|
+
profileId: parsed.values.profile,
|
|
5632
|
+
tasks,
|
|
5633
|
+
suite: suite.suite,
|
|
5634
|
+
limit,
|
|
5635
|
+
repeats,
|
|
5636
|
+
budgetUsd: budget,
|
|
5637
|
+
model: parsed.values.model,
|
|
5638
|
+
settingSources: parsed.values["setting-sources"],
|
|
5639
|
+
adapterId: HARNESS_ADAPTERS[harness],
|
|
5640
|
+
harness,
|
|
5641
|
+
runner: runnerFor(harness, {
|
|
5570
5642
|
model: parsed.values.model,
|
|
5571
|
-
settingSources: parsed.values["setting-sources"]
|
|
5572
|
-
|
|
5573
|
-
|
|
5574
|
-
|
|
5575
|
-
|
|
5643
|
+
settingSources: parsed.values["setting-sources"]
|
|
5644
|
+
}),
|
|
5645
|
+
// The runner is injected here so the harness can be chosen, which would otherwise disable
|
|
5646
|
+
// the auto-probe. A real campaign always talks to a real gateway, so it must always probe.
|
|
5647
|
+
probe: true,
|
|
5648
|
+
outDir: parsed.values.out ?? join9(process.cwd(), ".plus62", "eval")
|
|
5649
|
+
});
|
|
5650
|
+
const runs = report.results.reduce(
|
|
5651
|
+
(total, result) => total + result.baseline.runs.length + result.treatment.runs.length,
|
|
5652
|
+
0
|
|
5653
|
+
);
|
|
5654
|
+
console.log(
|
|
5655
|
+
`eval: ${report.results.length} task(s) x ${report.repeats} repeat(s) = ${runs} runs, cost $${report.total_cost_usd.toFixed(4)}`
|
|
5656
|
+
);
|
|
5657
|
+
if (report.aborted) {
|
|
5658
|
+
const cause = report.aborted.kind === "quota" ? "the provider refused further requests" : "consecutive runs were answered with nothing at all - the harness or route is not answering";
|
|
5576
5659
|
console.log(
|
|
5577
|
-
`
|
|
5660
|
+
`ABORTED after ${report.aborted.afterRuns} run(s): ${cause} (${report.aborted.reason.slice(0, 120)}). Not measured: ${report.aborted.remainingTasks.join(", ") || `${report.aborted.partialTask} (partial)`}. Nothing further was spent.`
|
|
5578
5661
|
);
|
|
5579
|
-
|
|
5580
|
-
|
|
5581
|
-
|
|
5582
|
-
|
|
5583
|
-
|
|
5584
|
-
|
|
5585
|
-
|
|
5662
|
+
}
|
|
5663
|
+
for (const result of report.results) {
|
|
5664
|
+
const state = (arm) => {
|
|
5665
|
+
if (arm.consensus === "error") return `ERROR: ${arm.runs[0]?.error ?? "no runs"}`;
|
|
5666
|
+
const passed = arm.runStatuses.filter((status) => status === "pass").length;
|
|
5667
|
+
const findings = [...new Set(arm.screens.findings)];
|
|
5668
|
+
const detail = findings.length > 0 ? ` (${findings.join("; ")})` : "";
|
|
5669
|
+
return `${arm.consensus} ${passed}/${arm.runs.length} med $${arm.median.costUsd.toFixed(4)}${detail}`;
|
|
5670
|
+
};
|
|
5586
5671
|
console.log(
|
|
5587
|
-
|
|
5672
|
+
`- ${result.task.id}: baseline=${state(result.baseline)}, treatment=${state(result.treatment)}`
|
|
5588
5673
|
);
|
|
5589
|
-
|
|
5590
|
-
|
|
5591
|
-
|
|
5592
|
-
|
|
5593
|
-
|
|
5594
|
-
|
|
5674
|
+
}
|
|
5675
|
+
const { baseline, treatment } = report.summary;
|
|
5676
|
+
if (report.summary.tasks_with_checks > 0) {
|
|
5677
|
+
console.log(
|
|
5678
|
+
`screens: baseline ${baseline.pass} pass / ${baseline.fail} fail / ${baseline.unknown} unknown; treatment ${treatment.pass} pass / ${treatment.fail} fail / ${treatment.unknown} unknown`
|
|
5679
|
+
);
|
|
5680
|
+
}
|
|
5681
|
+
const failedRun = report.results.flatMap((result) => [...result.baseline.runs, ...result.treatment.runs]).find((run) => run.error);
|
|
5682
|
+
if (failedRun) {
|
|
5683
|
+
process.exitCode = 1;
|
|
5684
|
+
} else if (parsed.values.strict && treatment.fail > 0) {
|
|
5685
|
+
console.error(`strict: ${treatment.fail} treatment screen(s) failed across repeats`);
|
|
5686
|
+
process.exitCode = 1;
|
|
5595
5687
|
}
|
|
5596
5688
|
} else {
|
|
5597
5689
|
printUsage();
|
|
5598
5690
|
process.exitCode = 1;
|
|
5599
5691
|
}
|
|
5600
|
-
} else if (command === "eval") {
|
|
5601
|
-
const suite = loadSuite(process.cwd(), parsed.values.suite);
|
|
5602
|
-
const limit = parsed.values.limit ? Number.parseInt(parsed.values.limit, 10) : void 0;
|
|
5603
|
-
const repeats = parsed.values.repeats ? Number.parseInt(parsed.values.repeats, 10) : 1;
|
|
5604
|
-
const budget = Number.parseFloat(parsed.values["budget-usd"]);
|
|
5605
|
-
if (limit !== void 0 && (!Number.isInteger(limit) || limit <= 0)) {
|
|
5606
|
-
throw new Error("--limit must be a positive integer");
|
|
5607
|
-
}
|
|
5608
|
-
if (!Number.isInteger(repeats) || repeats < 1) {
|
|
5609
|
-
throw new Error("--repeats must be a positive integer");
|
|
5610
|
-
}
|
|
5611
|
-
if (!Number.isFinite(budget) || budget <= 0) {
|
|
5612
|
-
throw new Error("--budget-usd must be a positive number");
|
|
5613
|
-
}
|
|
5614
|
-
let tasks = suite.tasks;
|
|
5615
|
-
if (parsed.values.task.length > 0) {
|
|
5616
|
-
const known = new Set(suite.tasks.map((task) => task.id));
|
|
5617
|
-
const unknown = parsed.values.task.filter((id) => !known.has(id));
|
|
5618
|
-
if (unknown.length > 0) {
|
|
5619
|
-
throw new Error(`Unknown task id(s): ${unknown.join(", ")} in suite ${suite.suite}`);
|
|
5620
|
-
}
|
|
5621
|
-
tasks = suite.tasks.filter((task) => parsed.values.task.includes(task.id));
|
|
5622
|
-
}
|
|
5623
|
-
const harness = harnessOf(parsed.values.harness);
|
|
5624
|
-
const report = runEval({
|
|
5625
|
-
sourceRoot: process.cwd(),
|
|
5626
|
-
profileId: parsed.values.profile,
|
|
5627
|
-
tasks,
|
|
5628
|
-
suite: suite.suite,
|
|
5629
|
-
limit,
|
|
5630
|
-
repeats,
|
|
5631
|
-
budgetUsd: budget,
|
|
5632
|
-
model: parsed.values.model,
|
|
5633
|
-
settingSources: parsed.values["setting-sources"],
|
|
5634
|
-
adapterId: HARNESS_ADAPTERS[harness],
|
|
5635
|
-
harness,
|
|
5636
|
-
runner: runnerFor(harness, {
|
|
5637
|
-
model: parsed.values.model,
|
|
5638
|
-
settingSources: parsed.values["setting-sources"]
|
|
5639
|
-
}),
|
|
5640
|
-
// The runner is injected here so the harness can be chosen, which would otherwise disable
|
|
5641
|
-
// the auto-probe. A real campaign always talks to a real gateway, so it must always probe.
|
|
5642
|
-
probe: true,
|
|
5643
|
-
outDir: parsed.values.out ?? join9(process.cwd(), ".plus62", "eval")
|
|
5644
|
-
});
|
|
5645
|
-
const runs = report.results.reduce(
|
|
5646
|
-
(total, result) => total + result.baseline.runs.length + result.treatment.runs.length,
|
|
5647
|
-
0
|
|
5648
|
-
);
|
|
5649
|
-
console.log(
|
|
5650
|
-
`eval: ${report.results.length} task(s) x ${report.repeats} repeat(s) = ${runs} runs, cost $${report.total_cost_usd.toFixed(4)}`
|
|
5651
|
-
);
|
|
5652
|
-
if (report.aborted) {
|
|
5653
|
-
const cause = report.aborted.kind === "quota" ? "the provider refused further requests" : "consecutive runs were answered with nothing at all - the harness or route is not answering";
|
|
5654
|
-
console.log(
|
|
5655
|
-
`ABORTED after ${report.aborted.afterRuns} run(s): ${cause} (${report.aborted.reason.slice(0, 120)}). Not measured: ${report.aborted.remainingTasks.join(", ") || `${report.aborted.partialTask} (partial)`}. Nothing further was spent.`
|
|
5656
|
-
);
|
|
5657
|
-
}
|
|
5658
|
-
for (const result of report.results) {
|
|
5659
|
-
const state = (arm) => {
|
|
5660
|
-
if (arm.consensus === "error") return `ERROR: ${arm.runs[0]?.error ?? "no runs"}`;
|
|
5661
|
-
const passed = arm.runStatuses.filter((status) => status === "pass").length;
|
|
5662
|
-
const findings = [...new Set(arm.screens.findings)];
|
|
5663
|
-
const detail = findings.length > 0 ? ` (${findings.join("; ")})` : "";
|
|
5664
|
-
return `${arm.consensus} ${passed}/${arm.runs.length} med $${arm.median.costUsd.toFixed(4)}${detail}`;
|
|
5665
|
-
};
|
|
5666
|
-
console.log(
|
|
5667
|
-
`- ${result.task.id}: baseline=${state(result.baseline)}, treatment=${state(result.treatment)}`
|
|
5668
|
-
);
|
|
5669
|
-
}
|
|
5670
|
-
const { baseline, treatment } = report.summary;
|
|
5671
|
-
if (report.summary.tasks_with_checks > 0) {
|
|
5672
|
-
console.log(
|
|
5673
|
-
`screens: baseline ${baseline.pass} pass / ${baseline.fail} fail / ${baseline.unknown} unknown; treatment ${treatment.pass} pass / ${treatment.fail} fail / ${treatment.unknown} unknown`
|
|
5674
|
-
);
|
|
5675
|
-
}
|
|
5676
|
-
const failedRun = report.results.flatMap((result) => [...result.baseline.runs, ...result.treatment.runs]).find((run) => run.error);
|
|
5677
|
-
if (failedRun) {
|
|
5678
|
-
process.exitCode = 1;
|
|
5679
|
-
} else if (parsed.values.strict && treatment.fail > 0) {
|
|
5680
|
-
console.error(`strict: ${treatment.fail} treatment screen(s) failed across repeats`);
|
|
5681
|
-
process.exitCode = 1;
|
|
5682
|
-
}
|
|
5683
|
-
} else {
|
|
5684
|
-
printUsage();
|
|
5685
|
-
process.exitCode = 1;
|
|
5686
5692
|
}
|
|
5687
5693
|
} catch (error) {
|
|
5688
5694
|
if (error instanceof BuildError || error instanceof InstallError) {
|