@tachikomagundam/abathur 0.2.2 → 0.2.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -142,7 +142,11 @@ Abathur ships an official opencode plugin adapter. The harness remains the
142
142
  orchestrator, but once set up an opencode session can also drive the CLI
143
143
  directly. Two install routes:
144
144
 
145
- **Route A one command (tool + `/abathur` slash command).**
145
+ Both routes deliver the tool **and** the `/abathur` slash command; pick by
146
+ taste — A needs no edit of your opencode config and works fully offline, B
147
+ needs no file copying and removes with one deleted line.
148
+
149
+ **Route A — one command (file-copy installer).**
146
150
 
147
151
  ```bash
148
152
  npm i -g @tachikomagundam/abathur
@@ -150,7 +154,7 @@ abathur opencode install # copies plugin assets into ~/.config/opencode/
150
154
  # restart opencode
151
155
  ```
152
156
 
153
- **Route B — config only (tool, no slash command).** Put the package name in
157
+ **Route B — config only (auto-downloaded from npm).** Put the package name in
154
158
  your opencode config and restart:
155
159
 
156
160
  ```jsonc
@@ -159,11 +163,16 @@ your opencode config and restart:
159
163
 
160
164
  opencode then downloads the package from npm at startup into its own cache
161
165
  (`~/.cache/opencode/packages/…`) and loads the package's `./server` export —
162
- no file copying, no uninstall step (delete the line). Two honest limits: the
163
- `abathur` **CLI** still must exist on `PATH` (or `ABATHUR_BIN`) because the
164
- tool spawns it, so keep the `npm i -g` install; and slash commands are plain
165
- markdown files that plugins cannot register upstream, so `/abathur` is Route A
166
- only.
166
+ no file copying, no uninstall step (delete the line). Since 0.2.3 the plugin
167
+ also self-registers `/abathur` through its `config` hook: slash commands are
168
+ entries in the merged config's `command` map, and a plugin may add one there
169
+ the upstream `opencode-acp` plugin registers `/acp` the same way (the
170
+ 0.2.0–0.2.2 note here claiming plugins cannot register slash commands was
171
+ wrong; corrected). If Route A's copied `commands/abathur.md` exists it stays
172
+ authoritative — the injection never overwrites it, and the name-keyed command
173
+ map means `/abathur` can never appear twice. One honest limit stands for both
174
+ routes: the tool spawns the `abathur` **CLI**, which must exist on `PATH` (or
175
+ via `ABATHUR_BIN`) — keep the `npm i -g` install.
167
176
 
168
177
  Either route gives the session:
169
178
 
@@ -176,9 +185,10 @@ Either route gives the session:
176
185
  so the allowlist limits typos and UX, not capability. `promote` and
177
186
  `tombstone` are deliberately NOT reachable through the tool: they are
178
187
  human gates and must be run in a terminal.
179
- - the user command **`/abathur <args…>`** (Route A only) — a slash command
180
- that tells the agent to translate its arguments into a tool call and report
181
- the exit code.
188
+ - the user command **`/abathur <args…>`** — a slash command that tells the
189
+ agent to translate its arguments into a tool call and report the exit code.
190
+ Route A provides it as the copied `commands/abathur.md` file; Route B gets
191
+ it from the plugin's `config` hook at startup.
182
192
 
183
193
  Route A ships its own manager commands: `abathur opencode status` shows each
184
194
  target's path, installed/packaged sha256, and state (up-to-date / outdated /
@@ -430,7 +440,10 @@ abathur run --genome toy-smoke --dry-run # 计划 + requires[] 探针,零
430
440
  Abathur 自带官方 opencode 插件适配器。工装仍是编排者,但配置之后,opencode
431
441
  会话也可以直接驱动 CLI。安装有两条路线:
432
442
 
433
- **路线 A——一条命令(工具 + `/abathur` 斜杠命令)。**
443
+ 两条路线都交付工具**和** `/abathur` 斜杠命令,按喜好挑选:A 不必改动你的
444
+ opencode 配置、且完全离线可用;B 不复制任何文件、删掉一行即卸载。
445
+
446
+ **路线 A——一条命令(文件复制安装器)。**
434
447
 
435
448
  ```bash
436
449
  npm i -g @tachikomagundam/abathur
@@ -438,7 +451,7 @@ abathur opencode install # 把插件资产复制进 ~/.config/opencode/
438
451
  # 重启 opencode
439
452
  ```
440
453
 
441
- **路线 B——纯配置(只有工具,没有斜杠命令)。** 把包名写进 opencode 配置再
454
+ **路线 B——纯配置(自动从 npm 下载)。** 把包名写进 opencode 配置再
442
455
  重启:
443
456
 
444
457
  ```jsonc
@@ -447,10 +460,14 @@ abathur opencode install # 把插件资产复制进 ~/.config/opencode/
447
460
 
448
461
  opencode 会在启动时自行从 npm 把包下载到它自己的缓存
449
462
  (`~/.cache/opencode/packages/…`)并加载该包的 `./server` 导出——不复制文件,
450
- 也没有 uninstall 步骤(删掉那行即可)。两点诚实的限制:工具 spawn 的是
451
- `abathur` **CLI**,所以 CLI 仍必须在 `PATH`(或 `ABATHUR_BIN`)上可达,
452
- `npm i -g` 不能省;而斜杠命令在上游只是 markdown 文件、插件无法注册,
453
- 所以 `/abathur` 只有路线 A 提供。
463
+ 也没有 uninstall 步骤(删掉那行即可)。自 0.2.3 起,插件还会通过它的
464
+ `config` 钩子自注册 `/abathur`:斜杠命令本质是合并后配置的 `command`
465
+ 映射里的条目,插件可以在钩子里添加一条——上游的 `opencode-acp` 插件正是
466
+ 这样注册 `/acp` 的(0.2.0–0.2.2 在此声称"插件无法注册斜杠命令"是错的,
467
+ 已更正)。若路线 A 复制的 `commands/abathur.md` 存在,它保持权威——注入
468
+ 绝不覆盖它,而且命令映射按名字键控,`/abathur` 绝不会出现两份。对两条
469
+ 路线同样成立的一个诚实限制:工具 spawn 的是 `abathur` **CLI**,CLI 必须
470
+ 在 `PATH`(或 `ABATHUR_BIN`)上可达,`npm i -g` 不能省。
454
471
 
455
472
  任一路线都会让会话获得:
456
473
 
@@ -461,8 +478,9 @@ opencode 会在启动时自行从 npm 把包下载到它自己的缓存
461
478
  bash 等同的权限——`run` 与 `genome` 按设计就会 spawn 变异器/引擎二进制——
462
479
  所以允许清单限制的是笔误与体验,而非能力。`promote` 与 `tombstone` 刻意
463
480
  不可经由工具触达:它们是人类闸门,必须在终端里运行。
464
- - 用户命令 **`/abathur <参数…>`**(仅路线 A)——斜杠命令,指示智能体把参数
465
- 翻译成工具调用并回报退出码。
481
+ - 用户命令 **`/abathur <参数…>`**——斜杠命令,指示智能体把参数
482
+ 翻译成工具调用并回报退出码。路线 A 由复制的 `commands/abathur.md`
483
+ 文件提供;路线 B 由插件的 `config` 钩子在启动时注册。
466
484
 
467
485
  路线 A 自带管理命令:`abathur opencode status` 逐目标打印路径、已安装/随包
468
486
  sha256 与状态
@@ -8,6 +8,11 @@
8
8
  //
9
9
  // ${ABATHUR_HISTORIAN_REPO} absolute path of the historian bench repo
10
10
  // (contains scenarios/, rubric.md, seed_sandbox.sh)
11
+ // — repoPath ONLY: the S4 engine seam delivers the
12
+ // bench's ACTIVE TREE (incumbent repoPath /
13
+ // candidate worktree) to the commands below as the
14
+ // {repoRoot} template var, so unit/seed paths must
15
+ // NOT bake a materialized repo literal here.
11
16
  // ${ABATHUR_REPO} absolute path of THIS Abathur checkout
12
17
  // (hosts graders/historian/*)
13
18
  // ${ABATHUR_WIKI_BASE} canonical wiki root, e.g. http://localhost:3000
@@ -47,11 +52,11 @@
47
52
  { "id": "scenario-08", "path": "scenarios/08-current-state-g5.md", "split": "train" },
48
53
  { "id": "scenario-09", "path": "scenarios/09-timeline-week-groups.md", "split": "val" }
49
54
  ],
50
- "seedCommand": "bash ${ABATHUR_REPO}/graders/historian/seed-wrapped.sh ${ABATHUR_HISTORIAN_REPO}/seed_sandbox.sh",
55
+ "seedCommand": "bash ${ABATHUR_REPO}/graders/historian/seed-wrapped.sh {repoRoot}/seed_sandbox.sh",
51
56
  "resetCommand": "bash ${ABATHUR_REPO}/graders/historian/reset-sandbox.sh",
52
- "runCommand": "bash ${ABATHUR_REPO}/graders/historian/run-scenario.sh {unit.id} ${ABATHUR_HISTORIAN_REPO}/{unit.path}",
53
- "graderCommand": "node ${ABATHUR_REPO}/graders/historian/grader.mjs {unit.id} ${ABATHUR_HISTORIAN_REPO}/{unit.path} ${ABATHUR_WIKI_BASE}",
54
- "agentModel": "bailian-token-plan/qwen3.8-flash",
57
+ "runCommand": "bash ${ABATHUR_REPO}/graders/historian/run-scenario.sh {unit.id} {repoRoot}/{unit.path}",
58
+ "graderCommand": "node ${ABATHUR_REPO}/graders/historian/grader.mjs {unit.id} {repoRoot}/{unit.path} ${ABATHUR_WIKI_BASE}",
59
+ "agentModel": "<provider>/<model-id>", // "<provider>/<model-id> as registered in your opencode config"
55
60
  // Real agent runs, not unit tests: 1200s caps one stuck scenario without
56
61
  // poisoning the generation (timeout => unscored/inconclusive, excluded by
57
62
  // the stats layer). seed+reset hooks share a FIXED 60s adapter budget
@@ -10,11 +10,29 @@
10
10
  // excluded from score distributions (hr discipline), not zeroed.
11
11
  import { spawn } from "node:child_process";
12
12
  import { cannotAnswer } from "../exit.js";
13
- export function unitVars(unit, sandboxDir) {
14
- return { "unit.path": unit.path, "unit.id": unit.id, sandbox: sandboxDir, workdir: sandboxDir };
13
+ /**
14
+ * `{repoRoot}` = the bench's ACTIVE TREE: the genome repoPath for the incumbent,
15
+ * the sealed candidate worktree for a candidate (run-loop.ts swaps spec.repoPath
16
+ * per target; adapters resolve it through effectiveRepoPath at construction).
17
+ * Omitting repoRoot keeps `{repoRoot}` an UNKNOWN placeholder — fail-closed
18
+ * exit 2, never a silently half-substituted command.
19
+ */
20
+ export function unitVars(unit, sandboxDir, repoRoot) {
21
+ const vars = {
22
+ "unit.path": unit.path,
23
+ "unit.id": unit.id,
24
+ sandbox: sandboxDir,
25
+ workdir: sandboxDir,
26
+ };
27
+ if (repoRoot !== undefined)
28
+ vars["repoRoot"] = repoRoot;
29
+ return vars;
15
30
  }
16
- export function sandboxVars(sandboxDir) {
17
- return { sandbox: sandboxDir, workdir: sandboxDir };
31
+ export function sandboxVars(sandboxDir, repoRoot) {
32
+ const vars = { sandbox: sandboxDir, workdir: sandboxDir };
33
+ if (repoRoot !== undefined)
34
+ vars["repoRoot"] = repoRoot;
35
+ return vars;
18
36
  }
19
37
  const PLACEHOLDER = /\{([A-Za-z0-9_.]+)\}/g;
20
38
  /**
@@ -32,6 +32,7 @@ import { cannotAnswer, ExitSignal } from "../exit.js";
32
32
  import { resolveConfigDir } from "../config.js";
33
33
  import { fingerprint } from "../core/ids.js";
34
34
  import { acquireGenomeLock, Ledger } from "../core/ledger.js";
35
+ import { effectiveRepoPath } from "../core/spec.js";
35
36
  import { childStatus, firstLine, inconclusive, parseGraderLine, renderCommand, runChild, sandboxVars, unitVars, ZERO_METRICS, } from "./adapter.js";
36
37
  import { buildManifest, gateVal, mirrorSandboxHome, parseRunMeta, sandboxHomeDir, transcriptPathFor, writeManifest, } from "./fixture-support.js";
37
38
  import { probeEngines, resolveOpencodeBin } from "./fixture-probe.js";
@@ -52,7 +53,10 @@ export class FixtureScenariosAdapter {
52
53
  if (spec.bench.type !== "opencode-fixture-scenarios") {
53
54
  cannotAnswer(`fixture adapter: bench.type '${spec.bench.type}' is not "opencode-fixture-scenarios"`, "use ToyBenchAdapter for bench.type 'toy'");
54
55
  }
55
- const root = path.resolve(spec.repoPath);
56
+ // Active-tree resolution follows the run-loop notion (run-loop.ts:142): a
57
+ // concrete repoPath resolves exactly as before; a `${VAR}` env literal
58
+ // resolves at THIS fs boundary or exits 2 naming the variable.
59
+ const root = effectiveRepoPath(spec.repoPath);
56
60
  if (!statSync(root, { throwIfNoEntry: false })?.isDirectory()) {
57
61
  cannotAnswer(`fixture: repoPath '${spec.repoPath}' is not a readable directory`);
58
62
  }
@@ -84,7 +88,7 @@ export class FixtureScenariosAdapter {
84
88
  const transcript = transcriptPathFor(sandboxDir, unit.id);
85
89
  mkdirSync(path.dirname(transcript), { recursive: true });
86
90
  const outcome = await runChild({
87
- argv: renderCommand(this.spec.bench.runCommand, unitVars(unit, sandboxDir)),
91
+ argv: renderCommand(this.spec.bench.runCommand, unitVars(unit, sandboxDir, this.repoRoot)),
88
92
  cwd: sandboxDir,
89
93
  timeoutS,
90
94
  env: this.sandboxEnv(sandboxDir, unit),
@@ -109,7 +113,7 @@ export class FixtureScenariosAdapter {
109
113
  cannotAnswer("fixture adapter: score() called before any reset()/seed()/run() — no sandbox yet", "drive the adapter in reset→seed→run→score order");
110
114
  }
111
115
  const outcome = await runChild({
112
- argv: renderCommand(this.spec.bench.graderCommand, unitVars(unit, sandbox)),
116
+ argv: renderCommand(this.spec.bench.graderCommand, unitVars(unit, sandbox, this.repoRoot)),
113
117
  cwd: sandbox,
114
118
  timeoutS: this.spec.bench.timeoutS,
115
119
  env: this.sandboxEnv(sandbox),
@@ -211,7 +215,7 @@ export class FixtureScenariosAdapter {
211
215
  if (template === undefined)
212
216
  return;
213
217
  const outcome = await runChild({
214
- argv: renderCommand(template, sandboxVars(sandboxDir)),
218
+ argv: renderCommand(template, sandboxVars(sandboxDir, this.repoRoot)),
215
219
  cwd: sandboxDir,
216
220
  timeoutS: HOOK_TIMEOUT_S,
217
221
  env: this.sandboxEnv(sandboxDir),
package/dist/bench/toy.js CHANGED
@@ -53,7 +53,7 @@ export class ToyBenchAdapter {
53
53
  async run(unit, sandboxDir, timeoutS) {
54
54
  this.activeSandbox = sandboxDir;
55
55
  const outcome = await runChild({
56
- argv: renderCommand(this.spec.bench.runCommand, unitVars(unit, sandboxDir)),
56
+ argv: renderCommand(this.spec.bench.runCommand, unitVars(unit, sandboxDir, this.repoRoot)),
57
57
  cwd: sandboxDir,
58
58
  timeoutS,
59
59
  ...(this.opts.onChild === undefined ? {} : { onChild: this.opts.onChild }),
@@ -75,7 +75,7 @@ export class ToyBenchAdapter {
75
75
  cannotAnswer("toy adapter: score() called before any reset()/seed()/run() — no sandbox yet", "drive the adapter in reset→seed→run→score order");
76
76
  }
77
77
  const outcome = await runChild({
78
- argv: renderCommand(this.spec.bench.graderCommand, unitVars(unit, sandbox)),
78
+ argv: renderCommand(this.spec.bench.graderCommand, unitVars(unit, sandbox, this.repoRoot)),
79
79
  cwd: sandbox,
80
80
  timeoutS: this.spec.bench.timeoutS,
81
81
  ...(this.opts.onChild === undefined ? {} : { onChild: this.opts.onChild }),
@@ -109,7 +109,7 @@ export class ToyBenchAdapter {
109
109
  if (template === undefined)
110
110
  return;
111
111
  const outcome = await runChild({
112
- argv: renderCommand(template, sandboxVars(sandboxDir)),
112
+ argv: renderCommand(template, sandboxVars(sandboxDir, this.repoRoot)),
113
113
  cwd: sandboxDir,
114
114
  timeoutS: HOOK_TIMEOUT_S,
115
115
  ...(this.opts.onChild === undefined ? {} : { onChild: this.opts.onChild }),
@@ -15,6 +15,7 @@ import { tryGit } from "../util/git.js";
15
15
  import { readRegistry } from "../core/genome.js";
16
16
  import { LEDGER_KIND_GRAFT_IMPORT, decodeGraftImport, listGraftQueue, } from "../core/graft-support.js";
17
17
  import { resolveUniqueEntry } from "./run.js";
18
+ import { effectiveRepoPath } from "../core/spec.js";
18
19
  const KIND_GENERATION = "generation_complete";
19
20
  const KIND_PROMOTE = "promote";
20
21
  const KIND_TOMBSTONE = "tombstone";
@@ -113,7 +114,11 @@ async function runStatus(args) {
113
114
  renderQueue(queue);
114
115
  return EXIT_OK;
115
116
  }
116
- const repo = entry.spec.repoPath;
117
+ // Resolve at the fs seam exactly like run/genome/kernel (run-loop.ts:142):
118
+ // feeding the stored UNRESOLVED `${VAR}` literal to git/ledger paths makes
119
+ // Node report the missing cwd as the misleading `spawn git ENOENT`; an unset
120
+ // variable is now a clean exit 2 that names it.
121
+ const repo = effectiveRepoPath(entry.spec.repoPath);
117
122
  const records = readLedgerRecords(ledgerPath(repo));
118
123
  const gens = generations(records);
119
124
  writeStdout(`genome: ${flat(entry.label)} (${entry.fingerprint}) repo: ${flat(repo, 200)}`);
@@ -4,6 +4,7 @@ import path from "node:path";
4
4
  import { cannotAnswer } from "../../exit.js";
5
5
  import { runId } from "../ids.js";
6
6
  import { buildManifest } from "../kernel.js";
7
+ import { effectiveRepoPath } from "../spec.js";
7
8
  import { budgetExhausted } from "../stats.js";
8
9
  import { ToyBenchAdapter } from "../../bench/toy.js";
9
10
  import { FixtureScenariosAdapter } from "../../bench/fixture.js";
@@ -164,7 +165,7 @@ export async function benchTarget(o) {
164
165
  spent,
165
166
  provenance: provenance ?? { benchType: o.spec.bench.type, versions: [] },
166
167
  complete,
167
- manifest: buildManifest(path.resolve(o.spec.repoPath), o.spec.kernel.immutableGlobs),
168
+ manifest: buildManifest(effectiveRepoPath(o.spec.repoPath), o.spec.kernel.immutableGlobs),
168
169
  failures,
169
170
  };
170
171
  }
@@ -31,3 +31,27 @@ test("renderCommand: quote-aware argv, placeholders, fail-closed junk (exit 2)",
31
31
  expectExit2(() => renderCommand(" ", unitVars(unit, "/s")), "empty");
32
32
  expectExit2(() => renderCommand('node "oops', unitVars(unit, "/s")), "unbalanced quote");
33
33
  });
34
+ // S4 engine seam: {repoRoot} = the bench's ACTIVE TREE, threaded through the
35
+ // shared CommandVars constructors so both adapters resolve it identically.
36
+ // Without a threaded repoRoot the placeholder must stay FAIL-CLOSED (exit 2),
37
+ // exactly like any unknown placeholder — never a half-substituted command.
38
+ test("{repoRoot}: unitVars/sandboxVars carry the active tree into run/grader/hook templates", () => {
39
+ const unit = { id: "u1", path: "scenarios/a.md", split: "train" };
40
+ assert.deepEqual(renderCommand("bash run-scenario.sh {unit.id} {repoRoot}/{unit.path}", unitVars(unit, "/sbx", "/active/tree")), ["bash", "run-scenario.sh", "u1", "/active/tree/scenarios/a.md"]);
41
+ assert.deepEqual(renderCommand("bash seed-wrapped.sh {repoRoot}/seed_sandbox.sh", sandboxVars("/sbx", "/active/tree")), ["bash", "seed-wrapped.sh", "/active/tree/seed_sandbox.sh"]);
42
+ // pre-existing keys untouched by the seam
43
+ assert.deepEqual(unitVars(unit, "/sbx", "/r"), {
44
+ "unit.path": "scenarios/a.md",
45
+ "unit.id": "u1",
46
+ sandbox: "/sbx",
47
+ workdir: "/sbx",
48
+ repoRoot: "/r",
49
+ });
50
+ assert.deepEqual(sandboxVars("/sbx", "/r"), { sandbox: "/sbx", workdir: "/sbx", repoRoot: "/r" });
51
+ });
52
+ test("{repoRoot} unthreaded ⇒ fail-closed exit 2 (unknown placeholder behavior UNCHANGED)", () => {
53
+ const unit = { id: "u1", path: "p", split: "train" };
54
+ expectExit2(() => renderCommand("node x {repoRoot}", unitVars(unit, "/s")), "unknown placeholder");
55
+ expectExit2(() => renderCommand("node x {repoRoot}", sandboxVars("/s")), "unknown placeholder");
56
+ expectExit2(() => renderCommand("node x {repoRoot}", { worktree: "/w", brief: "/b" }), "unknown placeholder");
57
+ });
@@ -62,9 +62,13 @@ fi
62
62
  echo "ran $unit"
63
63
  echo '{"tokensEst":123,"turns":4}'
64
64
  `;
65
+ // Unique sleep duration: node --test runs files concurrently and this test's
66
+ // orphan check scans the whole machine's `ps` output — "sleep 31.7" is the
67
+ // shipped toy genome's hang marker (bench-toy.test.ts); reusing it here made
68
+ // the two files' checks see each other's live sleeps (CI-verified 5/5 collision).
65
69
  const BIN_HANG = `#!/usr/bin/env bash
66
70
  if [ "\${1:-}" = "--version" ]; then echo "1.2.3"; exit 0; fi
67
- exec sleep 31.7
71
+ exec sleep 29.31
68
72
  `;
69
73
  const BIN_NO_VERSION = `#!/usr/bin/env bash
70
74
  if [ "\${1:-}" = "--version" ]; then echo "unsupported flag" >&2; exit 1; fi
@@ -296,7 +300,7 @@ test("(E) hanging unit: group kill at timeoutS, zero orphans", async (t) => {
296
300
  assert.equal(run.exitCode, null);
297
301
  assert.match(run.note ?? "", /killed after 1s: process group SIGKILL/);
298
302
  const ps = spawnSync("ps", ["-eo", "args"], { encoding: "utf8" });
299
- assert.ok(!ps.stdout.includes("sleep 31.7"), "orphaned sleep survived the group kill");
303
+ assert.ok(!ps.stdout.includes("sleep 29.31"), "orphaned sleep survived the group kill");
300
304
  });
301
305
  // ------------------------------------------------------------------ AC (F)
302
306
  test("(F) infra_failed is recorded with a distinct status/note, never a silent 0", async (t) => {
@@ -0,0 +1,197 @@
1
+ // S4 engine seam — the {repoRoot} command variable. The mutation channel's
2
+ // delivery hinge: run/grader/seed/reset templates must resolve {repoRoot} to
3
+ // the bench's ACTIVE TREE — the genome repoPath for the incumbent, the sealed
4
+ // candidate worktree for a candidate (run-loop.ts swaps `spec.repoPath` before
5
+ // benching each candidate; the adapter must follow that same notion, including
6
+ // effectiveRepoPath's env-literal resolution at the FS boundary).
7
+ // Unknown placeholders stay fail-closed (exit 2). Offline: capture scripts
8
+ // record the rendered argv; the fake opencode only answers --version.
9
+ import assert from "node:assert/strict";
10
+ import { chmodSync, cpSync, mkdirSync, readFileSync, writeFileSync } from "node:fs";
11
+ import { mkdtemp, rm, writeFile } from "node:fs/promises";
12
+ import * as os from "node:os";
13
+ import path from "node:path";
14
+ import { test } from "node:test";
15
+ import { ExitSignal } from "../exit.js";
16
+ import { parseGenomeSpecDocument } from "../core/spec.js";
17
+ import { FixtureScenariosAdapter } from "../bench/fixture.js";
18
+ // ------------------------------------------------------------------- helpers
19
+ async function freshDir(t, prefix) {
20
+ const dir = await mkdtemp(path.join(os.tmpdir(), `abathur-${prefix}-`));
21
+ t.after(() => rm(dir, { recursive: true, force: true }));
22
+ return dir;
23
+ }
24
+ async function writeScript(dir, name, body) {
25
+ const filePath = path.join(dir, name);
26
+ await writeFile(filePath, body, "utf8");
27
+ chmodSync(filePath, 0o755);
28
+ return filePath;
29
+ }
30
+ /** Records `argv[mode, out, ...rest]`: rest joined per line into `out`; prints the metric/score line. */
31
+ const CAPTURE = `#!/usr/bin/env node
32
+ import { writeFileSync } from "node:fs";
33
+ const [mode, out, ...rest] = process.argv.slice(2);
34
+ writeFileSync(out, rest.join("\\n") + "\\n", "utf8");
35
+ if (mode === "score") {
36
+ process.stdout.write(JSON.stringify({ unit: "one", score: 1, pass: true, metrics: { tokensEst: 3, turns: 1 } }) + "\\n");
37
+ } else {
38
+ process.stdout.write(JSON.stringify({ tokensEst: 3, turns: 1 }) + "\\n");
39
+ }
40
+ `;
41
+ const FAKE_OC = `#!/usr/bin/env bash
42
+ if [ "\${1:-}" = "--version" ]; then echo "1.2.3"; exit 0; fi
43
+ exit 0
44
+ `;
45
+ const UNIT = { id: "one", path: "scenarios/one.md", split: "train" };
46
+ const VAL_UNIT = { id: "v1", path: "scenarios/v1.md", split: "val" };
47
+ async function seamFixture(t, genDir) {
48
+ const root = await freshDir(t, "seam-ad");
49
+ const capPath = await writeScript(root, "cap.mjs", CAPTURE);
50
+ const binDir = await freshDir(t, "seam-bin");
51
+ const binPath = await writeScript(binDir, "oc-fake", FAKE_OC);
52
+ const home = await freshDir(t, "seam-home");
53
+ const configDir = await freshDir(t, "seam-cfg");
54
+ const outDir = path.join(root, "caps");
55
+ mkdirSync(outDir, { recursive: true });
56
+ const out = (name) => path.join(outDir, `${name}.txt`);
57
+ const spec = parseGenomeSpecDocument({
58
+ label: "seam-germ",
59
+ repoPath: genDir,
60
+ bench: {
61
+ type: "opencode-fixture-scenarios",
62
+ units: [UNIT, VAL_UNIT],
63
+ runCommand: `node ${capPath} run ${out("run")} {repoRoot} {unit.path}`,
64
+ seedCommand: `node ${capPath} seed ${out("seed")} {repoRoot}`,
65
+ resetCommand: `node ${capPath} reset ${out("reset")} {repoRoot}`,
66
+ graderCommand: `node ${capPath} score ${out("grader")} {repoRoot}`,
67
+ agentModel: "test/agent",
68
+ timeoutS: 30,
69
+ stats: { halfWidth: 0.15, minEffect: 0.1, nReps: { initial: 1, max: 2 } },
70
+ },
71
+ budget: { maxCandidates: 2, maxModelCalls: 4, maxTokens: 10000, maxWallS: 600 },
72
+ kernel: { immutableGlobs: [] },
73
+ requires: [{ cmd: "node", args: ["--version"], probeExit: 0 }],
74
+ opencodeBinVersion: { minVersion: "1.0.0" },
75
+ }, "<test>");
76
+ const sandbox = path.join(root, "sandbox");
77
+ return {
78
+ spec,
79
+ genDir,
80
+ sandbox,
81
+ home,
82
+ cap: capPath,
83
+ binPath,
84
+ out,
85
+ adapter: (s) => new FixtureScenariosAdapter(s, { env: { HOME: home }, home, configDir, opencodeBin: binPath }),
86
+ };
87
+ }
88
+ /** Minimal genome dir (the fixture adapter only requires repoPath to be a directory). */
89
+ async function genomeTree(t, name = "germ") {
90
+ const genRoot = await freshDir(t, `seam-${name}`);
91
+ const genDir = path.join(genRoot, "germ");
92
+ mkdirSync(path.join(genDir, "scenarios"), { recursive: true });
93
+ writeFileSync(path.join(genDir, "scenarios", "one.md"), "# one\n\n## Brief\ndo it\n", "utf8");
94
+ writeFileSync(path.join(genDir, "scenarios", "v1.md"), "# v1\n\n## Brief\ndo it val\n", "utf8");
95
+ return genDir;
96
+ }
97
+ function readCap(file) {
98
+ return readFileSync(file, "utf8").trim();
99
+ }
100
+ function isExit2Matching(needle) {
101
+ return (cause) => cause instanceof ExitSignal && cause.code === 2 && needle.test(cause.message);
102
+ }
103
+ // --------------------------------------------------------------------- pins
104
+ test("{repoRoot} renders to the incumbent genome repoPath in run + grader + hooks", async (t) => {
105
+ const genDir = await genomeTree(t);
106
+ const f = await seamFixture(t, genDir);
107
+ const adapter = f.adapter(f.spec);
108
+ try {
109
+ await adapter.reset(f.sandbox);
110
+ await adapter.seed(f.sandbox);
111
+ const run = await adapter.run(UNIT, f.sandbox, 30);
112
+ assert.equal(run.status, "ok");
113
+ const score = await adapter.score(UNIT);
114
+ assert.equal(score.kind, "scored");
115
+ assert.equal(readCap(f.out("run")), `${genDir}\nscenarios/one.md`);
116
+ assert.equal(readCap(f.out("grader")), genDir);
117
+ assert.equal(readCap(f.out("seed")), genDir);
118
+ assert.equal(readCap(f.out("reset")), genDir);
119
+ }
120
+ finally {
121
+ adapter.release();
122
+ }
123
+ });
124
+ test("{repoRoot} follows the candidate worktree when spec.repoPath is swapped (run-loop.ts pattern)", async (t) => {
125
+ const genDir = await genomeTree(t);
126
+ const f = await seamFixture(t, genDir);
127
+ // simulate run-loop.ts:279 — benchTarget receives { ...spec, repoPath: worktreePath }
128
+ const worktree = path.join(path.dirname(genDir), "wt");
129
+ cpSync(genDir, worktree, { recursive: true });
130
+ mkdirSync(path.join(worktree, "abathur-notes"), { recursive: true });
131
+ writeFileSync(path.join(worktree, "abathur-notes", "card.md"), "planted run card\n", "utf8");
132
+ const candidate = f.adapter({ ...f.spec, repoPath: worktree });
133
+ try {
134
+ const sandbox = path.join(f.sandbox, "cand");
135
+ await candidate.reset(sandbox);
136
+ await candidate.seed(sandbox);
137
+ const run = await candidate.run(UNIT, sandbox, 30);
138
+ assert.equal(run.status, "ok");
139
+ const score = await candidate.score(UNIT);
140
+ assert.equal(score.kind, "scored");
141
+ assert.equal(readCap(f.out("run")), `${worktree}\nscenarios/one.md`);
142
+ assert.equal(readCap(f.out("grader")), worktree);
143
+ }
144
+ finally {
145
+ candidate.release();
146
+ }
147
+ });
148
+ test("fixture ctor resolves an env-literal repoPath via effectiveRepoPath (run-loop.ts:142 pattern)", async (t) => {
149
+ const genDir = await genomeTree(t, "lit");
150
+ const f = await seamFixture(t, genDir);
151
+ const litSpec = { ...f.spec, repoPath: "${ABATHUR_SEAM_LIT_REPO}" };
152
+ const base = {
153
+ env: { HOME: f.home },
154
+ home: f.home,
155
+ configDir: path.join(f.home, ".config", "abathur"),
156
+ opencodeBin: f.binPath,
157
+ };
158
+ delete process.env["ABATHUR_SEAM_LIT_REPO"];
159
+ try {
160
+ // unset env ⇒ clean exit 2 NAMING the variable (never a garbage-path spawn error)
161
+ assert.throws(() => new FixtureScenariosAdapter(litSpec, base), isExit2Matching(/requires env var ABATHUR_SEAM_LIT_REPO/));
162
+ // set env ⇒ resolves at the FS boundary; {repoRoot} renders to the resolved tree
163
+ process.env["ABATHUR_SEAM_LIT_REPO"] = genDir;
164
+ const adapter = new FixtureScenariosAdapter(litSpec, base);
165
+ try {
166
+ const sandbox = path.join(f.sandbox, "lit");
167
+ await adapter.reset(sandbox);
168
+ await adapter.seed(sandbox);
169
+ const run = await adapter.run(UNIT, sandbox, 30);
170
+ assert.equal(run.status, "ok");
171
+ assert.equal(readCap(f.out("run")), `${genDir}\nscenarios/one.md`);
172
+ }
173
+ finally {
174
+ adapter.release();
175
+ }
176
+ }
177
+ finally {
178
+ delete process.env["ABATHUR_SEAM_LIT_REPO"];
179
+ }
180
+ });
181
+ test("fail-closed UNCHANGED: unknown placeholder through the adapter is still exit 2", async (t) => {
182
+ const genDir = await genomeTree(t, "unk");
183
+ const f = await seamFixture(t, genDir);
184
+ const bad = {
185
+ ...f.spec,
186
+ bench: { ...f.spec.bench, runCommand: `node ${f.cap} run ${f.out("bad")} {nope}` },
187
+ };
188
+ const adapter = f.adapter(bad);
189
+ try {
190
+ await adapter.reset(f.sandbox);
191
+ await adapter.seed(f.sandbox);
192
+ await assert.rejects(() => adapter.run(UNIT, f.sandbox, 30), isExit2Matching(/unknown placeholder/));
193
+ }
194
+ finally {
195
+ adapter.release();
196
+ }
197
+ });