@doidor/agentrig 0.9.0 → 0.10.0

package/README.md

@@ -108,48 +108,83 @@ the generated files. Commit them so remote agents (and teammates' tools) pick th
 This is a first-class feature, not an afterthought — and it's **repo-specific and runnable by you**.
 The eval kit installs into your repo (`.agentrig/eval/`) and is tailored to it during `init`, so you
 can measure whether AgentRig actually helps *here*. `agentrig eval` **defaults to the full agentic,
-harness-on run**; add `--static` for the fast no-model audit. Two layers:
+harness-on run**; add `--static` for the fast no-model audit. Three layers, each making a different
+**bounded** claim:
 
-- **Static audit (deterministic, no model).** Maps each principle to a structural check in
-  `.agentrig/eval/checks.json`, scored `0 / 0.5 / 1.0`, producing a **Harness Score**.
+- **Layer A1 — install completeness** (deterministic, no model). Every canonical artifact present
+  where the manifest says it should be. This is what CI gates on.
+
+- **Layer A2 — quality probes** (deterministic, no model). Cheap content sanity: parseable YAML,
+  no unfilled `{{PLACEHOLDER}}` in `AGENTS.md`, every skill carries the required frontmatter,
+  developer/reviewer use **different model families** (not just different ids), `axes.json` has an
+  issue code per axis. Diagnostic — surfaces drift without failing the build.
 
   ```bash
-  agentrig eval --static            # or: node .agentrig/eval/static-audit.mjs
-  agentrig eval --static --min 80   # CI gate: non-zero exit below 80%
+  agentrig eval --static            # both A1 and A2; prints sections separately
+  agentrig eval --static --min 80   # CI gate on A1 (Install Completeness)
   ```
 
-- **Dynamic behavioral eval (agentic, independent judge — the default).** Runs benchmark scenarios
-  (`.agentrig/eval/scenarios/*.md`) through the harness and scores the results with an **independent
-  judge model**. Scoring is rigorous: a registry (`axes.json`) of bounded **issue codes per axis**,
-  strict `0/0.5/1.0` tiers, mandatory evidence, confidence-gated rollups recomputed from the axis
-  data, and three **lifecycle rubrics** (`spec` / `run` / `review`). `score.mjs` validates everything
-  and never lets a judge invent codes. Runs are sandboxed (no push, no PR, no merge).
+- **Layer B — dynamic behavioral eval** (agentic, **isolated** producer + judge, fixture-based).
+  For each scenario in `.agentrig/eval/scenarios/*/`:
+  1. Seed a throwaway worktree from `scenarios/<id>/fixture/`.
+  2. Producer model runs `prompt.md` in the worktree.
+  3. **Deterministic oracle** (`scenarios/<id>/oracle.yml`) scores hard axes (correctness, tests,
+     scope, regression_risk) by running commands and inspecting the diff. No LLM.
+  4. **Judge model** — a *different family* from the producer — runs in its own cwd with
+     `prompt.md` + `diff.patch` + `transcript.md` + `oracle.json`. It does NOT see the producer's
+     worktree or reasoning trace. It writes scores to a JSON file the orchestrator validates.
+  5. `score.mjs save` enforces tier (`0`/`0.5`/`1.0`), issue code, evidence, **veto axes**
+     (correctness, gate_compliance — cosmetics can never paper over a real regression), and
+     **producer/judge family divergence** (override is recorded so reviewers can spot it).
 
   ```bash
-  agentrig eval                                  # the default: full agentic, harness-on run
-  agentrig eval --dynamic --scenario add-small-feature --timeout 60
-  agentrig eval --rubric                         # print exactly what's measured (axes, codes, scenarios)
+  agentrig eval --dynamic                                    # defaults: developer.yml + reviewer.yml models
+  agentrig eval --dynamic --producer-model claude-sonnet-4.6 --judge-model gpt-5.5   # explicit override
+  agentrig eval --dynamic --scenario fix-failing-test --n 5
+  agentrig eval --rubric          # print rubric (axes, codes, scenarios) without running
   node .agentrig/eval/score.mjs report
   ```
 
-  **What's being evaluated?** The rubric lives in `.agentrig/eval/RUBRIC.md` and the machine-readable
-  registry in `.agentrig/eval/axes.json`; scenarios are in `.agentrig/eval/scenarios/`. Run
-  `agentrig eval --rubric` to print the rubric types, every axis and its issue codes, and the
-  installed scenarios.
+  By default, the producer model is read from `.agentrig/agents/developer.yml` and the judge from
+  `.agentrig/agents/reviewer.yml` — the install-completeness audit already enforces these come
+  from different model families. Override with explicit flags or `AGENTRIG_PRODUCER_MODEL` /
+  `AGENTRIG_JUDGE_MODEL` env vars.
 
-### Does the harness actually help? (harness lift)
+### Does the harness actually help? (statistical harness lift)
 
-Prove the harness earns its keep in *your* repo by running a scenario **with** and **without** it:
+Prove the harness earns its keep in *your* repo by running each scenario **with** and **without**
+it, **with multiple paired trials** (single-trial deltas are coin flips):
 
 ```bash
-agentrig eval --dynamic --scenario <id> --variant harness    # harness ON
-agentrig eval --dynamic --scenario <id> --variant baseline   # bare agent, no AGENTS.md/rules/skills
+agentrig eval --dynamic --scenario <id> --variant harness  --n 5
+agentrig eval --dynamic --scenario <id> --variant baseline --n 5
 node .agentrig/eval/score.mjs compare --scenario <id> --baseline baseline
 ```
 
-`compare --baseline` prints the per-axis and aggregate **delta** with a `HELPS`/`HURTS` verdict — so
-you can see, and track over time, whether installing AgentRig measurably improves agent behavior here
-(and whether each rule/skill/prompt change is a real improvement or a regression).
+`compare --baseline` pairs trial *i* of harness with trial *i* of baseline, computes the **median
+delta** and a **binomial sign-test p-value**, and prints one of three verdicts:
+
+- **HELPS** — p < 0.05 and median > 0.05 (the harness measurably improves behavior here)
+- **HURTS** — p < 0.05 and median < -0.05 (regression — investigate before merging the change)
+- **INCONCLUSIVE** — n < 3, p ≥ 0.05, or effect smaller than ±0.05 (need more trials)
+
+A "HELPS" verdict on a real fixture, in a different model family than the judge, is the only thing
+that justifies the line "AgentRig improved agent behavior on this repo." Anything less is honest
+**inconclusive**.
+
+### Calibrating the judge
+
+A lazy judge that returns 1.0 on every axis passes every save validation but tells you nothing.
+`.agentrig/eval/calibration/` ships hand-labeled rubric instances; `score.mjs calibrate` runs the
+judge over them and reports % within ±0.5 tier + signed bias. `agentrig doctor` flags any judge
+below **80% agreement**. See `.agentrig/eval/calibration/README.md` for the format.
+
+### Scope honesty
+
+The static audit verifies install completeness, not "is this harness good." The dynamic eval, with
+a calibrated judge and ≥5 paired trials, gives you a statistical signal of behavior change between
+variants — but it's still measuring agent + model behavior on synthetic fixtures, not your real
+PR workload. Treat green checks as evidence, not certainty.
 
 ## Dashboard
 
@@ -160,7 +195,7 @@ CLI:
 - **Agent roster** — every role and the model it runs on.
 - **Live GitHub tasks** — open issues/PRs carrying each harness label, grouped by workflow state and
   showing assignees, fetched via the `gh` CLI (degrades gracefully when `gh` is absent/unauthed).
-- **Harness Score** — the latest static-audit score and any weak principles.
+- **Install Completeness + Quality Probes** — the latest static-audit scores and any weak principles.
 - **Evals** — the latest dynamic-eval summary.
 - **Limits** — the hard caps from the state machine.
 
@@ -191,7 +226,7 @@ customize (like `AGENTS.md`), preserving your repo-specific facts.
 | `agentrig init [path]` | Investigate + install a tailored harness, then compile surfaces. **Non-destructive by default** — preserves existing AGENTS.md / .mcp.json / .agents/rules; `--force` to overwrite |
 | `agentrig compile [path]` | Project AGENTS.md + rules into every agent surface (local + remote) |
 | `agentrig update [path]` | Re-sync the latest best practices (re-compiles surfaces) |
-| `agentrig eval [path] [--static\|--rubric] [--scenario id] [--variant name]` | Evaluate the harness (default: agentic; `--rubric` shows what's measured) |
+| `agentrig eval [path] [--static\|--rubric] [--scenario id] [--variant name] [--n trials] [--producer-model id] [--judge-model id]` | Evaluate the harness (default: agentic; `--static` for the cheap CI-safe audit; `--rubric` shows what's measured) |
 | `agentrig dashboard [path] [--html [file]] [--no-tasks] [--json]` | Roster, live GitHub tasks, score, evals |
 | `agentrig doctor [path] [--json]` | Health check (installed? agent reachable? score?) |
 

package/dist/agent/copilot.js

@@ -37,6 +37,37 @@ export class CopilotProvider {
             await client.stop().catch(() => undefined);
         }
     }
+    async validateModel(modelId) {
+        const client = new CopilotClient({
+            logLevel: "none",
+            env: { ...process.env, NODE_NO_WARNINGS: "1" },
+        });
+        try {
+            await client.start();
+            const models = await client.listModels();
+            const ids = models.map((m) => m.id);
+            if (ids.includes(modelId))
+                return { ok: true };
+            // "did you mean…" — same family by prefix, sorted alphabetically.
+            const prefix = modelId.split(/[-.]/)[0].toLowerCase();
+            const near = ids.filter((id) => id.toLowerCase().startsWith(prefix)).sort();
+            return {
+                ok: false,
+                available: ids.sort(),
+                detail: near.length
+                    ? `Did you mean: ${near.slice(0, 5).join(", ")}?`
+                    : `Available: ${ids.slice(0, 8).join(", ")}${ids.length > 8 ? ", …" : ""}`,
+            };
+        }
+        catch (err) {
+            // If the listing call itself fails (offline, auth issue), don't block — let
+            // startConversation surface the real error.
+            return { ok: true, detail: `validateModel skipped: ${err.message}` };
+        }
+        finally {
+            await client.stop().catch(() => undefined);
+        }
+    }
     async startConversation(options) {
         const client = new CopilotClient({
             workingDirectory: options.cwd,
@@ -44,11 +75,21 @@ export class CopilotProvider {
             env: { ...process.env, NODE_NO_WARNINGS: "1" },
         });
         await client.start();
-        const session = await client.createSession({
-            ...(options.model ? { model: options.model } : {}),
-            onPermissionRequest: approveAll,
-            ...(options.systemMessage ? { systemMessage: { content: options.systemMessage } } : {}),
-        });
+        // From here on, if anything throws (typically createSession failing because the model id is
+        // unknown for this account), we MUST stop the client — otherwise its child runtime keeps the
+        // event loop alive and `agentrig eval --dynamic` hangs after the report prints.
+        let session;
+        try {
+            session = await client.createSession({
+                ...(options.model ? { model: options.model } : {}),
+                onPermissionRequest: approveAll,
+                ...(options.systemMessage ? { systemMessage: { content: options.systemMessage } } : {}),
+            });
+        }
+        catch (err) {
+            await client.stop().catch(() => undefined);
+            throw err;
+        }
         const emit = options.onEvent;
         if (emit) {
             session.on("assistant.intent", (event) => {

package/dist/agent/copilot.js.map

@@ -1 +1 @@
-{"version":3,"file":"copilot.js","sourceRoot":"","sources":["../../src/agent/copilot.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,aAAa,EAAE,UAAU,EAAE,MAAM,qBAAqB,CAAC;AAEhE,OAAO,EACL,iBAAiB,GAKlB,MAAM,eAAe,CAAC;AAEvB,SAAS,aAAa,CAAC,IAAyC;IAC9D,IAAI,CAAC,IAAI;QAAE,OAAO,SAAS,CAAC;IAC5B,KAAK,MAAM,GAAG,IAAI,CAAC,SAAS,EAAE,MAAM,EAAE,UAAU,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE,QAAQ,CAAC,EAAE,CAAC;QACvF,MAAM,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC;QACpB,IAAI,OAAO,CAAC,KAAK,QAAQ,IAAI,CAAC,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAC1C,OAAO,GAAG,GAAG,IAAI,CAAC,CAAC,MAAM,GAAG,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;QAC9D,CAAC;IACH,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED;;;GAGG;AACH,MAAM,OAAO,eAAe;IACjB,IAAI,GAAG,SAAS,CAAC;IAE1B,KAAK,CAAC,SAAS;QACb,MAAM,MAAM,GAAG,IAAI,aAAa,CAAC;YAC/B,QAAQ,EAAE,MAAM;YAChB,GAAG,EAAE,EAAE,GAAG,OAAO,CAAC,GAAG,EAAE,gBAAgB,EAAE,GAAG,EAAE;SAC/C,CAAC,CAAC;QACH,IAAI,CAAC;YACH,MAAM,MAAM,CAAC,KAAK,EAAE,CAAC;YACrB,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,aAAa,EAAE,CAAC;YAC5C,IAAI,CAAC,MAAM,CAAC,eAAe,EAAE,CAAC;gBAC5B,OAAO,EAAE,EAAE,EAAE,KAAK,EAAE,MAAM,EAAE,oEAAoE,EAAE,CAAC;YACrG,CAAC;YACD,OAAO,EAAE,EAAE,EAAE,IAAI,EAAE,MAAM,EAAE,oBAAoB,MAAM,CAAC,KAAK,IAAI,MAAM,KAAK,MAAM,CAAC,QAAQ,IAAI,MAAM,GAAG,EAAE,CAAC;QAC3G,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,OAAO,EAAE,EAAE,EAAE,KAAK,EAAE,MAAM,EAAE,oCAAqC,GAAa,CAAC,OAAO,EAAE,EAAE,CAAC;QAC7F,CAAC;gBAAS,CAAC;YACT,MAAM,MAAM,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;QAC7C,CAAC;IACH,CAAC;IAED,KAAK,CAAC,iBAAiB,CAAC,OAA4B;QAClD,MAAM,MAAM,GAAG,IAAI,aAAa,CAAC;YAC/B,gBAAgB,EAAE,OAAO,CAAC,GAAG;YAC7B,QAAQ,EAAE,MAAM;YAChB,GAAG,EAAE,EAAE,GAAG,OAAO,CAAC,GAAG,EAAE,gBAAgB,EAAE,GAAG,EAAE;SAC/C,CAAC,CAAC;QACH,MAAM,MAAM,CAAC,KAAK,EAAE,CAAC;QAErB,MAAM,OAAO,GAAmB,MAAM,MAAM,CAAC,aAAa,CAAC;YACzD,GAAG,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,OAAO,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YAClD,mBAAmB,EAAE,UAAU;YAC/B,GAAG,CAAC,OAAO,CAAC,aAAa,CAAC,CAAC,CAAC,EAAE,aAAa,EAAE,EAAE,OAAO,EAAE,OAAO,CAAC,aAAa,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;SACxF,CAAC,CAAC;QAEH,MAAM,IAAI,GAAG,OAAO,CAAC,OAAO,CAAC;QAC7B,IAAI,IAAI,EAAE,CAAC;YACT,OAAO,CAAC,EAAE,CAAC,kBAAkB,EAAE,CAAC,KAAK,EAAE,EAAE;gBACvC,MAAM,MAAM,GAAI,KAAK,CAAC,IAAwC,EAAE,MAAM,CAAC;gBACvE,IAAI,MAAM;oBAAE,IAAI,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,MAAM,EAAE,CAAC,CAAC;YACrD,CAAC,CAAC,CAAC;YACH,OAAO,CAAC,EAAE,CAAC,qBAAqB,EAAE,CAAC,KAAK,EAAE,EAAE;gBAC1C,MAAM,OAAO,GAAI,KAAK,CAAC,IAAyC,EAAE,OAAO,CAAC;gBAC1E,IAAI,OAAO;oBAAE,IAAI,CAAC,EAAE,IAAI,EAAE,WAAW,EAAE,IAAI,EAAE,OAAO,EAAE,CAAC,CAAC;YAC1D,CAAC,CAAC,CAAC;YACH,OAAO,CAAC,EAAE,CAAC,sBAAsB,EAAE,CAAC,KAAK,EAAE,EAAE;gBAC3C,MAAM,IAAI,GAAG,KAAK,CAAC,IAA8E,CAAC;gBAClG,IAAI,IAAI,EAAE,QAAQ,EAAE,CAAC;oBACnB,MAAM,MAAM,GAAG,aAAa,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;oBAC7C,IAAI,CAAC,EAAE,IAAI,EAAE,YAAY,EAAE,IAAI,EAAE,IAAI,CAAC,QAAQ,EAAE,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC;gBACnF,CAAC;YACH,CAAC,CAAC,CAAC;YACH,OAAO,CAAC,EAAE,CAAC,yBAAyB,EAAE,CAAC,KAAK,EAAE,EAAE;gBAC9C,MAAM,IAAI,GAAG,KAAK,CAAC,IAAuE,CAAC;gBAC3F,IAAI,CAAC;oBACH,IAAI,EAAE,WAAW;oBACjB,IAAI,EAAE,IAAI,EAAE,OAAO,KAAK,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,EAAE,OAAO,IAAI,QAAQ,CAAC,CAAC,CAAC,IAAI;oBACtE,EAAE,EAAE,IAAI,EAAE,OAAO,KAAK,KAAK;iBAC5B,CAAC,CAAC;YACL,CAAC,CAAC,CAAC;YACH,OAAO,CAAC,EAAE,CAAC,mBAAmB,EAAE,CAAC,KAAK,EAAE,EAAE;gBACxC,MAAM,OAAO,GAAG,KAAK,CAAC,IAAI,EAAE,OAAO,CAAC;gBACpC,IAAI,OAAO;oBAAE,IAAI,CAAC,EAAE,IAAI,EAAE,WAAW,EAAE,IAAI,EAAE,OAAO,EAAE,CAAC,CAAC;YAC1D,CAAC,CAAC,CAAC;YACH,OAAO,CAAC,EAAE,CAAC,0BAA0B,EAAE,GAAG,EAAE;gBAC1C,IAAI,CAAC,EAAE,IAAI,EAAE,YAAY,EAAE,IAAI,EAAE,qBAAqB,EAAE,CAAC,CAAC;YAC5D,CAAC,CAAC,CAAC;QACL,CAAC;QAED,MAAM,YAAY,GAAG,OAAO,CAAC,YAAY,IAAI,CAAC,GAAG,EAAE,GAAG,IAAI,CAAC;QAC3D,MAAM,KAAK,GAAG,OAAO,CAAC,KAAK,IAAI,EAAE,GAAG,EAAE,GAAG,IAAI,CAAC;QAE9C,OAAO;YACL,oFAAoF;YACpF,uFAAuF;YACvF,oDAAoD;YACpD,IAAI,CAAC,MAAc;gBACjB,OAAO,IAAI,OAAO,CAAS,CAAC,OAAO,EAAE,MAAM,EAAE,EAAE;oBAC7C,IAAI,YAAY,GAAG,EAAE,CAAC;oBACtB,IAAI,YAAY,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;oBAC9B,MAAM,SAAS,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;oBAC7B,IAAI,OAAO,GAAG,KAAK,CAAC;oBAEpB,MAAM,WAAW,GAAG,OAAO,CAAC,EAAE,CAAC,GAAG,EAAE;wBAClC,YAAY,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;oBAC5B,CAAC,CAAC,CAAC;oBACH,MAAM,UAAU,GAAG,OAAO,CAAC,EAAE,CAAC,mBAAmB,EAAE,CAAC,KAAK,EAAE,EAAE;wBAC3D,MAAM,OAAO,GAAG,KAAK,CAAC,IAAI,EAAE,OAAO,CAAC;wBACpC,IAAI,OAAO;4BAAE,YAAY,GAAG,OAAO,CAAC;oBACtC,CAAC,CAAC,CAAC;oBACH,MAAM,OAAO,GAAG,OAAO,CAAC,EAAE,CAAC,cAAc,EAAE,GAAG,EAAE,CAAC,MAAM,CAAC,GAAG,EAAE,CAAC,OAAO,CAAC,YAAY,CAAC,CAAC,CAAC,CAAC;oBAEtF,MAAM,KAAK,GAAG,WAAW,CAAC,GAAG,EAAE;wBAC7B,MAAM,OAAO,GAAG,IAAI,CAAC,GAAG,EAAE,GAAG,YAAY,CAAC;wBAC1C,MAAM,QAAQ,GAAG,IAAI,CAAC,GAAG,EAAE,GAAG,SAAS,CAAC;wBACxC,IAAI,OAAO,IAAI,YAAY,EAAE,CAAC;4BAC5B,MAAM,CAAC,GAAG,EAAE;gCACV,KAAK,OAAO,CAAC,KAAK,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;gCAC5C,MAAM,CAAC,IAAI,iBAAiB,CAAC,wBAAwB,GAAG,IAAI,CAAC,KAAK,CAAC,YAAY,GAAG,IAAI,CAAC,GAAG,GAAG,EAAE,YAAY,CAAC,CAAC,CAAC;4BAChH,CAAC,CAAC,CAAC;wBACL,CAAC;6BAAM,IAAI,QAAQ,IAAI,KAAK,EAAE,CAAC;4BAC7B,MAAM,CAAC,GAAG,EAAE;gCACV,KAAK,OAAO,CAAC,KAAK,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;gCAC5C,MAAM,CAAC,IAAI,iBAAiB,CAAC,4BAA4B,GAAG,IAAI,CAAC,KAAK,CAAC,KAAK,GAAG,KAAK,CAAC,GAAG,GAAG,EAAE,UAAU,CAAC,CAAC,CAAC;4BAC5G,CAAC,CAAC,CAAC;wBACL,CAAC;oBACH,CAAC,EAAE,IAAI,CAAC,CAAC;oBACT,IAAI,KAAK,CAAC,KAAK;wBAAE,KAAK,CAAC,KAAK,EAAE,CAAC;oBAE/B,SAAS,MAAM,CAAC,MAAkB;wBAChC,IAAI,OAAO;4BAAE,OAAO;wBACpB,OAAO,GAAG,IAAI,CAAC;wBACf,aAAa,CAAC,KAAK,CAAC,CAAC;wBACrB,WAAW,EAAE,CAAC;wBACd,UAAU,EAAE,CAAC;wBACb,OAAO,EAAE,CAAC;wBACV,MAAM,EAAE,CAAC;oBACX,CAAC;oBAED,OAAO,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,MAAM,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC;gBACrE,CAAC,CAAC,CAAC;YACL,CAAC;YACD,KAAK,CAAC,GAAG;gBACP,MAAM,OAAO,CAAC,UAAU,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;gBAClD,MAAM,MAAM,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;YAC7C,CAAC;SACF,CAAC;IACJ,CAAC;CACF"}
+{"version":3,"file":"copilot.js","sourceRoot":"","sources":["../../src/agent/copilot.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,aAAa,EAAE,UAAU,EAAE,MAAM,qBAAqB,CAAC;AAEhE,OAAO,EACL,iBAAiB,GAMlB,MAAM,eAAe,CAAC;AAEvB,SAAS,aAAa,CAAC,IAAyC;IAC9D,IAAI,CAAC,IAAI;QAAE,OAAO,SAAS,CAAC;IAC5B,KAAK,MAAM,GAAG,IAAI,CAAC,SAAS,EAAE,MAAM,EAAE,UAAU,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE,QAAQ,CAAC,EAAE,CAAC;QACvF,MAAM,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC;QACpB,IAAI,OAAO,CAAC,KAAK,QAAQ,IAAI,CAAC,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YAC1C,OAAO,GAAG,GAAG,IAAI,CAAC,CAAC,MAAM,GAAG,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,GAAG,GAAG,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;QAC9D,CAAC;IACH,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED;;;GAGG;AACH,MAAM,OAAO,eAAe;IACjB,IAAI,GAAG,SAAS,CAAC;IAE1B,KAAK,CAAC,SAAS;QACb,MAAM,MAAM,GAAG,IAAI,aAAa,CAAC;YAC/B,QAAQ,EAAE,MAAM;YAChB,GAAG,EAAE,EAAE,GAAG,OAAO,CAAC,GAAG,EAAE,gBAAgB,EAAE,GAAG,EAAE;SAC/C,CAAC,CAAC;QACH,IAAI,CAAC;YACH,MAAM,MAAM,CAAC,KAAK,EAAE,CAAC;YACrB,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,aAAa,EAAE,CAAC;YAC5C,IAAI,CAAC,MAAM,CAAC,eAAe,EAAE,CAAC;gBAC5B,OAAO,EAAE,EAAE,EAAE,KAAK,EAAE,MAAM,EAAE,oEAAoE,EAAE,CAAC;YACrG,CAAC;YACD,OAAO,EAAE,EAAE,EAAE,IAAI,EAAE,MAAM,EAAE,oBAAoB,MAAM,CAAC,KAAK,IAAI,MAAM,KAAK,MAAM,CAAC,QAAQ,IAAI,MAAM,GAAG,EAAE,CAAC;QAC3G,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,OAAO,EAAE,EAAE,EAAE,KAAK,EAAE,MAAM,EAAE,oCAAqC,GAAa,CAAC,OAAO,EAAE,EAAE,CAAC;QAC7F,CAAC;gBAAS,CAAC;YACT,MAAM,MAAM,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;QAC7C,CAAC;IACH,CAAC;IAED,KAAK,CAAC,aAAa,CAAC,OAAe;QACjC,MAAM,MAAM,GAAG,IAAI,aAAa,CAAC;YAC/B,QAAQ,EAAE,MAAM;YAChB,GAAG,EAAE,EAAE,GAAG,OAAO,CAAC,GAAG,EAAE,gBAAgB,EAAE,GAAG,EAAE;SAC/C,CAAC,CAAC;QACH,IAAI,CAAC;YACH,MAAM,MAAM,CAAC,KAAK,EAAE,CAAC;YACrB,MAAM,MAAM,GAAG,MAAM,MAAM,CAAC,UAAU,EAAE,CAAC;YACzC,MAAM,GAAG,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;YACpC,IAAI,GAAG,CAAC,QAAQ,CAAC,OAAO,CAAC;gBAAE,OAAO,EAAE,EAAE,EAAE,IAAI,EAAE,CAAC;YAC/C,kEAAkE;YAClE,MAAM,MAAM,GAAG,OAAO,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,CAAE,CAAC,WAAW,EAAE,CAAC;YACvD,MAAM,IAAI,GAAG,GAAG,CAAC,MAAM,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE,CAAC,WAAW,EAAE,CAAC,UAAU,CAAC,MAAM,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;YAC5E,OAAO;gBACL,EAAE,EAAE,KAAK;gBACT,SAAS,EAAE,GAAG,CAAC,IAAI,EAAE;gBACrB,MAAM,EAAE,IAAI,CAAC,MAAM;oBACjB,CAAC,CAAC,iBAAiB,IAAI,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG;oBACjD,CAAC,CAAC,cAAc,GAAG,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,GAAG,GAAG,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,EAAE;aAC7E,CAAC;QACJ,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,4EAA4E;YAC5E,4CAA4C;YAC5C,OAAO,EAAE,EAAE,EAAE,IAAI,EAAE,MAAM,EAAE,0BAA2B,GAAa,CAAC,OAAO,EAAE,EAAE,CAAC;QAClF,CAAC;gBAAS,CAAC;YACT,MAAM,MAAM,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;QAC7C,CAAC;IACH,CAAC;IAED,KAAK,CAAC,iBAAiB,CAAC,OAA4B;QAClD,MAAM,MAAM,GAAG,IAAI,aAAa,CAAC;YAC/B,gBAAgB,EAAE,OAAO,CAAC,GAAG;YAC7B,QAAQ,EAAE,MAAM;YAChB,GAAG,EAAE,EAAE,GAAG,OAAO,CAAC,GAAG,EAAE,gBAAgB,EAAE,GAAG,EAAE;SAC/C,CAAC,CAAC;QACH,MAAM,MAAM,CAAC,KAAK,EAAE,CAAC;QAErB,4FAA4F;QAC5F,6FAA6F;QAC7F,gFAAgF;QAChF,IAAI,OAAuB,CAAC;QAC5B,IAAI,CAAC;YACH,OAAO,GAAG,MAAM,MAAM,CAAC,aAAa,CAAC;gBACnC,GAAG,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,OAAO,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;gBAClD,mBAAmB,EAAE,UAAU;gBAC/B,GAAG,CAAC,OAAO,CAAC,aAAa,CAAC,CAAC,CAAC,EAAE,aAAa,EAAE,EAAE,OAAO,EAAE,OAAO,CAAC,aAAa,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;aACxF,CAAC,CAAC;QACL,CAAC;QAAC,OAAO,GAAG,EAAE,CAAC;YACb,MAAM,MAAM,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;YAC3C,MAAM,GAAG,CAAC;QACZ,CAAC;QAED,MAAM,IAAI,GAAG,OAAO,CAAC,OAAO,CAAC;QAC7B,IAAI,IAAI,EAAE,CAAC;YACT,OAAO,CAAC,EAAE,CAAC,kBAAkB,EAAE,CAAC,KAAK,EAAE,EAAE;gBACvC,MAAM,MAAM,GAAI,KAAK,CAAC,IAAwC,EAAE,MAAM,CAAC;gBACvE,IAAI,MAAM;oBAAE,IAAI,CAAC,EAAE,IAAI,EAAE,QAAQ,EAAE,IAAI,EAAE,MAAM,EAAE,CAAC,CAAC;YACrD,CAAC,CAAC,CAAC;YACH,OAAO,CAAC,EAAE,CAAC,qBAAqB,EAAE,CAAC,KAAK,EAAE,EAAE;gBAC1C,MAAM,OAAO,GAAI,KAAK,CAAC,IAAyC,EAAE,OAAO,CAAC;gBAC1E,IAAI,OAAO;oBAAE,IAAI,CAAC,EAAE,IAAI,EAAE,WAAW,EAAE,IAAI,EAAE,OAAO,EAAE,CAAC,CAAC;YAC1D,CAAC,CAAC,CAAC;YACH,OAAO,CAAC,EAAE,CAAC,sBAAsB,EAAE,CAAC,KAAK,EAAE,EAAE;gBAC3C,MAAM,IAAI,GAAG,KAAK,CAAC,IAA8E,CAAC;gBAClG,IAAI,IAAI,EAAE,QAAQ,EAAE,CAAC;oBACnB,MAAM,MAAM,GAAG,aAAa,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;oBAC7C,IAAI,CAAC,EAAE,IAAI,EAAE,YAAY,EAAE,IAAI,EAAE,IAAI,CAAC,QAAQ,EAAE,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC;gBACnF,CAAC;YACH,CAAC,CAAC,CAAC;YACH,OAAO,CAAC,EAAE,CAAC,yBAAyB,EAAE,CAAC,KAAK,EAAE,EAAE;gBAC9C,MAAM,IAAI,GAAG,KAAK,CAAC,IAAuE,CAAC;gBAC3F,IAAI,CAAC;oBACH,IAAI,EAAE,WAAW;oBACjB,IAAI,EAAE,IAAI,EAAE,OAAO,KAAK,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,EAAE,OAAO,IAAI,QAAQ,CAAC,CAAC,CAAC,IAAI;oBACtE,EAAE,EAAE,IAAI,EAAE,OAAO,KAAK,KAAK;iBAC5B,CAAC,CAAC;YACL,CAAC,CAAC,CAAC;YACH,OAAO,CAAC,EAAE,CAAC,mBAAmB,EAAE,CAAC,KAAK,EAAE,EAAE;gBACxC,MAAM,OAAO,GAAG,KAAK,CAAC,IAAI,EAAE,OAAO,CAAC;gBACpC,IAAI,OAAO;oBAAE,IAAI,CAAC,EAAE,IAAI,EAAE,WAAW,EAAE,IAAI,EAAE,OAAO,EAAE,CAAC,CAAC;YAC1D,CAAC,CAAC,CAAC;YACH,OAAO,CAAC,EAAE,CAAC,0BAA0B,EAAE,GAAG,EAAE;gBAC1C,IAAI,CAAC,EAAE,IAAI,EAAE,YAAY,EAAE,IAAI,EAAE,qBAAqB,EAAE,CAAC,CAAC;YAC5D,CAAC,CAAC,CAAC;QACL,CAAC;QAED,MAAM,YAAY,GAAG,OAAO,CAAC,YAAY,IAAI,CAAC,GAAG,EAAE,GAAG,IAAI,CAAC;QAC3D,MAAM,KAAK,GAAG,OAAO,CAAC,KAAK,IAAI,EAAE,GAAG,EAAE,GAAG,IAAI,CAAC;QAE9C,OAAO;YACL,oFAAoF;YACpF,uFAAuF;YACvF,oDAAoD;YACpD,IAAI,CAAC,MAAc;gBACjB,OAAO,IAAI,OAAO,CAAS,CAAC,OAAO,EAAE,MAAM,EAAE,EAAE;oBAC7C,IAAI,YAAY,GAAG,EAAE,CAAC;oBACtB,IAAI,YAAY,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;oBAC9B,MAAM,SAAS,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;oBAC7B,IAAI,OAAO,GAAG,KAAK,CAAC;oBAEpB,MAAM,WAAW,GAAG,OAAO,CAAC,EAAE,CAAC,GAAG,EAAE;wBAClC,YAAY,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;oBAC5B,CAAC,CAAC,CAAC;oBACH,MAAM,UAAU,GAAG,OAAO,CAAC,EAAE,CAAC,mBAAmB,EAAE,CAAC,KAAK,EAAE,EAAE;wBAC3D,MAAM,OAAO,GAAG,KAAK,CAAC,IAAI,EAAE,OAAO,CAAC;wBACpC,IAAI,OAAO;4BAAE,YAAY,GAAG,OAAO,CAAC;oBACtC,CAAC,CAAC,CAAC;oBACH,MAAM,OAAO,GAAG,OAAO,CAAC,EAAE,CAAC,cAAc,EAAE,GAAG,EAAE,CAAC,MAAM,CAAC,GAAG,EAAE,CAAC,OAAO,CAAC,YAAY,CAAC,CAAC,CAAC,CAAC;oBAEtF,MAAM,KAAK,GAAG,WAAW,CAAC,GAAG,EAAE;wBAC7B,MAAM,OAAO,GAAG,IAAI,CAAC,GAAG,EAAE,GAAG,YAAY,CAAC;wBAC1C,MAAM,QAAQ,GAAG,IAAI,CAAC,GAAG,EAAE,GAAG,SAAS,CAAC;wBACxC,IAAI,OAAO,IAAI,YAAY,EAAE,CAAC;4BAC5B,MAAM,CAAC,GAAG,EAAE;gCACV,KAAK,OAAO,CAAC,KAAK,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;gCAC5C,MAAM,CAAC,IAAI,iBAAiB,CAAC,wBAAwB,GAAG,IAAI,CAAC,KAAK,CAAC,YAAY,GAAG,IAAI,CAAC,GAAG,GAAG,EAAE,YAAY,CAAC,CAAC,CAAC;4BAChH,CAAC,CAAC,CAAC;wBACL,CAAC;6BAAM,IAAI,QAAQ,IAAI,KAAK,EAAE,CAAC;4BAC7B,MAAM,CAAC,GAAG,EAAE;gCACV,KAAK,OAAO,CAAC,KAAK,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;gCAC5C,MAAM,CAAC,IAAI,iBAAiB,CAAC,4BAA4B,GAAG,IAAI,CAAC,KAAK,CAAC,KAAK,GAAG,KAAK,CAAC,GAAG,GAAG,EAAE,UAAU,CAAC,CAAC,CAAC;4BAC5G,CAAC,CAAC,CAAC;wBACL,CAAC;oBACH,CAAC,EAAE,IAAI,CAAC,CAAC;oBACT,IAAI,KAAK,CAAC,KAAK;wBAAE,KAAK,CAAC,KAAK,EAAE,CAAC;oBAE/B,SAAS,MAAM,CAAC,MAAkB;wBAChC,IAAI,OAAO;4BAAE,OAAO;wBACpB,OAAO,GAAG,IAAI,CAAC;wBACf,aAAa,CAAC,KAAK,CAAC,CAAC;wBACrB,WAAW,EAAE,CAAC;wBACd,UAAU,EAAE,CAAC;wBACb,OAAO,EAAE,CAAC;wBACV,MAAM,EAAE,CAAC;oBACX,CAAC;oBAED,OAAO,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,GAAG,EAAE,EAAE,CAAC,MAAM,CAAC,GAAG,EAAE,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC;gBACrE,CAAC,CAAC,CAAC;YACL,CAAC;YACD,KAAK,CAAC,GAAG;gBACP,MAAM,OAAO,CAAC,UAAU,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;gBAClD,MAAM,MAAM,CAAC,IAAI,EAAE,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;YAC7C,CAAC;SACF,CAAC;IACJ,CAAC;CACF"}

package/dist/cli.js

@@ -15,6 +15,9 @@ const BOOLEAN_FLAGS = new Set([
     "static",
     "dynamic",
     "rubric",
+    "scaffold",
+    "allow-same-family",
+    "include-bundled",
     "json",
     "no-tasks",
     "verbose",
@@ -69,16 +72,25 @@ ${color.bold("Commands:")}
   eval [path]      Evaluate the harness itself (defaults to the full agentic, harness-on run)
                      --static   fast deterministic structural audit, no model (use in CI)
                      --rubric   print what's evaluated (rubric axes, issue codes, scenarios)
-                     --scenario <id>   run one scenario only (e.g. fix-failing-test)
-                     --variant <name>  label this run (default 'harness'; use 'baseline' for harness-OFF)
-                     --timeout <min>   absolute cap per agent turn (default 45)
+                     --scaffold generate repo-specific scenarios via an agent turn, using the
+                                bundled generic scenarios as templates (--scaffold-count N, default 2)
+                     --include-bundled   include the generic 'bundled' template scenarios in the run
+                                         (default: only repo-specific scenarios run)
+                     --scenario <id>          run one scenario only (e.g. fix-failing-test)
+                     --variant <name>         label this run (default 'harness'; 'baseline' = harness OFF)
+                     --producer-model <id>    producer model (default: developer.yml model)
+                     --judge-model <id>       judge model (default: reviewer.yml model — different family)
+                     --allow-same-family      override the producer/judge family check (recorded in results)
+                     --n <int>                trials per scenario (default 1 single, 5 in baseline mode)
+                     --seed <int>             reproducibility seed (passed through where supported)
+                     --timeout <min>          absolute cap per agent turn (default 45)
   doctor [path]    Quick health check (installed? agent reachable? score?)
   dashboard [path] Show agent roster, live GitHub tasks, harness score, and evals
                      --html [file]  write a self-contained HTML dashboard
                      --no-tasks     skip live GitHub lookups (offline)
 
 ${color.bold("Options:")}
-  --model <id>     Model to use for agentic steps (e.g. claude-sonnet-4.5, gpt-5)
+  --model <id>     Model to use for agentic steps (e.g. claude-sonnet-4.6, gpt-5.5)
   --dry-run        Show what would happen without writing or calling the model
   --force          (init) overwrite existing user files (off by default; init is non-destructive)
   --skip-agent     Install/update the canonical harness without the agentic steps
@@ -139,7 +151,15 @@ async function main() {
                     mode,
                     json: Boolean(flags.json),
                     rubric: Boolean(flags.rubric),
+                    scaffold: Boolean(flags.scaffold),
+                    ...(flags["scaffold-count"] != null ? { scaffoldCount: Number(flags["scaffold-count"]) } : {}),
+                    ...(flags["include-bundled"] ? { includeBundled: true } : {}),
                     ...(model ? { model } : {}),
+                    ...(typeof flags["producer-model"] === "string" ? { producerModel: flags["producer-model"] } : {}),
+                    ...(typeof flags["judge-model"] === "string" ? { judgeModel: flags["judge-model"] } : {}),
+                    ...(flags["allow-same-family"] ? { allowSameFamily: true } : {}),
+                    ...(flags.n != null ? { trials: Number(flags.n) } : {}),
+                    ...(flags.seed != null ? { seed: Number(flags.seed) } : {}),
                     ...(flags.min != null ? { min: Number(flags.min) } : {}),
                     ...(typeof flags.scenario === "string" ? { scenario: flags.scenario } : {}),
                     ...(typeof flags.variant === "string" ? { variant: flags.variant } : {}),
@@ -170,5 +190,10 @@ async function main() {
         return 1;
     }
 }
-main().then((code) => process.exit(code));
+// Setting process.exitCode (instead of calling process.exit immediately) lets pending
+// stdout writes drain when output is piped to another process — required because the
+// JSON audit report can exceed the macOS pipe buffer and would otherwise be truncated.
+main().then((code) => {
+    process.exitCode = code;
+});
 //# sourceMappingURL=cli.js.map

package/dist/cli.js.map

@@ -1 +1 @@
-{"version":3,"file":"cli.js","sourceRoot":"","sources":["../src/cli.ts"],"names":[],"mappings":";AACA,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AACpC,OAAO,EAAE,UAAU,EAAE,GAAG,EAAE,KAAK,EAAE,MAAM,kBAAkB,CAAC;AAC1D,OAAO,EAAE,WAAW,EAAE,MAAM,oBAAoB,CAAC;AACjD,OAAO,EAAE,aAAa,EAAE,MAAM,sBAAsB,CAAC;AACrD,OAAO,EAAE,WAAW,EAAE,MAAM,oBAAoB,CAAC;AACjD,OAAO,EAAE,aAAa,EAAE,MAAM,sBAAsB,CAAC;AACrD,OAAO,EAAE,gBAAgB,EAAE,MAAM,yBAAyB,CAAC;AAC3D,OAAO,EAAE,cAAc,EAAE,MAAM,uBAAuB,CAAC;AACvD,OAAO,GAAG,MAAM,cAAc,CAAC;AAQ/B,MAAM,aAAa,GAAG,IAAI,GAAG,CAAC;IAC5B,SAAS;IACT,MAAM;IACN,YAAY;IACZ,QAAQ;IACR,SAAS;IACT,QAAQ;IACR,MAAM;IACN,UAAU;IACV,SAAS;IACT,KAAK;IACL,MAAM;IACN,SAAS;CACV,CAAC,CAAC;AAEH,SAAS,SAAS,CAAC,IAAc;IAC/B,MAAM,WAAW,GAAa,EAAE,CAAC;IACjC,MAAM,KAAK,GAAqC,EAAE,CAAC;IACnD,IAAI,OAA2B,CAAC;IAChC,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACrC,MAAM,CAAC,GAAG,IAAI,CAAC,CAAC,CAAE,CAAC;QACnB,IAAI,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,EAAE,CAAC;YACvB,MAAM,GAAG,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC;YACvB,MAAM,IAAI,GAAG,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;YACzB,IAAI,CAAC,aAAa,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,IAAI,KAAK,SAAS,IAAI,CAAC,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE,CAAC;gBAC3E,KAAK,CAAC,GAAG,CAAC,GAAG,IAAI,CAAC;gBAClB,CAAC,EAAE,CAAC;YACN,CAAC;iBAAM,CAAC;gBACN,KAAK,CAAC,GAAG,CAAC,GAAG,IAAI,CAAC;YACpB,CAAC;QACH,CAAC;aAAM,IAAI,CAAC,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE,CAAC;YAC7B,KAAK,MAAM,EAAE,IAAI,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC;gBAAE,KAAK,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC;QAChD,CAAC;aAAM,IAAI,OAAO,KAAK,SAAS,EAAE,CAAC;YACjC,OAAO,GAAG,CAAC,CAAC;QACd,CAAC;aAAM,CAAC;YACN,WAAW,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QACtB,CAAC;IACH,CAAC;IACD,OAAO,EAAE,OAAO,EAAE,WAAW,EAAE,KAAK,EAAE,CAAC;AACzC,CAAC;AAED,MAAM,IAAI,GAAG,GAAG,KAAK,CAAC,IAAI,CAAC,UAAU,CAAC;;EAEpC,KAAK,CAAC,IAAI,CAAC,QAAQ,CAAC;;EAEpB,KAAK,CAAC,IAAI,CAAC,WAAW,CAAC;;;;;;;;;;;;;;;;;;;;;EAqBvB,KAAK,CAAC,IAAI,CAAC,UAAU,CAAC;;;;;;;;;;;;;;;EAetB,KAAK,CAAC,IAAI,CAAC,cAAc,CAAC;;;EAG1B,KAAK,CAAC,GAAG,CAAC,mGAAmG,CAAC,EAAE,CAAC;AAEnH,KAAK,UAAU,IAAI;IACjB,MAAM,EAAE,OAAO,EAAE,WAAW,EAAE,KAAK,EAAE,GAAG,SAAS,CAAC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;IAEzE,IAAI,KAAK,CAAC,OAAO,IAAI,KAAK,CAAC,CAAC,IAAI,OAAO,KAAK,SAAS,EAAE,CAAC;QACtD,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;QACzB,OAAO,CAAC,CAAC;IACX,CAAC;IACD,IAAI,KAAK,CAAC,IAAI,IAAI,KAAK,CAAC,CAAC,IAAI,OAAO,KAAK,MAAM,EAAE,CAAC;QAChD,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QAClB,OAAO,CAAC,CAAC;IACX,CAAC;IACD,IAAI,CAAC,OAAO,EAAE,CAAC;QACb,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QAClB,OAAO,CAAC,CAAC;IACX,CAAC;IAED,UAAU,CAAC,OAAO,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC;IACnC,MAAM,QAAQ,GAAG,OAAO,CAAC,WAAW,CAAC,CAAC,CAAC,IAAI,GAAG,CAAC,CAAC;IAChD,MAAM,KAAK,GAAG,OAAO,KAAK,CAAC,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC;IAExE,IAAI,CAAC;QACH,QAAQ,OAAO,EAAE,CAAC;YAChB,KAAK,MAAM;gBACT,OAAO,MAAM,WAAW,CAAC,QAAQ,EAAE;oBACjC,MAAM,EAAE,OAAO,CAAC,KAAK,CAAC,SAAS,CAAC,CAAC;oBACjC,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAC3B,GAAG,EAAE,OAAO,CAAC,KAAK,CAAC,GAAG,CAAC;oBACvB,OAAO,EAAE,OAAO,CAAC,KAAK,CAAC,OAAO,CAAC;oBAC/B,SAAS,EAAE,OAAO,CAAC,KAAK,CAAC,YAAY,CAAC,CAAC;oBACvC,KAAK,EAAE,OAAO,CAAC,KAAK,CAAC,KAAK,CAAC;iBAC5B,CAAC,CAAC;YACL,KAAK,QAAQ;gBACX,OAAO,MAAM,aAAa,CAAC,QAAQ,EAAE;oBACnC,MAAM,EAAE,OAAO,CAAC,KAAK,CAAC,SAAS,CAAC,CAAC;oBACjC,IAAI,EAAE,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC;oBACzB,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAC3B,OAAO,EAAE,OAAO,CAAC,KAAK,CAAC,OAAO,CAAC;oBAC/B,SAAS,EAAE,OAAO,CAAC,KAAK,CAAC,YAAY,CAAC,CAAC;iBACxC,CAAC,CAAC;YACL,KAAK,MAAM,CAAC,CAAC,CAAC;gBACZ,gGAAgG;gBAChG,MAAM,IAAI,GAAG,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,SAAS,CAAC;gBACjD,OAAO,MAAM,WAAW,CAAC,QAAQ,EAAE;oBACjC,IAAI;oBACJ,IAAI,EAAE,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC;oBACzB,MAAM,EAAE,OAAO,CAAC,KAAK,CAAC,MAAM,CAAC;oBAC7B,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAC3B,GAAG,CAAC,KAAK,CAAC,GAAG,IAAI,IAAI,CAAC,CAAC,CAAC,EAAE,GAAG,EAAE,MAAM,CAAC,KAAK,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBACxD,GAAG,CAAC,OAAO,KAAK,CAAC,QAAQ,KAAK,QAAQ,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,KAAK,CAAC,QAAQ,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAC3E,GAAG,CAAC,OAAO,KAAK,CAAC,OAAO,KAAK,QAAQ,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBACxE,GAAG,CAAC,KAAK,CAAC,OAAO,IAAI,IAAI,CAAC,CAAC,CAAC,EAAE,cAAc,EAAE,MAAM,CAAC,KAAK,CAAC,OAAO,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAC3E,OAAO,EAAE,OAAO,CAAC,KAAK,CAAC,OAAO,CAAC;iBAChC,CAAC,CAAC;YACL,CAAC;YACD,KAAK,QAAQ;gBACX,OAAO,MAAM,aAAa,CAAC,QAAQ,EAAE,EAAE,IAAI,EAAE,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;YACtE,KAAK,SAAS;gBACZ,OAAO,cAAc,CAAC,QAAQ,EAAE,EAAE,IAAI,EAAE,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;YACjE,KAAK,WAAW;gBACd,OAAO,gBAAgB,CAAC,QAAQ,EAAE;oBAChC,IAAI,EAAE,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC;oBACzB,OAAO,EAAE,OAAO,CAAC,KAAK,CAAC,UAAU,CAAC,CAAC;oBACnC,GAAG,CAAC,KAAK,CAAC,IAAI,IAAI,IAAI,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;iBACpD,CAAC,CAAC;YACL;gBACE,GAAG,CAAC,KAAK,CAAC,oBAAoB,OAAO,EAAE,CAAC,CAAC;gBACzC,OAAO,CAAC,GAAG,CAAC,KAAK,IAAI,EAAE,CAAC,CAAC;gBACzB,OAAO,CAAC,CAAC;QACb,CAAC;IACH,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,GAAG,CAAC,KAAK,CAAE,GAAa,CAAC,OAAO,CAAC,CAAC;QAClC,IAAI,KAAK,CAAC,OAAO;YAAE,OAAO,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;QACtC,OAAO,CAAC,CAAC;IACX,CAAC;AACH,CAAC;AAED,IAAI,EAAE,CAAC,IAAI,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,OAAO,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC"}
+{"version":3,"file":"cli.js","sourceRoot":"","sources":["../src/cli.ts"],"names":[],"mappings":";AACA,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AACpC,OAAO,EAAE,UAAU,EAAE,GAAG,EAAE,KAAK,EAAE,MAAM,kBAAkB,CAAC;AAC1D,OAAO,EAAE,WAAW,EAAE,MAAM,oBAAoB,CAAC;AACjD,OAAO,EAAE,aAAa,EAAE,MAAM,sBAAsB,CAAC;AACrD,OAAO,EAAE,WAAW,EAAE,MAAM,oBAAoB,CAAC;AACjD,OAAO,EAAE,aAAa,EAAE,MAAM,sBAAsB,CAAC;AACrD,OAAO,EAAE,gBAAgB,EAAE,MAAM,yBAAyB,CAAC;AAC3D,OAAO,EAAE,cAAc,EAAE,MAAM,uBAAuB,CAAC;AACvD,OAAO,GAAG,MAAM,cAAc,CAAC;AAQ/B,MAAM,aAAa,GAAG,IAAI,GAAG,CAAC;IAC5B,SAAS;IACT,MAAM;IACN,YAAY;IACZ,QAAQ;IACR,SAAS;IACT,QAAQ;IACR,UAAU;IACV,mBAAmB;IACnB,iBAAiB;IACjB,MAAM;IACN,UAAU;IACV,SAAS;IACT,KAAK;IACL,MAAM;IACN,SAAS;CACV,CAAC,CAAC;AAEH,SAAS,SAAS,CAAC,IAAc;IAC/B,MAAM,WAAW,GAAa,EAAE,CAAC;IACjC,MAAM,KAAK,GAAqC,EAAE,CAAC;IACnD,IAAI,OAA2B,CAAC;IAChC,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACrC,MAAM,CAAC,GAAG,IAAI,CAAC,CAAC,CAAE,CAAC;QACnB,IAAI,CAAC,CAAC,UAAU,CAAC,IAAI,CAAC,EAAE,CAAC;YACvB,MAAM,GAAG,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC;YACvB,MAAM,IAAI,GAAG,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;YACzB,IAAI,CAAC,aAAa,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,IAAI,KAAK,SAAS,IAAI,CAAC,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE,CAAC;gBAC3E,KAAK,CAAC,GAAG,CAAC,GAAG,IAAI,CAAC;gBAClB,CAAC,EAAE,CAAC;YACN,CAAC;iBAAM,CAAC;gBACN,KAAK,CAAC,GAAG,CAAC,GAAG,IAAI,CAAC;YACpB,CAAC;QACH,CAAC;aAAM,IAAI,CAAC,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE,CAAC;YAC7B,KAAK,MAAM,EAAE,IAAI,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC;gBAAE,KAAK,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC;QAChD,CAAC;aAAM,IAAI,OAAO,KAAK,SAAS,EAAE,CAAC;YACjC,OAAO,GAAG,CAAC,CAAC;QACd,CAAC;aAAM,CAAC;YACN,WAAW,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;QACtB,CAAC;IACH,CAAC;IACD,OAAO,EAAE,OAAO,EAAE,WAAW,EAAE,KAAK,EAAE,CAAC;AACzC,CAAC;AAED,MAAM,IAAI,GAAG,GAAG,KAAK,CAAC,IAAI,CAAC,UAAU,CAAC;;EAEpC,KAAK,CAAC,IAAI,CAAC,QAAQ,CAAC;;EAEpB,KAAK,CAAC,IAAI,CAAC,WAAW,CAAC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;EA8BvB,KAAK,CAAC,IAAI,CAAC,UAAU,CAAC;;;;;;;;;;;;;;;EAetB,KAAK,CAAC,IAAI,CAAC,cAAc,CAAC;;;EAG1B,KAAK,CAAC,GAAG,CAAC,mGAAmG,CAAC,EAAE,CAAC;AAEnH,KAAK,UAAU,IAAI;IACjB,MAAM,EAAE,OAAO,EAAE,WAAW,EAAE,KAAK,EAAE,GAAG,SAAS,CAAC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;IAEzE,IAAI,KAAK,CAAC,OAAO,IAAI,KAAK,CAAC,CAAC,IAAI,OAAO,KAAK,SAAS,EAAE,CAAC;QACtD,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC;QACzB,OAAO,CAAC,CAAC;IACX,CAAC;IACD,IAAI,KAAK,CAAC,IAAI,IAAI,KAAK,CAAC,CAAC,IAAI,OAAO,KAAK,MAAM,EAAE,CAAC;QAChD,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QAClB,OAAO,CAAC,CAAC;IACX,CAAC;IACD,IAAI,CAAC,OAAO,EAAE,CAAC;QACb,OAAO,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;QAClB,OAAO,CAAC,CAAC;IACX,CAAC;IAED,UAAU,CAAC,OAAO,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC;IACnC,MAAM,QAAQ,GAAG,OAAO,CAAC,WAAW,CAAC,CAAC,CAAC,IAAI,GAAG,CAAC,CAAC;IAChD,MAAM,KAAK,GAAG,OAAO,KAAK,CAAC,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC;IAExE,IAAI,CAAC;QACH,QAAQ,OAAO,EAAE,CAAC;YAChB,KAAK,MAAM;gBACT,OAAO,MAAM,WAAW,CAAC,QAAQ,EAAE;oBACjC,MAAM,EAAE,OAAO,CAAC,KAAK,CAAC,SAAS,CAAC,CAAC;oBACjC,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAC3B,GAAG,EAAE,OAAO,CAAC,KAAK,CAAC,GAAG,CAAC;oBACvB,OAAO,EAAE,OAAO,CAAC,KAAK,CAAC,OAAO,CAAC;oBAC/B,SAAS,EAAE,OAAO,CAAC,KAAK,CAAC,YAAY,CAAC,CAAC;oBACvC,KAAK,EAAE,OAAO,CAAC,KAAK,CAAC,KAAK,CAAC;iBAC5B,CAAC,CAAC;YACL,KAAK,QAAQ;gBACX,OAAO,MAAM,aAAa,CAAC,QAAQ,EAAE;oBACnC,MAAM,EAAE,OAAO,CAAC,KAAK,CAAC,SAAS,CAAC,CAAC;oBACjC,IAAI,EAAE,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC;oBACzB,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAC3B,OAAO,EAAE,OAAO,CAAC,KAAK,CAAC,OAAO,CAAC;oBAC/B,SAAS,EAAE,OAAO,CAAC,KAAK,CAAC,YAAY,CAAC,CAAC;iBACxC,CAAC,CAAC;YACL,KAAK,MAAM,CAAC,CAAC,CAAC;gBACZ,gGAAgG;gBAChG,MAAM,IAAI,GAAG,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,SAAS,CAAC;gBACjD,OAAO,MAAM,WAAW,CAAC,QAAQ,EAAE;oBACjC,IAAI;oBACJ,IAAI,EAAE,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC;oBACzB,MAAM,EAAE,OAAO,CAAC,KAAK,CAAC,MAAM,CAAC;oBAC7B,QAAQ,EAAE,OAAO,CAAC,KAAK,CAAC,QAAQ,CAAC;oBACjC,GAAG,CAAC,KAAK,CAAC,gBAAgB,CAAC,IAAI,IAAI,CAAC,CAAC,CAAC,EAAE,aAAa,EAAE,MAAM,CAAC,KAAK,CAAC,gBAAgB,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAC9F,GAAG,CAAC,KAAK,CAAC,iBAAiB,CAAC,CAAC,CAAC,CAAC,EAAE,cAAc,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAC7D,GAAG,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAC3B,GAAG,CAAC,OAAO,KAAK,CAAC,gBAAgB,CAAC,KAAK,QAAQ,CAAC,CAAC,CAAC,EAAE,aAAa,EAAE,KAAK,CAAC,gBAAgB,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAClG,GAAG,CAAC,OAAO,KAAK,CAAC,aAAa,CAAC,KAAK,QAAQ,CAAC,CAAC,CAAC,EAAE,UAAU,EAAE,KAAK,CAAC,aAAa,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBACzF,GAAG,CAAC,KAAK,CAAC,mBAAmB,CAAC,CAAC,CAAC,CAAC,EAAE,eAAe,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAChE,GAAG,CAAC,KAAK,CAAC,CAAC,IAAI,IAAI,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBACvD,GAAG,CAAC,KAAK,CAAC,IAAI,IAAI,IAAI,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAC3D,GAAG,CAAC,KAAK,CAAC,GAAG,IAAI,IAAI,CAAC,CAAC,CAAC,EAAE,GAAG,EAAE,MAAM,CAAC,KAAK,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBACxD,GAAG,CAAC,OAAO,KAAK,CAAC,QAAQ,KAAK,QAAQ,CAAC,CAAC,CAAC,EAAE,QAAQ,EAAE,KAAK,CAAC,QAAQ,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAC3E,GAAG,CAAC,OAAO,KAAK,CAAC,OAAO,KAAK,QAAQ,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,KAAK,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBACxE,GAAG,CAAC,KAAK,CAAC,OAAO,IAAI,IAAI,CAAC,CAAC,CAAC,EAAE,cAAc,EAAE,MAAM,CAAC,KAAK,CAAC,OAAO,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBAC3E,OAAO,EAAE,OAAO,CAAC,KAAK,CAAC,OAAO,CAAC;iBAChC,CAAC,CAAC;YACL,CAAC;YACD,KAAK,QAAQ;gBACX,OAAO,MAAM,aAAa,CAAC,QAAQ,EAAE,EAAE,IAAI,EAAE,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;YACtE,KAAK,SAAS;gBACZ,OAAO,cAAc,CAAC,QAAQ,EAAE,EAAE,IAAI,EAAE,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;YACjE,KAAK,WAAW;gBACd,OAAO,gBAAgB,CAAC,QAAQ,EAAE;oBAChC,IAAI,EAAE,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC;oBACzB,OAAO,EAAE,OAAO,CAAC,KAAK,CAAC,UAAU,CAAC,CAAC;oBACnC,GAAG,CAAC,KAAK,CAAC,IAAI,IAAI,IAAI,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,KAAK,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;iBACpD,CAAC,CAAC;YACL;gBACE,GAAG,CAAC,KAAK,CAAC,oBAAoB,OAAO,EAAE,CAAC,CAAC;gBACzC,OAAO,CAAC,GAAG,CAAC,KAAK,IAAI,EAAE,CAAC,CAAC;gBACzB,OAAO,CAAC,CAAC;QACb,CAAC;IACH,CAAC;IAAC,OAAO,GAAG,EAAE,CAAC;QACb,GAAG,CAAC,KAAK,CAAE,GAAa,CAAC,OAAO,CAAC,CAAC;QAClC,IAAI,KAAK,CAAC,OAAO;YAAE,OAAO,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;QACtC,OAAO,CAAC,CAAC;IACX,CAAC;AACH,CAAC;AAED,sFAAsF;AACtF,qFAAqF;AACrF,uFAAuF;AACvF,IAAI,EAAE,CAAC,IAAI,CAAC,CAAC,IAAI,EAAE,EAAE;IACnB,OAAO,CAAC,QAAQ,GAAG,IAAI,CAAC;AAC1B,CAAC,CAAC,CAAC"}

package/dist/commands/doctor.js

@@ -1,8 +1,36 @@
+import { existsSync, readdirSync, readFileSync } from "node:fs";
 import { auditHarness } from "../core/audit.js";
 import { readState, isInstalled } from "../core/state.js";
 import { validateSetupSteps } from "../core/setupsteps.js";
 import { color, log } from "../core/logger.js";
 import { getProvider } from "../agent/index.js";
+import { join } from "../core/fsutil.js";
+/** Aggregate calibration results per judge (mirror of score.mjs calibrate --report). Returns
+ *  the worst judge (lowest agreement) so doctor can decide whether to flag a warning. */
+function summarizeCalibration(repoRoot) {
+    const dir = join(repoRoot, ".agentrig/eval/calibration/results");
+    if (!existsSync(dir))
+        return [];
+    const byJudge = new Map();
+    for (const f of readdirSync(dir).filter((f) => f.endsWith(".json"))) {
+        try {
+            const rec = JSON.parse(readFileSync(join(dir, f), "utf8"));
+            const j = rec.judgeModel ?? "unknown";
+            const acc = byJudge.get(j) ?? { agreementSum: 0, biasSum: 0, n: 0 };
+            acc.agreementSum += rec.agreement ?? 0;
+            acc.biasSum += rec.bias ?? 0;
+            acc.n += 1;
+            byJudge.set(j, acc);
+        }
+        catch { /* skip */ }
+    }
+    return [...byJudge.entries()].map(([judge, v]) => ({
+        judge,
+        n: v.n,
+        agreement: v.n ? v.agreementSum / v.n : 0,
+        bias: v.n ? v.biasSum / v.n : 0,
+    }));
+}
 /** Quick health check: is a harness installed, is the agent reachable, and what is the score. */
 export async function doctorCommand(repoRoot, options) {
     const installed = isInstalled(repoRoot);
@@ -11,30 +39,47 @@ export async function doctorCommand(repoRoot, options) {
     const setup = validateSetupSteps(repoRoot);
     const provider = getProvider();
     const pre = await provider.preflight();
+    const calibration = summarizeCalibration(repoRoot);
+    // Worst judge below 80% agreement = a real concern (P5 gate). Missing calibration = warning only.
+    const CALIB_THRESHOLD = 0.8;
+    const worstJudge = calibration.sort((a, b) => a.agreement - b.agreement)[0] ?? null;
+    const calibrationOk = !worstJudge || worstJudge.agreement >= CALIB_THRESHOLD;
     if (options.json) {
         console.log(JSON.stringify({
             installed,
             knowledgeVersion: state?.knowledgeVersion ?? null,
-            harnessScore: report.harnessScore,
+            installCompleteness: report.harnessScore,
+            qualityProbes: report.qualityScore,
             setupSteps: setup,
             agent: { provider: provider.name, ...pre },
+            calibration: { threshold: CALIB_THRESHOLD, judges: calibration, ok: calibrationOk },
         }, null, 2));
-        return installed && pre.ok ? 0 : 1;
+        return installed && pre.ok && calibrationOk ? 0 : 1;
     }
     log.info(color.bold("AgentRig — doctor\n"));
-    log.info(`  harness installed : ${installed ? color.green("yes") : color.red("no")}`);
+    log.info(`  harness installed     : ${installed ? color.green("yes") : color.red("no")}`);
     if (state)
-        log.info(`  knowledge version : ${state.knowledgeVersion}`);
-    log.info(`  harness score     : ${report.harnessScore}%`);
+        log.info(`  knowledge version     : ${state.knowledgeVersion}`);
+    log.info(`  install completeness  : ${report.harnessScore}%`);
+    log.info(`  quality probes        : ${report.qualityScore}%`);
     const setupStatus = !setup.present
         ? color.dim("not present")
         : setup.ok
             ? color.green(setup.warnings.length ? `valid (${setup.warnings.length} warning)` : "valid")
             : color.red(`${setup.errors.length} error(s)`);
-    log.info(`  setup-steps.yml   : ${setupStatus}`);
-    log.info(`  agent (${provider.name})     : ${pre.ok ? color.green(pre.detail) : color.red(pre.detail)}`);
+    log.info(`  setup-steps.yml       : ${setupStatus}`);
+    log.info(`  agent (${provider.name})         : ${pre.ok ? color.green(pre.detail) : color.red(pre.detail)}`);
+    if (calibration.length === 0) {
+        log.info(`  judge calibration     : ${color.dim("no calibration runs yet (see .agentrig/eval/calibration/README.md)")}`);
+    }
+    else {
+        for (const c of calibration.sort((a, b) => a.agreement - b.agreement)) {
+            const mark = c.agreement >= CALIB_THRESHOLD ? color.green("ok") : color.red(`below ${CALIB_THRESHOLD * 100}% threshold`);
+            log.info(`  judge ${c.judge.padEnd(15)} : ${(c.agreement * 100).toFixed(1)}% agreement, bias ${c.bias.toFixed(2)}  ${mark}`);
+        }
+    }
     if (!installed)
         log.info(color.dim("\n  Run `agentrig init` to install a harness."));
-    return installed && pre.ok ? 0 : 1;
+    return installed && pre.ok && calibrationOk ? 0 : 1;
 }
 //# sourceMappingURL=doctor.js.map

package/dist/commands/doctor.js.map

@@ -1 +1 @@
-{"version":3,"file":"doctor.js","sourceRoot":"","sources":["../../src/commands/doctor.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,MAAM,kBAAkB,CAAC;AAChD,OAAO,EAAE,SAAS,EAAE,WAAW,EAAE,MAAM,kBAAkB,CAAC;AAC1D,OAAO,EAAE,kBAAkB,EAAE,MAAM,uBAAuB,CAAC;AAC3D,OAAO,EAAE,KAAK,EAAE,GAAG,EAAE,MAAM,mBAAmB,CAAC;AAC/C,OAAO,EAAE,WAAW,EAAE,MAAM,mBAAmB,CAAC;AAMhD,iGAAiG;AACjG,MAAM,CAAC,KAAK,UAAU,aAAa,CAAC,QAAgB,EAAE,OAAsB;IAC1E,MAAM,SAAS,GAAG,WAAW,CAAC,QAAQ,CAAC,CAAC;IACxC,MAAM,KAAK,GAAG,SAAS,CAAC,QAAQ,CAAC,CAAC;IAClC,MAAM,MAAM,GAAG,YAAY,CAAC,QAAQ,CAAC,CAAC;IACtC,MAAM,KAAK,GAAG,kBAAkB,CAAC,QAAQ,CAAC,CAAC;IAC3C,MAAM,QAAQ,GAAG,WAAW,EAAE,CAAC;IAC/B,MAAM,GAAG,GAAG,MAAM,QAAQ,CAAC,SAAS,EAAE,CAAC;IAEvC,IAAI,OAAO,CAAC,IAAI,EAAE,CAAC;QACjB,OAAO,CAAC,GAAG,CACT,IAAI,CAAC,SAAS,CACZ;YACE,SAAS;YACT,gBAAgB,EAAE,KAAK,EAAE,gBAAgB,IAAI,IAAI;YACjD,YAAY,EAAE,MAAM,CAAC,YAAY;YACjC,UAAU,EAAE,KAAK;YACjB,KAAK,EAAE,EAAE,QAAQ,EAAE,QAAQ,CAAC,IAAI,EAAE,GAAG,GAAG,EAAE;SAC3C,EACD,IAAI,EACJ,CAAC,CACF,CACF,CAAC;QACF,OAAO,SAAS,IAAI,GAAG,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IACrC,CAAC;IAED,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,qBAAqB,CAAC,CAAC,CAAC;IAC5C,GAAG,CAAC,IAAI,CAAC,yBAAyB,SAAS,CAAC,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACtF,IAAI,KAAK;QAAE,GAAG,CAAC,IAAI,CAAC,yBAAyB,KAAK,CAAC,gBAAgB,EAAE,CAAC,CAAC;IACvE,GAAG,CAAC,IAAI,CAAC,yBAAyB,MAAM,CAAC,YAAY,GAAG,CAAC,CAAC;IAC1D,MAAM,WAAW,GAAG,CAAC,KAAK,CAAC,OAAO;QAChC,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,aAAa,CAAC;QAC1B,CAAC,CAAC,KAAK,CAAC,EAAE;YACR,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,KAAK,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC,UAAU,KAAK,CAAC,QAAQ,CAAC,MAAM,WAAW,CAAC,CAAC,CAAC,OAAO,CAAC;YAC3F,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,GAAG,KAAK,CAAC,MAAM,CAAC,MAAM,WAAW,CAAC,CAAC;IACnD,GAAG,CAAC,IAAI,CAAC,yBAAyB,WAAW,EAAE,CAAC,CAAC;IACjD,GAAG,CAAC,IAAI,CAAC,YAAY,QAAQ,CAAC,IAAI,WAAW,GAAG,CAAC,EAAE,CAAC,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC;IACzG,IAAI,CAAC,SAAS;QAAE,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,+CAA+C,CAAC,CAAC,CAAC;IACrF,OAAO,SAAS,IAAI,GAAG,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;AACrC,CAAC"}
+{"version":3,"file":"doctor.js","sourceRoot":"","sources":["../../src/commands/doctor.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,UAAU,EAAE,WAAW,EAAE,YAAY,EAAE,MAAM,SAAS,CAAC;AAChE,OAAO,EAAE,YAAY,EAAE,MAAM,kBAAkB,CAAC;AAChD,OAAO,EAAE,SAAS,EAAE,WAAW,EAAE,MAAM,kBAAkB,CAAC;AAC1D,OAAO,EAAE,kBAAkB,EAAE,MAAM,uBAAuB,CAAC;AAC3D,OAAO,EAAE,KAAK,EAAE,GAAG,EAAE,MAAM,mBAAmB,CAAC;AAC/C,OAAO,EAAE,WAAW,EAAE,MAAM,mBAAmB,CAAC;AAChD,OAAO,EAAE,IAAI,EAAE,MAAM,mBAAmB,CAAC;AAMzC;yFACyF;AACzF,SAAS,oBAAoB,CAAC,QAAgB;IAC5C,MAAM,GAAG,GAAG,IAAI,CAAC,QAAQ,EAAE,oCAAoC,CAAC,CAAC;IACjE,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC;QAAE,OAAO,EAAE,CAAC;IAChC,MAAM,OAAO,GAAG,IAAI,GAAG,EAAgE,CAAC;IACxF,KAAK,MAAM,CAAC,IAAI,WAAW,CAAC,GAAG,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC,EAAE,CAAC;QACpE,IAAI,CAAC;YACH,MAAM,GAAG,GAAG,IAAI,CAAC,KAAK,CAAC,YAAY,CAAC,IAAI,CAAC,GAAG,EAAE,CAAC,CAAC,EAAE,MAAM,CAAC,CAAC,CAAC;YAC3D,MAAM,CAAC,GAAG,GAAG,CAAC,UAAU,IAAI,SAAS,CAAC;YACtC,MAAM,GAAG,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,YAAY,EAAE,CAAC,EAAE,OAAO,EAAE,CAAC,EAAE,CAAC,EAAE,CAAC,EAAE,CAAC;YACpE,GAAG,CAAC,YAAY,IAAI,GAAG,CAAC,SAAS,IAAI,CAAC,CAAC;YACvC,GAAG,CAAC,OAAO,IAAI,GAAG,CAAC,IAAI,IAAI,CAAC,CAAC;YAC7B,GAAG,CAAC,CAAC,IAAI,CAAC,CAAC;YACX,OAAO,CAAC,GAAG,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC;QACtB,CAAC;QAAC,MAAM,CAAC,CAAC,UAAU,CAAC,CAAC;IACxB,CAAC;IACD,OAAO,CAAC,GAAG,OAAO,CAAC,OAAO,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,KAAK,EAAE,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;QACjD,KAAK;QACL,CAAC,EAAE,CAAC,CAAC,CAAC;QACN,SAAS,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,YAAY,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QACzC,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,OAAO,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;KAChC,CAAC,CAAC,CAAC;AACN,CAAC;AAED,iGAAiG;AACjG,MAAM,CAAC,KAAK,UAAU,aAAa,CAAC,QAAgB,EAAE,OAAsB;IAC1E,MAAM,SAAS,GAAG,WAAW,CAAC,QAAQ,CAAC,CAAC;IACxC,MAAM,KAAK,GAAG,SAAS,CAAC,QAAQ,CAAC,CAAC;IAClC,MAAM,MAAM,GAAG,YAAY,CAAC,QAAQ,CAAC,CAAC;IACtC,MAAM,KAAK,GAAG,kBAAkB,CAAC,QAAQ,CAAC,CAAC;IAC3C,MAAM,QAAQ,GAAG,WAAW,EAAE,CAAC;IAC/B,MAAM,GAAG,GAAG,MAAM,QAAQ,CAAC,SAAS,EAAE,CAAC;IACvC,MAAM,WAAW,GAAG,oBAAoB,CAAC,QAAQ,CAAC,CAAC;IACnD,kGAAkG;IAClG,MAAM,eAAe,GAAG,GAAG,CAAC;IAC5B,MAAM,UAAU,GAAG,WAAW,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,SAAS,GAAG,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,IAAI,IAAI,CAAC;IACpF,MAAM,aAAa,GAAG,CAAC,UAAU,IAAI,UAAU,CAAC,SAAS,IAAI,eAAe,CAAC;IAE7E,IAAI,OAAO,CAAC,IAAI,EAAE,CAAC;QACjB,OAAO,CAAC,GAAG,CACT,IAAI,CAAC,SAAS,CACZ;YACE,SAAS;YACT,gBAAgB,EAAE,KAAK,EAAE,gBAAgB,IAAI,IAAI;YACjD,mBAAmB,EAAE,MAAM,CAAC,YAAY;YACxC,aAAa,EAAE,MAAM,CAAC,YAAY;YAClC,UAAU,EAAE,KAAK;YACjB,KAAK,EAAE,EAAE,QAAQ,EAAE,QAAQ,CAAC,IAAI,EAAE,GAAG,GAAG,EAAE;YAC1C,WAAW,EAAE,EAAE,SAAS,EAAE,eAAe,EAAE,MAAM,EAAE,WAAW,EAAE,EAAE,EAAE,aAAa,EAAE;SACpF,EACD,IAAI,EACJ,CAAC,CACF,CACF,CAAC;QACF,OAAO,SAAS,IAAI,GAAG,CAAC,EAAE,IAAI,aAAa,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IACtD,CAAC;IAED,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,qBAAqB,CAAC,CAAC,CAAC;IAC5C,GAAG,CAAC,IAAI,CAAC,6BAA6B,SAAS,CAAC,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IAC1F,IAAI,KAAK;QAAE,GAAG,CAAC,IAAI,CAAC,6BAA6B,KAAK,CAAC,gBAAgB,EAAE,CAAC,CAAC;IAC3E,GAAG,CAAC,IAAI,CAAC,6BAA6B,MAAM,CAAC,YAAY,GAAG,CAAC,CAAC;IAC9D,GAAG,CAAC,IAAI,CAAC,6BAA6B,MAAM,CAAC,YAAY,GAAG,CAAC,CAAC;IAC9D,MAAM,WAAW,GAAG,CAAC,KAAK,CAAC,OAAO;QAChC,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,aAAa,CAAC;QAC1B,CAAC,CAAC,KAAK,CAAC,EAAE;YACR,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,KAAK,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC,UAAU,KAAK,CAAC,QAAQ,CAAC,MAAM,WAAW,CAAC,CAAC,CAAC,OAAO,CAAC;YAC3F,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,GAAG,KAAK,CAAC,MAAM,CAAC,MAAM,WAAW,CAAC,CAAC;IACnD,GAAG,CAAC,IAAI,CAAC,6BAA6B,WAAW,EAAE,CAAC,CAAC;IACrD,GAAG,CAAC,IAAI,CAAC,YAAY,QAAQ,CAAC,IAAI,eAAe,GAAG,CAAC,EAAE,CAAC,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC;IAE7G,IAAI,WAAW,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC7B,GAAG,CAAC,IAAI,CAAC,6BAA6B,KAAK,CAAC,GAAG,CAAC,oEAAoE,CAAC,EAAE,CAAC,CAAC;IAC3H,CAAC;SAAM,CAAC;QACN,KAAK,MAAM,CAAC,IAAI,WAAW,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,SAAS,GAAG,CAAC,CAAC,SAAS,CAAC,EAAE,CAAC;YACtE,MAAM,IAAI,GAAG,CAAC,CAAC,SAAS,IAAI,eAAe,CAAC,CAAC,CAAC,KAAK,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,GAAG,CAAC,SAAS,eAAe,GAAG,GAAG,aAAa,CAAC,CAAC;YACzH,GAAG,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC,KAAK,CAAC,MAAM,CAAC,EAAE,CAAC,MAAM,CAAC,CAAC,CAAC,SAAS,GAAG,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,qBAAqB,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,KAAK,IAAI,EAAE,CAAC,CAAC;QAC/H,CAAC;IACH,CAAC;IAED,IAAI,CAAC,SAAS;QAAE,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,+CAA+C,CAAC,CAAC,CAAC;IACrF,OAAO,SAAS,IAAI,GAAG,CAAC,EAAE,IAAI,aAAa,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;AACtD,CAAC"}