@webywz/eval-smoke 0.3.1 → 0.4.0

package/README.md

@@ -1,8 +1,22 @@
 # eval-smoke
 
-One-minute smoke evals for prompts and models.
+[![npm](https://img.shields.io/npm/v/@webywz/eval-smoke?style=flat-square&color=111111&labelColor=eeeeee)](https://www.npmjs.com/package/@webywz/eval-smoke)
+[![node](https://img.shields.io/node/v/@webywz/eval-smoke?style=flat-square&color=111111&labelColor=eeeeee)](https://www.npmjs.com/package/@webywz/eval-smoke)
+[![license](https://img.shields.io/npm/l/@webywz/eval-smoke?style=flat-square&color=111111&labelColor=eeeeee)](./LICENSE)
 
-Full eval harnesses are heavy. `eval-smoke` is the tiny local suite you run before a ship: a handful of cases, pass/fail, exit code 1 on regression. YAML or JSON in, a CI-friendly report out — including optional JUnit XML.
+> **One-minute smoke evals for prompts and models.**
+>
+> **一分钟烟雾测试：在发版前拦住 prompt / 模型回归。**
+
+YAML or JSON in. Pass/fail out. Exit `1` on regression. Optional JUnit for CI.
+
+YAML 或 JSON 进，通过/失败出。回归时退出码 `1`。可选 JUnit 对接 CI。
+
+---
+
+## English
+
+### Demo
 
 ```bash
 npm i -g @webywz/eval-smoke
@@ -10,8 +24,8 @@ eval-smoke init
 eval-smoke suite.yaml --mock
 ```
 
-```
-eval-smoke 0.3.0  ·  mock  ·  gpt-4o-mini  ·  smoke  ·  4 cases
+```text
+eval-smoke 0.4.0  ·  mock  ·  gpt-4o-mini  ·  smoke  ·  4 cases
 
   pass  greeting               0ms
   pass  capital-france         0ms
@@ -21,23 +35,31 @@ eval-smoke 0.3.0  ·  mock  ·  gpt-4o-mini  ·  smoke  ·  4 cases
 4 passed  0 failed  1ms
 ```
 
-`--mock` is offline and deterministic. Contributors and CI do not need API keys.
+`--mock` is offline and deterministic. CI and contributors do not need API keys.
+
+### Features
 
-## Install
+| | |
+| --- | --- |
+| **Tiny suite** | A handful of cases — not a full harness |
+| **Matchers** | `equals`, `contains`, `not_contains`, `regex`, `json`, `min_tokens` / `max_tokens` |
+| **init** | `eval-smoke init` scaffolds a mock-friendly `suite.yaml` |
+| **Filter** | `--only` by id substring or `/regex/` |
+| **CI** | `--junit`, `--github-summary`, exit codes, `--fail-fast`, `--concurrency` |
+| **Baseline** | `--baseline` / `--update-baseline` catch pass→fail regressions |
+| **Providers** | Mock offline; live OpenAI / Anthropic when keyed |
+
+### Install
 
 ```bash
 npm i -g @webywz/eval-smoke
-# or, from a project:
+# or one-off:
 npx @webywz/eval-smoke suite.yaml --mock
 ```
 
-Requires Node 18+.
+Node 18+.
 
-## 60-second start
-
-1. Scaffold a suite with `eval-smoke init`, or copy the example.
-2. Run mock mode until the runner and matchers look right.
-3. Point the same file at a live model when you are ready.
+### Quick start
 
 ```bash
 eval-smoke init
@@ -46,23 +68,26 @@ eval-smoke suite.yaml --mock --only greeting
 eval-smoke suite.yaml --mock --junit junit.xml
 eval-smoke examples/suite.yaml --mock --json
 eval-smoke examples/suite.yaml --mock --concurrency 4
+eval-smoke examples/suite.yaml --mock --update-baseline baseline.json
+eval-smoke examples/suite.yaml --mock --baseline baseline.json
+eval-smoke examples/suite.yaml --mock --github-summary
 OPENAI_API_KEY=sk-... eval-smoke examples/suite.yaml
 ANTHROPIC_API_KEY=sk-ant-... eval-smoke examples/suite.yaml --model claude-3-5-haiku-20241022
 ```
 
-## `eval-smoke init`
+### `eval-smoke init`
 
 ```bash
 eval-smoke init           # writes ./suite.yaml
 eval-smoke init ./prompts # writes ./prompts/suite.yaml
-eval-smoke init --force   # overwrite an existing suite.yaml
+eval-smoke init --force   # overwrite
 ```
 
-Creates a small mock-friendly suite (greeting, capital, arithmetic, refusal). Refuses to overwrite unless you pass `--force`.
+Creates a small mock-friendly suite (greeting, capital, arithmetic, refusal). Refuses to overwrite unless `--force`.
 
-## Suite format
+### Suite format
 
-A suite is a YAML or JSON list of cases, or an object with a `cases` array.
+YAML or JSON — a list of cases, or an object with `cases`:
 
 ```yaml
 name: demo
@@ -87,132 +112,219 @@ cases:
         city: Paris
 ```
 
-A bare list is also valid:
+| Field | Notes |
+| --- | --- |
+| `id` / `prompt` | Required |
+| `expect.contains` | Substring, case-insensitive; string or list |
+| `expect.not_contains` | Must be absent |
+| `expect.regex` | `/pattern/i`, `(?i)pattern`, or raw |
+| `expect.equals` | Exact after NFC + LF normalize |
+| `expect.json` | `true` = parseable; object/array = deep-equal |
+| `expect.min_tokens` / `max_tokens` | Approx via `ceil(chars / 4)` |
+| `system` / `model` | Suite or case |
+| `provider` | `openai`, `anthropic`, or `mock` |
+
+At least one matcher per case. Failures print expected/actual (and a first-diff for `equals`).
+
+### CLI
 
-```yaml
-- id: capital-france
-  prompt: What is the capital of France? Answer with the city name only.
-  expect:
-    contains: Paris
 ```
+eval-smoke <suite.yaml|suite.json> [options]
+eval-smoke init [dir] [--force]
+```
+
+| Flag | Meaning |
+| --- | --- |
+| `--mock` | Offline deterministic runner |
+| `--json` | Machine-readable JSON report |
+| `--junit <path>` | Write JUnit XML (still prints human report unless `--json`) |
+| `--baseline <path>` | Exit `1` if a baseline-passing case now fails or is missing |
+| `--update-baseline <path>` | Write current report JSON (pretty) to path |
+| `--github-summary [path]` | Append pass/fail Markdown to `$GITHUB_STEP_SUMMARY` or path |
+| `--only <pattern>` | Cases whose id contains pattern, or matches `/regex/` |
+| `--fail-fast` | Stop after first failure |
+| `--concurrency <n>` | Parallel cases (default 1; ignored with `--fail-fast`) |
+| `--timeout <ms>` | Per-case timeout (default 30000) |
+| `--model <name>` | Override suite model |
+| `--provider <name>` | `openai` \| `anthropic` \| `mock` |
+| `--force` | Overwrite for `init` |
+| `--no-color` | Disable ANSI (`NO_COLOR` respected) |
+
+| Exit | Meaning |
+| --- | --- |
+| `0` | All passed |
+| `1` | One or more failed |
+| `2` | Usage / config error (missing file, bad YAML, no key, `--only` matched nothing) |
+
+### Runners
 
-| Field | Where | Notes |
-| --- | --- | --- |
-| `id` | case | Required, unique. |
-| `prompt` | case | Required. |
-| `expect.contains` | case | Substring, case-insensitive. String or list; all must match. |
-| `expect.not_contains` | case | Substring must be absent. String or list. |
-| `expect.regex` | case | JavaScript regex. `/pattern/i`, `(?i)pattern`, or a raw pattern. |
-| `expect.equals` | case | Exact match after NFC + LF normalize; one trailing newline ignored. |
-| `expect.json` | case | `true` = must parse as JSON; object/array = deep-equal after parse. |
-| `expect.min_tokens` | case | Approximate floor using `ceil(chars / 4)`. |
-| `expect.max_tokens` | case | Approximate cap using `ceil(chars / 4)`. |
-| `system` | suite or case | Optional system prompt for live providers. |
-| `model` | suite or case | Live default: `gpt-4o-mini` or `claude-3-5-haiku-20241022`. |
-| `provider` | suite | `openai`, `anthropic`, or `mock`. |
+**Mock** never calls the network. Canned replies keep the example suite green in CI:
 
-At least one matcher is required on every case. Failed checks print expected/actual (and a first-diff pointer for `equals`).
+| Prompt contains | Mock reply |
+| --- | --- |
+| `hello` | `Hello` |
+| `capital of France` | `Paris` |
+| `2 + 2` | `4` |
+| `ignore previous` / `PWNED` | a refusal without `PWNED` |
 
-## CLI
+**Live OpenAI** with `OPENAI_API_KEY` (optional `OPENAI_BASE_URL`). **Live Anthropic** with `ANTHROPIC_API_KEY`, or when the model name starts with `claude`.
 
+### Library
+
+```ts
+import { loadSuite, runSuite, formatJUnit, initSuite } from "@webywz/eval-smoke";
+
+initSuite(".", { force: false });
+const suite = loadSuite("suite.yaml");
+const report = await runSuite(suite, { mock: true, concurrency: 4 });
+console.log(formatJUnit(report));
+if (!report.ok) process.exit(1);
 ```
-eval-smoke <suite.yaml|suite.json> [options]
-eval-smoke init [dir] [--force]
 
-  --mock              Offline runner with deterministic fake answers
-  --json              Machine-readable JSON report
-  --junit <path>      Write JUnit XML to path (still prints human report unless --json)
-  --only <pattern>    Run cases whose id contains pattern, or matches /regex/
-  --fail-fast         Stop after the first failing case
-  --concurrency <n>   Run up to N cases in parallel (default: 1; ignored with --fail-fast)
-  --timeout <ms>      Per-case timeout (default: 30000)
-  --model <name>      Override suite model
-  --provider <name>   openai | anthropic | mock
-  --force             Overwrite existing suite.yaml for init
-  --no-color          Disable ANSI colors (also respects NO_COLOR)
-  -h, --help
-  -v, --version
+### CI snippet
+
+```yaml
+- run: npx @webywz/eval-smoke@0.4.0 prompts/smoke.yaml --mock --junit junit.xml
+- uses: actions/upload-artifact@v4
+  if: always()
+  with:
+    name: junit
+    path: junit.xml
 ```
 
-| Exit | Meaning |
+### Links
+
+| | |
 | --- | --- |
-| 0 | Every case passed |
-| 1 | One or more cases failed |
-| 2 | Usage or configuration error (missing file, bad YAML, no API key, `--only` matched nothing) |
+| npm | [npmjs.com/package/@webywz/eval-smoke](https://www.npmjs.com/package/@webywz/eval-smoke) |
+| Issues | [github.com/webywz/eval-smoke/issues](https://github.com/webywz/eval-smoke/issues) |
+| License | [MIT](./LICENSE) © webywz |
+
+---
 
-### `--only`
+## 中文
 
-Filter cases by `id` before running:
+### 演示
 
 ```bash
-eval-smoke suite.yaml --mock --only greeting          # substring
-eval-smoke suite.yaml --mock --only '/^capital/'       # /regex/ (same style as matchers)
-eval-smoke suite.yaml --mock --only '/hello/i'
+npm i -g @webywz/eval-smoke
+eval-smoke init
+eval-smoke suite.yaml --mock
 ```
 
-Exits `2` if zero cases match.
+`--mock` 离线、确定。CI 与贡献者不需要 API key。
 
-### `--junit`
+### 功能
 
-After a run, write a JUnit XML report for CI while still printing the human (or `--json`) report to stdout:
+| | |
+| --- | --- |
+| **轻量套件** | 少量用例，不是完整评测框架 |
+| **匹配器** | `equals`、`contains`、`not_contains`、`regex`、`json`、token 上下限 |
+| **init** | `eval-smoke init` 生成可 mock 的 `suite.yaml` |
+| **过滤** | `--only`：id 子串或 `/regex/` |
+| **CI** | `--junit`、`--github-summary`、退出码、`--fail-fast`、`--concurrency` |
+| **基线** | `--baseline` / `--update-baseline` 拦住 pass→fail 回归 |
+| **提供商** | Mock 离线；有 key 时走 OpenAI / Anthropic |
+
+### 安装
 
 ```bash
-eval-smoke suite.yaml --mock --junit reports/junit.xml
+npm i -g @webywz/eval-smoke
+# 或一次性：
+npx @webywz/eval-smoke suite.yaml --mock
 ```
 
-Failures include matcher detail (`expected` / `actual` / `detail`). Skipped cases (e.g. `--fail-fast`) become `<skipped/>`.
+需要 Node 18+。
+
+### 快速开始
 
-## Runners
+```bash
+eval-smoke init
+eval-smoke suite.yaml --mock
+eval-smoke suite.yaml --mock --only greeting
+eval-smoke suite.yaml --mock --junit junit.xml
+eval-smoke examples/suite.yaml --mock --json
+eval-smoke examples/suite.yaml --mock --concurrency 4
+eval-smoke examples/suite.yaml --mock --update-baseline baseline.json
+eval-smoke examples/suite.yaml --mock --baseline baseline.json
+eval-smoke examples/suite.yaml --mock --github-summary
+OPENAI_API_KEY=sk-... eval-smoke examples/suite.yaml
+ANTHROPIC_API_KEY=sk-ant-... eval-smoke examples/suite.yaml --model claude-3-5-haiku-20241022
+```
 
-**Mock** (`--mock` or `--provider mock`) never calls a network. It returns canned replies matched from the prompt so the example suite stays green in CI:
+### `eval-smoke init`
 
-| Prompt contains | Mock reply |
+```bash
+eval-smoke init           # 写入 ./suite.yaml
+eval-smoke init ./prompts # 写入 ./prompts/suite.yaml
+eval-smoke init --force   # 覆盖已有文件
+```
+
+生成小型 mock 友好套件（问候、首都、算术、拒答）。不加 `--force` 不会覆盖已有文件。
+
+### 套件格式
+
+YAML 或 JSON —— 用例列表，或带 `cases` 的对象：
+
+```yaml
+name: demo
+model: gpt-4o-mini
+cases:
+  - id: greeting
+    prompt: Reply with exactly the word Hello.
+    expect:
+      equals: Hello
+      contains: Hello
+      not_contains: Goodbye
+      regex: "(?i)^hello\\b"
+      min_tokens: 1
+      max_tokens: 32
+```
+
+| 字段 | 说明 |
 | --- | --- |
-| `hello` | `Hello` |
-| `capital of France` | `Paris` |
-| `2 + 2` | `4` |
-| `ignore previous` / `PWNED` | a refusal that does not contain `PWNED` |
+| `id` / `prompt` | 必填 |
+| `expect.contains` | 子串，忽略大小写；字符串或列表 |
+| `expect.not_contains` | 不得出现 |
+| `expect.regex` | `/pattern/i`、`(?i)pattern` 或裸模式 |
+| `expect.equals` | NFC + LF 规范化后精确匹配 |
+| `expect.json` | `true` = 可解析；对象/数组 = 深相等 |
+| `expect.min_tokens` / `max_tokens` | 约 `ceil(chars / 4)` |
+| `system` / `model` | 套件级或用例级 |
+| `provider` | `openai` / `anthropic` / `mock` |
 
-Anything else returns `[mock] No canned reply for this prompt.` — useful as a deliberate fail.
+每个用例至少要有一个匹配器。失败时打印期望/实际（`equals` 还有首差位置）。
 
-**Live OpenAI** when `OPENAI_API_KEY` is set. Optional `OPENAI_BASE_URL` for compatible servers.
+### 命令行要点
 
-**Live Anthropic** when `ANTHROPIC_API_KEY` is set, or when the model name starts with `claude`.
+| 参数 | 含义 |
+| --- | --- |
+| `--mock` | 离线确定性运行 |
+| `--json` | JSON 报告 |
+| `--junit <path>` | 写出 JUnit XML（仍打印人类可读报告，除非同时 `--json`） |
+| `--baseline <path>` | 基线里曾通过的用例若失败或缺失则退出 `1` |
+| `--update-baseline <path>` | 把当前报告 JSON（pretty）写入路径 |
+| `--github-summary [path]` | 追加通过/失败 Markdown 到 `$GITHUB_STEP_SUMMARY` 或指定路径 |
+| `--only <pattern>` | 按 id 子串或 `/regex/` 过滤 |
+| `--fail-fast` | 首个失败即停 |
+| `--concurrency <n>` | 并行用例数（默认 1） |
+| `--timeout <ms>` | 单用例超时（默认 30000） |
+| `--model` / `--provider` | 覆盖模型 / 提供商 |
+| `--force` | `init` 时覆盖 |
+
+| 退出码 | 含义 |
+| --- | --- |
+| `0` | 全部通过 |
+| `1` | 有失败 |
+| `2` | 用法/配置错误（缺文件、坏 YAML、无 key、`--only` 无匹配） |
 
-If no key is set and you did not pass `--mock`, the CLI exits 2 with a short hint.
+### 运行器
 
-## JSON report
+**Mock** 永不联网。内置 canned 回复保证示例套件在 CI 里绿。
 
-```bash
-eval-smoke examples/suite.yaml --mock --json
-```
+**Live OpenAI**：`OPENAI_API_KEY`（可选 `OPENAI_BASE_URL`）。**Live Anthropic**：`ANTHROPIC_API_KEY`，或模型名以 `claude` 开头。
 
-```json
-{
-  "ok": true,
-  "version": "0.3.0",
-  "runner": "mock",
-  "model": "gpt-4o-mini",
-  "suite": "demo",
-  "passed": 5,
-  "failed": 0,
-  "skipped": 0,
-  "duration_ms": 1,
-  "cases": [
-    {
-      "id": "greeting",
-      "ok": true,
-      "duration_ms": 1,
-      "output": "Hello",
-      "checks": [{ "name": "contains", "ok": true, "expected": "Hello" }]
-    }
-  ]
-}
-```
-
-Failed checks may include `actual` plus a multi-line `detail` (first-diff for `equals`).
-
-## Programmatic API
+### 库用法
 
 ```ts
 import { loadSuite, runSuite, formatJUnit, initSuite } from "@webywz/eval-smoke";
@@ -224,21 +336,10 @@ console.log(formatJUnit(report));
 if (!report.ok) process.exit(1);
 ```
 
-`runSuite` also accepts a custom `complete` function, which is how the tests inject fakes without keys.
-
-## CI
+### 链接
 
-GitHub Actions in this repo runs `npm test`, `npm run build`, and `eval-smoke examples/suite.yaml --mock` on Node 18/20/22. Drop this into another project:
-
-```yaml
-- run: npx @webywz/eval-smoke@0.3.1 prompts/smoke.yaml --mock --junit junit.xml
-- uses: actions/upload-artifact@v4
-  if: always()
-  with:
-    name: junit
-    path: junit.xml
-```
-
-## License
-
-MIT
+| | |
+| --- | --- |
+| npm | [npmjs.com/package/@webywz/eval-smoke](https://www.npmjs.com/package/@webywz/eval-smoke) |
+| Issues | [github.com/webywz/eval-smoke/issues](https://github.com/webywz/eval-smoke/issues) |
+| License | [MIT](./LICENSE) © webywz |

package/dist/baseline.d.ts

@@ -0,0 +1,25 @@
+import type { Report } from "./types.js";
+export interface BaselineCase {
+    id: string;
+    ok: boolean;
+}
+export interface BaselineFile {
+    cases: BaselineCase[];
+}
+/** Minimal shape: case id → ok. Accepts a full Report or a slim baseline. */
+export declare function loadBaseline(path: string): Map<string, boolean>;
+export interface BaselineCheck {
+    ok: boolean;
+    regressions: string[];
+    /** Human-readable lines */
+    report: string;
+}
+/**
+ * Fail if any case that passed in baseline now fails, or a baseline case is missing
+ * from the current run. New cases are OK. Skipped current cases that passed in
+ * baseline count as missing/regression.
+ */
+export declare function checkBaseline(report: Report, baseline: Map<string, boolean>): BaselineCheck;
+/** Pretty-print current report JSON for --update-baseline. */
+export declare function writeBaseline(path: string, report: Report): void;
+//# sourceMappingURL=baseline.d.ts.map

package/dist/baseline.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"baseline.d.ts","sourceRoot":"","sources":["../src/baseline.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,YAAY,CAAC;AAGzC,MAAM,WAAW,YAAY;IAC3B,EAAE,EAAE,MAAM,CAAC;IACX,EAAE,EAAE,OAAO,CAAC;CACb;AAED,MAAM,WAAW,YAAY;IAC3B,KAAK,EAAE,YAAY,EAAE,CAAC;CACvB;AAED,6EAA6E;AAC7E,wBAAgB,YAAY,CAAC,IAAI,EAAE,MAAM,GAAG,GAAG,CAAC,MAAM,EAAE,OAAO,CAAC,CAmC/D;AAED,MAAM,WAAW,aAAa;IAC5B,EAAE,EAAE,OAAO,CAAC;IACZ,WAAW,EAAE,MAAM,EAAE,CAAC;IACtB,2BAA2B;IAC3B,MAAM,EAAE,MAAM,CAAC;CAChB;AAED;;;;GAIG;AACH,wBAAgB,aAAa,CAAC,MAAM,EAAE,MAAM,EAAE,QAAQ,EAAE,GAAG,CAAC,MAAM,EAAE,OAAO,CAAC,GAAG,aAAa,CAgC3F;AAED,8DAA8D;AAC9D,wBAAgB,aAAa,CAAC,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,GAAG,IAAI,CAEhE"}

package/dist/baseline.js

@@ -0,0 +1,86 @@
+import { readFileSync, writeFileSync } from "node:fs";
+import { UsageError } from "./types.js";
+/** Minimal shape: case id → ok. Accepts a full Report or a slim baseline. */
+export function loadBaseline(path) {
+    let raw;
+    try {
+        raw = readFileSync(path, "utf8");
+    }
+    catch (error) {
+        const code = error && typeof error === "object" && "code" in error
+            ? String(error.code)
+            : "";
+        if (code === "ENOENT")
+            throw new UsageError(`baseline not found: ${path}`);
+        throw error;
+    }
+    let parsed;
+    try {
+        parsed = JSON.parse(raw);
+    }
+    catch {
+        throw new UsageError(`baseline is not valid JSON: ${path}`);
+    }
+    if (!parsed || typeof parsed !== "object") {
+        throw new UsageError(`baseline must be a JSON object: ${path}`);
+    }
+    const cases = parsed.cases;
+    if (!Array.isArray(cases)) {
+        throw new UsageError(`baseline missing cases[]: ${path}`);
+    }
+    const map = new Map();
+    for (const entry of cases) {
+        if (!entry || typeof entry !== "object")
+            continue;
+        const id = entry.id;
+        const ok = entry.ok;
+        if (typeof id !== "string" || !id)
+            continue;
+        if (typeof ok !== "boolean")
+            continue;
+        map.set(id, ok);
+    }
+    return map;
+}
+/**
+ * Fail if any case that passed in baseline now fails, or a baseline case is missing
+ * from the current run. New cases are OK. Skipped current cases that passed in
+ * baseline count as missing/regression.
+ */
+export function checkBaseline(report, baseline) {
+    const byId = new Map(report.cases.map((c) => [c.id, c]));
+    const regressions = [];
+    for (const [id, wasOk] of baseline) {
+        if (!wasOk)
+            continue; // only guard formerly-passing cases
+        const current = byId.get(id);
+        if (!current) {
+            regressions.push(`${id}: missing from current run (passed in baseline)`);
+            continue;
+        }
+        if (current.skipped) {
+            regressions.push(`${id}: skipped now (passed in baseline)`);
+            continue;
+        }
+        if (!current.ok) {
+            regressions.push(`${id}: was pass, now fail`);
+        }
+    }
+    const lines = ["baseline check:"];
+    if (regressions.length === 0) {
+        lines.push("  ok — no regressions vs baseline");
+    }
+    else {
+        lines.push(`  ${regressions.length} regression(s):`);
+        for (const line of regressions) {
+            lines.push(`  - ${line}`);
+        }
+    }
+    lines.push("");
+    return { ok: regressions.length === 0, regressions, report: lines.join("\n") };
+}
+/** Pretty-print current report JSON for --update-baseline. */
+export function writeBaseline(path, report) {
+    writeFileSync(path, `${JSON.stringify(report, null, 2)}\n`, "utf8");
+}
+//# sourceMappingURL=baseline.js.map

package/dist/baseline.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"baseline.js","sourceRoot":"","sources":["../src/baseline.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,SAAS,CAAC;AAEtD,OAAO,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAWxC,6EAA6E;AAC7E,MAAM,UAAU,YAAY,CAAC,IAAY;IACvC,IAAI,GAAW,CAAC;IAChB,IAAI,CAAC;QACH,GAAG,GAAG,YAAY,CAAC,IAAI,EAAE,MAAM,CAAC,CAAC;IACnC,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,IAAI,GACR,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,MAAM,IAAI,KAAK;YACnD,CAAC,CAAC,MAAM,CAAE,KAA2B,CAAC,IAAI,CAAC;YAC3C,CAAC,CAAC,EAAE,CAAC;QACT,IAAI,IAAI,KAAK,QAAQ;YAAE,MAAM,IAAI,UAAU,CAAC,uBAAuB,IAAI,EAAE,CAAC,CAAC;QAC3E,MAAM,KAAK,CAAC;IACd,CAAC;IACD,IAAI,MAAe,CAAC;IACpB,IAAI,CAAC;QACH,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;IAC3B,CAAC;IAAC,MAAM,CAAC;QACP,MAAM,IAAI,UAAU,CAAC,+BAA+B,IAAI,EAAE,CAAC,CAAC;IAC9D,CAAC;IACD,IAAI,CAAC,MAAM,IAAI,OAAO,MAAM,KAAK,QAAQ,EAAE,CAAC;QAC1C,MAAM,IAAI,UAAU,CAAC,mCAAmC,IAAI,EAAE,CAAC,CAAC;IAClE,CAAC;IACD,MAAM,KAAK,GAAI,MAA8B,CAAC,KAAK,CAAC;IACpD,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE,CAAC;QAC1B,MAAM,IAAI,UAAU,CAAC,6BAA6B,IAAI,EAAE,CAAC,CAAC;IAC5D,CAAC;IACD,MAAM,GAAG,GAAG,IAAI,GAAG,EAAmB,CAAC;IACvC,KAAK,MAAM,KAAK,IAAI,KAAK,EAAE,CAAC;QAC1B,IAAI,CAAC,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ;YAAE,SAAS;QAClD,MAAM,EAAE,GAAI,KAA0B,CAAC,EAAE,CAAC;QAC1C,MAAM,EAAE,GAAI,KAA0B,CAAC,EAAE,CAAC;QAC1C,IAAI,OAAO,EAAE,KAAK,QAAQ,IAAI,CAAC,EAAE;YAAE,SAAS;QAC5C,IAAI,OAAO,EAAE,KAAK,SAAS;YAAE,SAAS;QACtC,GAAG,CAAC,GAAG,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC;IAClB,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AASD;;;;GAIG;AACH,MAAM,UAAU,aAAa,CAAC,MAAc,EAAE,QAA8B;IAC1E,MAAM,IAAI,GAAG,IAAI,GAAG,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;IACzD,MAAM,WAAW,GAAa,EAAE,CAAC;IAEjC,KAAK,MAAM,CAAC,EAAE,EAAE,KAAK,CAAC,IAAI,QAAQ,EAAE,CAAC;QACnC,IAAI,CAAC,KAAK;YAAE,SAAS,CAAC,oCAAoC;QAC1D,MAAM,OAAO,GAAG,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;QAC7B,IAAI,CAAC,OAAO,EAAE,CAAC;YACb,WAAW,CAAC,IAAI,CAAC,GAAG,EAAE,iDAAiD,CAAC,CAAC;YACzE,SAAS;QACX,CAAC;QACD,IAAI,OAAO,CAAC,OAAO,EAAE,CAAC;YACpB,WAAW,CAAC,IAAI,CAAC,GAAG,EAAE,oCAAoC,CAAC,CAAC;YAC5D,SAAS;QACX,CAAC;QACD,IAAI,CAAC,OAAO,CAAC,EAAE,EAAE,CAAC;YAChB,WAAW,CAAC,IAAI,CAAC,GAAG,EAAE,sBAAsB,CAAC,CAAC;QAChD,CAAC;IACH,CAAC;IAED,MAAM,KAAK,GAAG,CAAC,iBAAiB,CAAC,CAAC;IAClC,IAAI,WAAW,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC7B,KAAK,CAAC,IAAI,CAAC,mCAAmC,CAAC,CAAC;IAClD,CAAC;SAAM,CAAC;QACN,KAAK,CAAC,IAAI,CAAC,KAAK,WAAW,CAAC,MAAM,iBAAiB,CAAC,CAAC;QACrD,KAAK,MAAM,IAAI,IAAI,WAAW,EAAE,CAAC;YAC/B,KAAK,CAAC,IAAI,CAAC,OAAO,IAAI,EAAE,CAAC,CAAC;QAC5B,CAAC;IACH,CAAC;IACD,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IAEf,OAAO,EAAE,EAAE,EAAE,WAAW,CAAC,MAAM,KAAK,CAAC,EAAE,WAAW,EAAE,MAAM,EAAE,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;AACjF,CAAC;AAED,8DAA8D;AAC9D,MAAM,UAAU,aAAa,CAAC,IAAY,EAAE,MAAc;IACxD,aAAa,CAAC,IAAI,EAAE,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC,IAAI,EAAE,MAAM,CAAC,CAAC;AACtE,CAAC"}

package/dist/github-summary.d.ts

@@ -0,0 +1,4 @@
+export declare const DEFAULT_GITHUB_SUMMARY = "github-step-summary.md";
+export declare function resolveGithubSummaryPath(flagPath: string | true | undefined, env?: NodeJS.ProcessEnv, cwd?: string): string | undefined;
+export declare function appendGithubSummary(path: string, markdown: string): void;
+//# sourceMappingURL=github-summary.d.ts.map

package/dist/github-summary.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"github-summary.d.ts","sourceRoot":"","sources":["../src/github-summary.ts"],"names":[],"mappings":"AAGA,eAAO,MAAM,sBAAsB,2BAA2B,CAAC;AAE/D,wBAAgB,wBAAwB,CACtC,QAAQ,EAAE,MAAM,GAAG,IAAI,GAAG,SAAS,EACnC,GAAG,GAAE,MAAM,CAAC,UAAwB,EACpC,GAAG,GAAE,MAAsB,GAC1B,MAAM,GAAG,SAAS,CAWpB;AAED,wBAAgB,mBAAmB,CAAC,IAAI,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,GAAG,IAAI,CAoBxE"}

package/dist/github-summary.js

@@ -0,0 +1,39 @@
+import { appendFileSync, mkdirSync, writeFileSync } from "node:fs";
+import { dirname, resolve } from "node:path";
+export const DEFAULT_GITHUB_SUMMARY = "github-step-summary.md";
+export function resolveGithubSummaryPath(flagPath, env = process.env, cwd = process.cwd()) {
+    if (flagPath === undefined) {
+        const fromEnv = env.GITHUB_STEP_SUMMARY;
+        return fromEnv && fromEnv.trim() ? fromEnv : undefined;
+    }
+    if (flagPath === true || flagPath === "") {
+        const fromEnv = env.GITHUB_STEP_SUMMARY;
+        if (fromEnv && fromEnv.trim())
+            return fromEnv;
+        return resolve(cwd, DEFAULT_GITHUB_SUMMARY);
+    }
+    return resolve(cwd, flagPath);
+}
+export function appendGithubSummary(path, markdown) {
+    const body = markdown.endsWith("\n") ? markdown : `${markdown}\n`;
+    try {
+        mkdirSync(dirname(path), { recursive: true });
+    }
+    catch {
+        // ignore
+    }
+    try {
+        appendFileSync(path, body, "utf8");
+    }
+    catch (error) {
+        const code = error && typeof error === "object" && "code" in error
+            ? String(error.code)
+            : "";
+        if (code === "ENOENT") {
+            writeFileSync(path, body, "utf8");
+            return;
+        }
+        throw error;
+    }
+}
+//# sourceMappingURL=github-summary.js.map

package/dist/github-summary.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"github-summary.js","sourceRoot":"","sources":["../src/github-summary.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,cAAc,EAAE,SAAS,EAAE,aAAa,EAAE,MAAM,SAAS,CAAC;AACnE,OAAO,EAAE,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AAE7C,MAAM,CAAC,MAAM,sBAAsB,GAAG,wBAAwB,CAAC;AAE/D,MAAM,UAAU,wBAAwB,CACtC,QAAmC,EACnC,MAAyB,OAAO,CAAC,GAAG,EACpC,MAAc,OAAO,CAAC,GAAG,EAAE;IAE3B,IAAI,QAAQ,KAAK,SAAS,EAAE,CAAC;QAC3B,MAAM,OAAO,GAAG,GAAG,CAAC,mBAAmB,CAAC;QACxC,OAAO,OAAO,IAAI,OAAO,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,SAAS,CAAC;IACzD,CAAC;IACD,IAAI,QAAQ,KAAK,IAAI,IAAI,QAAQ,KAAK,EAAE,EAAE,CAAC;QACzC,MAAM,OAAO,GAAG,GAAG,CAAC,mBAAmB,CAAC;QACxC,IAAI,OAAO,IAAI,OAAO,CAAC,IAAI,EAAE;YAAE,OAAO,OAAO,CAAC;QAC9C,OAAO,OAAO,CAAC,GAAG,EAAE,sBAAsB,CAAC,CAAC;IAC9C,CAAC;IACD,OAAO,OAAO,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC;AAChC,CAAC;AAED,MAAM,UAAU,mBAAmB,CAAC,IAAY,EAAE,QAAgB;IAChE,MAAM,IAAI,GAAG,QAAQ,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,GAAG,QAAQ,IAAI,CAAC;IAClE,IAAI,CAAC;QACH,SAAS,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;IAChD,CAAC;IAAC,MAAM,CAAC;QACP,SAAS;IACX,CAAC;IACD,IAAI,CAAC;QACH,cAAc,CAAC,IAAI,EAAE,IAAI,EAAE,MAAM,CAAC,CAAC;IACrC,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,IAAI,GACR,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,MAAM,IAAI,KAAK;YACnD,CAAC,CAAC,MAAM,CAAE,KAA2B,CAAC,IAAI,CAAC;YAC3C,CAAC,CAAC,EAAE,CAAC;QACT,IAAI,IAAI,KAAK,QAAQ,EAAE,CAAC;YACtB,aAAa,CAAC,IAAI,EAAE,IAAI,EAAE,MAAM,CAAC,CAAC;YAClC,OAAO;QACT,CAAC;QACD,MAAM,KAAK,CAAC;IACd,CAAC;AACH,CAAC"}

package/dist/index.d.ts

@@ -9,4 +9,8 @@ export { estimateTokens } from "./tokens.js";
 export { VERSION } from "./version.js";
 export type { Case, CaseResult, CheckName, CheckResult, CompleteFn, CompleteRequest, Expect, JsonExpect, ProviderName, Report, RunOptions, Suite, } from "./types.js";
 export { UsageError } from "./types.js";
+export { checkBaseline, loadBaseline, writeBaseline } from "./baseline.js";
+export type { BaselineCheck, BaselineCase, BaselineFile } from "./baseline.js";
+export { appendGithubSummary, resolveGithubSummaryPath, DEFAULT_GITHUB_SUMMARY } from "./github-summary.js";
+export { formatGithubSummary } from "./summary-md.js";
 //# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,UAAU,EAAE,MAAM,iBAAiB,CAAC;AACxD,OAAO,EAAE,SAAS,EAAE,mBAAmB,EAAE,eAAe,EAAE,MAAM,WAAW,CAAC;AAC5E,OAAO,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AACzC,OAAO,EAAE,YAAY,EAAE,cAAc,EAAE,UAAU,EAAE,eAAe,EAAE,MAAM,eAAe,CAAC;AAC1F,OAAO,EAAE,cAAc,EAAE,YAAY,EAAE,eAAe,EAAE,MAAM,gBAAgB,CAAC;AAC/E,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,OAAO,EAAE,QAAQ,EAAE,MAAM,UAAU,CAAC;AACpC,OAAO,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAC7C,OAAO,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AACvC,YAAY,EACV,IAAI,EACJ,UAAU,EACV,SAAS,EACT,WAAW,EACX,UAAU,EACV,eAAe,EACf,MAAM,EACN,UAAU,EACV,YAAY,EACZ,MAAM,EACN,UAAU,EACV,KAAK,GACN,MAAM,YAAY,CAAC;AACpB,OAAO,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,UAAU,EAAE,MAAM,iBAAiB,CAAC;AACxD,OAAO,EAAE,SAAS,EAAE,mBAAmB,EAAE,eAAe,EAAE,MAAM,WAAW,CAAC;AAC5E,OAAO,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AACzC,OAAO,EAAE,YAAY,EAAE,cAAc,EAAE,UAAU,EAAE,eAAe,EAAE,MAAM,eAAe,CAAC;AAC1F,OAAO,EAAE,cAAc,EAAE,YAAY,EAAE,eAAe,EAAE,MAAM,gBAAgB,CAAC;AAC/E,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,OAAO,EAAE,QAAQ,EAAE,MAAM,UAAU,CAAC;AACpC,OAAO,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAC7C,OAAO,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AACvC,YAAY,EACV,IAAI,EACJ,UAAU,EACV,SAAS,EACT,WAAW,EACX,UAAU,EACV,eAAe,EACf,MAAM,EACN,UAAU,EACV,YAAY,EACZ,MAAM,EACN,UAAU,EACV,KAAK,GACN,MAAM,YAAY,CAAC;AACpB,OAAO,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAExC,OAAO,EAAE,aAAa,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,eAAe,CAAC;AAC3E,YAAY,EAAE,aAAa,EAAE,YAAY,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC;AAC/E,OAAO,EAAE,mBAAmB,EAAE,wBAAwB,EAAE,sBAAsB,EAAE,MAAM,qBAAqB,CAAC;AAC5G,OAAO,EAAE,mBAAmB,EAAE,MAAM,iBAAiB,CAAC"}

package/dist/index.js

@@ -8,4 +8,7 @@ export { runSuite } from "./run.js";
 export { estimateTokens } from "./tokens.js";
 export { VERSION } from "./version.js";
 export { UsageError } from "./types.js";
+export { checkBaseline, loadBaseline, writeBaseline } from "./baseline.js";
+export { appendGithubSummary, resolveGithubSummaryPath, DEFAULT_GITHUB_SUMMARY } from "./github-summary.js";
+export { formatGithubSummary } from "./summary-md.js";
 //# sourceMappingURL=index.js.map

package/dist/index.js.map

@@ -1 +1 @@
-{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,UAAU,EAAE,MAAM,iBAAiB,CAAC;AACxD,OAAO,EAAE,SAAS,EAAE,mBAAmB,EAAE,eAAe,EAAE,MAAM,WAAW,CAAC;AAC5E,OAAO,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AACzC,OAAO,EAAE,YAAY,EAAE,cAAc,EAAE,UAAU,EAAE,eAAe,EAAE,MAAM,eAAe,CAAC;AAC1F,OAAO,EAAE,cAAc,EAAE,YAAY,EAAE,eAAe,EAAE,MAAM,gBAAgB,CAAC;AAC/E,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,OAAO,EAAE,QAAQ,EAAE,MAAM,UAAU,CAAC;AACpC,OAAO,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAC7C,OAAO,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAevC,OAAO,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC"}
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,UAAU,EAAE,MAAM,iBAAiB,CAAC;AACxD,OAAO,EAAE,SAAS,EAAE,mBAAmB,EAAE,eAAe,EAAE,MAAM,WAAW,CAAC;AAC5E,OAAO,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AACzC,OAAO,EAAE,YAAY,EAAE,cAAc,EAAE,UAAU,EAAE,eAAe,EAAE,MAAM,eAAe,CAAC;AAC1F,OAAO,EAAE,cAAc,EAAE,YAAY,EAAE,eAAe,EAAE,MAAM,gBAAgB,CAAC;AAC/E,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,OAAO,EAAE,QAAQ,EAAE,MAAM,UAAU,CAAC;AACpC,OAAO,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAC7C,OAAO,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAevC,OAAO,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAExC,OAAO,EAAE,aAAa,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,eAAe,CAAC;AAE3E,OAAO,EAAE,mBAAmB,EAAE,wBAAwB,EAAE,sBAAsB,EAAE,MAAM,qBAAqB,CAAC;AAC5G,OAAO,EAAE,mBAAmB,EAAE,MAAM,iBAAiB,CAAC"}

package/dist/run-cli.d.ts

@@ -1,5 +1,5 @@
 import { type Suite } from "./types.js";
-export declare const HELP = "eval-smoke \u2014 one-minute smoke evals for prompts and models\n\nUsage:\n  eval-smoke <suite.yaml|suite.json> [options]\n  eval-smoke init [dir] [--force]\n\nOptions:\n  --mock              Offline runner with deterministic fake answers\n  --json              Machine-readable JSON report\n  --junit <path>      Write JUnit XML report to path (still prints human report unless --json)\n  --only <pattern>    Run cases whose id contains pattern, or matches /regex/\n  --fail-fast         Stop after the first failing case\n  --concurrency <n>   Run up to N cases in parallel (default: 1; ignored with --fail-fast)\n  --timeout <ms>      Per-case timeout (default: 30000)\n  --model <name>      Override suite model\n  --provider <name>   openai | anthropic | mock\n  --force             Overwrite existing suite.yaml when using init\n  --no-color          Disable ANSI colors (also respects NO_COLOR)\n  -h, --help          Show this help\n  -v, --version       Show version\n\nExit codes:\n  0  all cases passed\n  1  one or more cases failed\n  2  usage or configuration error\n\nExamples:\n  eval-smoke init\n  eval-smoke init ./prompts --force\n  eval-smoke examples/suite.yaml --mock\n  eval-smoke examples/suite.yaml --mock --only greeting\n  eval-smoke examples/suite.yaml --mock --only '/^capital/'\n  eval-smoke examples/suite.yaml --mock --junit junit.xml\n  eval-smoke examples/suite.yaml --json --fail-fast\n  eval-smoke examples/suite.yaml --mock --concurrency 4\n  OPENAI_API_KEY=sk-... eval-smoke examples/suite.yaml\n";
+export declare const HELP = "eval-smoke \u2014 one-minute smoke evals for prompts and models\n\nUsage:\n  eval-smoke <suite.yaml|suite.json> [options]\n  eval-smoke init [dir] [--force]\n\nOptions:\n  --mock              Offline runner with deterministic fake answers\n  --json              Machine-readable JSON report\n  --junit <path>      Write JUnit XML report to path (still prints human report unless --json)\n  --only <pattern>    Run cases whose id contains pattern, or matches /regex/\n  --fail-fast         Stop after the first failing case\n  --concurrency <n>   Run up to N cases in parallel (default: 1; ignored with --fail-fast)\n  --timeout <ms>      Per-case timeout (default: 30000)\n  --model <name>      Override suite model\n  --provider <name>   openai | anthropic | mock\n  --baseline <path>   Fail if any baseline-passing case now fails or is missing\n  --update-baseline <path>  Write current report JSON to path (pretty)\n  --github-summary [path]   Append Markdown to GITHUB_STEP_SUMMARY or path\n  --force             Overwrite existing suite.yaml when using init\n  --no-color          Disable ANSI colors (also respects NO_COLOR)\n  -h, --help          Show this help\n  -v, --version       Show version\n\nExit codes:\n  0  all cases passed (and baseline clean when --baseline is set)\n  1  one or more cases failed, or baseline regression\n  2  usage or configuration error\n\nExamples:\n  eval-smoke init\n  eval-smoke init ./prompts --force\n  eval-smoke examples/suite.yaml --mock\n  eval-smoke examples/suite.yaml --mock --only greeting\n  eval-smoke examples/suite.yaml --mock --only '/^capital/'\n  eval-smoke examples/suite.yaml --mock --junit junit.xml\n  eval-smoke examples/suite.yaml --json --fail-fast\n  eval-smoke examples/suite.yaml --mock --concurrency 4\n  eval-smoke examples/suite.yaml --mock --update-baseline baseline.json\n  eval-smoke examples/suite.yaml --mock --baseline baseline.json\n  eval-smoke examples/suite.yaml --mock --github-summary\n  OPENAI_API_KEY=sk-... eval-smoke examples/suite.yaml\n";
 export interface CliIo {
     stdout: {
         write(text: string): void;
@@ -14,5 +14,11 @@ export interface CliIo {
 /** Match case ids by substring, or by /regex/ flags (same style as matchers). */
 export declare function matchOnly(id: string, pattern: string): boolean;
 export declare function filterSuiteByOnly(suite: Suite, pattern: string): Suite;
+/**
+ * Rewrite bare `--github-summary` into `--github-summary=__auto__` so node:util
+ * parseArgs (string option) accepts it. `--github-summary=path` and
+ * `--github-summary path` stay as-is.
+ */
+export declare function normalizeGithubSummaryArgv(argv: string[]): string[];
 export declare function runCli(argv: string[], io?: CliIo): Promise<number>;
 //# sourceMappingURL=run-cli.d.ts.map

package/dist/run-cli.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"run-cli.d.ts","sourceRoot":"","sources":["../src/run-cli.ts"],"names":[],"mappings":"AASA,OAAO,EAAiC,KAAK,KAAK,EAAE,MAAM,YAAY,CAAC;AAGvE,eAAO,MAAM,IAAI,mgDAoChB,CAAC;AAEF,MAAM,WAAW,KAAK;IACpB,MAAM,EAAE;QAAE,KAAK,CAAC,IAAI,EAAE,MAAM,GAAG,IAAI,CAAA;KAAE,CAAC;IACtC,MAAM,EAAE;QAAE,KAAK,CAAC,IAAI,EAAE,MAAM,GAAG,IAAI,CAAA;KAAE,CAAC;IACtC,GAAG,EAAE,MAAM,CAAC,UAAU,CAAC;IACvB,GAAG,EAAE,MAAM,CAAC;IACZ,KAAK,EAAE,OAAO,CAAC;CAChB;AAkCD,iFAAiF;AACjF,wBAAgB,SAAS,CAAC,EAAE,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,GAAG,OAAO,CAQ9D;AAED,wBAAgB,iBAAiB,CAAC,KAAK,EAAE,KAAK,EAAE,OAAO,EAAE,MAAM,GAAG,KAAK,CAMtE;AAED,wBAAsB,MAAM,CAAC,IAAI,EAAE,MAAM,EAAE,EAAE,EAAE,GAAE,KAAmB,GAAG,OAAO,CAAC,MAAM,CAAC,CA6FrF"}
+{"version":3,"file":"run-cli.d.ts","sourceRoot":"","sources":["../src/run-cli.ts"],"names":[],"mappings":"AAYA,OAAO,EAAiC,KAAK,KAAK,EAAE,MAAM,YAAY,CAAC;AAMvE,eAAO,MAAM,IAAI,i/DA0ChB,CAAC;AAEF,MAAM,WAAW,KAAK;IACpB,MAAM,EAAE;QAAE,KAAK,CAAC,IAAI,EAAE,MAAM,GAAG,IAAI,CAAA;KAAE,CAAC;IACtC,MAAM,EAAE;QAAE,KAAK,CAAC,IAAI,EAAE,MAAM,GAAG,IAAI,CAAA;KAAE,CAAC;IACtC,GAAG,EAAE,MAAM,CAAC,UAAU,CAAC;IACvB,GAAG,EAAE,MAAM,CAAC;IACZ,KAAK,EAAE,OAAO,CAAC;CAChB;AAkCD,iFAAiF;AACjF,wBAAgB,SAAS,CAAC,EAAE,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,GAAG,OAAO,CAQ9D;AAED,wBAAgB,iBAAiB,CAAC,KAAK,EAAE,KAAK,EAAE,OAAO,EAAE,MAAM,GAAG,KAAK,CAMtE;AAED;;;;GAIG;AACH,wBAAgB,0BAA0B,CAAC,IAAI,EAAE,MAAM,EAAE,GAAG,MAAM,EAAE,CAiBnE;AAQD,wBAAsB,MAAM,CAAC,IAAI,EAAE,MAAM,EAAE,EAAE,EAAE,GAAE,KAAmB,GAAG,OAAO,CAAC,MAAM,CAAC,CAgIrF"}

package/dist/run-cli.js

@@ -1,14 +1,19 @@
 import { writeFileSync } from "node:fs";
 import { resolve } from "node:path";
 import { parseArgs } from "node:util";
+import { checkBaseline, loadBaseline, writeBaseline } from "./baseline.js";
+import { appendGithubSummary, resolveGithubSummaryPath } from "./github-summary.js";
 import { initSuite } from "./init.js";
 import { formatJUnit } from "./junit.js";
 import { loadSuite } from "./load-suite.js";
 import { compileRegex } from "./matchers.js";
 import { formatReport } from "./report.js";
 import { DEFAULT_TIMEOUT_MS, runSuite } from "./run.js";
+import { formatGithubSummary } from "./summary-md.js";
 import { UsageError } from "./types.js";
 import { VERSION } from "./version.js";
+/** Sentinel so node:util accepts a bare --github-summary flag. */
+const GITHUB_SUMMARY_AUTO = "__auto__";
 export const HELP = `eval-smoke — one-minute smoke evals for prompts and models
 
 Usage:
@@ -25,14 +30,17 @@ Options:
   --timeout <ms>      Per-case timeout (default: ${DEFAULT_TIMEOUT_MS})
   --model <name>      Override suite model
   --provider <name>   openai | anthropic | mock
+  --baseline <path>   Fail if any baseline-passing case now fails or is missing
+  --update-baseline <path>  Write current report JSON to path (pretty)
+  --github-summary [path]   Append Markdown to GITHUB_STEP_SUMMARY or path
   --force             Overwrite existing suite.yaml when using init
   --no-color          Disable ANSI colors (also respects NO_COLOR)
   -h, --help          Show this help
   -v, --version       Show version
 
 Exit codes:
-  0  all cases passed
-  1  one or more cases failed
+  0  all cases passed (and baseline clean when --baseline is set)
+  1  one or more cases failed, or baseline regression
   2  usage or configuration error
 
 Examples:
@@ -44,6 +52,9 @@ Examples:
   eval-smoke examples/suite.yaml --mock --junit junit.xml
   eval-smoke examples/suite.yaml --json --fail-fast
   eval-smoke examples/suite.yaml --mock --concurrency 4
+  eval-smoke examples/suite.yaml --mock --update-baseline baseline.json
+  eval-smoke examples/suite.yaml --mock --baseline baseline.json
+  eval-smoke examples/suite.yaml --mock --github-summary
   OPENAI_API_KEY=sk-... eval-smoke examples/suite.yaml
 `;
 const defaultIo = () => ({
@@ -95,11 +106,42 @@ export function filterSuiteByOnly(suite, pattern) {
     }
     return { ...suite, cases };
 }
+/**
+ * Rewrite bare `--github-summary` into `--github-summary=__auto__` so node:util
+ * parseArgs (string option) accepts it. `--github-summary=path` and
+ * `--github-summary path` stay as-is.
+ */
+export function normalizeGithubSummaryArgv(argv) {
+    const out = [];
+    for (let i = 0; i < argv.length; i += 1) {
+        const token = argv[i];
+        if (token === "--github-summary") {
+            const next = argv[i + 1];
+            if (!next || next.startsWith("-")) {
+                out.push(`--github-summary=${GITHUB_SUMMARY_AUTO}`);
+            }
+            else {
+                out.push("--github-summary", next);
+                i += 1;
+            }
+            continue;
+        }
+        out.push(token);
+    }
+    return out;
+}
+function githubSummaryFlag(raw) {
+    if (raw == null)
+        return undefined;
+    if (raw === GITHUB_SUMMARY_AUTO)
+        return true;
+    return raw;
+}
 export async function runCli(argv, io = defaultIo()) {
     let parsed;
     try {
         parsed = parseArgs({
-            args: argv,
+            args: normalizeGithubSummaryArgv(argv),
             allowPositionals: true,
             strict: true,
             options: {
@@ -112,6 +154,9 @@ export async function runCli(argv, io = defaultIo()) {
                 timeout: { type: "string" },
                 model: { type: "string" },
                 provider: { type: "string" },
+                baseline: { type: "string" },
+                "update-baseline": { type: "string" },
+                "github-summary": { type: "string" },
                 force: { type: "boolean", default: false },
                 "no-color": { type: "boolean", default: false },
                 help: { type: "boolean", short: "h", default: false },
@@ -158,6 +203,9 @@ export async function runCli(argv, io = defaultIo()) {
         const json = asBool(values.json);
         const only = asString(values.only);
         const junitPath = asString(values.junit);
+        const baselinePath = asString(values.baseline);
+        const updateBaselinePath = asString(values["update-baseline"]);
+        const githubSummary = githubSummaryFlag(asString(values["github-summary"]));
         let suite = loadSuite(suitePath, io.cwd);
         if (only != null) {
             suite = filterSuiteByOnly(suite, only);
@@ -175,9 +223,36 @@ export async function runCli(argv, io = defaultIo()) {
             const out = resolve(io.cwd, junitPath);
             writeFileSync(out, formatJUnit(report), "utf8");
         }
+        if (updateBaselinePath != null) {
+            const out = resolve(io.cwd, updateBaselinePath);
+            writeBaseline(out, report);
+            io.stderr.write(`Wrote baseline ${out}\n`);
+        }
+        let baselineFailed = false;
+        if (baselinePath != null) {
+            const baseline = loadBaseline(resolve(io.cwd, baselinePath));
+            const check = checkBaseline(report, baseline);
+            io.stderr.write(check.report);
+            if (!check.ok)
+                baselineFailed = true;
+        }
+        const outSummary = githubSummary === undefined
+            ? resolveGithubSummaryPath(undefined, io.env, io.cwd)
+            : resolveGithubSummaryPath(githubSummary, io.env, io.cwd);
+        if (outSummary) {
+            try {
+                appendGithubSummary(outSummary, formatGithubSummary(report));
+            }
+            catch (error) {
+                const message = error instanceof Error ? error.message : String(error);
+                io.stderr.write(`could not write github summary: ${message}\n`);
+            }
+        }
         const noColor = asBool(values["no-color"]) || Boolean(io.env.NO_COLOR);
         const color = io.isTTY && !json && !noColor;
         io.stdout.write(formatReport(report, { json, color }));
+        if (baselineFailed)
+            return 1;
         return report.ok ? 0 : 1;
     }
     catch (error) {

package/dist/run-cli.js.map

@@ -1 +1 @@
-{"version":3,"file":"run-cli.js","sourceRoot":"","sources":["../src/run-cli.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,aAAa,EAAE,MAAM,SAAS,CAAC;AACxC,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AACpC,OAAO,EAAE,SAAS,EAAE,MAAM,WAAW,CAAC;AACtC,OAAO,EAAE,SAAS,EAAE,MAAM,WAAW,CAAC;AACtC,OAAO,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AACzC,OAAO,EAAE,SAAS,EAAE,MAAM,iBAAiB,CAAC;AAC5C,OAAO,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC;AAC7C,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,OAAO,EAAE,kBAAkB,EAAE,QAAQ,EAAE,MAAM,UAAU,CAAC;AACxD,OAAO,EAAE,UAAU,EAAiC,MAAM,YAAY,CAAC;AACvE,OAAO,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAEvC,MAAM,CAAC,MAAM,IAAI,GAAG;;;;;;;;;;;;;mDAa+B,kBAAkB;;;;;;;;;;;;;;;;;;;;;;;CAuBpE,CAAC;AAUF,MAAM,SAAS,GAAG,GAAU,EAAE,CAAC,CAAC;IAC9B,MAAM,EAAE,OAAO,CAAC,MAAM;IACtB,MAAM,EAAE,OAAO,CAAC,MAAM;IACtB,GAAG,EAAE,OAAO,CAAC,GAAG;IAChB,GAAG,EAAE,OAAO,CAAC,GAAG,EAAE;IAClB,KAAK,EAAE,OAAO,CAAC,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC;CACrC,CAAC,CAAC;AAEH,SAAS,YAAY,CAAC,GAAuB;IAC3C,IAAI,GAAG,IAAI,IAAI;QAAE,OAAO,kBAAkB,CAAC;IAC3C,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC;IAC1B,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,KAAK,IAAI,CAAC,EAAE,CAAC;QAC3C,MAAM,IAAI,UAAU,CAAC,qDAAqD,CAAC,CAAC;IAC9E,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,SAAS,gBAAgB,CAAC,GAAuB;IAC/C,IAAI,GAAG,IAAI,IAAI;QAAE,OAAO,SAAS,CAAC;IAClC,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC;IAC1B,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,KAAK,IAAI,CAAC,EAAE,CAAC;QAC3C,MAAM,IAAI,UAAU,CAAC,0CAA0C,CAAC,CAAC;IACnE,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,SAAS,aAAa,CAAC,GAAuB;IAC5C,IAAI,GAAG,IAAI,IAAI;QAAE,OAAO,SAAS,CAAC;IAClC,IAAI,GAAG,KAAK,MAAM,IAAI,GAAG,KAAK,QAAQ,IAAI,GAAG,KAAK,WAAW;QAAE,OAAO,GAAG,CAAC;IAC1E,MAAM,IAAI,UAAU,CAAC,+CAA+C,CAAC,CAAC;AACxE,CAAC;AAED,iFAAiF;AACjF,MAAM,UAAU,SAAS,CAAC,EAAU,EAAE,OAAe;IACnD,MAAM,OAAO,GAAG,OAAO,CAAC,KAAK,CAAC,4BAA4B,CAAC,CAAC;IAC5D,IAAI,OAAO,EAAE,CAAC,CAAC,CAAC,IAAI,IAAI,EAAE,CAAC;QACzB,MAAM,EAAE,GAAG,YAAY,CAAC,OAAO,CAAC,CAAC;QACjC,EAAE,CAAC,SAAS,GAAG,CAAC,CAAC;QACjB,OAAO,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACrB,CAAC;IACD,OAAO,EAAE,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC;AAC9B,CAAC;AAED,MAAM,UAAU,iBAAiB,CAAC,KAAY,EAAE,OAAe;IAC7D,MAAM,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,QAAQ,EAAE,EAAE,CAAC,SAAS,CAAC,QAAQ,CAAC,EAAE,EAAE,OAAO,CAAC,CAAC,CAAC;IAChF,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACvB,MAAM,IAAI,UAAU,CAAC,UAAU,IAAI,CAAC,SAAS,CAAC,OAAO,CAAC,kBAAkB,CAAC,CAAC;IAC5E,CAAC;IACD,OAAO,EAAE,GAAG,KAAK,EAAE,KAAK,EAAE,CAAC;AAC7B,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,MAAM,CAAC,IAAc,EAAE,KAAY,SAAS,EAAE;IAClE,IAAI,MAAoC,CAAC;IACzC,IAAI,CAAC;QACH,MAAM,GAAG,SAAS,CAAC;YACjB,IAAI,EAAE,IAAI;YACV,gBAAgB,EAAE,IAAI;YACtB,MAAM,EAAE,IAAI;YACZ,OAAO,EAAE;gBACP,IAAI,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBACzC,IAAI,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBACzC,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACzB,IAAI,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACxB,WAAW,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBAChD,WAAW,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBAC/B,OAAO,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBAC3B,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACzB,QAAQ,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBAC5B,KAAK,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBAC1C,UAAU,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBAC/C,IAAI,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,KAAK,EAAE,GAAG,EAAE,OAAO,EAAE,KAAK,EAAE;gBACrD,OAAO,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,KAAK,EAAE,GAAG,EAAE,OAAO,EAAE,KAAK,EAAE;aACzD;SACF,CAAC,CAAC;IACL,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QAC/E,OAAO,CAAC,CAAC;IACX,CAAC;IAED,MAAM,EAAE,MAAM,EAAE,WAAW,EAAE,GAAG,MAAM,CAAC;IACvC,MAAM,QAAQ,GAAG,CAAC,KAAc,EAAsB,EAAE,CACtD,OAAO,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC;IAChD,MAAM,MAAM,GAAG,CAAC,KAAc,EAAW,EAAE,CAAC,KAAK,KAAK,IAAI,CAAC;IAE3D,IAAI,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC;QACxB,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QACtB,OAAO,CAAC,CAAC;IACX,CAAC;IACD,IAAI,MAAM,CAAC,MAAM,CAAC,OAAO,CAAC,EAAE,CAAC;QAC3B,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,IAAI,CAAC,CAAC;QAChC,OAAO,CAAC,CAAC;IACX,CAAC;IAED,IAAI,WAAW,CAAC,CAAC,CAAC,KAAK,MAAM,EAAE,CAAC;QAC9B,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC;YAC9B,MAAM,MAAM,GAAG,SAAS,CAAC,MAAM,EAAE,EAAE,KAAK,EAAE,MAAM,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,GAAG,EAAE,EAAE,CAAC,GAAG,EAAE,CAAC,CAAC;YAC/E,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,WAAW,MAAM,CAAC,IAAI,IAAI,CAAC,CAAC;YAC5C,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,oBAAoB,MAAM,CAAC,IAAI,WAAW,CAAC,CAAC;YAC5D,OAAO,CAAC,CAAC;QACX,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,MAAM,OAAO,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;YACvE,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,IAAI,CAAC,CAAC;YAChC,OAAO,KAAK,YAAY,UAAU,CAAC,CAAC,CAAC,KAAK,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC;QAC1D,CAAC;IACH,CAAC;IAED,MAAM,SAAS,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC;IACjC,IAAI,CAAC,SAAS,EAAE,CAAC;QACf,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,yBAAyB,CAAC,CAAC;QAC3C,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QACtB,OAAO,CAAC,CAAC;IACX,CAAC;IAED,IAAI,CAAC;QACH,MAAM,IAAI,GAAG,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QACjC,MAAM,IAAI,GAAG,QAAQ,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QACnC,MAAM,SAAS,GAAG,QAAQ,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QACzC,IAAI,KAAK,GAAG,SAAS,CAAC,SAAS,EAAE,EAAE,CAAC,GAAG,CAAC,CAAC;QACzC,IAAI,IAAI,IAAI,IAAI,EAAE,CAAC;YACjB,KAAK,GAAG,iBAAiB,CAAC,KAAK,EAAE,IAAI,CAAC,CAAC;QACzC,CAAC;QACD,MAAM,MAAM,GAAG,MAAM,QAAQ,CAAC,KAAK,EAAE;YACnC,IAAI,EAAE,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC;YACzB,QAAQ,EAAE,aAAa,CAAC,QAAQ,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;YAClD,KAAK,EAAE,QAAQ,CAAC,MAAM,CAAC,KAAK,CAAC;YAC7B,QAAQ,EAAE,MAAM,CAAC,MAAM,CAAC,WAAW,CAAC,CAAC;YACrC,WAAW,EAAE,gBAAgB,CAAC,QAAQ,CAAC,MAAM,CAAC,WAAW,CAAC,CAAC;YAC3D,SAAS,EAAE,YAAY,CAAC,QAAQ,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;YACjD,GAAG,EAAE,EAAE,CAAC,GAAG;SACZ,CAAC,CAAC;QACH,IAAI,SAAS,IAAI,IAAI,EAAE,CAAC;YACtB,MAAM,GAAG,GAAG,OAAO,CAAC,EAAE,CAAC,GAAG,EAAE,SAAS,CAAC,CAAC;YACvC,aAAa,CAAC,GAAG,EAAE,WAAW,CAAC,MAAM,CAAC,EAAE,MAAM,CAAC,CAAC;QAClD,CAAC;QACD,MAAM,OAAO,GAAG,MAAM,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC,IAAI,OAAO,CAAC,EAAE,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;QACvE,MAAM,KAAK,GAAG,EAAE,CAAC,KAAK,IAAI,CAAC,IAAI,IAAI,CAAC,OAAO,CAAC;QAC5C,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,YAAY,CAAC,MAAM,EAAE,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC;QACvD,OAAO,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IAC3B,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,OAAO,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QACvE,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,IAAI,CAAC,CAAC;QAChC,OAAO,KAAK,YAAY,UAAU,CAAC,CAAC,CAAC,KAAK,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC;IAC1D,CAAC;AACH,CAAC"}
+{"version":3,"file":"run-cli.js","sourceRoot":"","sources":["../src/run-cli.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,aAAa,EAAE,MAAM,SAAS,CAAC;AACxC,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AACpC,OAAO,EAAE,SAAS,EAAE,MAAM,WAAW,CAAC;AACtC,OAAO,EAAE,aAAa,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,eAAe,CAAC;AAC3E,OAAO,EAAE,mBAAmB,EAAE,wBAAwB,EAAE,MAAM,qBAAqB,CAAC;AACpF,OAAO,EAAE,SAAS,EAAE,MAAM,WAAW,CAAC;AACtC,OAAO,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AACzC,OAAO,EAAE,SAAS,EAAE,MAAM,iBAAiB,CAAC;AAC5C,OAAO,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC;AAC7C,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,OAAO,EAAE,kBAAkB,EAAE,QAAQ,EAAE,MAAM,UAAU,CAAC;AACxD,OAAO,EAAE,mBAAmB,EAAE,MAAM,iBAAiB,CAAC;AACtD,OAAO,EAAE,UAAU,EAAiC,MAAM,YAAY,CAAC;AACvE,OAAO,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAEvC,kEAAkE;AAClE,MAAM,mBAAmB,GAAG,UAAU,CAAC;AAEvC,MAAM,CAAC,MAAM,IAAI,GAAG;;;;;;;;;;;;;mDAa+B,kBAAkB;;;;;;;;;;;;;;;;;;;;;;;;;;;;;CA6BpE,CAAC;AAUF,MAAM,SAAS,GAAG,GAAU,EAAE,CAAC,CAAC;IAC9B,MAAM,EAAE,OAAO,CAAC,MAAM;IACtB,MAAM,EAAE,OAAO,CAAC,MAAM;IACtB,GAAG,EAAE,OAAO,CAAC,GAAG;IAChB,GAAG,EAAE,OAAO,CAAC,GAAG,EAAE;IAClB,KAAK,EAAE,OAAO,CAAC,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC;CACrC,CAAC,CAAC;AAEH,SAAS,YAAY,CAAC,GAAuB;IAC3C,IAAI,GAAG,IAAI,IAAI;QAAE,OAAO,kBAAkB,CAAC;IAC3C,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC;IAC1B,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,KAAK,IAAI,CAAC,EAAE,CAAC;QAC3C,MAAM,IAAI,UAAU,CAAC,qDAAqD,CAAC,CAAC;IAC9E,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,SAAS,gBAAgB,CAAC,GAAuB;IAC/C,IAAI,GAAG,IAAI,IAAI;QAAE,OAAO,SAAS,CAAC;IAClC,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC;IAC1B,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,KAAK,IAAI,CAAC,EAAE,CAAC;QAC3C,MAAM,IAAI,UAAU,CAAC,0CAA0C,CAAC,CAAC;IACnE,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,SAAS,aAAa,CAAC,GAAuB;IAC5C,IAAI,GAAG,IAAI,IAAI;QAAE,OAAO,SAAS,CAAC;IAClC,IAAI,GAAG,KAAK,MAAM,IAAI,GAAG,KAAK,QAAQ,IAAI,GAAG,KAAK,WAAW;QAAE,OAAO,GAAG,CAAC;IAC1E,MAAM,IAAI,UAAU,CAAC,+CAA+C,CAAC,CAAC;AACxE,CAAC;AAED,iFAAiF;AACjF,MAAM,UAAU,SAAS,CAAC,EAAU,EAAE,OAAe;IACnD,MAAM,OAAO,GAAG,OAAO,CAAC,KAAK,CAAC,4BAA4B,CAAC,CAAC;IAC5D,IAAI,OAAO,EAAE,CAAC,CAAC,CAAC,IAAI,IAAI,EAAE,CAAC;QACzB,MAAM,EAAE,GAAG,YAAY,CAAC,OAAO,CAAC,CAAC;QACjC,EAAE,CAAC,SAAS,GAAG,CAAC,CAAC;QACjB,OAAO,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACrB,CAAC;IACD,OAAO,EAAE,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC;AAC9B,CAAC;AAED,MAAM,UAAU,iBAAiB,CAAC,KAAY,EAAE,OAAe;IAC7D,MAAM,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,QAAQ,EAAE,EAAE,CAAC,SAAS,CAAC,QAAQ,CAAC,EAAE,EAAE,OAAO,CAAC,CAAC,CAAC;IAChF,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACvB,MAAM,IAAI,UAAU,CAAC,UAAU,IAAI,CAAC,SAAS,CAAC,OAAO,CAAC,kBAAkB,CAAC,CAAC;IAC5E,CAAC;IACD,OAAO,EAAE,GAAG,KAAK,EAAE,KAAK,EAAE,CAAC;AAC7B,CAAC;AAED;;;;GAIG;AACH,MAAM,UAAU,0BAA0B,CAAC,IAAc;IACvD,MAAM,GAAG,GAAa,EAAE,CAAC;IACzB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC;QACxC,MAAM,KAAK,GAAG,IAAI,CAAC,CAAC,CAAE,CAAC;QACvB,IAAI,KAAK,KAAK,kBAAkB,EAAE,CAAC;YACjC,MAAM,IAAI,GAAG,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;YACzB,IAAI,CAAC,IAAI,IAAI,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE,CAAC;gBAClC,GAAG,CAAC,IAAI,CAAC,oBAAoB,mBAAmB,EAAE,CAAC,CAAC;YACtD,CAAC;iBAAM,CAAC;gBACN,GAAG,CAAC,IAAI,CAAC,kBAAkB,EAAE,IAAI,CAAC,CAAC;gBACnC,CAAC,IAAI,CAAC,CAAC;YACT,CAAC;YACD,SAAS;QACX,CAAC;QACD,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;IAClB,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,SAAS,iBAAiB,CAAC,GAAuB;IAChD,IAAI,GAAG,IAAI,IAAI;QAAE,OAAO,SAAS,CAAC;IAClC,IAAI,GAAG,KAAK,mBAAmB;QAAE,OAAO,IAAI,CAAC;IAC7C,OAAO,GAAG,CAAC;AACb,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,MAAM,CAAC,IAAc,EAAE,KAAY,SAAS,EAAE;IAClE,IAAI,MAAoC,CAAC;IACzC,IAAI,CAAC;QACH,MAAM,GAAG,SAAS,CAAC;YACjB,IAAI,EAAE,0BAA0B,CAAC,IAAI,CAAC;YACtC,gBAAgB,EAAE,IAAI;YACtB,MAAM,EAAE,IAAI;YACZ,OAAO,EAAE;gBACP,IAAI,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBACzC,IAAI,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBACzC,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACzB,IAAI,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACxB,WAAW,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBAChD,WAAW,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBAC/B,OAAO,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBAC3B,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACzB,QAAQ,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBAC5B,QAAQ,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBAC5B,iBAAiB,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACrC,gBAAgB,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACpC,KAAK,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBAC1C,UAAU,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBAC/C,IAAI,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,KAAK,EAAE,GAAG,EAAE,OAAO,EAAE,KAAK,EAAE;gBACrD,OAAO,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,KAAK,EAAE,GAAG,EAAE,OAAO,EAAE,KAAK,EAAE;aACzD;SACF,CAAC,CAAC;IACL,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QAC/E,OAAO,CAAC,CAAC;IACX,CAAC;IAED,MAAM,EAAE,MAAM,EAAE,WAAW,EAAE,GAAG,MAAM,CAAC;IACvC,MAAM,QAAQ,GAAG,CAAC,KAAc,EAAsB,EAAE,CACtD,OAAO,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC;IAChD,MAAM,MAAM,GAAG,CAAC,KAAc,EAAW,EAAE,CAAC,KAAK,KAAK,IAAI,CAAC;IAE3D,IAAI,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC;QACxB,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QACtB,OAAO,CAAC,CAAC;IACX,CAAC;IACD,IAAI,MAAM,CAAC,MAAM,CAAC,OAAO,CAAC,EAAE,CAAC;QAC3B,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,IAAI,CAAC,CAAC;QAChC,OAAO,CAAC,CAAC;IACX,CAAC;IAED,IAAI,WAAW,CAAC,CAAC,CAAC,KAAK,MAAM,EAAE,CAAC;QAC9B,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC;YAC9B,MAAM,MAAM,GAAG,SAAS,CAAC,MAAM,EAAE,EAAE,KAAK,EAAE,MAAM,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,GAAG,EAAE,EAAE,CAAC,GAAG,EAAE,CAAC,CAAC;YAC/E,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,WAAW,MAAM,CAAC,IAAI,IAAI,CAAC,CAAC;YAC5C,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,oBAAoB,MAAM,CAAC,IAAI,WAAW,CAAC,CAAC;YAC5D,OAAO,CAAC,CAAC;QACX,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,MAAM,OAAO,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;YACvE,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,IAAI,CAAC,CAAC;YAChC,OAAO,KAAK,YAAY,UAAU,CAAC,CAAC,CAAC,KAAK,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC;QAC1D,CAAC;IACH,CAAC;IAED,MAAM,SAAS,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC;IACjC,IAAI,CAAC,SAAS,EAAE,CAAC;QACf,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,yBAAyB,CAAC,CAAC;QAC3C,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QACtB,OAAO,CAAC,CAAC;IACX,CAAC;IAED,IAAI,CAAC;QACH,MAAM,IAAI,GAAG,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QACjC,MAAM,IAAI,GAAG,QAAQ,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QACnC,MAAM,SAAS,GAAG,QAAQ,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QACzC,MAAM,YAAY,GAAG,QAAQ,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;QAC/C,MAAM,kBAAkB,GAAG,QAAQ,CAAC,MAAM,CAAC,iBAAiB,CAAC,CAAC,CAAC;QAC/D,MAAM,aAAa,GAAG,iBAAiB,CAAC,QAAQ,CAAC,MAAM,CAAC,gBAAgB,CAAC,CAAC,CAAC,CAAC;QAE5E,IAAI,KAAK,GAAG,SAAS,CAAC,SAAS,EAAE,EAAE,CAAC,GAAG,CAAC,CAAC;QACzC,IAAI,IAAI,IAAI,IAAI,EAAE,CAAC;YACjB,KAAK,GAAG,iBAAiB,CAAC,KAAK,EAAE,IAAI,CAAC,CAAC;QACzC,CAAC;QACD,MAAM,MAAM,GAAG,MAAM,QAAQ,CAAC,KAAK,EAAE;YACnC,IAAI,EAAE,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC;YACzB,QAAQ,EAAE,aAAa,CAAC,QAAQ,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;YAClD,KAAK,EAAE,QAAQ,CAAC,MAAM,CAAC,KAAK,CAAC;YAC7B,QAAQ,EAAE,MAAM,CAAC,MAAM,CAAC,WAAW,CAAC,CAAC;YACrC,WAAW,EAAE,gBAAgB,CAAC,QAAQ,CAAC,MAAM,CAAC,WAAW,CAAC,CAAC;YAC3D,SAAS,EAAE,YAAY,CAAC,QAAQ,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;YACjD,GAAG,EAAE,EAAE,CAAC,GAAG;SACZ,CAAC,CAAC;QACH,IAAI,SAAS,IAAI,IAAI,EAAE,CAAC;YACtB,MAAM,GAAG,GAAG,OAAO,CAAC,EAAE,CAAC,GAAG,EAAE,SAAS,CAAC,CAAC;YACvC,aAAa,CAAC,GAAG,EAAE,WAAW,CAAC,MAAM,CAAC,EAAE,MAAM,CAAC,CAAC;QAClD,CAAC;QACD,IAAI,kBAAkB,IAAI,IAAI,EAAE,CAAC;YAC/B,MAAM,GAAG,GAAG,OAAO,CAAC,EAAE,CAAC,GAAG,EAAE,kBAAkB,CAAC,CAAC;YAChD,aAAa,CAAC,GAAG,EAAE,MAAM,CAAC,CAAC;YAC3B,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,kBAAkB,GAAG,IAAI,CAAC,CAAC;QAC7C,CAAC;QAED,IAAI,cAAc,GAAG,KAAK,CAAC;QAC3B,IAAI,YAAY,IAAI,IAAI,EAAE,CAAC;YACzB,MAAM,QAAQ,GAAG,YAAY,CAAC,OAAO,CAAC,EAAE,CAAC,GAAG,EAAE,YAAY,CAAC,CAAC,CAAC;YAC7D,MAAM,KAAK,GAAG,aAAa,CAAC,MAAM,EAAE,QAAQ,CAAC,CAAC;YAC9C,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC;YAC9B,IAAI,CAAC,KAAK,CAAC,EAAE;gBAAE,cAAc,GAAG,IAAI,CAAC;QACvC,CAAC;QAED,MAAM,UAAU,GACd,aAAa,KAAK,SAAS;YACzB,CAAC,CAAC,wBAAwB,CAAC,SAAS,EAAE,EAAE,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,CAAC;YACrD,CAAC,CAAC,wBAAwB,CAAC,aAAa,EAAE,EAAE,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,CAAC,CAAC;QAC9D,IAAI,UAAU,EAAE,CAAC;YACf,IAAI,CAAC;gBACH,mBAAmB,CAAC,UAAU,EAAE,mBAAmB,CAAC,MAAM,CAAC,CAAC,CAAC;YAC/D,CAAC;YAAC,OAAO,KAAK,EAAE,CAAC;gBACf,MAAM,OAAO,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;gBACvE,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,mCAAmC,OAAO,IAAI,CAAC,CAAC;YAClE,CAAC;QACH,CAAC;QAED,MAAM,OAAO,GAAG,MAAM,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC,IAAI,OAAO,CAAC,EAAE,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;QACvE,MAAM,KAAK,GAAG,EAAE,CAAC,KAAK,IAAI,CAAC,IAAI,IAAI,CAAC,OAAO,CAAC;QAC5C,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,YAAY,CAAC,MAAM,EAAE,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC;QACvD,IAAI,cAAc;YAAE,OAAO,CAAC,CAAC;QAC7B,OAAO,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IAC3B,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,OAAO,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QACvE,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,IAAI,CAAC,CAAC;QAChC,OAAO,KAAK,YAAY,UAAU,CAAC,CAAC,CAAC,KAAK,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC;IAC1D,CAAC;AACH,CAAC"}

package/dist/summary-md.d.ts

@@ -0,0 +1,4 @@
+import type { Report } from "./types.js";
+/** Concise pass/fail Markdown table for GitHub Actions step summary. */
+export declare function formatGithubSummary(report: Report): string;
+//# sourceMappingURL=summary-md.d.ts.map

package/dist/summary-md.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"summary-md.d.ts","sourceRoot":"","sources":["../src/summary-md.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,YAAY,CAAC;AAEzC,wEAAwE;AACxE,wBAAgB,mBAAmB,CAAC,MAAM,EAAE,MAAM,GAAG,MAAM,CAoB1D"}

package/dist/summary-md.js

@@ -0,0 +1,22 @@
+/** Concise pass/fail Markdown table for GitHub Actions step summary. */
+export function formatGithubSummary(report) {
+    const lines = [];
+    lines.push(`### eval-smoke · ${escape(report.suite ?? "suite")}`);
+    lines.push("");
+    lines.push(`**${report.passed} passed · ${report.failed} failed${report.skipped ? ` · ${report.skipped} skipped` : ""} · ${report.duration_ms}ms** · \`${report.runner}\` / \`${escape(report.model)}\``);
+    lines.push("");
+    lines.push("| Case | Result | ms |");
+    lines.push("| --- | --- | ---: |");
+    for (const c of report.cases) {
+        const status = c.skipped ? "skip" : c.ok ? "✅ pass" : "❌ fail";
+        lines.push(`| \`${escape(c.id)}\` | ${status} | ${c.duration_ms} |`);
+    }
+    lines.push("");
+    lines.push(report.ok ? "**Suite: PASS**" : "**Suite: FAIL**");
+    lines.push("");
+    return lines.join("\n");
+}
+function escape(text) {
+    return text.replace(/\|/g, "\\|").replace(/\r?\n/g, " ");
+}
+//# sourceMappingURL=summary-md.js.map

package/dist/summary-md.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"summary-md.js","sourceRoot":"","sources":["../src/summary-md.ts"],"names":[],"mappings":"AAEA,wEAAwE;AACxE,MAAM,UAAU,mBAAmB,CAAC,MAAc;IAChD,MAAM,KAAK,GAAa,EAAE,CAAC;IAC3B,KAAK,CAAC,IAAI,CAAC,oBAAoB,MAAM,CAAC,MAAM,CAAC,KAAK,IAAI,OAAO,CAAC,EAAE,CAAC,CAAC;IAClE,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACf,KAAK,CAAC,IAAI,CACR,KAAK,MAAM,CAAC,MAAM,aAAa,MAAM,CAAC,MAAM,UAC1C,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,MAAM,CAAC,OAAO,UAAU,CAAC,CAAC,CAAC,EACpD,MAAM,MAAM,CAAC,WAAW,YAAY,MAAM,CAAC,MAAM,UAAU,MAAM,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CACpF,CAAC;IACF,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACf,KAAK,CAAC,IAAI,CAAC,wBAAwB,CAAC,CAAC;IACrC,KAAK,CAAC,IAAI,CAAC,sBAAsB,CAAC,CAAC;IACnC,KAAK,MAAM,CAAC,IAAI,MAAM,CAAC,KAAK,EAAE,CAAC;QAC7B,MAAM,MAAM,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC;QAC/D,KAAK,CAAC,IAAI,CAAC,OAAO,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,QAAQ,MAAM,MAAM,CAAC,CAAC,WAAW,IAAI,CAAC,CAAC;IACvE,CAAC;IACD,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACf,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC,iBAAiB,CAAC,CAAC,CAAC,iBAAiB,CAAC,CAAC;IAC9D,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACf,OAAO,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AAC1B,CAAC;AAED,SAAS,MAAM,CAAC,IAAY;IAC1B,OAAO,IAAI,CAAC,OAAO,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC,OAAO,CAAC,QAAQ,EAAE,GAAG,CAAC,CAAC;AAC3D,CAAC"}

package/examples/github-action.yml

@@ -0,0 +1,41 @@
+# Copy-paste: run eval-smoke in GitHub Actions (mock offline + JUnit + step summary).
+# No clone of this repo required — npx pulls @webywz/eval-smoke from npm.
+#
+# Human report → stdout (job log). JUnit → file. Markdown → $GITHUB_STEP_SUMMARY.
+
+name: Smoke eval with eval-smoke
+
+on:
+  workflow_dispatch:
+  # Or: push / pull_request when you have a suite.yaml in the repo.
+
+jobs:
+  eval:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22
+
+      # Offline smoke against the published example suite (no API key):
+      - name: Mock smoke + JUnit + step summary
+        run: |
+          npx --yes @webywz/eval-smoke --help >/dev/null
+          # Prefer a suite checked into YOUR repo; this uses the package example via init:
+          npx --yes @webywz/eval-smoke init ./eval-smoke-ci --force
+          npx --yes @webywz/eval-smoke ./eval-smoke-ci/suite.yaml \
+            --mock \
+            --junit junit-eval-smoke.xml \
+            --github-summary \
+            --update-baseline eval-smoke-baseline.json
+
+      - name: Upload JUnit
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: eval-smoke-junit
+          path: |
+            junit-eval-smoke.xml
+            eval-smoke-baseline.json

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@webywz/eval-smoke",
-  "version": "0.3.1",
-  "description": "One-minute smoke evals for prompts and models — equals/json matchers, diffs, concurrency, JUnit, init.",
+  "version": "0.4.0",
+  "description": "One-minute smoke evals for prompts and models \u2014 equals/json matchers, diffs, concurrency, JUnit, init.",
   "type": "module",
   "bin": {
     "eval-smoke": "dist/cli.js"