npm - @webywz/eval-smoke - Versions diffs - 0.3.1 → 0.5.0 - Mend

@webywz/eval-smoke 0.3.1 → 0.5.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (27) hide show

package/README.md +232 -127
package/dist/baseline.d.ts +25 -0
package/dist/baseline.d.ts.map +1 -0
package/dist/baseline.js +86 -0
package/dist/baseline.js.map +1 -0
package/dist/github-summary.d.ts +4 -0
package/dist/github-summary.d.ts.map +1 -0
package/dist/github-summary.js +39 -0
package/dist/github-summary.js.map +1 -0
package/dist/html-report.d.ts +8 -0
package/dist/html-report.d.ts.map +1 -0
package/dist/html-report.js +196 -0
package/dist/html-report.js.map +1 -0
package/dist/index.d.ts +5 -0
package/dist/index.d.ts.map +1 -1
package/dist/index.js +4 -0
package/dist/index.js.map +1 -1
package/dist/run-cli.d.ts +7 -1
package/dist/run-cli.d.ts.map +1 -1
package/dist/run-cli.js +87 -3
package/dist/run-cli.js.map +1 -1
package/dist/summary-md.d.ts +4 -0
package/dist/summary-md.d.ts.map +1 -0
package/dist/summary-md.js +22 -0
package/dist/summary-md.js.map +1 -0
package/examples/github-action.yml +41 -0
package/package.json +1 -1

package/README.md CHANGED Viewed

@@ -1,8 +1,22 @@
 # eval-smoke
-One-minute smoke evals for prompts and models.
+[![npm](https://img.shields.io/npm/v/@webywz/eval-smoke?style=flat-square&color=111111&labelColor=eeeeee)](https://www.npmjs.com/package/@webywz/eval-smoke)
+[![node](https://img.shields.io/node/v/@webywz/eval-smoke?style=flat-square&color=111111&labelColor=eeeeee)](https://www.npmjs.com/package/@webywz/eval-smoke)
+[![license](https://img.shields.io/npm/l/@webywz/eval-smoke?style=flat-square&color=111111&labelColor=eeeeee)](./LICENSE)
-Full eval harnesses are heavy. `eval-smoke` is the tiny local suite you run before a ship: a handful of cases, pass/fail, exit code 1 on regression. YAML or JSON in, a CI-friendly report out — including optional JUnit XML.
+> **One-minute smoke evals for prompts and models.**
+>
+> **一分钟烟雾测试：在发版前拦住 prompt / 模型回归。**
+YAML or JSON in. Pass/fail out. Exit `1` on regression. Optional JUnit for CI.
+YAML 或 JSON 进，通过/失败出。回归时退出码 `1`。可选 JUnit 对接 CI。
+---
+## English
+### Demo
 ```bash
 npm i -g @webywz/eval-smoke
@@ -10,8 +24,8 @@ eval-smoke init
 eval-smoke suite.yaml --mock
 ```
-```
-eval-smoke 0.3.0  ·  mock  ·  gpt-4o-mini  ·  smoke  ·  4 cases
+```text
+eval-smoke 0.5.0  ·  mock  ·  gpt-4o-mini  ·  smoke  ·  4 cases
   pass  greeting               0ms
   pass  capital-france         0ms
@@ -21,48 +35,60 @@ eval-smoke 0.3.0  ·  mock  ·  gpt-4o-mini  ·  smoke  ·  4 cases
 4 passed  0 failed  1ms
 ```
-`--mock` is offline and deterministic. Contributors and CI do not need API keys.
+`--mock` is offline and deterministic. CI and contributors do not need API keys.
+### Features
-## Install
+| | |
+| --- | --- |
+| **Tiny suite** | A handful of cases — not a full harness |
+| **Matchers** | `equals`, `contains`, `not_contains`, `regex`, `json`, `min_tokens` / `max_tokens` |
+| **init** | `eval-smoke init` scaffolds a mock-friendly `suite.yaml` |
+| **Filter** | `--only` by id substring or `/regex/` |
+| **CI** | `--junit`, `--html`, `--github-summary`, exit codes, `--fail-fast`, `--concurrency` |
+| **Baseline** | `--baseline` / `--update-baseline` catch pass→fail regressions |
+| **Providers** | Mock offline; live OpenAI / Anthropic when keyed |
+### Install
 ```bash
 npm i -g @webywz/eval-smoke
-# or, from a project:
+# or one-off:
 npx @webywz/eval-smoke suite.yaml --mock
 ```
-Requires Node 18+.
+Node 18+.
-## 60-second start
-1. Scaffold a suite with `eval-smoke init`, or copy the example.
-2. Run mock mode until the runner and matchers look right.
-3. Point the same file at a live model when you are ready.
+### Quick start
 ```bash
 eval-smoke init
 eval-smoke suite.yaml --mock
 eval-smoke suite.yaml --mock --only greeting
 eval-smoke suite.yaml --mock --junit junit.xml
+eval-smoke suite.yaml --mock --html report.html
 eval-smoke examples/suite.yaml --mock --json
 eval-smoke examples/suite.yaml --mock --concurrency 4
+eval-smoke examples/suite.yaml --mock --update-baseline baseline.json
+eval-smoke examples/suite.yaml --mock --baseline baseline.json
+eval-smoke examples/suite.yaml --mock --github-summary
 OPENAI_API_KEY=sk-... eval-smoke examples/suite.yaml
 ANTHROPIC_API_KEY=sk-ant-... eval-smoke examples/suite.yaml --model claude-3-5-haiku-20241022
 ```
-## `eval-smoke init`
+### `eval-smoke init`
 ```bash
 eval-smoke init           # writes ./suite.yaml
 eval-smoke init ./prompts # writes ./prompts/suite.yaml
-eval-smoke init --force   # overwrite an existing suite.yaml
+eval-smoke init --force   # overwrite
 ```
-Creates a small mock-friendly suite (greeting, capital, arithmetic, refusal). Refuses to overwrite unless you pass `--force`.
+Creates a small mock-friendly suite (greeting, capital, arithmetic, refusal). Refuses to overwrite unless `--force`.
-## Suite format
+### Suite format
-A suite is a YAML or JSON list of cases, or an object with a `cases` array.
+YAML or JSON — a list of cases, or an object with `cases`:
 ```yaml
 name: demo
@@ -87,132 +113,222 @@ cases:
         city: Paris
 ```
-A bare list is also valid:
+| Field | Notes |
+| --- | --- |
+| `id` / `prompt` | Required |
+| `expect.contains` | Substring, case-insensitive; string or list |
+| `expect.not_contains` | Must be absent |
+| `expect.regex` | `/pattern/i`, `(?i)pattern`, or raw |
+| `expect.equals` | Exact after NFC + LF normalize |
+| `expect.json` | `true` = parseable; object/array = deep-equal |
+| `expect.min_tokens` / `max_tokens` | Approx via `ceil(chars / 4)` |
+| `system` / `model` | Suite or case |
+| `provider` | `openai`, `anthropic`, or `mock` |
+At least one matcher per case. Failures print expected/actual (and a first-diff for `equals`).
+### CLI
-```yaml
-- id: capital-france
-  prompt: What is the capital of France? Answer with the city name only.
-  expect:
-    contains: Paris
 ```
+eval-smoke <suite.yaml|suite.json> [options]
+eval-smoke init [dir] [--force]
+```
+| Flag | Meaning |
+| --- | --- |
+| `--mock` | Offline deterministic runner |
+| `--json` | Machine-readable JSON report |
+| `--junit <path>` | Write JUnit XML (still prints human report unless `--json`) |
+| `--html <path>` | Write self-contained HTML report (offline, no CDN) |
+| `--baseline <path>` | Exit `1` if a baseline-passing case now fails or is missing |
+| `--update-baseline <path>` | Write current report JSON (pretty) to path |
+| `--github-summary [path]` | Append pass/fail Markdown to `$GITHUB_STEP_SUMMARY` or path |
+| `--only <pattern>` | Cases whose id contains pattern, or matches `/regex/` |
+| `--fail-fast` | Stop after first failure |
+| `--concurrency <n>` | Parallel cases (default 1; ignored with `--fail-fast`) |
+| `--timeout <ms>` | Per-case timeout (default 30000) |
+| `--model <name>` | Override suite model |
+| `--provider <name>` | `openai` \| `anthropic` \| `mock` |
+| `--force` | Overwrite for `init` |
+| `--no-color` | Disable ANSI (`NO_COLOR` respected) |
+| Exit | Meaning |
+| --- | --- |
+| `0` | All passed |
+| `1` | One or more failed |
+| `2` | Usage / config error (missing file, bad YAML, no key, `--only` matched nothing) |
+### Runners
-| Field | Where | Notes |
-| --- | --- | --- |
-| `id` | case | Required, unique. |
-| `prompt` | case | Required. |
-| `expect.contains` | case | Substring, case-insensitive. String or list; all must match. |
-| `expect.not_contains` | case | Substring must be absent. String or list. |
-| `expect.regex` | case | JavaScript regex. `/pattern/i`, `(?i)pattern`, or a raw pattern. |
-| `expect.equals` | case | Exact match after NFC + LF normalize; one trailing newline ignored. |
-| `expect.json` | case | `true` = must parse as JSON; object/array = deep-equal after parse. |
-| `expect.min_tokens` | case | Approximate floor using `ceil(chars / 4)`. |
-| `expect.max_tokens` | case | Approximate cap using `ceil(chars / 4)`. |
-| `system` | suite or case | Optional system prompt for live providers. |
-| `model` | suite or case | Live default: `gpt-4o-mini` or `claude-3-5-haiku-20241022`. |
-| `provider` | suite | `openai`, `anthropic`, or `mock`. |
+**Mock** never calls the network. Canned replies keep the example suite green in CI:
-At least one matcher is required on every case. Failed checks print expected/actual (and a first-diff pointer for `equals`).
+| Prompt contains | Mock reply |
+| --- | --- |
+| `hello` | `Hello` |
+| `capital of France` | `Paris` |
+| `2 + 2` | `4` |
+| `ignore previous` / `PWNED` | a refusal without `PWNED` |
-## CLI
+**Live OpenAI** with `OPENAI_API_KEY` (optional `OPENAI_BASE_URL`). **Live Anthropic** with `ANTHROPIC_API_KEY`, or when the model name starts with `claude`.
+### Library
+```ts
+import { loadSuite, runSuite, formatJUnit, initSuite } from "@webywz/eval-smoke";
+initSuite(".", { force: false });
+const suite = loadSuite("suite.yaml");
+const report = await runSuite(suite, { mock: true, concurrency: 4 });
+console.log(formatJUnit(report));
+if (!report.ok) process.exit(1);
 ```
-eval-smoke <suite.yaml|suite.json> [options]
-eval-smoke init [dir] [--force]
-  --mock              Offline runner with deterministic fake answers
-  --json              Machine-readable JSON report
-  --junit <path>      Write JUnit XML to path (still prints human report unless --json)
-  --only <pattern>    Run cases whose id contains pattern, or matches /regex/
-  --fail-fast         Stop after the first failing case
-  --concurrency <n>   Run up to N cases in parallel (default: 1; ignored with --fail-fast)
-  --timeout <ms>      Per-case timeout (default: 30000)
-  --model <name>      Override suite model
-  --provider <name>   openai | anthropic | mock
-  --force             Overwrite existing suite.yaml for init
-  --no-color          Disable ANSI colors (also respects NO_COLOR)
-  -h, --help
-  -v, --version
+### CI snippet
+```yaml
+- run: npx @webywz/eval-smoke@0.5.0 prompts/smoke.yaml --mock --junit junit.xml
+- uses: actions/upload-artifact@v4
+  if: always()
+  with:
+    name: junit
+    path: junit.xml
 ```
-| Exit | Meaning |
+### Links
+| | |
 | --- | --- |
-| 0 | Every case passed |
-| 1 | One or more cases failed |
-| 2 | Usage or configuration error (missing file, bad YAML, no API key, `--only` matched nothing) |
+| npm | [npmjs.com/package/@webywz/eval-smoke](https://www.npmjs.com/package/@webywz/eval-smoke) |
+| Issues | [github.com/webywz/eval-smoke/issues](https://github.com/webywz/eval-smoke/issues) |
+| License | [MIT](./LICENSE) © webywz |
+---
-### `--only`
+## 中文
-Filter cases by `id` before running:
+### 演示
 ```bash
-eval-smoke suite.yaml --mock --only greeting          # substring
-eval-smoke suite.yaml --mock --only '/^capital/'       # /regex/ (same style as matchers)
-eval-smoke suite.yaml --mock --only '/hello/i'
+npm i -g @webywz/eval-smoke
+eval-smoke init
+eval-smoke suite.yaml --mock
 ```
-Exits `2` if zero cases match.
+`--mock` 离线、确定。CI 与贡献者不需要 API key。
-### `--junit`
+### 功能
-After a run, write a JUnit XML report for CI while still printing the human (or `--json`) report to stdout:
+| | |
+| --- | --- |
+| **轻量套件** | 少量用例，不是完整评测框架 |
+| **匹配器** | `equals`、`contains`、`not_contains`、`regex`、`json`、token 上下限 |
+| **init** | `eval-smoke init` 生成可 mock 的 `suite.yaml` |
+| **过滤** | `--only`：id 子串或 `/regex/` |
+| **CI** | `--junit`、`--html`、`--github-summary`、退出码、`--fail-fast`、`--concurrency` |
+| **基线** | `--baseline` / `--update-baseline` 拦住 pass→fail 回归 |
+| **提供商** | Mock 离线；有 key 时走 OpenAI / Anthropic |
+### 安装
 ```bash
-eval-smoke suite.yaml --mock --junit reports/junit.xml
+npm i -g @webywz/eval-smoke
+# 或一次性：
+npx @webywz/eval-smoke suite.yaml --mock
 ```
-Failures include matcher detail (`expected` / `actual` / `detail`). Skipped cases (e.g. `--fail-fast`) become `<skipped/>`.
+需要 Node 18+。
+### 快速开始
-## Runners
+```bash
+eval-smoke init
+eval-smoke suite.yaml --mock
+eval-smoke suite.yaml --mock --only greeting
+eval-smoke suite.yaml --mock --junit junit.xml
+eval-smoke suite.yaml --mock --html report.html
+eval-smoke examples/suite.yaml --mock --json
+eval-smoke examples/suite.yaml --mock --concurrency 4
+eval-smoke examples/suite.yaml --mock --update-baseline baseline.json
+eval-smoke examples/suite.yaml --mock --baseline baseline.json
+eval-smoke examples/suite.yaml --mock --github-summary
+OPENAI_API_KEY=sk-... eval-smoke examples/suite.yaml
+ANTHROPIC_API_KEY=sk-ant-... eval-smoke examples/suite.yaml --model claude-3-5-haiku-20241022
+```
-**Mock** (`--mock` or `--provider mock`) never calls a network. It returns canned replies matched from the prompt so the example suite stays green in CI:
+### `eval-smoke init`
-| Prompt contains | Mock reply |
+```bash
+eval-smoke init           # 写入 ./suite.yaml
+eval-smoke init ./prompts # 写入 ./prompts/suite.yaml
+eval-smoke init --force   # 覆盖已有文件
+```
+生成小型 mock 友好套件（问候、首都、算术、拒答）。不加 `--force` 不会覆盖已有文件。
+### 套件格式
+YAML 或 JSON —— 用例列表，或带 `cases` 的对象：
+```yaml
+name: demo
+model: gpt-4o-mini
+cases:
+  - id: greeting
+    prompt: Reply with exactly the word Hello.
+    expect:
+      equals: Hello
+      contains: Hello
+      not_contains: Goodbye
+      regex: "(?i)^hello\\b"
+      min_tokens: 1
+      max_tokens: 32
+```
+| 字段 | 说明 |
 | --- | --- |
-| `hello` | `Hello` |
-| `capital of France` | `Paris` |
-| `2 + 2` | `4` |
-| `ignore previous` / `PWNED` | a refusal that does not contain `PWNED` |
+| `id` / `prompt` | 必填 |
+| `expect.contains` | 子串，忽略大小写；字符串或列表 |
+| `expect.not_contains` | 不得出现 |
+| `expect.regex` | `/pattern/i`、`(?i)pattern` 或裸模式 |
+| `expect.equals` | NFC + LF 规范化后精确匹配 |
+| `expect.json` | `true` = 可解析；对象/数组 = 深相等 |
+| `expect.min_tokens` / `max_tokens` | 约 `ceil(chars / 4)` |
+| `system` / `model` | 套件级或用例级 |
+| `provider` | `openai` / `anthropic` / `mock` |
-Anything else returns `[mock] No canned reply for this prompt.` — useful as a deliberate fail.
+每个用例至少要有一个匹配器。失败时打印期望/实际（`equals` 还有首差位置）。
-**Live OpenAI** when `OPENAI_API_KEY` is set. Optional `OPENAI_BASE_URL` for compatible servers.
+### 命令行要点
-**Live Anthropic** when `ANTHROPIC_API_KEY` is set, or when the model name starts with `claude`.
+| 参数 | 含义 |
+| --- | --- |
+| `--mock` | 离线确定性运行 |
+| `--json` | JSON 报告 |
+| `--junit <path>` | 写出 JUnit XML（仍打印人类可读报告，除非同时 `--json`） |
+| `--html <path>` | 写入自包含 HTML 报告（离线、无 CDN） |
+| `--baseline <path>` | 基线里曾通过的用例若失败或缺失则退出 `1` |
+| `--update-baseline <path>` | 把当前报告 JSON（pretty）写入路径 |
+| `--github-summary [path]` | 追加通过/失败 Markdown 到 `$GITHUB_STEP_SUMMARY` 或指定路径 |
+| `--only <pattern>` | 按 id 子串或 `/regex/` 过滤 |
+| `--fail-fast` | 首个失败即停 |
+| `--concurrency <n>` | 并行用例数（默认 1） |
+| `--timeout <ms>` | 单用例超时（默认 30000） |
+| `--model` / `--provider` | 覆盖模型 / 提供商 |
+| `--force` | `init` 时覆盖 |
+| 退出码 | 含义 |
+| --- | --- |
+| `0` | 全部通过 |
+| `1` | 有失败 |
+| `2` | 用法/配置错误（缺文件、坏 YAML、无 key、`--only` 无匹配） |
-If no key is set and you did not pass `--mock`, the CLI exits 2 with a short hint.
+### 运行器
-## JSON report
+**Mock** 永不联网。内置 canned 回复保证示例套件在 CI 里绿。
-```bash
-eval-smoke examples/suite.yaml --mock --json
-```
+**Live OpenAI**：`OPENAI_API_KEY`（可选 `OPENAI_BASE_URL`）。**Live Anthropic**：`ANTHROPIC_API_KEY`，或模型名以 `claude` 开头。
-```json
-{
-  "ok": true,
-  "version": "0.3.0",
-  "runner": "mock",
-  "model": "gpt-4o-mini",
-  "suite": "demo",
-  "passed": 5,
-  "failed": 0,
-  "skipped": 0,
-  "duration_ms": 1,
-  "cases": [
-    {
-      "id": "greeting",
-      "ok": true,
-      "duration_ms": 1,
-      "output": "Hello",
-      "checks": [{ "name": "contains", "ok": true, "expected": "Hello" }]
-    }
-  ]
-}
-```
-Failed checks may include `actual` plus a multi-line `detail` (first-diff for `equals`).
-## Programmatic API
+### 库用法
 ```ts
 import { loadSuite, runSuite, formatJUnit, initSuite } from "@webywz/eval-smoke";
@@ -224,21 +340,10 @@ console.log(formatJUnit(report));
 if (!report.ok) process.exit(1);
 ```
-`runSuite` also accepts a custom `complete` function, which is how the tests inject fakes without keys.
-## CI
+### 链接
-GitHub Actions in this repo runs `npm test`, `npm run build`, and `eval-smoke examples/suite.yaml --mock` on Node 18/20/22. Drop this into another project:
-```yaml
-- run: npx @webywz/eval-smoke@0.3.1 prompts/smoke.yaml --mock --junit junit.xml
-- uses: actions/upload-artifact@v4
-  if: always()
-  with:
-    name: junit
-    path: junit.xml
-```
-## License
-MIT
+| | |
+| --- | --- |
+| npm | [npmjs.com/package/@webywz/eval-smoke](https://www.npmjs.com/package/@webywz/eval-smoke) |
+| Issues | [github.com/webywz/eval-smoke/issues](https://github.com/webywz/eval-smoke/issues) |
+| License | [MIT](./LICENSE) © webywz |

package/dist/baseline.d.ts ADDED Viewed

@@ -0,0 +1,25 @@
+import type { Report } from "./types.js";
+export interface BaselineCase {
+    id: string;
+    ok: boolean;
+}
+export interface BaselineFile {
+    cases: BaselineCase[];
+}
+/** Minimal shape: case id → ok. Accepts a full Report or a slim baseline. */
+export declare function loadBaseline(path: string): Map<string, boolean>;
+export interface BaselineCheck {
+    ok: boolean;
+    regressions: string[];
+    /** Human-readable lines */
+    report: string;
+}
+/**
+ * Fail if any case that passed in baseline now fails, or a baseline case is missing
+ * from the current run. New cases are OK. Skipped current cases that passed in
+ * baseline count as missing/regression.
+ */
+export declare function checkBaseline(report: Report, baseline: Map<string, boolean>): BaselineCheck;
+/** Pretty-print current report JSON for --update-baseline. */
+export declare function writeBaseline(path: string, report: Report): void;
+//# sourceMappingURL=baseline.d.ts.map

package/dist/baseline.d.ts.map ADDED Viewed

@@ -0,0 +1 @@

+ {"version":3,"file":"baseline.d.ts","sourceRoot":"","sources":["../src/baseline.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,YAAY,CAAC;AAGzC,MAAM,WAAW,YAAY;IAC3B,EAAE,EAAE,MAAM,CAAC;IACX,EAAE,EAAE,OAAO,CAAC;CACb;AAED,MAAM,WAAW,YAAY;IAC3B,KAAK,EAAE,YAAY,EAAE,CAAC;CACvB;AAED,6EAA6E;AAC7E,wBAAgB,YAAY,CAAC,IAAI,EAAE,MAAM,GAAG,GAAG,CAAC,MAAM,EAAE,OAAO,CAAC,CAmC/D;AAED,MAAM,WAAW,aAAa;IAC5B,EAAE,EAAE,OAAO,CAAC;IACZ,WAAW,EAAE,MAAM,EAAE,CAAC;IACtB,2BAA2B;IAC3B,MAAM,EAAE,MAAM,CAAC;CAChB;AAED;;;;GAIG;AACH,wBAAgB,aAAa,CAAC,MAAM,EAAE,MAAM,EAAE,QAAQ,EAAE,GAAG,CAAC,MAAM,EAAE,OAAO,CAAC,GAAG,aAAa,CAgC3F;AAED,8DAA8D;AAC9D,wBAAgB,aAAa,CAAC,IAAI,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,GAAG,IAAI,CAEhE"}

package/dist/baseline.js ADDED Viewed

@@ -0,0 +1,86 @@
+import { readFileSync, writeFileSync } from "node:fs";
+import { UsageError } from "./types.js";
+/** Minimal shape: case id → ok. Accepts a full Report or a slim baseline. */
+export function loadBaseline(path) {
+    let raw;
+    try {
+        raw = readFileSync(path, "utf8");
+    }
+    catch (error) {
+        const code = error && typeof error === "object" && "code" in error
+            ? String(error.code)
+            : "";
+        if (code === "ENOENT")
+            throw new UsageError(`baseline not found: ${path}`);
+        throw error;
+    }
+    let parsed;
+    try {
+        parsed = JSON.parse(raw);
+    }
+    catch {
+        throw new UsageError(`baseline is not valid JSON: ${path}`);
+    }
+    if (!parsed || typeof parsed !== "object") {
+        throw new UsageError(`baseline must be a JSON object: ${path}`);
+    }
+    const cases = parsed.cases;
+    if (!Array.isArray(cases)) {
+        throw new UsageError(`baseline missing cases[]: ${path}`);
+    }
+    const map = new Map();
+    for (const entry of cases) {
+        if (!entry || typeof entry !== "object")
+            continue;
+        const id = entry.id;
+        const ok = entry.ok;
+        if (typeof id !== "string" || !id)
+            continue;
+        if (typeof ok !== "boolean")
+            continue;
+        map.set(id, ok);
+    }
+    return map;
+}
+/**
+ * Fail if any case that passed in baseline now fails, or a baseline case is missing
+ * from the current run. New cases are OK. Skipped current cases that passed in
+ * baseline count as missing/regression.
+ */
+export function checkBaseline(report, baseline) {
+    const byId = new Map(report.cases.map((c) => [c.id, c]));
+    const regressions = [];
+    for (const [id, wasOk] of baseline) {
+        if (!wasOk)
+            continue; // only guard formerly-passing cases
+        const current = byId.get(id);
+        if (!current) {
+            regressions.push(`${id}: missing from current run (passed in baseline)`);
+            continue;
+        }
+        if (current.skipped) {
+            regressions.push(`${id}: skipped now (passed in baseline)`);
+            continue;
+        }
+        if (!current.ok) {
+            regressions.push(`${id}: was pass, now fail`);
+        }
+    }
+    const lines = ["baseline check:"];
+    if (regressions.length === 0) {
+        lines.push("  ok — no regressions vs baseline");
+    }
+    else {
+        lines.push(`  ${regressions.length} regression(s):`);
+        for (const line of regressions) {
+            lines.push(`  - ${line}`);
+        }
+    }
+    lines.push("");
+    return { ok: regressions.length === 0, regressions, report: lines.join("\n") };
+}
+/** Pretty-print current report JSON for --update-baseline. */
+export function writeBaseline(path, report) {
+    writeFileSync(path, `${JSON.stringify(report, null, 2)}\n`, "utf8");
+}
+//# sourceMappingURL=baseline.js.map

package/dist/baseline.js.map ADDED Viewed

@@ -0,0 +1 @@

+ {"version":3,"file":"baseline.js","sourceRoot":"","sources":["../src/baseline.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,SAAS,CAAC;AAEtD,OAAO,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAWxC,6EAA6E;AAC7E,MAAM,UAAU,YAAY,CAAC,IAAY;IACvC,IAAI,GAAW,CAAC;IAChB,IAAI,CAAC;QACH,GAAG,GAAG,YAAY,CAAC,IAAI,EAAE,MAAM,CAAC,CAAC;IACnC,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,IAAI,GACR,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,MAAM,IAAI,KAAK;YACnD,CAAC,CAAC,MAAM,CAAE,KAA2B,CAAC,IAAI,CAAC;YAC3C,CAAC,CAAC,EAAE,CAAC;QACT,IAAI,IAAI,KAAK,QAAQ;YAAE,MAAM,IAAI,UAAU,CAAC,uBAAuB,IAAI,EAAE,CAAC,CAAC;QAC3E,MAAM,KAAK,CAAC;IACd,CAAC;IACD,IAAI,MAAe,CAAC;IACpB,IAAI,CAAC;QACH,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;IAC3B,CAAC;IAAC,MAAM,CAAC;QACP,MAAM,IAAI,UAAU,CAAC,+BAA+B,IAAI,EAAE,CAAC,CAAC;IAC9D,CAAC;IACD,IAAI,CAAC,MAAM,IAAI,OAAO,MAAM,KAAK,QAAQ,EAAE,CAAC;QAC1C,MAAM,IAAI,UAAU,CAAC,mCAAmC,IAAI,EAAE,CAAC,CAAC;IAClE,CAAC;IACD,MAAM,KAAK,GAAI,MAA8B,CAAC,KAAK,CAAC;IACpD,IAAI,CAAC,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,EAAE,CAAC;QAC1B,MAAM,IAAI,UAAU,CAAC,6BAA6B,IAAI,EAAE,CAAC,CAAC;IAC5D,CAAC;IACD,MAAM,GAAG,GAAG,IAAI,GAAG,EAAmB,CAAC;IACvC,KAAK,MAAM,KAAK,IAAI,KAAK,EAAE,CAAC;QAC1B,IAAI,CAAC,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ;YAAE,SAAS;QAClD,MAAM,EAAE,GAAI,KAA0B,CAAC,EAAE,CAAC;QAC1C,MAAM,EAAE,GAAI,KAA0B,CAAC,EAAE,CAAC;QAC1C,IAAI,OAAO,EAAE,KAAK,QAAQ,IAAI,CAAC,EAAE;YAAE,SAAS;QAC5C,IAAI,OAAO,EAAE,KAAK,SAAS;YAAE,SAAS;QACtC,GAAG,CAAC,GAAG,CAAC,EAAE,EAAE,EAAE,CAAC,CAAC;IAClB,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AASD;;;;GAIG;AACH,MAAM,UAAU,aAAa,CAAC,MAAc,EAAE,QAA8B;IAC1E,MAAM,IAAI,GAAG,IAAI,GAAG,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;IACzD,MAAM,WAAW,GAAa,EAAE,CAAC;IAEjC,KAAK,MAAM,CAAC,EAAE,EAAE,KAAK,CAAC,IAAI,QAAQ,EAAE,CAAC;QACnC,IAAI,CAAC,KAAK;YAAE,SAAS,CAAC,oCAAoC;QAC1D,MAAM,OAAO,GAAG,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;QAC7B,IAAI,CAAC,OAAO,EAAE,CAAC;YACb,WAAW,CAAC,IAAI,CAAC,GAAG,EAAE,iDAAiD,CAAC,CAAC;YACzE,SAAS;QACX,CAAC;QACD,IAAI,OAAO,CAAC,OAAO,EAAE,CAAC;YACpB,WAAW,CAAC,IAAI,CAAC,GAAG,EAAE,oCAAoC,CAAC,CAAC;YAC5D,SAAS;QACX,CAAC;QACD,IAAI,CAAC,OAAO,CAAC,EAAE,EAAE,CAAC;YAChB,WAAW,CAAC,IAAI,CAAC,GAAG,EAAE,sBAAsB,CAAC,CAAC;QAChD,CAAC;IACH,CAAC;IAED,MAAM,KAAK,GAAG,CAAC,iBAAiB,CAAC,CAAC;IAClC,IAAI,WAAW,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC7B,KAAK,CAAC,IAAI,CAAC,mCAAmC,CAAC,CAAC;IAClD,CAAC;SAAM,CAAC;QACN,KAAK,CAAC,IAAI,CAAC,KAAK,WAAW,CAAC,MAAM,iBAAiB,CAAC,CAAC;QACrD,KAAK,MAAM,IAAI,IAAI,WAAW,EAAE,CAAC;YAC/B,KAAK,CAAC,IAAI,CAAC,OAAO,IAAI,EAAE,CAAC,CAAC;QAC5B,CAAC;IACH,CAAC;IACD,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IAEf,OAAO,EAAE,EAAE,EAAE,WAAW,CAAC,MAAM,KAAK,CAAC,EAAE,WAAW,EAAE,MAAM,EAAE,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;AACjF,CAAC;AAED,8DAA8D;AAC9D,MAAM,UAAU,aAAa,CAAC,IAAY,EAAE,MAAc;IACxD,aAAa,CAAC,IAAI,EAAE,GAAG,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC,IAAI,EAAE,MAAM,CAAC,CAAC;AACtE,CAAC"}

package/dist/github-summary.d.ts ADDED Viewed

@@ -0,0 +1,4 @@
+export declare const DEFAULT_GITHUB_SUMMARY = "github-step-summary.md";
+export declare function resolveGithubSummaryPath(flagPath: string | true | undefined, env?: NodeJS.ProcessEnv, cwd?: string): string | undefined;
+export declare function appendGithubSummary(path: string, markdown: string): void;
+//# sourceMappingURL=github-summary.d.ts.map

package/dist/github-summary.d.ts.map ADDED Viewed

	@@ -0,0 +1 @@
1	+ {"version":3,"file":"github-summary.d.ts","sourceRoot":"","sources":["../src/github-summary.ts"],"names":[],"mappings":"AAGA,eAAO,MAAM,sBAAsB,2BAA2B,CAAC;AAE/D,wBAAgB,wBAAwB,CACtC,QAAQ,EAAE,MAAM,GAAG,IAAI,GAAG,SAAS,EACnC,GAAG,GAAE,MAAM,CAAC,UAAwB,EACpC,GAAG,GAAE,MAAsB,GAC1B,MAAM,GAAG,SAAS,CAWpB;AAED,wBAAgB,mBAAmB,CAAC,IAAI,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,GAAG,IAAI,CAoBxE"}

package/dist/github-summary.js ADDED Viewed

@@ -0,0 +1,39 @@
+import { appendFileSync, mkdirSync, writeFileSync } from "node:fs";
+import { dirname, resolve } from "node:path";
+export const DEFAULT_GITHUB_SUMMARY = "github-step-summary.md";
+export function resolveGithubSummaryPath(flagPath, env = process.env, cwd = process.cwd()) {
+    if (flagPath === undefined) {
+        const fromEnv = env.GITHUB_STEP_SUMMARY;
+        return fromEnv && fromEnv.trim() ? fromEnv : undefined;
+    }
+    if (flagPath === true || flagPath === "") {
+        const fromEnv = env.GITHUB_STEP_SUMMARY;
+        if (fromEnv && fromEnv.trim())
+            return fromEnv;
+        return resolve(cwd, DEFAULT_GITHUB_SUMMARY);
+    }
+    return resolve(cwd, flagPath);
+}
+export function appendGithubSummary(path, markdown) {
+    const body = markdown.endsWith("\n") ? markdown : `${markdown}\n`;
+    try {
+        mkdirSync(dirname(path), { recursive: true });
+    }
+    catch {
+        // ignore
+    }
+    try {
+        appendFileSync(path, body, "utf8");
+    }
+    catch (error) {
+        const code = error && typeof error === "object" && "code" in error
+            ? String(error.code)
+            : "";
+        if (code === "ENOENT") {
+            writeFileSync(path, body, "utf8");
+            return;
+        }
+        throw error;
+    }
+}
+//# sourceMappingURL=github-summary.js.map

package/dist/github-summary.js.map ADDED Viewed

@@ -0,0 +1 @@

+ {"version":3,"file":"github-summary.js","sourceRoot":"","sources":["../src/github-summary.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,cAAc,EAAE,SAAS,EAAE,aAAa,EAAE,MAAM,SAAS,CAAC;AACnE,OAAO,EAAE,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AAE7C,MAAM,CAAC,MAAM,sBAAsB,GAAG,wBAAwB,CAAC;AAE/D,MAAM,UAAU,wBAAwB,CACtC,QAAmC,EACnC,MAAyB,OAAO,CAAC,GAAG,EACpC,MAAc,OAAO,CAAC,GAAG,EAAE;IAE3B,IAAI,QAAQ,KAAK,SAAS,EAAE,CAAC;QAC3B,MAAM,OAAO,GAAG,GAAG,CAAC,mBAAmB,CAAC;QACxC,OAAO,OAAO,IAAI,OAAO,CAAC,IAAI,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,SAAS,CAAC;IACzD,CAAC;IACD,IAAI,QAAQ,KAAK,IAAI,IAAI,QAAQ,KAAK,EAAE,EAAE,CAAC;QACzC,MAAM,OAAO,GAAG,GAAG,CAAC,mBAAmB,CAAC;QACxC,IAAI,OAAO,IAAI,OAAO,CAAC,IAAI,EAAE;YAAE,OAAO,OAAO,CAAC;QAC9C,OAAO,OAAO,CAAC,GAAG,EAAE,sBAAsB,CAAC,CAAC;IAC9C,CAAC;IACD,OAAO,OAAO,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC;AAChC,CAAC;AAED,MAAM,UAAU,mBAAmB,CAAC,IAAY,EAAE,QAAgB;IAChE,MAAM,IAAI,GAAG,QAAQ,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,GAAG,QAAQ,IAAI,CAAC;IAClE,IAAI,CAAC;QACH,SAAS,CAAC,OAAO,CAAC,IAAI,CAAC,EAAE,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC;IAChD,CAAC;IAAC,MAAM,CAAC;QACP,SAAS;IACX,CAAC;IACD,IAAI,CAAC;QACH,cAAc,CAAC,IAAI,EAAE,IAAI,EAAE,MAAM,CAAC,CAAC;IACrC,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,IAAI,GACR,KAAK,IAAI,OAAO,KAAK,KAAK,QAAQ,IAAI,MAAM,IAAI,KAAK;YACnD,CAAC,CAAC,MAAM,CAAE,KAA2B,CAAC,IAAI,CAAC;YAC3C,CAAC,CAAC,EAAE,CAAC;QACT,IAAI,IAAI,KAAK,QAAQ,EAAE,CAAC;YACtB,aAAa,CAAC,IAAI,EAAE,IAAI,EAAE,MAAM,CAAC,CAAC;YAClC,OAAO;QACT,CAAC;QACD,MAAM,KAAK,CAAC;IACd,CAAC;AACH,CAAC"}

package/dist/html-report.d.ts ADDED Viewed

@@ -0,0 +1,8 @@
+import type { Report } from "./types.js";
+/** Escape text for safe insertion into HTML text/attr contexts. */
+export declare function escapeHtml(text: string): string;
+/**
+ * Self-contained dark HTML report for an eval-smoke run (inline CSS+JS, no CDN).
+ */
+export declare function formatHtmlReport(report: Report): string;
+//# sourceMappingURL=html-report.d.ts.map

package/dist/html-report.d.ts.map ADDED Viewed

	@@ -0,0 +1 @@
1	+ {"version":3,"file":"html-report.d.ts","sourceRoot":"","sources":["../src/html-report.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAA2B,MAAM,EAAE,MAAM,YAAY,CAAC;AAElE,mEAAmE;AACnE,wBAAgB,UAAU,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,CAO/C;AA8CD;;GAEG;AACH,wBAAgB,gBAAgB,CAAC,MAAM,EAAE,MAAM,GAAG,MAAM,CAgJvD"}

package/dist/html-report.js ADDED Viewed

@@ -0,0 +1,196 @@
+/** Escape text for safe insertion into HTML text/attr contexts. */
+export function escapeHtml(text) {
+    return text
+        .replace(/&/g, "&amp;")
+        .replace(/</g, "&lt;")
+        .replace(/>/g, "&gt;")
+        .replace(/"/g, "&quot;")
+        .replace(/'/g, "&#39;");
+}
+function truncate(text, max = 400) {
+    const compact = text.replace(/\s+/g, " ").trim();
+    if (compact.length <= max)
+        return compact;
+    return `${compact.slice(0, max - 1)}…`;
+}
+function failedChecks(result) {
+    return (result.checks ?? []).filter((check) => !check.ok);
+}
+function statusLabel(result) {
+    if (result.skipped)
+        return "skip";
+    return result.ok ? "pass" : "fail";
+}
+function statusClass(result) {
+    if (result.skipped)
+        return "skip";
+    return result.ok ? "pass" : "fail";
+}
+function failureDetails(result) {
+    const parts = [];
+    if (result.error)
+        parts.push(result.error);
+    for (const check of failedChecks(result)) {
+        const detail = check.detail ?? "failed";
+        parts.push(`${check.name}: ${detail}`);
+        if (check.expected != null && !detail.includes("expected:")) {
+            parts.push(`expected: ${truncate(check.expected, 200)}`);
+        }
+        if (check.actual != null && !detail.includes("actual:")) {
+            parts.push(`actual: ${truncate(check.actual, 200)}`);
+        }
+    }
+    if (result.output &&
+        !result.ok &&
+        failedChecks(result).every((c) => c.name !== "equals")) {
+        parts.push(`output: ${truncate(result.output)}`);
+    }
+    if (!parts.length)
+        return "";
+    return `<details><summary>details</summary><pre>${escapeHtml(parts.join("\n"))}</pre></details>`;
+}
+/**
+ * Self-contained dark HTML report for an eval-smoke run (inline CSS+JS, no CDN).
+ */
+export function formatHtmlReport(report) {
+    const suite = escapeHtml(report.suite ?? "suite");
+    const version = escapeHtml(report.version);
+    const runner = escapeHtml(report.runner);
+    const model = escapeHtml(report.model);
+    const okClass = report.ok ? "ok" : "bad";
+    const rows = report.cases
+        .map((result) => {
+        const st = statusClass(result);
+        const details = failureDetails(result);
+        return `<tr class="${st}">
+  <td><span class="badge ${st}">${statusLabel(result)}</span></td>
+  <td><code>${escapeHtml(result.id)}</code></td>
+  <td class="num">${result.duration_ms}ms</td>
+  <td class="detail">${details}</td>
+</tr>`;
+    })
+        .join("\n");
+    return `<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="utf-8"/>
+<meta name="viewport" content="width=device-width, initial-scale=1"/>
+<title>eval-smoke · ${suite}</title>
+<style>
+:root {
+  --bg: #0a0a0a;
+  --panel: #111111;
+  --border: #222222;
+  --text: #eeeeee;
+  --muted: #888888;
+  --pass: #7ee787;
+  --fail: #ff7b72;
+  --skip: #d2a8ff;
+  --font: ui-monospace, SFMono-Regular, Menlo, Consolas, monospace;
+}
+* { box-sizing: border-box; }
+html, body {
+  margin: 0; padding: 0;
+  background: var(--bg); color: var(--text);
+  font: 13px/1.45 var(--font);
+}
+body { padding: 24px 20px 48px; max-width: 960px; margin: 0 auto; }
+header.top {
+  border: 1px solid var(--border); background: var(--panel);
+  padding: 16px 18px; margin-bottom: 16px;
+}
+header.top h1 { margin: 0 0 6px; font-size: 15px; font-weight: 600; }
+header.top .meta { color: var(--muted); font-size: 12px; }
+header.top .summary { margin-top: 10px; font-size: 13px; }
+header.top.ok .summary { color: var(--pass); }
+header.top.bad .summary { color: var(--fail); }
+.stats {
+  display: grid; grid-template-columns: repeat(4, minmax(0, 1fr));
+  gap: 8px; margin: 0 0 18px;
+}
+.stat {
+  border: 1px solid var(--border); background: var(--panel);
+  padding: 10px 12px;
+}
+.stat .label { color: var(--muted); font-size: 11px; text-transform: uppercase; letter-spacing: 0.06em; }
+.stat .value { margin-top: 4px; font-size: 16px; }
+table {
+  width: 100%; border-collapse: collapse;
+  border: 1px solid var(--border); background: var(--panel);
+}
+th, td {
+  text-align: left; padding: 8px 10px; border-bottom: 1px solid var(--border);
+  vertical-align: top;
+}
+th { color: var(--muted); font-weight: 500; font-size: 11px; text-transform: uppercase; }
+td.num { text-align: right; color: var(--muted); white-space: nowrap; }
+.badge {
+  display: inline-block; min-width: 3.2em; text-align: center;
+  padding: 1px 6px; border: 1px solid var(--border);
+}
+.badge.pass { color: var(--pass); border-color: #1e3d28; }
+.badge.fail { color: var(--fail); border-color: #5a1d1d; }
+.badge.skip { color: var(--skip); border-color: #3a2a55; }
+tr.fail { background: #140a0a; }
+details { margin-top: 4px; }
+summary { cursor: pointer; color: var(--muted); user-select: none; }
+pre {
+  margin: 6px 0 0; padding: 10px; overflow: auto;
+  background: #050505; border: 1px solid var(--border);
+  white-space: pre-wrap; word-break: break-word; color: #d8d8d8;
+}
+.toolbar { margin: 0 0 12px; display: flex; gap: 8px; }
+.toolbar button {
+  background: var(--panel); color: var(--text); border: 1px solid var(--border);
+  padding: 6px 10px; font: inherit; cursor: pointer;
+}
+.toolbar button:hover { border-color: #444; }
+footer {
+  margin-top: 28px; color: var(--muted); font-size: 11px;
+  border-top: 1px solid var(--border); padding-top: 12px;
+}
+code { font: inherit; }
+@media (max-width: 640px) {
+  .stats { grid-template-columns: repeat(2, minmax(0, 1fr)); }
+}
+</style>
+</head>
+<body>
+<header class="top ${okClass}">
+  <h1>eval-smoke ${version}</h1>
+  <div class="meta">${runner} · ${model} · ${suite} · ${report.cases.length} cases</div>
+  <div class="summary">${report.passed} passed · ${report.failed} failed${report.skipped ? ` · ${report.skipped} skipped` : ""} · ${report.duration_ms}ms</div>
+</header>
+<section class="stats" aria-label="stats">
+  <div class="stat"><div class="label">passed</div><div class="value">${report.passed}</div></div>
+  <div class="stat"><div class="label">failed</div><div class="value">${report.failed}</div></div>
+  <div class="stat"><div class="label">skipped</div><div class="value">${report.skipped}</div></div>
+  <div class="stat"><div class="label">duration</div><div class="value">${report.duration_ms}ms</div></div>
+</section>
+<div class="toolbar">
+  <button type="button" id="expand-all">Expand failures</button>
+  <button type="button" id="collapse-all">Collapse</button>
+</div>
+<table>
+<thead><tr><th>Status</th><th>Case</th><th>Duration</th><th>Details</th></tr></thead>
+<tbody>
+${rows || `<tr><td colspan="4" style="color:var(--muted)">no cases</td></tr>`}
+</tbody>
+</table>
+<footer>Generated by eval-smoke · self-contained · works offline (file://)</footer>
+<script>
+(function () {
+  var expand = document.getElementById("expand-all");
+  var collapse = document.getElementById("collapse-all");
+  function setAll(open) {
+    document.querySelectorAll("details").forEach(function (d) { d.open = open; });
+  }
+  if (expand) expand.addEventListener("click", function () { setAll(true); });
+  if (collapse) collapse.addEventListener("click", function () { setAll(false); });
+})();
+</script>
+</body>
+</html>
+`;
+}
+//# sourceMappingURL=html-report.js.map

package/dist/html-report.js.map ADDED Viewed

@@ -0,0 +1 @@

+ {"version":3,"file":"html-report.js","sourceRoot":"","sources":["../src/html-report.ts"],"names":[],"mappings":"AAEA,mEAAmE;AACnE,MAAM,UAAU,UAAU,CAAC,IAAY;IACrC,OAAO,IAAI;SACR,OAAO,CAAC,IAAI,EAAE,OAAO,CAAC;SACtB,OAAO,CAAC,IAAI,EAAE,MAAM,CAAC;SACrB,OAAO,CAAC,IAAI,EAAE,MAAM,CAAC;SACrB,OAAO,CAAC,IAAI,EAAE,QAAQ,CAAC;SACvB,OAAO,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC;AAC5B,CAAC;AAED,SAAS,QAAQ,CAAC,IAAY,EAAE,GAAG,GAAG,GAAG;IACvC,MAAM,OAAO,GAAG,IAAI,CAAC,OAAO,CAAC,MAAM,EAAE,GAAG,CAAC,CAAC,IAAI,EAAE,CAAC;IACjD,IAAI,OAAO,CAAC,MAAM,IAAI,GAAG;QAAE,OAAO,OAAO,CAAC;IAC1C,OAAO,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC,EAAE,GAAG,GAAG,CAAC,CAAC,GAAG,CAAC;AACzC,CAAC;AAED,SAAS,YAAY,CAAC,MAAkB;IACtC,OAAO,CAAC,MAAM,CAAC,MAAM,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC,KAAK,CAAC,EAAE,CAAC,CAAC;AAC5D,CAAC;AAED,SAAS,WAAW,CAAC,MAAkB;IACrC,IAAI,MAAM,CAAC,OAAO;QAAE,OAAO,MAAM,CAAC;IAClC,OAAO,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,MAAM,CAAC;AACrC,CAAC;AAED,SAAS,WAAW,CAAC,MAAkB;IACrC,IAAI,MAAM,CAAC,OAAO;QAAE,OAAO,MAAM,CAAC;IAClC,OAAO,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,MAAM,CAAC;AACrC,CAAC;AAED,SAAS,cAAc,CAAC,MAAkB;IACxC,MAAM,KAAK,GAAa,EAAE,CAAC;IAC3B,IAAI,MAAM,CAAC,KAAK;QAAE,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;IAC3C,KAAK,MAAM,KAAK,IAAI,YAAY,CAAC,MAAM,CAAC,EAAE,CAAC;QACzC,MAAM,MAAM,GAAG,KAAK,CAAC,MAAM,IAAI,QAAQ,CAAC;QACxC,KAAK,CAAC,IAAI,CAAC,GAAG,KAAK,CAAC,IAAI,KAAK,MAAM,EAAE,CAAC,CAAC;QACvC,IAAI,KAAK,CAAC,QAAQ,IAAI,IAAI,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,WAAW,CAAC,EAAE,CAAC;YAC5D,KAAK,CAAC,IAAI,CAAC,aAAa,QAAQ,CAAC,KAAK,CAAC,QAAQ,EAAE,GAAG,CAAC,EAAE,CAAC,CAAC;QAC3D,CAAC;QACD,IAAI,KAAK,CAAC,MAAM,IAAI,IAAI,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,SAAS,CAAC,EAAE,CAAC;YACxD,KAAK,CAAC,IAAI,CAAC,WAAW,QAAQ,CAAC,KAAK,CAAC,MAAM,EAAE,GAAG,CAAC,EAAE,CAAC,CAAC;QACvD,CAAC;IACH,CAAC;IACD,IACE,MAAM,CAAC,MAAM;QACb,CAAC,MAAM,CAAC,EAAE;QACV,YAAY,CAAC,MAAM,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,KAAK,QAAQ,CAAC,EACtD,CAAC;QACD,KAAK,CAAC,IAAI,CAAC,WAAW,QAAQ,CAAC,MAAM,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC;IACnD,CAAC;IACD,IAAI,CAAC,KAAK,CAAC,MAAM;QAAE,OAAO,EAAE,CAAC;IAC7B,OAAO,2CAA2C,UAAU,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,kBAAkB,CAAC;AACnG,CAAC;AAED;;GAEG;AACH,MAAM,UAAU,gBAAgB,CAAC,MAAc;IAC7C,MAAM,KAAK,GAAG,UAAU,CAAC,MAAM,CAAC,KAAK,IAAI,OAAO,CAAC,CAAC;IAClD,MAAM,OAAO,GAAG,UAAU,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;IAC3C,MAAM,MAAM,GAAG,UAAU,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;IACzC,MAAM,KAAK,GAAG,UAAU,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;IACvC,MAAM,OAAO,GAAG,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,KAAK,CAAC;IAEzC,MAAM,IAAI,GAAG,MAAM,CAAC,KAAK;SACtB,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE;QACd,MAAM,EAAE,GAAG,WAAW,CAAC,MAAM,CAAC,CAAC;QAC/B,MAAM,OAAO,GAAG,cAAc,CAAC,MAAM,CAAC,CAAC;QACvC,OAAO,cAAc,EAAE;2BACF,EAAE,KAAK,WAAW,CAAC,MAAM,CAAC;cACvC,UAAU,CAAC,MAAM,CAAC,EAAE,CAAC;oBACf,MAAM,CAAC,WAAW;uBACf,OAAO;MACxB,CAAC;IACH,CAAC,CAAC;SACD,IAAI,CAAC,IAAI,CAAC,CAAC;IAEd,OAAO;;;;;sBAKa,KAAK;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;qBAiFN,OAAO;mBACT,OAAO;sBACJ,MAAM,MAAM,KAAK,MAAM,KAAK,MAAM,MAAM,CAAC,KAAK,CAAC,MAAM;yBAClD,MAAM,CAAC,MAAM,aAAa,MAAM,CAAC,MAAM,UAC5D,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,MAAM,CAAC,OAAO,UAAU,CAAC,CAAC,CAAC,EACpD,MAAM,MAAM,CAAC,WAAW;;;wEAG8C,MAAM,CAAC,MAAM;wEACb,MAAM,CAAC,MAAM;yEACZ,MAAM,CAAC,OAAO;0EACb,MAAM,CAAC,WAAW;;;;;;;;;EAS1F,IAAI,IAAI,mEAAmE;;;;;;;;;;;;;;;;;CAiB5E,CAAC;AACF,CAAC"}

package/dist/index.d.ts CHANGED Viewed

@@ -4,9 +4,14 @@ export { formatJUnit } from "./junit.js";
 export { evaluateCase, evaluateExpect, equalsDiff, normalizeEquals } from "./matchers.js";
 export { createComplete, mockComplete, resolveProvider } from "./providers.js";
 export { formatReport } from "./report.js";
+export { formatHtmlReport, escapeHtml } from "./html-report.js";
 export { runSuite } from "./run.js";
 export { estimateTokens } from "./tokens.js";
 export { VERSION } from "./version.js";
 export type { Case, CaseResult, CheckName, CheckResult, CompleteFn, CompleteRequest, Expect, JsonExpect, ProviderName, Report, RunOptions, Suite, } from "./types.js";
 export { UsageError } from "./types.js";
+export { checkBaseline, loadBaseline, writeBaseline } from "./baseline.js";
+export type { BaselineCheck, BaselineCase, BaselineFile } from "./baseline.js";
+export { appendGithubSummary, resolveGithubSummaryPath, DEFAULT_GITHUB_SUMMARY } from "./github-summary.js";
+export { formatGithubSummary } from "./summary-md.js";
 //# sourceMappingURL=index.d.ts.map

package/dist/index.d.ts.map CHANGED Viewed

	@@ -1 +1 @@
1	- {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,UAAU,EAAE,MAAM,iBAAiB,CAAC;AACxD,OAAO,EAAE,SAAS,EAAE,mBAAmB,EAAE,eAAe,EAAE,MAAM,WAAW,CAAC;AAC5E,OAAO,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AACzC,OAAO,EAAE,YAAY,EAAE,cAAc,EAAE,UAAU,EAAE,eAAe,EAAE,MAAM,eAAe,CAAC;AAC1F,OAAO,EAAE,cAAc,EAAE,YAAY,EAAE,eAAe,EAAE,MAAM,gBAAgB,CAAC;AAC/E,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,OAAO,EAAE,QAAQ,EAAE,MAAM,UAAU,CAAC;AACpC,OAAO,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAC7C,OAAO,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AACvC,YAAY,EACV,IAAI,EACJ,UAAU,EACV,SAAS,EACT,WAAW,EACX,UAAU,EACV,eAAe,EACf,MAAM,EACN,UAAU,EACV,YAAY,EACZ,MAAM,EACN,UAAU,EACV,KAAK,GACN,MAAM,YAAY,CAAC;AACpB,OAAO,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC"}
1	+ {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,UAAU,EAAE,MAAM,iBAAiB,CAAC;AACxD,OAAO,EAAE,SAAS,EAAE,mBAAmB,EAAE,eAAe,EAAE,MAAM,WAAW,CAAC;AAC5E,OAAO,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AACzC,OAAO,EAAE,YAAY,EAAE,cAAc,EAAE,UAAU,EAAE,eAAe,EAAE,MAAM,eAAe,CAAC;AAC1F,OAAO,EAAE,cAAc,EAAE,YAAY,EAAE,eAAe,EAAE,MAAM,gBAAgB,CAAC;AAC/E,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,OAAO,EAAE,gBAAgB,EAAE,UAAU,EAAE,MAAM,kBAAkB,CAAC;AAChE,OAAO,EAAE,QAAQ,EAAE,MAAM,UAAU,CAAC;AACpC,OAAO,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAC7C,OAAO,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AACvC,YAAY,EACV,IAAI,EACJ,UAAU,EACV,SAAS,EACT,WAAW,EACX,UAAU,EACV,eAAe,EACf,MAAM,EACN,UAAU,EACV,YAAY,EACZ,MAAM,EACN,UAAU,EACV,KAAK,GACN,MAAM,YAAY,CAAC;AACpB,OAAO,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAExC,OAAO,EAAE,aAAa,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,eAAe,CAAC;AAC3E,YAAY,EAAE,aAAa,EAAE,YAAY,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC;AAC/E,OAAO,EAAE,mBAAmB,EAAE,wBAAwB,EAAE,sBAAsB,EAAE,MAAM,qBAAqB,CAAC;AAC5G,OAAO,EAAE,mBAAmB,EAAE,MAAM,iBAAiB,CAAC"}

package/dist/index.js CHANGED Viewed

@@ -4,8 +4,12 @@ export { formatJUnit } from "./junit.js";
 export { evaluateCase, evaluateExpect, equalsDiff, normalizeEquals } from "./matchers.js";
 export { createComplete, mockComplete, resolveProvider } from "./providers.js";
 export { formatReport } from "./report.js";
+export { formatHtmlReport, escapeHtml } from "./html-report.js";
 export { runSuite } from "./run.js";
 export { estimateTokens } from "./tokens.js";
 export { VERSION } from "./version.js";
 export { UsageError } from "./types.js";
+export { checkBaseline, loadBaseline, writeBaseline } from "./baseline.js";
+export { appendGithubSummary, resolveGithubSummaryPath, DEFAULT_GITHUB_SUMMARY } from "./github-summary.js";
+export { formatGithubSummary } from "./summary-md.js";
 //# sourceMappingURL=index.js.map

package/dist/index.js.map CHANGED Viewed

	@@ -1 +1 @@
1	- {"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,UAAU,EAAE,MAAM,iBAAiB,CAAC;AACxD,OAAO,EAAE,SAAS,EAAE,mBAAmB,EAAE,eAAe,EAAE,MAAM,WAAW,CAAC;AAC5E,OAAO,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AACzC,OAAO,EAAE,YAAY,EAAE,cAAc,EAAE,UAAU,EAAE,eAAe,EAAE,MAAM,eAAe,CAAC;AAC1F,OAAO,EAAE,cAAc,EAAE,YAAY,EAAE,eAAe,EAAE,MAAM,gBAAgB,CAAC;AAC/E,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,OAAO,EAAE,QAAQ,EAAE,MAAM,UAAU,CAAC;AACpC,OAAO,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAC7C,OAAO,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAevC,OAAO,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC"}
1	+ {"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,UAAU,EAAE,MAAM,iBAAiB,CAAC;AACxD,OAAO,EAAE,SAAS,EAAE,mBAAmB,EAAE,eAAe,EAAE,MAAM,WAAW,CAAC;AAC5E,OAAO,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AACzC,OAAO,EAAE,YAAY,EAAE,cAAc,EAAE,UAAU,EAAE,eAAe,EAAE,MAAM,eAAe,CAAC;AAC1F,OAAO,EAAE,cAAc,EAAE,YAAY,EAAE,eAAe,EAAE,MAAM,gBAAgB,CAAC;AAC/E,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,OAAO,EAAE,gBAAgB,EAAE,UAAU,EAAE,MAAM,kBAAkB,CAAC;AAChE,OAAO,EAAE,QAAQ,EAAE,MAAM,UAAU,CAAC;AACpC,OAAO,EAAE,cAAc,EAAE,MAAM,aAAa,CAAC;AAC7C,OAAO,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAevC,OAAO,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAExC,OAAO,EAAE,aAAa,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,eAAe,CAAC;AAE3E,OAAO,EAAE,mBAAmB,EAAE,wBAAwB,EAAE,sBAAsB,EAAE,MAAM,qBAAqB,CAAC;AAC5G,OAAO,EAAE,mBAAmB,EAAE,MAAM,iBAAiB,CAAC"}

package/dist/run-cli.d.ts CHANGED Viewed

@@ -1,5 +1,5 @@
 import { type Suite } from "./types.js";
-export declare const HELP = "eval-smoke \u2014 one-minute smoke evals for prompts and models\n\nUsage:\n  eval-smoke <suite.yaml|suite.json> [options]\n  eval-smoke init [dir] [--force]\n\nOptions:\n  --mock              Offline runner with deterministic fake answers\n  --json              Machine-readable JSON report\n  --junit <path>      Write JUnit XML report to path (still prints human report unless --json)\n  --only <pattern>    Run cases whose id contains pattern, or matches /regex/\n  --fail-fast         Stop after the first failing case\n  --concurrency <n>   Run up to N cases in parallel (default: 1; ignored with --fail-fast)\n  --timeout <ms>      Per-case timeout (default: 30000)\n  --model <name>      Override suite model\n  --provider <name>   openai | anthropic | mock\n  --force             Overwrite existing suite.yaml when using init\n  --no-color          Disable ANSI colors (also respects NO_COLOR)\n  -h, --help          Show this help\n  -v, --version       Show version\n\nExit codes:\n  0  all cases passed\n  1  one or more cases failed\n  2  usage or configuration error\n\nExamples:\n  eval-smoke init\n  eval-smoke init ./prompts --force\n  eval-smoke examples/suite.yaml --mock\n  eval-smoke examples/suite.yaml --mock --only greeting\n  eval-smoke examples/suite.yaml --mock --only '/^capital/'\n  eval-smoke examples/suite.yaml --mock --junit junit.xml\n  eval-smoke examples/suite.yaml --json --fail-fast\n  eval-smoke examples/suite.yaml --mock --concurrency 4\n  OPENAI_API_KEY=sk-... eval-smoke examples/suite.yaml\n";
+export declare const HELP = "eval-smoke \u2014 one-minute smoke evals for prompts and models\n\nUsage:\n  eval-smoke <suite.yaml|suite.json> [options]\n  eval-smoke init [dir] [--force]\n\nOptions:\n  --mock              Offline runner with deterministic fake answers\n  --json              Machine-readable JSON report\n  --junit <path>      Write JUnit XML report to path (still prints human report unless --json)\n  --html <path>       Write self-contained HTML report to path (offline, no CDN)\n  --only <pattern>    Run cases whose id contains pattern, or matches /regex/\n  --fail-fast         Stop after the first failing case\n  --concurrency <n>   Run up to N cases in parallel (default: 1; ignored with --fail-fast)\n  --timeout <ms>      Per-case timeout (default: 30000)\n  --model <name>      Override suite model\n  --provider <name>   openai | anthropic | mock\n  --baseline <path>   Fail if any baseline-passing case now fails or is missing\n  --update-baseline <path>  Write current report JSON to path (pretty)\n  --github-summary [path]   Append Markdown to GITHUB_STEP_SUMMARY or path\n  --force             Overwrite existing suite.yaml when using init\n  --no-color          Disable ANSI colors (also respects NO_COLOR)\n  -h, --help          Show this help\n  -v, --version       Show version\n\nExit codes:\n  0  all cases passed (and baseline clean when --baseline is set)\n  1  one or more cases failed, or baseline regression\n  2  usage or configuration error\n\nExamples:\n  eval-smoke init\n  eval-smoke init ./prompts --force\n  eval-smoke examples/suite.yaml --mock\n  eval-smoke examples/suite.yaml --mock --only greeting\n  eval-smoke examples/suite.yaml --mock --only '/^capital/'\n  eval-smoke examples/suite.yaml --mock --junit junit.xml\n  eval-smoke examples/suite.yaml --mock --html report.html\n  eval-smoke examples/suite.yaml --json --fail-fast\n  eval-smoke examples/suite.yaml --mock --concurrency 4\n  eval-smoke examples/suite.yaml --mock --update-baseline baseline.json\n  eval-smoke examples/suite.yaml --mock --baseline baseline.json\n  eval-smoke examples/suite.yaml --mock --github-summary\n  OPENAI_API_KEY=sk-... eval-smoke examples/suite.yaml\n";
 export interface CliIo {
     stdout: {
         write(text: string): void;
@@ -14,5 +14,11 @@ export interface CliIo {
 /** Match case ids by substring, or by /regex/ flags (same style as matchers). */
 export declare function matchOnly(id: string, pattern: string): boolean;
 export declare function filterSuiteByOnly(suite: Suite, pattern: string): Suite;
+/**
+ * Rewrite bare `--github-summary` into `--github-summary=__auto__` so node:util
+ * parseArgs (string option) accepts it. `--github-summary=path` and
+ * `--github-summary path` stay as-is.
+ */
+export declare function normalizeGithubSummaryArgv(argv: string[]): string[];
 export declare function runCli(argv: string[], io?: CliIo): Promise<number>;
 //# sourceMappingURL=run-cli.d.ts.map

package/dist/run-cli.d.ts.map CHANGED Viewed

	@@ -1 +1 @@
1	- {"version":3,"file":"run-cli.d.ts","sourceRoot":"","sources":["../src/run-cli.ts"],"names":[],"mappings":"~~AASA~~,OAAO,EAAiC,KAAK,KAAK,EAAE,MAAM,YAAY,CAAC;~~AAGvE~~,eAAO,MAAM,IAAI,~~mgDAoChB,~~CAAC;AAEF,MAAM,WAAW,KAAK;IACpB,MAAM,EAAE;QAAE,KAAK,CAAC,IAAI,EAAE,MAAM,GAAG,IAAI,CAAA;KAAE,CAAC;IACtC,MAAM,EAAE;QAAE,KAAK,CAAC,IAAI,EAAE,MAAM,GAAG,IAAI,CAAA;KAAE,CAAC;IACtC,GAAG,EAAE,MAAM,CAAC,UAAU,CAAC;IACvB,GAAG,EAAE,MAAM,CAAC;IACZ,KAAK,EAAE,OAAO,CAAC;CAChB;AAkCD,iFAAiF;AACjF,wBAAgB,SAAS,CAAC,EAAE,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,GAAG,OAAO,CAQ9D;AAED,wBAAgB,iBAAiB,CAAC,KAAK,EAAE,KAAK,EAAE,OAAO,EAAE,MAAM,GAAG,KAAK,CAMtE;AAED,wBAAsB,MAAM,CAAC,IAAI,EAAE,MAAM,EAAE,EAAE,EAAE,GAAE,KAAmB,GAAG,OAAO,CAAC,MAAM,CAAC,~~CA6FrF~~"}
1	+ {"version":3,"file":"run-cli.d.ts","sourceRoot":"","sources":["../src/run-cli.ts"],"names":[],"mappings":"AAaA,OAAO,EAAiC,KAAK,KAAK,EAAE,MAAM,YAAY,CAAC;AAMvE,eAAO,MAAM,IAAI,+nEA4ChB,CAAC;AAEF,MAAM,WAAW,KAAK;IACpB,MAAM,EAAE;QAAE,KAAK,CAAC,IAAI,EAAE,MAAM,GAAG,IAAI,CAAA;KAAE,CAAC;IACtC,MAAM,EAAE;QAAE,KAAK,CAAC,IAAI,EAAE,MAAM,GAAG,IAAI,CAAA;KAAE,CAAC;IACtC,GAAG,EAAE,MAAM,CAAC,UAAU,CAAC;IACvB,GAAG,EAAE,MAAM,CAAC;IACZ,KAAK,EAAE,OAAO,CAAC;CAChB;AAkCD,iFAAiF;AACjF,wBAAgB,SAAS,CAAC,EAAE,EAAE,MAAM,EAAE,OAAO,EAAE,MAAM,GAAG,OAAO,CAQ9D;AAED,wBAAgB,iBAAiB,CAAC,KAAK,EAAE,KAAK,EAAE,OAAO,EAAE,MAAM,GAAG,KAAK,CAMtE;AAED;;;;GAIG;AACH,wBAAgB,0BAA0B,CAAC,IAAI,EAAE,MAAM,EAAE,GAAG,MAAM,EAAE,CAiBnE;AAQD,wBAAsB,MAAM,CAAC,IAAI,EAAE,MAAM,EAAE,EAAE,EAAE,GAAE,KAAmB,GAAG,OAAO,CAAC,MAAM,CAAC,CAsIrF"}

package/dist/run-cli.js CHANGED Viewed

@@ -1,14 +1,20 @@
 import { writeFileSync } from "node:fs";
 import { resolve } from "node:path";
 import { parseArgs } from "node:util";
+import { checkBaseline, loadBaseline, writeBaseline } from "./baseline.js";
+import { appendGithubSummary, resolveGithubSummaryPath } from "./github-summary.js";
 import { initSuite } from "./init.js";
 import { formatJUnit } from "./junit.js";
 import { loadSuite } from "./load-suite.js";
 import { compileRegex } from "./matchers.js";
+import { formatHtmlReport } from "./html-report.js";
 import { formatReport } from "./report.js";
 import { DEFAULT_TIMEOUT_MS, runSuite } from "./run.js";
+import { formatGithubSummary } from "./summary-md.js";
 import { UsageError } from "./types.js";
 import { VERSION } from "./version.js";
+/** Sentinel so node:util accepts a bare --github-summary flag. */
+const GITHUB_SUMMARY_AUTO = "__auto__";
 export const HELP = `eval-smoke — one-minute smoke evals for prompts and models
 Usage:
@@ -19,20 +25,24 @@ Options:
   --mock              Offline runner with deterministic fake answers
   --json              Machine-readable JSON report
   --junit <path>      Write JUnit XML report to path (still prints human report unless --json)
+  --html <path>       Write self-contained HTML report to path (offline, no CDN)
   --only <pattern>    Run cases whose id contains pattern, or matches /regex/
   --fail-fast         Stop after the first failing case
   --concurrency <n>   Run up to N cases in parallel (default: 1; ignored with --fail-fast)
   --timeout <ms>      Per-case timeout (default: ${DEFAULT_TIMEOUT_MS})
   --model <name>      Override suite model
   --provider <name>   openai | anthropic | mock
+  --baseline <path>   Fail if any baseline-passing case now fails or is missing
+  --update-baseline <path>  Write current report JSON to path (pretty)
+  --github-summary [path]   Append Markdown to GITHUB_STEP_SUMMARY or path
   --force             Overwrite existing suite.yaml when using init
   --no-color          Disable ANSI colors (also respects NO_COLOR)
   -h, --help          Show this help
   -v, --version       Show version
 Exit codes:
-  0  all cases passed
-  1  one or more cases failed
+  0  all cases passed (and baseline clean when --baseline is set)
+  1  one or more cases failed, or baseline regression
   2  usage or configuration error
 Examples:
@@ -42,8 +52,12 @@ Examples:
   eval-smoke examples/suite.yaml --mock --only greeting
   eval-smoke examples/suite.yaml --mock --only '/^capital/'
   eval-smoke examples/suite.yaml --mock --junit junit.xml
+  eval-smoke examples/suite.yaml --mock --html report.html
   eval-smoke examples/suite.yaml --json --fail-fast
   eval-smoke examples/suite.yaml --mock --concurrency 4
+  eval-smoke examples/suite.yaml --mock --update-baseline baseline.json
+  eval-smoke examples/suite.yaml --mock --baseline baseline.json
+  eval-smoke examples/suite.yaml --mock --github-summary
   OPENAI_API_KEY=sk-... eval-smoke examples/suite.yaml
 `;
 const defaultIo = () => ({
@@ -95,23 +109,58 @@ export function filterSuiteByOnly(suite, pattern) {
     }
     return { ...suite, cases };
 }
+/**
+ * Rewrite bare `--github-summary` into `--github-summary=__auto__` so node:util
+ * parseArgs (string option) accepts it. `--github-summary=path` and
+ * `--github-summary path` stay as-is.
+ */
+export function normalizeGithubSummaryArgv(argv) {
+    const out = [];
+    for (let i = 0; i < argv.length; i += 1) {
+        const token = argv[i];
+        if (token === "--github-summary") {
+            const next = argv[i + 1];
+            if (!next || next.startsWith("-")) {
+                out.push(`--github-summary=${GITHUB_SUMMARY_AUTO}`);
+            }
+            else {
+                out.push("--github-summary", next);
+                i += 1;
+            }
+            continue;
+        }
+        out.push(token);
+    }
+    return out;
+}
+function githubSummaryFlag(raw) {
+    if (raw == null)
+        return undefined;
+    if (raw === GITHUB_SUMMARY_AUTO)
+        return true;
+    return raw;
+}
 export async function runCli(argv, io = defaultIo()) {
     let parsed;
     try {
         parsed = parseArgs({
-            args: argv,
+            args: normalizeGithubSummaryArgv(argv),
             allowPositionals: true,
             strict: true,
             options: {
                 mock: { type: "boolean", default: false },
                 json: { type: "boolean", default: false },
                 junit: { type: "string" },
+                html: { type: "string" },
                 only: { type: "string" },
                 "fail-fast": { type: "boolean", default: false },
                 concurrency: { type: "string" },
                 timeout: { type: "string" },
                 model: { type: "string" },
                 provider: { type: "string" },
+                baseline: { type: "string" },
+                "update-baseline": { type: "string" },
+                "github-summary": { type: "string" },
                 force: { type: "boolean", default: false },
                 "no-color": { type: "boolean", default: false },
                 help: { type: "boolean", short: "h", default: false },
@@ -158,6 +207,10 @@ export async function runCli(argv, io = defaultIo()) {
         const json = asBool(values.json);
         const only = asString(values.only);
         const junitPath = asString(values.junit);
+        const htmlPath = asString(values.html);
+        const baselinePath = asString(values.baseline);
+        const updateBaselinePath = asString(values["update-baseline"]);
+        const githubSummary = githubSummaryFlag(asString(values["github-summary"]));
         let suite = loadSuite(suitePath, io.cwd);
         if (only != null) {
             suite = filterSuiteByOnly(suite, only);
@@ -175,9 +228,40 @@ export async function runCli(argv, io = defaultIo()) {
             const out = resolve(io.cwd, junitPath);
             writeFileSync(out, formatJUnit(report), "utf8");
         }
+        if (htmlPath != null) {
+            const out = resolve(io.cwd, htmlPath);
+            writeFileSync(out, formatHtmlReport(report), "utf8");
+        }
+        if (updateBaselinePath != null) {
+            const out = resolve(io.cwd, updateBaselinePath);
+            writeBaseline(out, report);
+            io.stderr.write(`Wrote baseline ${out}\n`);
+        }
+        let baselineFailed = false;
+        if (baselinePath != null) {
+            const baseline = loadBaseline(resolve(io.cwd, baselinePath));
+            const check = checkBaseline(report, baseline);
+            io.stderr.write(check.report);
+            if (!check.ok)
+                baselineFailed = true;
+        }
+        const outSummary = githubSummary === undefined
+            ? resolveGithubSummaryPath(undefined, io.env, io.cwd)
+            : resolveGithubSummaryPath(githubSummary, io.env, io.cwd);
+        if (outSummary) {
+            try {
+                appendGithubSummary(outSummary, formatGithubSummary(report));
+            }
+            catch (error) {
+                const message = error instanceof Error ? error.message : String(error);
+                io.stderr.write(`could not write github summary: ${message}\n`);
+            }
+        }
         const noColor = asBool(values["no-color"]) || Boolean(io.env.NO_COLOR);
         const color = io.isTTY && !json && !noColor;
         io.stdout.write(formatReport(report, { json, color }));
+        if (baselineFailed)
+            return 1;
         return report.ok ? 0 : 1;
     }
     catch (error) {

package/dist/run-cli.js.map CHANGED Viewed

	@@ -1 +1 @@
1	- {"version":3,"file":"run-cli.js","sourceRoot":"","sources":["../src/run-cli.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,aAAa,EAAE,MAAM,SAAS,CAAC;AACxC,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AACpC,OAAO,EAAE,SAAS,EAAE,MAAM,WAAW,CAAC;AACtC,OAAO,EAAE,SAAS,EAAE,MAAM,WAAW,CAAC;AACtC,OAAO,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AACzC,OAAO,EAAE,SAAS,EAAE,MAAM,iBAAiB,CAAC;AAC5C,OAAO,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC;AAC7C,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,OAAO,EAAE,kBAAkB,EAAE,QAAQ,EAAE,MAAM,UAAU,CAAC;AACxD,OAAO,EAAE,UAAU,EAAiC,MAAM,YAAY,CAAC;AACvE,OAAO,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAEvC,MAAM,CAAC,MAAM,IAAI,GAAG~~;;;;;;;;;;;;;mDAa~~+B,kBAAkB~~;;;;;;;;;;;;;;;;;;;;;;;CAuBpE~~,CAAC;AAUF,MAAM,SAAS,GAAG,GAAU,EAAE,CAAC,CAAC;IAC9B,MAAM,EAAE,OAAO,CAAC,MAAM;IACtB,MAAM,EAAE,OAAO,CAAC,MAAM;IACtB,GAAG,EAAE,OAAO,CAAC,GAAG;IAChB,GAAG,EAAE,OAAO,CAAC,GAAG,EAAE;IAClB,KAAK,EAAE,OAAO,CAAC,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC;CACrC,CAAC,CAAC;AAEH,SAAS,YAAY,CAAC,GAAuB;IAC3C,IAAI,GAAG,IAAI,IAAI;QAAE,OAAO,kBAAkB,CAAC;IAC3C,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC;IAC1B,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,KAAK,IAAI,CAAC,EAAE,CAAC;QAC3C,MAAM,IAAI,UAAU,CAAC,qDAAqD,CAAC,CAAC;IAC9E,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,SAAS,gBAAgB,CAAC,GAAuB;IAC/C,IAAI,GAAG,IAAI,IAAI;QAAE,OAAO,SAAS,CAAC;IAClC,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC;IAC1B,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,KAAK,IAAI,CAAC,EAAE,CAAC;QAC3C,MAAM,IAAI,UAAU,CAAC,0CAA0C,CAAC,CAAC;IACnE,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,SAAS,aAAa,CAAC,GAAuB;IAC5C,IAAI,GAAG,IAAI,IAAI;QAAE,OAAO,SAAS,CAAC;IAClC,IAAI,GAAG,KAAK,MAAM,IAAI,GAAG,KAAK,QAAQ,IAAI,GAAG,KAAK,WAAW;QAAE,OAAO,GAAG,CAAC;IAC1E,MAAM,IAAI,UAAU,CAAC,+CAA+C,CAAC,CAAC;AACxE,CAAC;AAED,iFAAiF;AACjF,MAAM,UAAU,SAAS,CAAC,EAAU,EAAE,OAAe;IACnD,MAAM,OAAO,GAAG,OAAO,CAAC,KAAK,CAAC,4BAA4B,CAAC,CAAC;IAC5D,IAAI,OAAO,EAAE,CAAC,CAAC,CAAC,IAAI,IAAI,EAAE,CAAC;QACzB,MAAM,EAAE,GAAG,YAAY,CAAC,OAAO,CAAC,CAAC;QACjC,EAAE,CAAC,SAAS,GAAG,CAAC,CAAC;QACjB,OAAO,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACrB,CAAC;IACD,OAAO,EAAE,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC;AAC9B,CAAC;AAED,MAAM,UAAU,iBAAiB,CAAC,KAAY,EAAE,OAAe;IAC7D,MAAM,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,QAAQ,EAAE,EAAE,CAAC,SAAS,CAAC,QAAQ,CAAC,EAAE,EAAE,OAAO,CAAC,CAAC,CAAC;IAChF,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACvB,MAAM,IAAI,UAAU,CAAC,UAAU,IAAI,CAAC,SAAS,CAAC,OAAO,CAAC,kBAAkB,CAAC,CAAC;IAC5E,CAAC;IACD,OAAO,EAAE,GAAG,KAAK,EAAE,KAAK,EAAE,CAAC;AAC7B,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,MAAM,CAAC,IAAc,EAAE,KAAY,SAAS,EAAE;IAClE,IAAI,MAAoC,CAAC;IACzC,IAAI,CAAC;QACH,MAAM,GAAG,SAAS,CAAC;YACjB,IAAI,EAAE,IAAI;~~YACV~~,gBAAgB,EAAE,IAAI;YACtB,MAAM,EAAE,IAAI;YACZ,OAAO,EAAE;gBACP,IAAI,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBACzC,IAAI,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBACzC,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACzB,IAAI,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACxB,WAAW,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBAChD,WAAW,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBAC/B,OAAO,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBAC3B,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACzB,QAAQ,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBAC5B,KAAK,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBAC1C,UAAU,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBAC/C,IAAI,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,KAAK,EAAE,GAAG,EAAE,OAAO,EAAE,KAAK,EAAE;gBACrD,OAAO,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,KAAK,EAAE,GAAG,EAAE,OAAO,EAAE,KAAK,EAAE;aACzD;SACF,CAAC,CAAC;IACL,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QAC/E,OAAO,CAAC,CAAC;IACX,CAAC;IAED,MAAM,EAAE,MAAM,EAAE,WAAW,EAAE,GAAG,MAAM,CAAC;IACvC,MAAM,QAAQ,GAAG,CAAC,KAAc,EAAsB,EAAE,CACtD,OAAO,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC;IAChD,MAAM,MAAM,GAAG,CAAC,KAAc,EAAW,EAAE,CAAC,KAAK,KAAK,IAAI,CAAC;IAE3D,IAAI,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC;QACxB,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QACtB,OAAO,CAAC,CAAC;IACX,CAAC;IACD,IAAI,MAAM,CAAC,MAAM,CAAC,OAAO,CAAC,EAAE,CAAC;QAC3B,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,IAAI,CAAC,CAAC;QAChC,OAAO,CAAC,CAAC;IACX,CAAC;IAED,IAAI,WAAW,CAAC,CAAC,CAAC,KAAK,MAAM,EAAE,CAAC;QAC9B,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC;YAC9B,MAAM,MAAM,GAAG,SAAS,CAAC,MAAM,EAAE,EAAE,KAAK,EAAE,MAAM,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,GAAG,EAAE,EAAE,CAAC,GAAG,EAAE,CAAC,CAAC;YAC/E,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,WAAW,MAAM,CAAC,IAAI,IAAI,CAAC,CAAC;YAC5C,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,oBAAoB,MAAM,CAAC,IAAI,WAAW,CAAC,CAAC;YAC5D,OAAO,CAAC,CAAC;QACX,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,MAAM,OAAO,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;YACvE,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,IAAI,CAAC,CAAC;YAChC,OAAO,KAAK,YAAY,UAAU,CAAC,CAAC,CAAC,KAAK,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC;QAC1D,CAAC;IACH,CAAC;IAED,MAAM,SAAS,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC;IACjC,IAAI,CAAC,SAAS,EAAE,CAAC;QACf,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,yBAAyB,CAAC,CAAC;QAC3C,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QACtB,OAAO,CAAC,CAAC;IACX,CAAC;IAED,IAAI,CAAC;QACH,MAAM,IAAI,GAAG,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QACjC,MAAM,IAAI,GAAG,QAAQ,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QACnC,MAAM,SAAS,GAAG,QAAQ,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QACzC,IAAI,KAAK,GAAG,SAAS,CAAC,SAAS,EAAE,EAAE,CAAC,GAAG,CAAC,CAAC;QACzC,IAAI,IAAI,IAAI,IAAI,EAAE,CAAC;YACjB,KAAK,GAAG,iBAAiB,CAAC,KAAK,EAAE,IAAI,CAAC,CAAC;QACzC,CAAC;QACD,MAAM,MAAM,GAAG,MAAM,QAAQ,CAAC,KAAK,EAAE;YACnC,IAAI,EAAE,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC;YACzB,QAAQ,EAAE,aAAa,CAAC,QAAQ,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;YAClD,KAAK,EAAE,QAAQ,CAAC,MAAM,CAAC,KAAK,CAAC;YAC7B,QAAQ,EAAE,MAAM,CAAC,MAAM,CAAC,WAAW,CAAC,CAAC;YACrC,WAAW,EAAE,gBAAgB,CAAC,QAAQ,CAAC,MAAM,CAAC,WAAW,CAAC,CAAC;YAC3D,SAAS,EAAE,YAAY,CAAC,QAAQ,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;YACjD,GAAG,EAAE,EAAE,CAAC,GAAG;SACZ,CAAC,CAAC;QACH,IAAI,SAAS,IAAI,IAAI,EAAE,CAAC;YACtB,MAAM,GAAG,GAAG,OAAO,CAAC,EAAE,CAAC,GAAG,EAAE,SAAS,CAAC,CAAC;YACvC,aAAa,CAAC,GAAG,EAAE,WAAW,CAAC,MAAM,CAAC,EAAE,MAAM,CAAC,CAAC;QAClD,CAAC;QACD,MAAM,OAAO,GAAG,MAAM,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC,IAAI,OAAO,CAAC,EAAE,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;QACvE,MAAM,KAAK,GAAG,EAAE,CAAC,KAAK,IAAI,CAAC,IAAI,IAAI,CAAC,OAAO,CAAC;QAC5C,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,YAAY,CAAC,MAAM,EAAE,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC;QACvD,OAAO,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IAC3B,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,OAAO,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QACvE,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,IAAI,CAAC,CAAC;QAChC,OAAO,KAAK,YAAY,UAAU,CAAC,CAAC,CAAC,KAAK,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC;IAC1D,CAAC;AACH,CAAC"}
1	+ {"version":3,"file":"run-cli.js","sourceRoot":"","sources":["../src/run-cli.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,aAAa,EAAE,MAAM,SAAS,CAAC;AACxC,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AACpC,OAAO,EAAE,SAAS,EAAE,MAAM,WAAW,CAAC;AACtC,OAAO,EAAE,aAAa,EAAE,YAAY,EAAE,aAAa,EAAE,MAAM,eAAe,CAAC;AAC3E,OAAO,EAAE,mBAAmB,EAAE,wBAAwB,EAAE,MAAM,qBAAqB,CAAC;AACpF,OAAO,EAAE,SAAS,EAAE,MAAM,WAAW,CAAC;AACtC,OAAO,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AACzC,OAAO,EAAE,SAAS,EAAE,MAAM,iBAAiB,CAAC;AAC5C,OAAO,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC;AAC7C,OAAO,EAAE,gBAAgB,EAAE,MAAM,kBAAkB,CAAC;AACpD,OAAO,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AAC3C,OAAO,EAAE,kBAAkB,EAAE,QAAQ,EAAE,MAAM,UAAU,CAAC;AACxD,OAAO,EAAE,mBAAmB,EAAE,MAAM,iBAAiB,CAAC;AACtD,OAAO,EAAE,UAAU,EAAiC,MAAM,YAAY,CAAC;AACvE,OAAO,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAEvC,kEAAkE;AAClE,MAAM,mBAAmB,GAAG,UAAU,CAAC;AAEvC,MAAM,CAAC,MAAM,IAAI,GAAG;;;;;;;;;;;;;;mDAc+B,kBAAkB;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;CA8BpE,CAAC;AAUF,MAAM,SAAS,GAAG,GAAU,EAAE,CAAC,CAAC;IAC9B,MAAM,EAAE,OAAO,CAAC,MAAM;IACtB,MAAM,EAAE,OAAO,CAAC,MAAM;IACtB,GAAG,EAAE,OAAO,CAAC,GAAG;IAChB,GAAG,EAAE,OAAO,CAAC,GAAG,EAAE;IAClB,KAAK,EAAE,OAAO,CAAC,OAAO,CAAC,MAAM,CAAC,KAAK,CAAC;CACrC,CAAC,CAAC;AAEH,SAAS,YAAY,CAAC,GAAuB;IAC3C,IAAI,GAAG,IAAI,IAAI;QAAE,OAAO,kBAAkB,CAAC;IAC3C,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC;IAC1B,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,KAAK,IAAI,CAAC,EAAE,CAAC;QAC3C,MAAM,IAAI,UAAU,CAAC,qDAAqD,CAAC,CAAC;IAC9E,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,SAAS,gBAAgB,CAAC,GAAuB;IAC/C,IAAI,GAAG,IAAI,IAAI;QAAE,OAAO,SAAS,CAAC;IAClC,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC;IAC1B,IAAI,CAAC,MAAM,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,KAAK,IAAI,CAAC,EAAE,CAAC;QAC3C,MAAM,IAAI,UAAU,CAAC,0CAA0C,CAAC,CAAC;IACnE,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,SAAS,aAAa,CAAC,GAAuB;IAC5C,IAAI,GAAG,IAAI,IAAI;QAAE,OAAO,SAAS,CAAC;IAClC,IAAI,GAAG,KAAK,MAAM,IAAI,GAAG,KAAK,QAAQ,IAAI,GAAG,KAAK,WAAW;QAAE,OAAO,GAAG,CAAC;IAC1E,MAAM,IAAI,UAAU,CAAC,+CAA+C,CAAC,CAAC;AACxE,CAAC;AAED,iFAAiF;AACjF,MAAM,UAAU,SAAS,CAAC,EAAU,EAAE,OAAe;IACnD,MAAM,OAAO,GAAG,OAAO,CAAC,KAAK,CAAC,4BAA4B,CAAC,CAAC;IAC5D,IAAI,OAAO,EAAE,CAAC,CAAC,CAAC,IAAI,IAAI,EAAE,CAAC;QACzB,MAAM,EAAE,GAAG,YAAY,CAAC,OAAO,CAAC,CAAC;QACjC,EAAE,CAAC,SAAS,GAAG,CAAC,CAAC;QACjB,OAAO,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACrB,CAAC;IACD,OAAO,EAAE,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC;AAC9B,CAAC;AAED,MAAM,UAAU,iBAAiB,CAAC,KAAY,EAAE,OAAe;IAC7D,MAAM,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,QAAQ,EAAE,EAAE,CAAC,SAAS,CAAC,QAAQ,CAAC,EAAE,EAAE,OAAO,CAAC,CAAC,CAAC;IAChF,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QACvB,MAAM,IAAI,UAAU,CAAC,UAAU,IAAI,CAAC,SAAS,CAAC,OAAO,CAAC,kBAAkB,CAAC,CAAC;IAC5E,CAAC;IACD,OAAO,EAAE,GAAG,KAAK,EAAE,KAAK,EAAE,CAAC;AAC7B,CAAC;AAED;;;;GAIG;AACH,MAAM,UAAU,0BAA0B,CAAC,IAAc;IACvD,MAAM,GAAG,GAAa,EAAE,CAAC;IACzB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,IAAI,CAAC,MAAM,EAAE,CAAC,IAAI,CAAC,EAAE,CAAC;QACxC,MAAM,KAAK,GAAG,IAAI,CAAC,CAAC,CAAE,CAAC;QACvB,IAAI,KAAK,KAAK,kBAAkB,EAAE,CAAC;YACjC,MAAM,IAAI,GAAG,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;YACzB,IAAI,CAAC,IAAI,IAAI,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE,CAAC;gBAClC,GAAG,CAAC,IAAI,CAAC,oBAAoB,mBAAmB,EAAE,CAAC,CAAC;YACtD,CAAC;iBAAM,CAAC;gBACN,GAAG,CAAC,IAAI,CAAC,kBAAkB,EAAE,IAAI,CAAC,CAAC;gBACnC,CAAC,IAAI,CAAC,CAAC;YACT,CAAC;YACD,SAAS;QACX,CAAC;QACD,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;IAClB,CAAC;IACD,OAAO,GAAG,CAAC;AACb,CAAC;AAED,SAAS,iBAAiB,CAAC,GAAuB;IAChD,IAAI,GAAG,IAAI,IAAI;QAAE,OAAO,SAAS,CAAC;IAClC,IAAI,GAAG,KAAK,mBAAmB;QAAE,OAAO,IAAI,CAAC;IAC7C,OAAO,GAAG,CAAC;AACb,CAAC;AAED,MAAM,CAAC,KAAK,UAAU,MAAM,CAAC,IAAc,EAAE,KAAY,SAAS,EAAE;IAClE,IAAI,MAAoC,CAAC;IACzC,IAAI,CAAC;QACH,MAAM,GAAG,SAAS,CAAC;YACjB,IAAI,EAAE,0BAA0B,CAAC,IAAI,CAAC;YACtC,gBAAgB,EAAE,IAAI;YACtB,MAAM,EAAE,IAAI;YACZ,OAAO,EAAE;gBACP,IAAI,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBACzC,IAAI,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBACzC,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACzB,IAAI,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACxB,IAAI,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACxB,WAAW,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBAChD,WAAW,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBAC/B,OAAO,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBAC3B,KAAK,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACzB,QAAQ,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBAC5B,QAAQ,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBAC5B,iBAAiB,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACrC,gBAAgB,EAAE,EAAE,IAAI,EAAE,QAAQ,EAAE;gBACpC,KAAK,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBAC1C,UAAU,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,OAAO,EAAE,KAAK,EAAE;gBAC/C,IAAI,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,KAAK,EAAE,GAAG,EAAE,OAAO,EAAE,KAAK,EAAE;gBACrD,OAAO,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,KAAK,EAAE,GAAG,EAAE,OAAO,EAAE,KAAK,EAAE;aACzD;SACF,CAAC,CAAC;IACL,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QAC/E,OAAO,CAAC,CAAC;IACX,CAAC;IAED,MAAM,EAAE,MAAM,EAAE,WAAW,EAAE,GAAG,MAAM,CAAC;IACvC,MAAM,QAAQ,GAAG,CAAC,KAAc,EAAsB,EAAE,CACtD,OAAO,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC;IAChD,MAAM,MAAM,GAAG,CAAC,KAAc,EAAW,EAAE,CAAC,KAAK,KAAK,IAAI,CAAC;IAE3D,IAAI,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC;QACxB,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QACtB,OAAO,CAAC,CAAC;IACX,CAAC;IACD,IAAI,MAAM,CAAC,MAAM,CAAC,OAAO,CAAC,EAAE,CAAC;QAC3B,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,IAAI,CAAC,CAAC;QAChC,OAAO,CAAC,CAAC;IACX,CAAC;IAED,IAAI,WAAW,CAAC,CAAC,CAAC,KAAK,MAAM,EAAE,CAAC;QAC9B,IAAI,CAAC;YACH,MAAM,MAAM,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC;YAC9B,MAAM,MAAM,GAAG,SAAS,CAAC,MAAM,EAAE,EAAE,KAAK,EAAE,MAAM,CAAC,MAAM,CAAC,KAAK,CAAC,EAAE,GAAG,EAAE,EAAE,CAAC,GAAG,EAAE,CAAC,CAAC;YAC/E,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,WAAW,MAAM,CAAC,IAAI,IAAI,CAAC,CAAC;YAC5C,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,oBAAoB,MAAM,CAAC,IAAI,WAAW,CAAC,CAAC;YAC5D,OAAO,CAAC,CAAC;QACX,CAAC;QAAC,OAAO,KAAK,EAAE,CAAC;YACf,MAAM,OAAO,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;YACvE,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,IAAI,CAAC,CAAC;YAChC,OAAO,KAAK,YAAY,UAAU,CAAC,CAAC,CAAC,KAAK,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC;QAC1D,CAAC;IACH,CAAC;IAED,MAAM,SAAS,GAAG,WAAW,CAAC,CAAC,CAAC,CAAC;IACjC,IAAI,CAAC,SAAS,EAAE,CAAC;QACf,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,yBAAyB,CAAC,CAAC;QAC3C,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;QACtB,OAAO,CAAC,CAAC;IACX,CAAC;IAED,IAAI,CAAC;QACH,MAAM,IAAI,GAAG,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QACjC,MAAM,IAAI,GAAG,QAAQ,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QACnC,MAAM,SAAS,GAAG,QAAQ,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QACzC,MAAM,QAAQ,GAAG,QAAQ,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;QACvC,MAAM,YAAY,GAAG,QAAQ,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;QAC/C,MAAM,kBAAkB,GAAG,QAAQ,CAAC,MAAM,CAAC,iBAAiB,CAAC,CAAC,CAAC;QAC/D,MAAM,aAAa,GAAG,iBAAiB,CAAC,QAAQ,CAAC,MAAM,CAAC,gBAAgB,CAAC,CAAC,CAAC,CAAC;QAE5E,IAAI,KAAK,GAAG,SAAS,CAAC,SAAS,EAAE,EAAE,CAAC,GAAG,CAAC,CAAC;QACzC,IAAI,IAAI,IAAI,IAAI,EAAE,CAAC;YACjB,KAAK,GAAG,iBAAiB,CAAC,KAAK,EAAE,IAAI,CAAC,CAAC;QACzC,CAAC;QACD,MAAM,MAAM,GAAG,MAAM,QAAQ,CAAC,KAAK,EAAE;YACnC,IAAI,EAAE,MAAM,CAAC,MAAM,CAAC,IAAI,CAAC;YACzB,QAAQ,EAAE,aAAa,CAAC,QAAQ,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;YAClD,KAAK,EAAE,QAAQ,CAAC,MAAM,CAAC,KAAK,CAAC;YAC7B,QAAQ,EAAE,MAAM,CAAC,MAAM,CAAC,WAAW,CAAC,CAAC;YACrC,WAAW,EAAE,gBAAgB,CAAC,QAAQ,CAAC,MAAM,CAAC,WAAW,CAAC,CAAC;YAC3D,SAAS,EAAE,YAAY,CAAC,QAAQ,CAAC,MAAM,CAAC,OAAO,CAAC,CAAC;YACjD,GAAG,EAAE,EAAE,CAAC,GAAG;SACZ,CAAC,CAAC;QACH,IAAI,SAAS,IAAI,IAAI,EAAE,CAAC;YACtB,MAAM,GAAG,GAAG,OAAO,CAAC,EAAE,CAAC,GAAG,EAAE,SAAS,CAAC,CAAC;YACvC,aAAa,CAAC,GAAG,EAAE,WAAW,CAAC,MAAM,CAAC,EAAE,MAAM,CAAC,CAAC;QAClD,CAAC;QACD,IAAI,QAAQ,IAAI,IAAI,EAAE,CAAC;YACrB,MAAM,GAAG,GAAG,OAAO,CAAC,EAAE,CAAC,GAAG,EAAE,QAAQ,CAAC,CAAC;YACtC,aAAa,CAAC,GAAG,EAAE,gBAAgB,CAAC,MAAM,CAAC,EAAE,MAAM,CAAC,CAAC;QACvD,CAAC;QACD,IAAI,kBAAkB,IAAI,IAAI,EAAE,CAAC;YAC/B,MAAM,GAAG,GAAG,OAAO,CAAC,EAAE,CAAC,GAAG,EAAE,kBAAkB,CAAC,CAAC;YAChD,aAAa,CAAC,GAAG,EAAE,MAAM,CAAC,CAAC;YAC3B,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,kBAAkB,GAAG,IAAI,CAAC,CAAC;QAC7C,CAAC;QAED,IAAI,cAAc,GAAG,KAAK,CAAC;QAC3B,IAAI,YAAY,IAAI,IAAI,EAAE,CAAC;YACzB,MAAM,QAAQ,GAAG,YAAY,CAAC,OAAO,CAAC,EAAE,CAAC,GAAG,EAAE,YAAY,CAAC,CAAC,CAAC;YAC7D,MAAM,KAAK,GAAG,aAAa,CAAC,MAAM,EAAE,QAAQ,CAAC,CAAC;YAC9C,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC;YAC9B,IAAI,CAAC,KAAK,CAAC,EAAE;gBAAE,cAAc,GAAG,IAAI,CAAC;QACvC,CAAC;QAED,MAAM,UAAU,GACd,aAAa,KAAK,SAAS;YACzB,CAAC,CAAC,wBAAwB,CAAC,SAAS,EAAE,EAAE,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,CAAC;YACrD,CAAC,CAAC,wBAAwB,CAAC,aAAa,EAAE,EAAE,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,CAAC,CAAC;QAC9D,IAAI,UAAU,EAAE,CAAC;YACf,IAAI,CAAC;gBACH,mBAAmB,CAAC,UAAU,EAAE,mBAAmB,CAAC,MAAM,CAAC,CAAC,CAAC;YAC/D,CAAC;YAAC,OAAO,KAAK,EAAE,CAAC;gBACf,MAAM,OAAO,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;gBACvE,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,mCAAmC,OAAO,IAAI,CAAC,CAAC;YAClE,CAAC;QACH,CAAC;QAED,MAAM,OAAO,GAAG,MAAM,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC,IAAI,OAAO,CAAC,EAAE,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;QACvE,MAAM,KAAK,GAAG,EAAE,CAAC,KAAK,IAAI,CAAC,IAAI,IAAI,CAAC,OAAO,CAAC;QAC5C,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,YAAY,CAAC,MAAM,EAAE,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,CAAC,CAAC;QACvD,IAAI,cAAc;YAAE,OAAO,CAAC,CAAC;QAC7B,OAAO,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IAC3B,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,MAAM,OAAO,GAAG,KAAK,YAAY,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;QACvE,EAAE,CAAC,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,IAAI,CAAC,CAAC;QAChC,OAAO,KAAK,YAAY,UAAU,CAAC,CAAC,CAAC,KAAK,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,CAAC;IAC1D,CAAC;AACH,CAAC"}

package/dist/summary-md.d.ts ADDED Viewed

@@ -0,0 +1,4 @@
+import type { Report } from "./types.js";
+/** Concise pass/fail Markdown table for GitHub Actions step summary. */
+export declare function formatGithubSummary(report: Report): string;
+//# sourceMappingURL=summary-md.d.ts.map

package/dist/summary-md.d.ts.map ADDED Viewed

	@@ -0,0 +1 @@
1	+ {"version":3,"file":"summary-md.d.ts","sourceRoot":"","sources":["../src/summary-md.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,YAAY,CAAC;AAEzC,wEAAwE;AACxE,wBAAgB,mBAAmB,CAAC,MAAM,EAAE,MAAM,GAAG,MAAM,CAoB1D"}

package/dist/summary-md.js ADDED Viewed

@@ -0,0 +1,22 @@
+/** Concise pass/fail Markdown table for GitHub Actions step summary. */
+export function formatGithubSummary(report) {
+    const lines = [];
+    lines.push(`### eval-smoke · ${escape(report.suite ?? "suite")}`);
+    lines.push("");
+    lines.push(`**${report.passed} passed · ${report.failed} failed${report.skipped ? ` · ${report.skipped} skipped` : ""} · ${report.duration_ms}ms** · \`${report.runner}\` / \`${escape(report.model)}\``);
+    lines.push("");
+    lines.push("| Case | Result | ms |");
+    lines.push("| --- | --- | ---: |");
+    for (const c of report.cases) {
+        const status = c.skipped ? "skip" : c.ok ? "✅ pass" : "❌ fail";
+        lines.push(`| \`${escape(c.id)}\` | ${status} | ${c.duration_ms} |`);
+    }
+    lines.push("");
+    lines.push(report.ok ? "**Suite: PASS**" : "**Suite: FAIL**");
+    lines.push("");
+    return lines.join("\n");
+}
+function escape(text) {
+    return text.replace(/\|/g, "\\|").replace(/\r?\n/g, " ");
+}
+//# sourceMappingURL=summary-md.js.map

package/dist/summary-md.js.map ADDED Viewed

@@ -0,0 +1 @@

+ {"version":3,"file":"summary-md.js","sourceRoot":"","sources":["../src/summary-md.ts"],"names":[],"mappings":"AAEA,wEAAwE;AACxE,MAAM,UAAU,mBAAmB,CAAC,MAAc;IAChD,MAAM,KAAK,GAAa,EAAE,CAAC;IAC3B,KAAK,CAAC,IAAI,CAAC,oBAAoB,MAAM,CAAC,MAAM,CAAC,KAAK,IAAI,OAAO,CAAC,EAAE,CAAC,CAAC;IAClE,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACf,KAAK,CAAC,IAAI,CACR,KAAK,MAAM,CAAC,MAAM,aAAa,MAAM,CAAC,MAAM,UAC1C,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,MAAM,CAAC,OAAO,UAAU,CAAC,CAAC,CAAC,EACpD,MAAM,MAAM,CAAC,WAAW,YAAY,MAAM,CAAC,MAAM,UAAU,MAAM,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CACpF,CAAC;IACF,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACf,KAAK,CAAC,IAAI,CAAC,wBAAwB,CAAC,CAAC;IACrC,KAAK,CAAC,IAAI,CAAC,sBAAsB,CAAC,CAAC;IACnC,KAAK,MAAM,CAAC,IAAI,MAAM,CAAC,KAAK,EAAE,CAAC;QAC7B,MAAM,MAAM,GAAG,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC;QAC/D,KAAK,CAAC,IAAI,CAAC,OAAO,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,QAAQ,MAAM,MAAM,CAAC,CAAC,WAAW,IAAI,CAAC,CAAC;IACvE,CAAC;IACD,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACf,KAAK,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC,iBAAiB,CAAC,CAAC,CAAC,iBAAiB,CAAC,CAAC;IAC9D,KAAK,CAAC,IAAI,CAAC,EAAE,CAAC,CAAC;IACf,OAAO,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AAC1B,CAAC;AAED,SAAS,MAAM,CAAC,IAAY;IAC1B,OAAO,IAAI,CAAC,OAAO,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC,OAAO,CAAC,QAAQ,EAAE,GAAG,CAAC,CAAC;AAC3D,CAAC"}

package/examples/github-action.yml ADDED Viewed

@@ -0,0 +1,41 @@
+# Copy-paste: run eval-smoke in GitHub Actions (mock offline + JUnit + step summary).
+# No clone of this repo required — npx pulls @webywz/eval-smoke from npm.
+#
+# Human report → stdout (job log). JUnit → file. Markdown → $GITHUB_STEP_SUMMARY.
+name: Smoke eval with eval-smoke
+on:
+  workflow_dispatch:
+  # Or: push / pull_request when you have a suite.yaml in the repo.
+jobs:
+  eval:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 22
+      # Offline smoke against the published example suite (no API key):
+      - name: Mock smoke + JUnit + step summary
+        run: |
+          npx --yes @webywz/eval-smoke --help >/dev/null
+          # Prefer a suite checked into YOUR repo; this uses the package example via init:
+          npx --yes @webywz/eval-smoke init ./eval-smoke-ci --force
+          npx --yes @webywz/eval-smoke ./eval-smoke-ci/suite.yaml \
+            --mock \
+            --junit junit-eval-smoke.xml \
+            --github-summary \
+            --update-baseline eval-smoke-baseline.json
+      - name: Upload JUnit
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: eval-smoke-junit
+          path: |
+            junit-eval-smoke.xml
+            eval-smoke-baseline.json

package/package.json CHANGED Viewed

@@ -1,6 +1,6 @@
 {
   "name": "@webywz/eval-smoke",
-  "version": "0.3.1",
+  "version": "0.5.0",
   "description": "One-minute smoke evals for prompts and models — equals/json matchers, diffs, concurrency, JUnit, init.",
   "type": "module",
   "bin": {