agent-inspect 5.0.0 → 5.2.0

package/CHANGELOG.md

@@ -1,5 +1,17 @@
 # Changelog
 
+## 5.2.0
+
+### Minor Changes
+
+- 52c2539: Add CI quality gates: `agent-inspect gate` with suite/threshold checks, stable exit codes, and JUnit/GitHub step-summary artifacts.
+
+## 5.1.0
+
+### Minor Changes
+
+- 44e9684: Add cohort analysis v2: baseline/candidate comparison, grouping, metrics, and `agent-inspect cohort` CLI with JSON/Markdown/HTML reports.
+
 ## 5.0.0
 
 ### Minor Changes

package/docs/CLI.md

@@ -930,6 +930,71 @@ npx agent-inspect suite run --config fixtures/configs/outcome-suite.suite.json -
 
 Recipe: [trace-suite-basic](../examples/recipes/trace-suite-basic/README.md).
 
+### 6.26 `cohort`
+
+Compare **baseline vs candidate** trace cohorts for regressions (v5.1+). Reads local traces only — no agent replay, no model calls, no upload.
+
+```bash
+agent-inspect cohort --dir <path> --baseline <label> --candidate <label> [options]
+```
+
+Options:
+
+- `--dir <path>` — trace directory (default: `.agent-inspect/traces`)
+- `--baseline <label>` / `--candidate <label>` — cohort labels from `run_started.metadata`
+- `--cohort-key <key>` — metadata key for labels (default: `cohort`)
+- `--group-by <spec>` — `model`, `session`, `group`, or `metadata.<key>` (default: `model`)
+- `--metric <list>` — comma-separated metrics (`errorRate`, `duration`, `toolChoice`, `observationFailure`, …)
+- `--format <format>` — `markdown`, `json`, or `html` (default: `markdown`)
+- `-o, --output <dir>` — write `cohort-results.json`, `cohort-summary.md`, `cohort-report.html`
+- `--json` — print deterministic JSON result
+
+Exit code **1** when any comparison flags a regression.
+
+Example:
+
+```bash
+npx agent-inspect cohort \
+  --dir fixtures/cohorts/before-after \
+  --baseline before \
+  --candidate after \
+  --group-by model
+```
+
+Recipe: [cohort-baseline-candidate](../examples/recipes/cohort-baseline-candidate/README.md).
+
+### 6.27 `gate`
+
+Run **deterministic CI quality gates** over local traces or suite configs (v5.2+). No agent replay, no model calls, no upload.
+
+```bash
+agent-inspect gate --suite <path> [options]
+agent-inspect gate --dir <path> --max-error-rate <percent> [options]
+```
+
+Options:
+
+- `--dir <path>` — trace directory for threshold checks
+- `--suite <path>` — suite config (`.json`, `.js`, `.mjs`, `.cjs`)
+- `--max-error-rate <percent>` — maximum allowed error rate
+- `--max-p95-duration <ms>` — maximum allowed p95 run duration
+- `--forbid-tool <name>` — forbidden tool (repeatable or comma-separated)
+- `--require-observation <name>` — required passed observation (repeatable or comma-separated)
+- `--format <format>` — `markdown`, `json`, `html`, `junit`, or `github` (default: `markdown`)
+- `-o, --output <dir>` — write `gate-results.json`, `gate-summary.md`, `gate-report.html`, `junit.xml`, `github-step-summary.md`
+- `--json` — print deterministic JSON result
+
+Exit codes: **0** pass, **1** gate failed, **2** invalid config, **3** trace read failure, **4** unsupported format.
+
+Example:
+
+```bash
+npx agent-inspect gate --suite fixtures/configs/outcome-suite.suite.json --output ./gate-artifacts
+npx agent-inspect gate --dir fixtures/cohorts/before-after --max-error-rate 5 --forbid-tool deleteAccount
+```
+
+Recipe: [github-actions-gate](../examples/recipes/github-actions-gate/README.md).
+
 ## 7. Optional TUI behavior
 
 `view --tui` delegates to `@agent-inspect/tui` and requires an interactive terminal. If the package is not installed, the CLI prints a short install hint.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-inspect",
-  "version": "5.0.0",
+  "version": "5.2.0",
   "license": "MIT",
   "type": "module",
   "description": "Trace, check, and safely share TypeScript AI-agent runs locally — no account, no upload, metadata-only by default",