agent-inspect 4.4.0 → 5.1.0

package/CHANGELOG.md

@@ -1,5 +1,17 @@
 # Changelog
 
+## 5.1.0
+
+### Minor Changes
+
+- 44e9684: Add cohort analysis v2: baseline/candidate comparison, grouping, metrics, and `agent-inspect cohort` CLI with JSON/Markdown/HTML reports.
+
+## 5.0.0
+
+### Minor Changes
+
+- c5e3b16: Add trace suite config (`agent-inspect.suite.json`) with `suite init`, `validate`, `list`, `run`, and `report` commands for local CI trajectory checks.
+
 ## 4.4.0
 
 ### Minor Changes

package/docs/CLI.md

@@ -896,6 +896,73 @@ npx agent-inspect bundle --since 24h --dir ./.agent-inspect --json
 
 Recipe: [shareable-bundle-basic](../examples/recipes/shareable-bundle-basic/README.md).
 
+### 6.25 `suite`
+
+Define and run **local trace suites** for CI trajectory checks (v5.0+). Suites read existing traces only — no agent replay, no model calls, no upload.
+
+```bash
+agent-inspect suite init
+agent-inspect suite validate [--config path]
+agent-inspect suite list [--config path]
+agent-inspect suite run [--config path] [--json] [--markdown] [-o dir]
+agent-inspect suite report --input <suite-run.json> [--format markdown|json]
+```
+
+Default config file: `agent-inspect.suite.json` in the current directory.
+
+Options (shared):
+
+- `--config <path>` — suite config (`.json`, `.js`, `.mjs`, `.cjs`)
+
+`run` also supports:
+
+- `-o, --output <dir>` — write JSON run artifact (default: `.agent-inspect/suite-runs`)
+- `--json` / `--markdown` — structured or readable output
+
+Case fields include `requireTools`, `forbidTools`, `maxDurationMs`, and `expectedObservations` (outcome names that must be `passed`).
+
+Example:
+
+```bash
+npx agent-inspect suite init
+npx agent-inspect suite run --config fixtures/configs/outcome-suite.suite.json --json
+```
+
+Recipe: [trace-suite-basic](../examples/recipes/trace-suite-basic/README.md).
+
+### 6.26 `cohort`
+
+Compare **baseline vs candidate** trace cohorts for regressions (v5.1+). Reads local traces only — no agent replay, no model calls, no upload.
+
+```bash
+agent-inspect cohort --dir <path> --baseline <label> --candidate <label> [options]
+```
+
+Options:
+
+- `--dir <path>` — trace directory (default: `.agent-inspect/traces`)
+- `--baseline <label>` / `--candidate <label>` — cohort labels from `run_started.metadata`
+- `--cohort-key <key>` — metadata key for labels (default: `cohort`)
+- `--group-by <spec>` — `model`, `session`, `group`, or `metadata.<key>` (default: `model`)
+- `--metric <list>` — comma-separated metrics (`errorRate`, `duration`, `toolChoice`, `observationFailure`, …)
+- `--format <format>` — `markdown`, `json`, or `html` (default: `markdown`)
+- `-o, --output <dir>` — write `cohort-results.json`, `cohort-summary.md`, `cohort-report.html`
+- `--json` — print deterministic JSON result
+
+Exit code **1** when any comparison flags a regression.
+
+Example:
+
+```bash
+npx agent-inspect cohort \
+  --dir fixtures/cohorts/before-after \
+  --baseline before \
+  --candidate after \
+  --group-by model
+```
+
+Recipe: [cohort-baseline-candidate](../examples/recipes/cohort-baseline-candidate/README.md).
+
 ## 7. Optional TUI behavior
 
 `view --tui` delegates to `@agent-inspect/tui` and requires an interactive terminal. If the package is not installed, the CLI prints a short install hint.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-inspect",
-  "version": "4.4.0",
+  "version": "5.1.0",
   "license": "MIT",
   "type": "module",
   "description": "Trace, check, and safely share TypeScript AI-agent runs locally — no account, no upload, metadata-only by default",