agent-inspect 2.1.0 → 2.2.0

package/CHANGELOG.md

@@ -1,5 +1,21 @@
 # Changelog
 
+## 2.2.0
+
+### Minor Changes
+
+- efb3fef: Release v2.2.0 with local test reporter artifacts and CI summaries.
+
+  Adds the public optional `@agent-inspect/vitest` and `@agent-inspect/jest` reporter packages, the shared experimental `agent-inspect/reporters` helpers, and the `agent-inspect ci-summary` workflow for deterministic local reporter manifests and CI artifacts.
+
+## Unreleased
+
+### Draft v2.2.0 Notes
+
+- Prepared the v2.2 reporter and CI workflow release notes for local Vitest/Jest reporter artifacts, the shared `agent-inspect/reporters` manifest helpers, and `agent-inspect ci-summary`.
+- Reporter artifacts remain local-only and metadata-bounded. `ci-summary` reads reporter manifest JSON only, validates relative artifact paths, and does not read trace contents, upload artifacts, call GitHub APIs, or mutate repository state.
+- `@agent-inspect/vitest` and `@agent-inspect/jest` remain private/unpublished until maintainer first-publication setup is explicitly cleared before release prep.
+
 ## 2.1.0
 
 ### Minor Changes

package/README.md

@@ -22,7 +22,7 @@ agent-inspect gives those runs **structure**: an **execution tree** you can read
 
 ## Install
 
-Current npm release line: **2.0.x** for the existing public packages. v2.0.0 is the stable trace-contract release: small root API, schema 1.0 persisted writer path, v0.1/v0.2/v1.0 read compatibility, and explicit non-destructive migration workflow.
+Current npm release line: **2.1.x** for the existing public packages. v2.1.0 adds deterministic local eval and reusable redaction utilities on top of the stable v2 trace contract: small root API, schema 1.0 persisted writer path, v0.1/v0.2/v1.0 read compatibility, and explicit non-destructive migration workflow.
 
 ```bash
 npm install agent-inspect
@@ -240,6 +240,7 @@ AGENT_INSPECT=1 node eval-runner.mjs
 - **Redact local files** with `agent-inspect redact` or `@agent-inspect/redact` before creating shareable copies.
 - **Migrate explicitly** with `agent-inspect migrate <trace.jsonl> --to 1.0 --dry-run` or `--output <file>`; originals are never overwritten by default.
 - **Export share-safe copies** — `export --redaction-profile share` (or `strict`) writes local Markdown/HTML/OpenInference/OTLP JSON only.
+- **Create local CI artifacts** with `agent-inspect artifacts`, and summarize local test-reporter manifests with `agent-inspect ci-summary`.
 - **Parse structured logs** you already emit (JSON first-class; log4js best-effort).
 - **Optional LangChain adapter** — metadata-only by default; optional `persist: true` and `stream: true` streaming metadata (no full token capture by default).
 - **Optional AI SDK adapter** — experimental `@agent-inspect/ai-sdk` telemetry integration for AI SDK v6; metadata-only by default with `recordInputs: false` and `recordOutputs: false`.
@@ -308,6 +309,7 @@ More detail: [docs/LOGS.md](docs/LOGS.md) · [docs/LOG-TO-TREE-QUICKSTART.md](do
 | `report` | Markdown/HTML inspection report (what + timeline + tree) |
 | `check` / `scan` / `verify-safe` | Deterministic local trace checks and best-effort safety verification |
 | `artifacts` | Safe local CI artifact bundles and optional step-summary file output |
+| `ci-summary` | Summarize local Vitest/Jest reporter artifact manifests for CI |
 
 ![Timeline with slow-step focus for one run](https://raw.githubusercontent.com/rajudandigam/agent-inspect/main/docs/assets/demos/timeline.gif)
 
@@ -338,6 +340,8 @@ AgentInspect is the **local-first trace workbench** for TypeScript AI agents:
 
 Pass `enabled: false` to `inspectRun` for a no-trace passthrough. Use `maybeInspectRun` with `AGENT_INSPECT=1` to toggle tracing in eval or CI — see [docs/API.md](docs/API.md).
 
+**Shipped in 2.1.0:** deterministic local eval and redaction utilities. Linked release aligns `agent-inspect`, `@agent-inspect/ai-sdk`, `@agent-inspect/langchain`, `@agent-inspect/tui`, `@agent-inspect/openai-agents`, `@agent-inspect/redact`, and `@agent-inspect/eval` at **2.1.0**.
+
 **Shipped in 2.0.0:** stable root API contract, schema 1.0 persisted writer path, v0.1/v0.2/v1.0 read compatibility, and explicit trace migration workflow. Linked release aligns `agent-inspect`, `@agent-inspect/ai-sdk`, `@agent-inspect/langchain`, `@agent-inspect/tui`, and `@agent-inspect/openai-agents` at **2.0.0**.
 
 **Shipped in 1.9.0:** private harness workspace foundation, explain dry-run/local analysis, promoted adapter adoption paths, and the v2 root API slimming plan.
@@ -350,7 +354,7 @@ Pass `enabled: false` to `inspectRun` for a no-trace passthrough. Use `maybeInsp
 
 **Shipped in 1.5.0:** non-breaking subpath exports; `what` and `report` CLI; dual-format read path (v0.1 + v0.2 JSONL); [what-report-inspect recipe](examples/recipes/what-report-inspect/). Linked release aligns all three npm packages at **1.5.0**.
 
-**Roadmap beyond current release work:** v2.1 starts the eval/redact utility triangle, followed by reporters/CI, adapter hardening, sessions/MCP telemetry, guardrails, optional viewer/IDE surfaces, and conditional v3 extensibility. See [ROADMAP.md](ROADMAP.md).
+**Roadmap beyond current release work:** v2.2 prepares test reporters and CI workflows, followed by adapter hardening, sessions/MCP telemetry, guardrails, optional viewer/IDE surfaces, and conditional v3 extensibility. See [ROADMAP.md](ROADMAP.md).
 
 **Shipped in 1.4.0:** CI artifact recipe ([docs/CI-ARTIFACTS.md](docs/CI-ARTIFACTS.md)); `timeline`, `stats`, and `search` CLI; core helpers `buildRunTimeline`, `buildTraceStats`, `searchTraces`. Linked release aligns all three npm packages at **1.4.0**.
 
@@ -405,6 +409,12 @@ npx agent-inspect view <run-id> --tui
 
 The TUI is available as a separate optional package; its programmatic API is experimental, while the CLI integration (`view --tui`) is the intended usage. Details: [docs/ADAPTERS.md](docs/ADAPTERS.md).
 
+### Test reporter artifacts (`@agent-inspect/vitest`, `@agent-inspect/jest`)
+
+Optional Vitest/Jest reporter packages are implemented in the workspace for local failure artifacts, but remain private/unpublished until the maintainer clears first-publication setup for a v2.2 release. They write shared `schemaVersion: "0.1"` reporter manifests with safe relative artifact paths and bounded structural metadata. Use `agent-inspect ci-summary` to summarize those local manifests in CI without reading trace contents or calling GitHub APIs.
+
+Reporter artifact behavior and API details are documented in [docs/API.md](docs/API.md) and [docs/CI-ARTIFACTS.md](docs/CI-ARTIFACTS.md).
+
 ## Examples and recipes
 
 | Example | Shows |
@@ -431,7 +441,7 @@ The TUI is available as a separate optional package; its programmatic API is exp
 | [examples/recipes/eval-local-checks](examples/recipes/eval-local-checks) | v2.1 deterministic local eval checks |
 | [examples/recipes/redact-share-safe-file](examples/recipes/redact-share-safe-file) | v2.1 share-safe local redaction copy |
 | [examples/recipes/eval-ci-artifacts](examples/recipes/eval-ci-artifacts) | v2.1 eval before safe CI artifacts |
-| [examples/recipes/test-reporter-artifacts](examples/recipes/test-reporter-artifacts) | v1.8 Vitest/Jest reporter artifact patterns |
+| [examples/recipes/test-reporter-artifacts](examples/recipes/test-reporter-artifacts) | Vitest/Jest reporter artifact patterns |
 | [examples/recipes/what-report-inspect](examples/recipes/what-report-inspect/) | `what` + `report` inspection |
 | [examples/recipes/runtime-and-ingestion](examples/recipes/runtime-and-ingestion/) | v1.6 runtime writers + universal ingestion |
 

package/docs/API.md

@@ -222,9 +222,32 @@ No network writer, OpenTelemetry exporter, provider wrapper, or global monkey-pa
 
 Recipe: [examples/recipes/ai-sdk-local-telemetry](../examples/recipes/ai-sdk-local-telemetry/).
 
+## 11.1 Experimental `agent-inspect/reporters` APIs
+
+`agent-inspect/reporters` contains shared, dependency-free helpers for local test reporter artifacts. The subpath does not import Vitest, Jest, GitHub SDKs, provider SDKs, or upload clients.
+
+Import from `agent-inspect/reporters`:
+
+```ts
+import {
+  TRACE_ARTIFACT_MANIFEST_SCHEMA_VERSION,
+  createReporterArtifactPath,
+  createTraceArtifactManifest,
+  validateReporterArtifactPath,
+  type TraceArtifactManifest,
+} from "agent-inspect/reporters";
+```
+
+- **`TRACE_ARTIFACT_MANIFEST_SCHEMA_VERSION`**: currently `"0.1"` for local reporter manifests.
+- **`createTraceArtifactManifest(options)`**: clones, sorts, and deduplicates reporter results/artifacts into deterministic manifest JSON.
+- **`createReporterArtifactPath(options)`**: creates a safe relative artifact path under a caller-provided output directory.
+- **`validateReporterArtifactPath(options)`**: rejects empty, absolute, traversal, Windows-absolute, and symlink-escape style paths before reporters or `ci-summary` trust artifact links.
+
+The manifest records framework, generation time, bounded test results, artifact descriptors, redaction profile, and diagnostics. It is an artifact index only; it should not contain raw trace contents, prompts, model outputs, request/response bodies, headers, API keys, secrets, or full tool payloads.
+
 ## 12. Experimental `@agent-inspect/vitest` APIs
 
-`@agent-inspect/vitest` is an optional experimental workspace package for local Vitest failure artifacts. It remains private/unpublished. It does not add a Vitest dependency to root/core, does not upload artifacts, and does not infer trace relationships by timestamp.
+`@agent-inspect/vitest` is an optional experimental workspace package for local Vitest failure artifacts. It remains private/unpublished pending maintainer first-publication setup. It does not add a Vitest dependency to root/core, does not upload artifacts, and does not infer trace relationships by timestamp.
 
 Import from `@agent-inspect/vitest`:
 
@@ -237,6 +260,7 @@ import { createAgentInspectVitestReporter } from "@agent-inspect/vitest";
   - **`githubSummary`**: optional GitHub step-summary file path. The reporter appends bounded structural counts only and does not use the GitHub API.
   - **`retainSuccessful`**: `false`/undefined keeps no passing-test artifacts; `true` keeps up to `maxSuccessfulTraces`; a number keeps up to that many passing-test artifacts.
   - **`maxSuccessfulTraces`**: upper bound for passing-test artifacts, capped by the reporter.
+  - **`redactionProfile`**: manifest artifact profile, `local` (default), `share`, or `strict`.
   - **`resolveTrace(test)`**: optional explicit association resolver when task metadata is not convenient.
   - **`onDiagnostic(diagnostic)`**: observes non-fatal reporter/artifact failures.
   - **`getDiagnostics()`** and **`getArtifacts()`** expose reporter state for tests and custom harnesses.
@@ -252,11 +276,11 @@ ctx.task.meta.agentInspect = {
 };
 ```
 
-Artifacts are safe structural summaries. They include bounded test identity, status, trace run id, and trace filename, but they do not read or embed raw trace contents, prompts, generated outputs, request/response bodies, headers, API keys, secrets, or tool payloads. Reporter/artifact failures are diagnostics and do not replace original Vitest failures.
+Artifacts are safe structural summaries. The reporter writes a shared `schemaVersion: "0.1"` manifest wrapper with package metadata, generated time, framework, test results, artifact descriptors, relative paths, and redaction profile. It includes bounded test identity, status, trace run id, and trace filename, but it does not read or embed raw trace contents, prompts, generated outputs, request/response bodies, headers, API keys, secrets, or tool payloads. Reporter/artifact failures are diagnostics and do not replace original Vitest failures.
 
 ## 13. Experimental `@agent-inspect/jest` APIs
 
-`@agent-inspect/jest` is an optional experimental workspace package for local Jest failure artifacts. It remains private/unpublished. It does not add a Jest dependency to root/core, does not upload artifacts, and does not infer trace relationships by timestamp.
+`@agent-inspect/jest` is an optional experimental workspace package for local Jest failure artifacts. It remains private/unpublished pending maintainer first-publication setup. It does not add a Jest dependency to root/core, does not upload artifacts, and does not infer trace relationships by timestamp.
 
 Import from `@agent-inspect/jest`:
 
@@ -270,6 +294,7 @@ import { AgentInspectJestReporter, createAgentInspectJestReporter } from "@agent
   - **`githubSummary`**: optional GitHub step-summary file path. The reporter appends bounded structural counts only and does not use the GitHub API.
   - **`retainSuccessful`**: `false`/undefined keeps no passing-test artifacts; `true` keeps up to `maxSuccessfulTraces`; a number keeps up to that many passing-test artifacts.
   - **`maxSuccessfulTraces`**: upper bound for passing-test artifacts, capped by the reporter.
+  - **`redactionProfile`**: manifest artifact profile, `local` (default), `share`, or `strict`.
   - **`associations`**: explicit trace associations keyed by `file::fullName`, `basename::fullName`, or `fullName`.
   - **`resolveTrace(test)`**: optional explicit association resolver for normalized Jest assertion results.
   - **`onDiagnostic(diagnostic)`**: observes non-fatal reporter/artifact failures.
@@ -294,7 +319,7 @@ reporters: [
 ],
 ```
 
-Artifacts are safe structural summaries. They include bounded test identity, status, trace run id, and trace filename, but they do not read or embed raw trace contents, prompts, generated outputs, request/response bodies, headers, API keys, secrets, or tool payloads. Reporter/artifact failures are diagnostics and do not replace original Jest failures.
+Artifacts are safe structural summaries. The reporter writes a shared `schemaVersion: "0.1"` manifest wrapper with package metadata, generated time, framework, test results, artifact descriptors, relative paths, and redaction profile. It includes bounded test identity, status, trace run id, and trace filename, but it does not read or embed raw trace contents, prompts, generated outputs, request/response bodies, headers, API keys, secrets, or tool payloads. Reporter/artifact failures are diagnostics and do not replace original Jest failures.
 
 ## 14. Experimental `@agent-inspect/openai-agents` APIs
 

package/docs/CLI.md

@@ -33,6 +33,7 @@ Core commands:
 - `scan` — best-effort local safety scan for trace capture risks
 - `verify-safe` — best-effort local trace safety verification
 - `artifacts` — create safe local CI trace artifact bundles and optional step summaries
+- `ci-summary` — summarize local reporter artifact manifests for CI
 - `diff` — compare two manual traces (local, read-only)
 - `timeline` — chronological view of one run (local JSONL)
 - `stats` — local aggregate stats over a trace directory
@@ -489,7 +490,33 @@ npx agent-inspect artifacts candidate.jsonl --baseline baseline.jsonl --output-d
 
 Recipe and sample workflow: [examples/recipes/deterministic-ci-checks](../examples/recipes/deterministic-ci-checks/README.md)
 
-### 6.14 `diff`
+### 6.14 `ci-summary`
+
+Summarize local Vitest/Jest reporter artifact manifests into deterministic Markdown or JSON. This command reads shared `schemaVersion: "0.1"` manifest JSON files only, including the reporter package wrapper emitted by the workspace reporters. It does not read trace contents, rerun tests, upload artifacts, call GitHub APIs, or mutate repository state. `--output` and `--github-summary` write local files.
+
+```bash
+agent-inspect ci-summary <manifest...> [options]
+```
+
+Options:
+
+- `-o, --output <path>`: write the Markdown summary to a local file
+- `--github-summary <path>`: append the Markdown summary to a local file, such as `$GITHUB_STEP_SUMMARY`
+- `--json`: print deterministic JSON summary
+
+Example:
+
+```bash
+npx agent-inspect ci-summary .agent-inspect/jest-artifacts/tests/**/report.json \
+  --output ./artifacts/reporter-summary.md \
+  --github-summary "$GITHUB_STEP_SUMMARY"
+```
+
+Reporter artifact paths in the summary are kept relative and validated conservatively. The summary includes bounded package/framework metadata, test identity, status counts, trace filenames, artifact paths, redaction profiles, and diagnostic counts only.
+
+Recipe and sample workflow: [examples/recipes/github-actions-artifact](../examples/recipes/github-actions-artifact/README.md)
+
+### 6.15 `diff`
 
 Compare two manual trace runs. Diff is **local** and **read-only** (does not rerun agents).
 
@@ -553,7 +580,7 @@ Differences:
 
 More examples, including timing-only and structure-only diffs, are in `docs/DIFF.md`.
 
-### 6.15 `timeline`
+### 6.16 `timeline`
 
 Chronological step list for one manual trace. Read-only; does not mutate JSONL files.
 
@@ -569,7 +596,7 @@ Options:
 
 ![Timeline with slow-step focus](../assets/demos/timeline.gif)
 
-### 6.16 `stats`
+### 6.17 `stats`
 
 Local aggregate statistics over trace files in a directory. Read-only.
 
@@ -589,7 +616,7 @@ Options:
 
 Use `--correlation-id` or `--group-id` to filter runs by `run_started` metadata (see [API.md](./API.md)).
 
-### 6.17 `search`
+### 6.18 `search`
 
 Deterministic search over local traces (substring / exact filters). No semantic search.
 
@@ -619,7 +646,7 @@ npx agent-inspect search --duration ">100ms" --json
 
 ![Search traces by status error](../assets/demos/search.gif)
 
-### 6.18 `what`
+### 6.19 `what`
 
 Concise human-readable summary of one local trace run. Read-only; accepts v0.1 manual JSONL and v0.2 persisted-event JSONL through the shared dual-format normalization path. Vocabulary: [TRACE-VOCABULARY-V1.5.md](./proposals/TRACE-VOCABULARY-V1.5.md).
 
@@ -648,7 +675,7 @@ Outcome: Completed successfully.
 Slowest: plan (100ms, logic)
 ```
 
-### 6.19 `report`
+### 6.20 `report`
 
 Generate a local inspection report combining **what happened**, **timeline**, and **execution tree** sections. The command reads local v0.1 manual JSONL and v0.2 persisted-event JSONL through the shared dual-format normalization path without mutating them. Distinct from `export` (which targets shareable tree snapshots and standards formats).
 
@@ -673,7 +700,7 @@ Example:
 npx agent-inspect report minimal-success --dir fixtures/traces --format html -o report.html
 ```
 
-### 6.20 `explain`
+### 6.21 `explain`
 
 Explain a local trace using deterministic facts and local inference labels. This command reads through the same local reader pipeline as `open` / `check`; it does not call a model provider, upload traces, replay agents, or mutate input files.
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-inspect",
-  "version": "2.1.0",
+  "version": "2.2.0",
   "license": "MIT",
   "type": "module",
   "description": "Local-first execution-tree debugger for TypeScript AI agents",
@@ -105,6 +105,16 @@
         "types": "./packages/core/dist/checks.d.cts",
         "default": "./packages/core/dist/checks.cjs"
       }
+    },
+    "./reporters": {
+      "import": {
+        "types": "./packages/core/dist/reporters.d.ts",
+        "default": "./packages/core/dist/reporters.mjs"
+      },
+      "require": {
+        "types": "./packages/core/dist/reporters.d.cts",
+        "default": "./packages/core/dist/reporters.cjs"
+      }
     }
   },
   "bin": {