agent-inspect 6.31.0 → 6.31.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,5 +1,17 @@
1
1
  # Changelog
2
2
 
3
+ ## 6.31.2
4
+
5
+ ### Patch Changes
6
+
7
+ - 520c7c3: Clarify README Prevent path (trajectory without leading observation flags), AI SDK inspect/bundle commands, and Jest/Vitest explicit trace association in CI docs.
8
+
9
+ ## 6.31.1
10
+
11
+ ### Patch Changes
12
+
13
+ - 11a0159: Strict `allowedStatuses` validation (typos like `succes` no longer map to permissive `error`); portable `prepublishOnly` skip runner for Trusted Publish; LangChain `handleChainStart` parentage matches CallbackManager runtime arg order.
14
+
3
15
  ## 6.31.0
4
16
 
5
17
  ### Minor Changes
package/README.md CHANGED
@@ -74,10 +74,11 @@ Checks are deterministic and provider-free: the same trace and rules produce the
74
74
  ```bash
75
75
  npx agent-inspect check <run-id> --dir .agent-inspect \
76
76
  --preset trajectory \
77
- --required-tool retrieve_policy \
78
- --fail-on-observation failed
77
+ --required-tool retrieve_policy
79
78
  ```
80
79
 
80
+ Use `--fail-on-observation failed` only when the run records explicit OUTCOME events (for example the [broken-agent-debugging starter](https://github.com/rajudandigam/agent-inspect/tree/main/examples/starters/broken-agent-debugging)). Trajectory checks do not invent outcomes from tool/LLM/run success.
81
+
81
82
  A passing check exits `0`; a rule failure exits `1`. Invalid configuration and unreadable/unsupported inputs use separate documented exit codes. Use TraceContract, suites, cohorts, Vitest/Jest reporters, or `--evidence-on fail` when the workflow needs more than one CLI check.
82
83
 
83
84
  ### Share — create reviewable offline evidence
@@ -212,7 +213,7 @@ The root package is enough for custom capture, the CLI, checks, and Evidence wor
212
213
 
213
214
  ## Status and documentation
214
215
 
215
- **Current published baseline:** **6.31.0** · persisted schema `1.0` · Node.js `>=20` · MIT.
216
+ **Current published baseline:** **6.31.2** · persisted schema `1.0` · Node.js `>=20` · MIT.
216
217
 
217
218
  Legacy v0.1 and v0.2 traces remain readable. Check the npm badge and [changelog](CHANGELOG.md) for the current published version.
218
219
 
@@ -97,10 +97,15 @@ sensitive free text; redact before sharing. Full contract:
97
97
 
98
98
  ```bash
99
99
  npx agent-inspect list --dir .agent-inspect
100
- npx agent-inspect open .agent-inspect/<run>.jsonl
101
- npx agent-inspect check .agent-inspect/<run>.jsonl
100
+ npx agent-inspect view <run-id> --dir .agent-inspect --summary
101
+ npx agent-inspect check <run-id> --dir .agent-inspect --preset trajectory
102
+ npx agent-inspect verify-safe <run-id> --dir .agent-inspect
103
+ npx agent-inspect bundle <run-id> --dir .agent-inspect --profile share --out ./evidence
104
+ npx agent-inspect bundle verify ./evidence
102
105
  ```
103
106
 
107
+ Use `--fail-on-observation` only when the run records explicit OUTCOME events.
108
+
104
109
  ## Troubleshooting
105
110
 
106
111
  | Symptom | Fix |
@@ -56,6 +56,8 @@ npx agent-inspect ci-summary .agent-inspect/jest-artifacts/tests/**/report.json
56
56
 
57
57
  `ci-summary` writes local files only. It validates reporter artifact paths as relative paths and includes bounded structural metadata: package/framework, test status counts, trace filenames, artifact paths, redaction profile, and diagnostic counts.
58
58
 
59
+ **Jest / Vitest association:** reporters do **not** invent run↔test links from timestamps. Wire an explicit association (`withAgentInspectJestTrace`, reporter `associations` / `resolveTrace`, or the Vitest equivalents) or expect a `no-trace-association` diagnostic. See [`@agent-inspect/jest`](https://github.com/rajudandigam/agent-inspect/blob/main/packages/jest/README.md#trace-association) and [`@agent-inspect/vitest`](https://github.com/rajudandigam/agent-inspect/blob/main/packages/vitest/README.md).
60
+
59
61
  ```bash
60
62
  npx agent-inspect export <run-id> --dir ./.agent-inspect \
61
63
  --format markdown --redaction-profile share -o ./artifacts/trace.md
package/docs/COMPARE.md CHANGED
@@ -120,7 +120,7 @@ AgentInspect avoids SDK/collector setup for local debugging:
120
120
 
121
121
  ## v3.5 positioning (local inner loop)
122
122
 
123
- AgentInspect v3.5 is the **adoption release**. The npm map is one core package (`agent-inspect`: APIs + CLI) plus optional packages that stay out of the root dependency graph: framework adapters (`ai-sdk`, `openai-agents`, `langchain`, `mcp`, `adapter-sdk`), CI and quality gates (`vitest`, `jest`, `eval`, `guardrails`, `circuit`, `harness`), and inspection surfaces (`viewer`, `tui`, `mcp-server`, `redact`). See the [package map](../README.md#package-map).
123
+ AgentInspect v3.5 is the **adoption release**. The npm map is one core package (`agent-inspect`: APIs + CLI) plus optional packages that stay out of the root dependency graph: framework adapters (`ai-sdk`, `openai-agents`, `langchain`, `mcp`, `adapter-sdk`), CI and quality gates (`vitest`, `jest`, `eval`, `guardrails`, `circuit`, `harness`), and inspection surfaces (`viewer`, `tui`, `mcp-server`, `redact`). See the [package map](https://github.com/rajudandigam/agent-inspect#package-map).
124
124
 
125
125
  Use it when:
126
126
 
@@ -1,6 +1,6 @@
1
1
  # First trace in 5 minutes
2
2
 
3
- Goal: install → one trace → one check → one share-safe bundle.
3
+ Goal: install → one trace → one trajectory check → one share-safe Evidence bundle.
4
4
 
5
5
  **Docs site:** [https://agentinspect.vercel.app/docs/getting-started/](https://agentinspect.vercel.app/docs/getting-started/)
6
6
 
@@ -14,12 +14,15 @@ npx agent-inspect list --dir .agent-inspect
14
14
  Copy a `<run-id>` from `list`, then:
15
15
 
16
16
  ```bash
17
- npx agent-inspect report <run-id> --dir .agent-inspect
18
- npx agent-inspect check <run-id> --dir .agent-inspect
19
- npx agent-inspect bundle <run-id> --dir .agent-inspect --profile share
17
+ npx agent-inspect view <run-id> --dir .agent-inspect --summary
18
+ npx agent-inspect check <run-id> --dir .agent-inspect --preset trajectory
20
19
  npx agent-inspect verify-safe <run-id> --dir .agent-inspect
20
+ npx agent-inspect bundle <run-id> --dir .agent-inspect --profile share --out ./evidence
21
+ npx agent-inspect bundle verify ./evidence
21
22
  ```
22
23
 
24
+ `init` scaffolds files into the current directory (`agent-inspect.config.ts`, `.agent-inspect/`, and `examples/agent-inspect-demo.mjs`). It does **not** install dependencies or write a trace by itself. Use `--preset trajectory` for structural CI gates; `--fail-on-observation` belongs only in examples that record explicit OUTCOME events.
25
+
23
26
  ## Minutes 0–1: Install
24
27
 
25
28
  ```bash
@@ -28,9 +31,6 @@ npm install agent-inspect
28
31
  npx agent-inspect init --yes
29
32
  ```
30
33
 
31
- Creates `agent-inspect.config.ts`, `.agent-inspect/`, and `examples/agent-inspect-demo.mjs`.
32
- `init` scaffolds files; it does **not** write a trace by itself.
33
-
34
34
  Framework users: pick the correct capture path first — [CHOOSE-YOUR-CAPTURE-PATH.md](./CHOOSE-YOUR-CAPTURE-PATH.md).
35
35
 
36
36
  ## Minutes 1–2: Run
@@ -39,33 +39,36 @@ Framework users: pick the correct capture path first — [CHOOSE-YOUR-CAPTURE-PA
39
39
  node examples/agent-inspect-demo.mjs
40
40
  ```
41
41
 
42
- No API keys. Deterministic local trace.
42
+ No API keys. Deterministic local trace under `.agent-inspect/`.
43
43
 
44
44
  ## Minutes 2–3: Inspect
45
45
 
46
46
  ```bash
47
47
  npx agent-inspect list --dir .agent-inspect
48
- npx agent-inspect view <run-id> --dir .agent-inspect
49
- npx agent-inspect report <run-id> --dir .agent-inspect
48
+ npx agent-inspect view <run-id> --dir .agent-inspect --summary
50
49
  ```
51
50
 
51
+ Replace `<run-id>` with the ID printed by `list` (do not guess “latest file”).
52
+
52
53
  ## Minutes 3–4: Check
53
54
 
54
55
  ```bash
55
- npx agent-inspect check <run-id> --dir .agent-inspect
56
+ npx agent-inspect check <run-id> --dir .agent-inspect --preset trajectory
56
57
  ```
57
58
 
59
+ Expected exit code `0` for the keyless demo. Add `--required-tool <name>` only when that tool is part of the real expected workflow.
60
+
58
61
  ## Minutes 4–5: Share-safe artifact
59
62
 
60
63
  ```bash
61
- npx agent-inspect bundle <run-id> --dir .agent-inspect --profile share
62
64
  npx agent-inspect verify-safe <run-id> --dir .agent-inspect
63
- npx agent-inspect bundle verify .agent-inspect/bundles/<run-id>
65
+ npx agent-inspect bundle <run-id> --dir .agent-inspect --profile share --out ./evidence
66
+ npx agent-inspect bundle verify ./evidence
64
67
  ```
65
68
 
66
- Attach the share-profile bundle (or a redacted file) to a PR or issue — not raw traces.
69
+ `--out ./evidence` writes the Evidence v2 package to a known path; `bundle verify ./evidence` checks that exact directory. Do not use `--allow-unsafe` to force a share.
67
70
 
68
- Optional file redaction:
71
+ Optional file redaction before bundling:
69
72
 
70
73
  ```bash
71
74
  npx agent-inspect redact <run-id> --dir .agent-inspect --profile share -o redacted.jsonl
@@ -75,8 +78,9 @@ npx agent-inspect redact <run-id> --dir .agent-inspect --profile share -o redact
75
78
 
76
79
  | If you use… | Go to |
77
80
  | ----------- | ----- |
78
- | Broken agent demo (same answer, wrong path) | [broken-agent-debugging starter](../examples/starters/broken-agent-debugging/README.md) (`node prove-same-output-wrong-path.mjs`) |
79
- | Coding-agent MCP loop | [CODING-AGENT-LOOP.md](./CODING-AGENT-LOOP.md) · [coding-agent-debug-loop](../examples/starters/coding-agent-debug-loop/README.md) |
81
+ | Full install + instrumentation guide | [GETTING-STARTED.md](./GETTING-STARTED.md) (site: [/docs/getting-started/guide](/docs/getting-started/guide)) |
82
+ | Broken agent demo (same answer, wrong path) | [broken-agent-debugging starter](../examples/starters/broken-agent-debugging/README.md) |
83
+ | Coding-agent MCP loop | [CODING-AGENT-LOOP.md](./CODING-AGENT-LOOP.md) |
80
84
  | Contracts / CI gates | [TRACE-CONTRACTS.md](./TRACE-CONTRACTS.md) · [SUITES-COHORTS-GATES.md](./SUITES-COHORTS-GATES.md) |
81
85
  | AI SDK | [AI SDK adoption](./AI-SDK-ADOPTION.md) |
82
86
  | OpenAI Agents | [OpenAI Agents local](./OPENAI-AGENTS-LOCAL.md) |
@@ -14,7 +14,7 @@ pnpm add agent-inspect
14
14
 
15
15
  ### Quick bootstrap (v3.1+)
16
16
 
17
- See also [FIRST-TRACE-IN-5-MINUTES.md](./FIRST-TRACE-IN-5-MINUTES.md).
17
+ Prefer the five-minute path first: [FIRST-TRACE-IN-5-MINUTES.md](./FIRST-TRACE-IN-5-MINUTES.md) (site: [/docs/getting-started](/docs/getting-started)). This page is the longer install and instrumentation guide (site: [/docs/getting-started/guide](/docs/getting-started/guide)).
18
18
 
19
19
  ```bash
20
20
  npx agent-inspect init --yes
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "agent-inspect",
3
- "version": "6.31.0",
3
+ "version": "6.31.2",
4
4
  "license": "MIT",
5
5
  "type": "module",
6
6
  "description": "Local evidence debugger and trajectory-test toolkit for TypeScript AI agents — execution trees, TraceContract checks, Evidence v2, and read-only MCP",
@@ -234,7 +234,7 @@
234
234
  "test:coverage": "vitest run --coverage",
235
235
  "size": "size-limit --config size-limit.config.mjs",
236
236
  "test:all": "pnpm run typecheck && pnpm run linked-versions:check && pnpm run build && pnpm run test && pnpm run size",
237
- "prepublish:checks": "pnpm run typecheck && pnpm run test && pnpm run test:coverage && pnpm run build && pnpm run fixtures:check && pnpm run recipes:check && pnpm run size && pnpm run linked-versions:check && pnpm run repo:health && pnpm run pack:smoke",
237
+ "prepublish:checks": "pnpm run typecheck && pnpm run build && pnpm run test && pnpm run test:coverage && pnpm run fixtures:check && pnpm run recipes:check && pnpm run size && pnpm run linked-versions:check && pnpm run repo:health && pnpm run pack:smoke",
238
238
  "pack:dry-run": "pnpm run build && npm pack --dry-run",
239
239
  "pack:smoke": "pnpm run package-licenses:check && pnpm run build && node scripts/package-smoke.mjs && node scripts/packed-openai-agents-e2e.mjs && node scripts/packed-ai-sdk-e2e.mjs && node scripts/packed-mcp-e2e.mjs && node scripts/packed-quickstart-e2e.mjs && node scripts/packed-semantic-loop-e2e.mjs && node scripts/packed-swarm-loop-e2e.mjs && node scripts/evidence-ci-golden-paths.mjs",
240
240
  "linked-versions:check": "node scripts/check-linked-versions.mjs",
@@ -262,6 +262,8 @@
262
262
  "website:dev": "pnpm --filter @agent-inspect/website dev",
263
263
  "website:build": "pnpm --filter @agent-inspect/website build",
264
264
  "website:typecheck": "pnpm --filter @agent-inspect/website typecheck",
265
+ "website:check": "pnpm --filter @agent-inspect/website check",
266
+ "website:crawl": "pnpm --filter @agent-inspect/website crawl",
265
267
  "package-licenses:check": "node scripts/validate-package-licenses.mjs",
266
268
  "package-licenses:sync": "node scripts/sync-package-licenses.mjs",
267
269
  "demo-version-sync:check": "node scripts/check-demo-version-sync.mjs"
@@ -6351,11 +6351,40 @@ function contractFailFinding(ruleId, message, evidence, expected, actual) {
6351
6351
  evidence: [...evidence]
6352
6352
  };
6353
6353
  }
6354
+ var ALLOWED_STATUS_ALIASES = {
6355
+ ok: "ok",
6356
+ error: "error",
6357
+ running: "running",
6358
+ success: "ok",
6359
+ failed: "error"
6360
+ };
6361
+ function parseAllowedStatus(status) {
6362
+ return ALLOWED_STATUS_ALIASES[status];
6363
+ }
6354
6364
  function normalizeStatus(status) {
6355
- if (status === "ok" || status === "error" || status === "running") return status;
6356
- if (status === "success") return "ok";
6357
- if (status === "failed") return "error";
6358
- return "error";
6365
+ const parsed = parseAllowedStatus(status);
6366
+ if (parsed === void 0) {
6367
+ throw new TypeError(
6368
+ `Unknown run status ${JSON.stringify(status)}. Allowed: ok, error, running (aliases: success, failed).`
6369
+ );
6370
+ }
6371
+ return parsed;
6372
+ }
6373
+ function validateAllowedStatusesShape(allowedStatuses, pathPrefix = "run.allowedStatuses") {
6374
+ if (!allowedStatuses?.length) return [];
6375
+ const out = [];
6376
+ for (let i = 0; i < allowedStatuses.length; i++) {
6377
+ const status = allowedStatuses[i];
6378
+ if (parseAllowedStatus(status) === void 0) {
6379
+ out.push({
6380
+ code: "contract.run.allowedStatuses.unknown",
6381
+ severity: "error",
6382
+ message: `${pathPrefix}[${i}] has unknown status ${JSON.stringify(status)}. Allowed: ok, error, running (aliases: success, failed).`,
6383
+ path: `${pathPrefix}[${i}]`
6384
+ });
6385
+ }
6386
+ }
6387
+ return out;
6359
6388
  }
6360
6389
  function cloneBody(body) {
6361
6390
  return {
@@ -7022,6 +7051,7 @@ function evaluateBody(input, body, options = {}) {
7022
7051
  }
7023
7052
  function defineTraceContract(input) {
7024
7053
  const shapeErrors = [
7054
+ ...validateAllowedStatusesShape(input.run?.allowedStatuses),
7025
7055
  ...validateAlternativesShape(input.alternatives),
7026
7056
  ...validateScopeShape(input.scope),
7027
7057
  ...validateProvenanceShape(input.observations)
@@ -7052,6 +7082,7 @@ function defineTraceContract(input) {
7052
7082
  }
7053
7083
  function evaluateTraceContract(input, contract, options = {}) {
7054
7084
  const shapeErrors = [
7085
+ ...validateAllowedStatusesShape(contract.run?.allowedStatuses),
7055
7086
  ...validateAlternativesShape(contract.alternatives),
7056
7087
  ...validateScopeShape(contract.scope),
7057
7088
  ...validateProvenanceShape(contract.observations)
@@ -14854,5 +14885,5 @@ function renderGateReport(result, options = {}) {
14854
14885
  }
14855
14886
 
14856
14887
  export { ATTRIBUTION_CONFIDENCES, COHORT_METRIC_IDS, DEFAULT_SUITE_ARTIFACTS_DIR, EVIDENCE_FORMAT_VERSION, EVIDENCE_HTML_FILENAME, EVIDENCE_MANIFEST_FILENAME, Redactor, TraceDirectory, TraceReadError, TreeBuilder, aggregateBundleSafeStatus, aggregateSessionCheckResults, analyzeCohort, applyProfileMetadataCaps, assertBundlePathContained, assertEvidenceRelativePath, buildActivitySummary, buildBundleMetadata, buildBundleSummaryMarkdown, buildEvidenceCausalFailureViewHtml, buildEvidenceCiPackage, buildEvidenceCircuitViewHtml, buildEvidenceContractsViewHtml, buildEvidenceDiffViewHtml, buildEvidenceHtmlShell, buildEvidenceManifest, buildEvidenceOutcomesViewHtml, buildEvidenceProvenanceViewHtml, buildEvidenceSafetyViewHtml, buildEvidenceTimelineViewHtml, buildEvidenceToolsLlmViewHtml, buildEvidenceTreeViewHtml, buildLocalExplanation, buildPlaceholderArtifact, buildRunSummary, buildRunTimeline, buildRunWhatSummary, buildSessionIndex, buildTraceStats, buildZipArchive, bundleFailsOnSafety, bundleRunAssetRelativePath, collectTraceSchemaVersions, compactAttributes, createBaselineRegressionRule, createLlmUsageRule, createMaxStepDurationRule, createObservedOutcomeRule, createRequireCompletedRule, createRunDepthRule, createRunDurationRule, createRunStatusRule, createSafetyOversizedAttributeRule, createSafetyRawContentRule, createSafetyRedactionRule, createSafetySecretPatternRule, createStallDetectionRule, createStructureCycleRule, createStructureOrphanRule, createStructureParallelWidthRule, createStructureRelationshipRule, createToolUsageRule, defaultBundleOutputPath, defaultSuiteConfigTemplate, defineTraceContract, diffRuns, diffTraceEvents, enrichSessionRunRecord, escapeHtml, escapeMarkdown, evaluateTraceContract, extractMetadata, extractOutcomesFromTraceEvents, filterMetasBySessionScope, filterTraces, flattenTree, formatDuration2 as formatDuration, formatStepLabel, formatTimestamp, gateHasThresholds, getIndent, getTraceFilePath, inferEvidenceFileRole, isAgentInspectTrace, isAttributionConfidence, isPersistedInspectEvent, loadSessionRunRecords, loadSuiteConfig, loadTraceMetadataList, manualTraceEventsToComparableRun, nanoid, normalizeBundleOutputPath, openTrace, parseCohortMetricList, parseDuration, parseDurationFilter, parseGateList, parseTraceJsonl, persistedInspectEventsToTraceEvents, projectLogicalEvents, renderActivitySummaryHuman, renderCohortReport, renderErrorLine, renderGateReport, renderObservedOutcomesHtml, renderObservedOutcomesMarkdown, renderRunDiff, renderRunWhat, renderStepLine, renderSuiteReport, renderTimeline, renderTraceStats, resolveBundleRunIds, resolveRedactionProfile, resolveSuiteTemplate, resolveTraceDir, runGate, runSuite, runTraceChecks, safeString, sanitizeBundleRunId, searchTraces, serializeEvidenceManifest, sha256Hex, stableJson, summarizeObservedOutcomes, summarizeSemanticParity, traceEventToPersistedInspectEvent, truncateName, truncateStringForProfile, validateEvent, validateSuiteConfig, verifyEvidenceDirectory, zeroKinds };
14857
- //# sourceMappingURL=chunk-Y5Z4BXUO.mjs.map
14858
- //# sourceMappingURL=chunk-Y5Z4BXUO.mjs.map
14888
+ //# sourceMappingURL=chunk-SJ6FRNOE.mjs.map
14889
+ //# sourceMappingURL=chunk-SJ6FRNOE.mjs.map