agent-inspect 1.9.0 → 2.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +14 -0
- package/README.md +27 -10
- package/docs/ADAPTER-CONFORMANCE.md +1 -1
- package/docs/ADAPTERS.md +4 -0
- package/docs/API.md +76 -35
- package/docs/CLI.md +129 -11
- package/docs/COMPARE.md +5 -4
- package/docs/EXPORTS.md +1 -1
- package/docs/GETTING-STARTED.md +26 -7
- package/docs/KNOWN-ISSUES.md +15 -3
- package/docs/LIMITATIONS.md +9 -7
- package/docs/MIGRATION.md +8 -8
- package/docs/SCHEMA.md +29 -12
- package/package.json +2 -2
- package/packages/cli/dist/index.cjs +9808 -8624
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +9807 -8623
- package/packages/cli/dist/index.mjs.map +1 -1
- package/packages/core/dist/advanced.cjs +2047 -26
- package/packages/core/dist/advanced.cjs.map +1 -1
- package/packages/core/dist/advanced.d.cts +346 -196
- package/packages/core/dist/advanced.d.ts +346 -196
- package/packages/core/dist/advanced.mjs +628 -5
- package/packages/core/dist/advanced.mjs.map +1 -1
- package/packages/core/dist/checks.d.cts +1 -1
- package/packages/core/dist/checks.d.ts +1 -1
- package/packages/core/dist/chunk-2ZXEMWGK.mjs +549 -0
- package/packages/core/dist/chunk-2ZXEMWGK.mjs.map +1 -0
- package/packages/core/dist/{chunk-57S5D6HR.mjs → chunk-4JVGFNQH.mjs} +50 -6
- package/packages/core/dist/chunk-4JVGFNQH.mjs.map +1 -0
- package/packages/core/dist/{chunk-6QSLZCBJ.mjs → chunk-A4V2H3YU.mjs} +329 -163
- package/packages/core/dist/chunk-A4V2H3YU.mjs.map +1 -0
- package/packages/core/dist/{chunk-74XZ6N7Q.mjs → chunk-IZBJAZGF.mjs} +12 -4
- package/packages/core/dist/chunk-IZBJAZGF.mjs.map +1 -0
- package/packages/core/dist/{chunk-VTIB5MDK.mjs → chunk-NQZPZ2FN.mjs} +15 -10
- package/packages/core/dist/chunk-NQZPZ2FN.mjs.map +1 -0
- package/packages/core/dist/{chunk-TZISEVLQ.mjs → chunk-XUNJMAPD.mjs} +3 -3
- package/packages/core/dist/{chunk-TZISEVLQ.mjs.map → chunk-XUNJMAPD.mjs.map} +1 -1
- package/packages/core/dist/context-CSKnzpXR.d.cts +177 -0
- package/packages/core/dist/context-yv2VSDQF.d.ts +177 -0
- package/packages/core/dist/diff.d.cts +2 -2
- package/packages/core/dist/diff.d.ts +2 -2
- package/packages/core/dist/diff.mjs +473 -3
- package/packages/core/dist/diff.mjs.map +1 -1
- package/packages/core/dist/exporters.cjs +691 -0
- package/packages/core/dist/exporters.cjs.map +1 -1
- package/packages/core/dist/exporters.d.cts +24 -3
- package/packages/core/dist/exporters.d.ts +24 -3
- package/packages/core/dist/exporters.mjs +1113 -3
- package/packages/core/dist/exporters.mjs.map +1 -1
- package/packages/core/dist/index.cjs +1385 -8168
- package/packages/core/dist/index.cjs.map +1 -1
- package/packages/core/dist/index.d.cts +8 -350
- package/packages/core/dist/index.d.ts +8 -350
- package/packages/core/dist/index.mjs +18 -1478
- package/packages/core/dist/index.mjs.map +1 -1
- package/packages/core/dist/{log-config-DanPV3P9.d.cts → log-config-8aE5Kxtr.d.cts} +1 -1
- package/packages/core/dist/{log-config-BPHS4Sds.d.ts → log-config-DVimQJho.d.ts} +1 -1
- package/packages/core/dist/logs.d.cts +2 -2
- package/packages/core/dist/logs.d.ts +2 -2
- package/packages/core/dist/logs.mjs +785 -2
- package/packages/core/dist/logs.mjs.map +1 -1
- package/packages/core/dist/{persisted-inspect-event-Cw7TeYGr.d.ts → persisted-inspect-event-D-WpXeZX.d.ts} +14 -5
- package/packages/core/dist/{persisted-inspect-event-DHPfzUd8.d.cts → persisted-inspect-event-DHcHPUKv.d.cts} +14 -5
- package/packages/core/dist/persisted.cjs +11 -1
- package/packages/core/dist/persisted.cjs.map +1 -1
- package/packages/core/dist/persisted.d.cts +4 -4
- package/packages/core/dist/persisted.d.ts +4 -4
- package/packages/core/dist/persisted.mjs +150 -3
- package/packages/core/dist/persisted.mjs.map +1 -1
- package/packages/core/dist/readers.cjs +27 -10
- package/packages/core/dist/readers.cjs.map +1 -1
- package/packages/core/dist/readers.d.cts +1 -1
- package/packages/core/dist/readers.d.ts +1 -1
- package/packages/core/dist/readers.mjs +1580 -4
- package/packages/core/dist/readers.mjs.map +1 -1
- package/packages/core/dist/{types-Ap9uMdx_.d.ts → types-D-Y1kOU-.d.ts} +2 -2
- package/packages/core/dist/{types-B2-BU5CS.d.cts → types-DgMN3qow.d.cts} +2 -2
- package/packages/core/dist/writers.cjs +49 -3
- package/packages/core/dist/writers.cjs.map +1 -1
- package/packages/core/dist/writers.d.cts +1 -1
- package/packages/core/dist/writers.d.ts +1 -1
- package/packages/core/dist/writers.mjs +341 -4
- package/packages/core/dist/writers.mjs.map +1 -1
- package/packages/core/dist/chunk-57S5D6HR.mjs.map +0 -1
- package/packages/core/dist/chunk-6QSLZCBJ.mjs.map +0 -1
- package/packages/core/dist/chunk-6SZPTECC.mjs +0 -342
- package/packages/core/dist/chunk-6SZPTECC.mjs.map +0 -1
- package/packages/core/dist/chunk-74XZ6N7Q.mjs.map +0 -1
- package/packages/core/dist/chunk-HR7G62IE.mjs +0 -785
- package/packages/core/dist/chunk-HR7G62IE.mjs.map +0 -1
- package/packages/core/dist/chunk-TFLPUZ56.mjs +0 -1571
- package/packages/core/dist/chunk-TFLPUZ56.mjs.map +0 -1
- package/packages/core/dist/chunk-U2BGPESY.mjs +0 -150
- package/packages/core/dist/chunk-U2BGPESY.mjs.map +0 -1
- package/packages/core/dist/chunk-VTIB5MDK.mjs.map +0 -1
- package/packages/core/dist/chunk-Y56BPA3B.mjs +0 -990
- package/packages/core/dist/chunk-Y56BPA3B.mjs.map +0 -1
- package/packages/core/dist/chunk-YWAOOXLR.mjs +0 -475
- package/packages/core/dist/chunk-YWAOOXLR.mjs.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,19 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 2.1.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- 1e5e889: Release v2.1.0 with deterministic local eval and redaction utilities.
|
|
8
|
+
|
|
9
|
+
Adds the public optional `@agent-inspect/redact` and `@agent-inspect/eval` packages, root CLI redaction and eval workflows, shared redaction profiles/findings, deterministic local eval checks, and adoption recipes for local eval, share-safe traces, and CI artifacts.
|
|
10
|
+
|
|
11
|
+
## 2.0.0
|
|
12
|
+
|
|
13
|
+
### Major Changes
|
|
14
|
+
|
|
15
|
+
- 90fa75e: Release v2.0.0 with the stable root API contract, schema 1.0 persisted InspectEvent writer path, v0.1/v0.2/v1.0 reader compatibility, and explicit trace migration workflow.
|
|
16
|
+
|
|
3
17
|
## 1.9.0
|
|
4
18
|
|
|
5
19
|
### Minor Changes
|
package/README.md
CHANGED
|
@@ -22,7 +22,7 @@ agent-inspect gives those runs **structure**: an **execution tree** you can read
|
|
|
22
22
|
|
|
23
23
|
## Install
|
|
24
24
|
|
|
25
|
-
Current npm release: **
|
|
25
|
+
Current npm release line: **2.0.x** for the existing public packages. v2.0.0 is the stable trace-contract release: small root API, schema 1.0 persisted writer path, v0.1/v0.2/v1.0 read compatibility, and explicit non-destructive migration workflow.
|
|
26
26
|
|
|
27
27
|
```bash
|
|
28
28
|
npm install agent-inspect
|
|
@@ -194,6 +194,7 @@ Use the root import for stable beginner APIs:
|
|
|
194
194
|
|
|
195
195
|
```ts
|
|
196
196
|
import {
|
|
197
|
+
createInspector,
|
|
197
198
|
observe,
|
|
198
199
|
inspectRun,
|
|
199
200
|
maybeInspectRun,
|
|
@@ -212,7 +213,7 @@ import { diffTraceEvents } from "agent-inspect/diff";
|
|
|
212
213
|
import { exportMarkdown } from "agent-inspect/exporters";
|
|
213
214
|
import { parseLogsToTrees } from "agent-inspect/logs";
|
|
214
215
|
import { traceEventsToPersistedInspectEvents } from "agent-inspect/persisted";
|
|
215
|
-
import {
|
|
216
|
+
import { createInspectorRuntime } from "agent-inspect/advanced";
|
|
216
217
|
```
|
|
217
218
|
|
|
218
219
|
**Env-gated tracing** (eval harnesses, CI): use `maybeInspectRun` and set `AGENT_INSPECT=1` when you want a trace — otherwise no files are written.
|
|
@@ -227,20 +228,24 @@ await maybeInspectRun("eval-case-42", async () => runAgent());
|
|
|
227
228
|
AGENT_INSPECT=1 node eval-runner.mjs
|
|
228
229
|
```
|
|
229
230
|
|
|
230
|
-
## What you can do today
|
|
231
|
+
## What you can do today
|
|
231
232
|
|
|
232
233
|
- **Trace manually** with `inspectRun`, `step`, `step.llm`, `step.tool`, and `observe` — local JSONL under `.agent-inspect/` by default.
|
|
233
234
|
- **Toggle tracing** with `maybeInspectRun` and `AGENT_INSPECT=1` in eval harnesses or CI.
|
|
235
|
+
- **Use an isolated inspector** with `createInspector()` and explicit local writers for tests/adapters.
|
|
234
236
|
- **Correlate runs** with optional `correlationId`, `requestId`, `decisionId`, and `groupId` on `run_started` metadata.
|
|
235
237
|
- **Redact before disk** with default key-based redaction, or choose `redactionProfile`: `local`, `share`, or `strict`.
|
|
236
|
-
- **Inspect from the CLI** — `list`, `view`, `clean`, `logs`, `tail`, `export`, `open`, `diff`, `timeline`, `stats`, `search`, `what`, `report`.
|
|
238
|
+
- **Inspect from the CLI** — `list`, `view`, `clean`, `logs`, `tail`, `export`, `open`, `migrate`, `eval`, `redact`, `diff`, `timeline`, `stats`, `search`, `what`, `report`.
|
|
239
|
+
- **Run local evals** with `agent-inspect eval` or `@agent-inspect/eval`; built-in checks are deterministic heuristics over local traces, not model judges.
|
|
240
|
+
- **Redact local files** with `agent-inspect redact` or `@agent-inspect/redact` before creating shareable copies.
|
|
241
|
+
- **Migrate explicitly** with `agent-inspect migrate <trace.jsonl> --to 1.0 --dry-run` or `--output <file>`; originals are never overwritten by default.
|
|
237
242
|
- **Export share-safe copies** — `export --redaction-profile share` (or `strict`) writes local Markdown/HTML/OpenInference/OTLP JSON only.
|
|
238
243
|
- **Parse structured logs** you already emit (JSON first-class; log4js best-effort).
|
|
239
244
|
- **Optional LangChain adapter** — metadata-only by default; optional `persist: true` and `stream: true` streaming metadata (no full token capture by default).
|
|
240
245
|
- **Optional AI SDK adapter** — experimental `@agent-inspect/ai-sdk` telemetry integration for AI SDK v6; metadata-only by default with `recordInputs: false` and `recordOutputs: false`.
|
|
241
246
|
- **Optional OpenAI Agents adapter** — experimental `@agent-inspect/openai-agents` trace processor for local OpenAI Agents JS trace processing.
|
|
242
247
|
- **Optional TUI** — `view --tui` when `@agent-inspect/tui` is installed.
|
|
243
|
-
- **Persisted-event foundation
|
|
248
|
+
- **Persisted-event foundation** — v0.1/v0.2/v1.0 AgentInspect JSONL remains readable; `createInspector()` and built-in writers use the schema 1.0 persisted path.
|
|
244
249
|
- **Experimental subpaths** — `agent-inspect/readers`, `/writers`, `/checks`, `/diff`, `/exporters`, `/logs`, `/persisted`, and `/advanced` for advanced local workflows.
|
|
245
250
|
|
|
246
251
|
Nothing uploads traces by default. Review exports before sharing — see [safe trace sharing](docs/SAFE-TRACE-SHARING.md).
|
|
@@ -292,6 +297,9 @@ More detail: [docs/LOGS.md](docs/LOGS.md) · [docs/LOG-TO-TREE-QUICKSTART.md](do
|
|
|
292
297
|
| `tail` | Watch structured logs while the app runs |
|
|
293
298
|
| `export` | Write Markdown / HTML / OpenInference-compatible JSON / OTLP JSON **locally** |
|
|
294
299
|
| `open` | Read AgentInspect JSONL, OpenInference JSON, or OTLP JSON locally |
|
|
300
|
+
| `migrate` | Convert a local AgentInspect JSONL file to schema 1.0 with dry-run or explicit output |
|
|
301
|
+
| `eval` | Deterministic local evals over existing traces |
|
|
302
|
+
| `redact` | Redact a local JSON/JSONL file or trace copy |
|
|
295
303
|
| `diff` | Compare two local runs (read-only) |
|
|
296
304
|
| `timeline` | Chronological view of one run |
|
|
297
305
|
| `stats` | Local aggregates over a trace directory |
|
|
@@ -312,22 +320,28 @@ Full flags and behavior: [docs/CLI.md](docs/CLI.md).
|
|
|
312
320
|
- Debug a **failed tool call** or thrown error in a support or ops agent.
|
|
313
321
|
- See **which step dominated latency** in a multi-step planner or RAG pipeline.
|
|
314
322
|
- **Diff two runs** after a prompt, model, or routing change (see [diff examples](docs/DIFF.md)).
|
|
323
|
+
- Run **local eval checks** over a trace before sharing or creating CI artifacts.
|
|
324
|
+
- **Redact** a local trace/file before attaching it to a PR, issue, or support thread.
|
|
315
325
|
- Point **`logs`** / **`tail`** at existing job or service logs to get a **local execution view** without shipping data upstream.
|
|
316
326
|
- **Export** a run to Markdown for a PR, postmortem, or internal thread — use `--redaction-profile share` for share-safe copies, then review before sharing.
|
|
317
327
|
- Keep traces **on disk** while still using enterprise observability elsewhere.
|
|
318
328
|
|
|
319
|
-
## Stable foundation
|
|
329
|
+
## Stable foundation
|
|
320
330
|
|
|
321
|
-
|
|
331
|
+
AgentInspect is the **local-first trace workbench** for TypeScript AI agents:
|
|
322
332
|
|
|
323
333
|
- Instrument runs with `inspectRun` and `step`
|
|
324
|
-
- Write **local JSONL traces** (`schemaVersion: "0.1"`
|
|
334
|
+
- Write and read **local JSONL traces** (`schemaVersion: "0.1"` manual traces remain readable; schema 1.0 persisted rows are the v2 writer target)
|
|
325
335
|
- Inspect with **`list`**, **`view`**, **`clean`**, **`logs`**, **`tail`**, **`export`**, **`diff`**, **`timeline`**, **`stats`**, **`search`**
|
|
326
336
|
|
|
327
|
-
**Stable APIs:** `inspectRun()`, `maybeInspectRun()`, `step()`, `step.llm()`, `step.tool()`, `observe()`, `getCurrentCorrelationMetadata()`.
|
|
337
|
+
**Stable root APIs:** `createInspector()`, `inspectRun()`, `maybeInspectRun()`, `step()`, `step.llm()`, `step.tool()`, `observe()`, `getCurrentCorrelationMetadata()`.
|
|
328
338
|
|
|
329
339
|
Pass `enabled: false` to `inspectRun` for a no-trace passthrough. Use `maybeInspectRun` with `AGENT_INSPECT=1` to toggle tracing in eval or CI — see [docs/API.md](docs/API.md).
|
|
330
340
|
|
|
341
|
+
**Shipped in 2.0.0:** stable root API contract, schema 1.0 persisted writer path, v0.1/v0.2/v1.0 read compatibility, and explicit trace migration workflow. Linked release aligns `agent-inspect`, `@agent-inspect/ai-sdk`, `@agent-inspect/langchain`, `@agent-inspect/tui`, and `@agent-inspect/openai-agents` at **2.0.0**.
|
|
342
|
+
|
|
343
|
+
**Shipped in 1.9.0:** private harness workspace foundation, explain dry-run/local analysis, promoted adapter adoption paths, and the v2 root API slimming plan.
|
|
344
|
+
|
|
331
345
|
**Shipped in 1.8.0:** experimental deterministic checks (`agent-inspect/checks` and `agent-inspect check`), safe-sharing workflows (`scan`, `verify-safe`, safe artifacts), and first public `@agent-inspect/openai-agents` package. Linked release aligns `agent-inspect`, `@agent-inspect/ai-sdk`, `@agent-inspect/langchain`, `@agent-inspect/tui`, and `@agent-inspect/openai-agents` at **1.8.0**.
|
|
332
346
|
|
|
333
347
|
**Shipped in 1.7.0:** experimental `@agent-inspect/ai-sdk` telemetry integration for AI SDK v6 with a local no-network [ai-sdk-local-telemetry recipe](examples/recipes/ai-sdk-local-telemetry/), adapter conformance fixtures, OpenAI Agents/LangGraph support decisions, and local-first adapter docs. Examples keep `recordInputs: false`, `recordOutputs: false`, metadata-only capture, and no upload behavior. Linked release aligns `agent-inspect`, `@agent-inspect/ai-sdk`, `@agent-inspect/langchain`, and `@agent-inspect/tui` at **1.7.0**.
|
|
@@ -336,7 +350,7 @@ Pass `enabled: false` to `inspectRun` for a no-trace passthrough. Use `maybeInsp
|
|
|
336
350
|
|
|
337
351
|
**Shipped in 1.5.0:** non-breaking subpath exports; `what` and `report` CLI; dual-format read path (v0.1 + v0.2 JSONL); [what-report-inspect recipe](examples/recipes/what-report-inspect/). Linked release aligns all three npm packages at **1.5.0**.
|
|
338
352
|
|
|
339
|
-
**Roadmap beyond current release work:**
|
|
353
|
+
**Roadmap beyond current release work:** v2.1 starts the eval/redact utility triangle, followed by reporters/CI, adapter hardening, sessions/MCP telemetry, guardrails, optional viewer/IDE surfaces, and conditional v3 extensibility. See [ROADMAP.md](ROADMAP.md).
|
|
340
354
|
|
|
341
355
|
**Shipped in 1.4.0:** CI artifact recipe ([docs/CI-ARTIFACTS.md](docs/CI-ARTIFACTS.md)); `timeline`, `stats`, and `search` CLI; core helpers `buildRunTimeline`, `buildTraceStats`, `searchTraces`. Linked release aligns all three npm packages at **1.4.0**.
|
|
342
356
|
|
|
@@ -414,6 +428,9 @@ The TUI is available as a separate optional package; its programmatic API is exp
|
|
|
414
428
|
| [examples/recipes/parallel-tools](examples/recipes/parallel-tools) | Parallel tools |
|
|
415
429
|
| [examples/recipes/github-actions-artifact](examples/recipes/github-actions-artifact) | CI trace artifacts |
|
|
416
430
|
| [examples/recipes/deterministic-ci-checks](examples/recipes/deterministic-ci-checks) | v1.8 checks, baseline, and safe CI artifacts |
|
|
431
|
+
| [examples/recipes/eval-local-checks](examples/recipes/eval-local-checks) | v2.1 deterministic local eval checks |
|
|
432
|
+
| [examples/recipes/redact-share-safe-file](examples/recipes/redact-share-safe-file) | v2.1 share-safe local redaction copy |
|
|
433
|
+
| [examples/recipes/eval-ci-artifacts](examples/recipes/eval-ci-artifacts) | v2.1 eval before safe CI artifacts |
|
|
417
434
|
| [examples/recipes/test-reporter-artifacts](examples/recipes/test-reporter-artifacts) | v1.8 Vitest/Jest reporter artifact patterns |
|
|
418
435
|
| [examples/recipes/what-report-inspect](examples/recipes/what-report-inspect/) | `what` + `report` inspection |
|
|
419
436
|
| [examples/recipes/runtime-and-ingestion](examples/recipes/runtime-and-ingestion/) | v1.6 runtime writers + universal ingestion |
|
|
@@ -22,7 +22,7 @@ Executable shared assertions live in `packages/core/test/adapter-executable-conf
|
|
|
22
22
|
| --- | --- | --- | --- | --- |
|
|
23
23
|
| AI SDK | `@agent-inspect/ai-sdk` | implemented experimental; v1.8 correctness hardening pending | AI SDK telemetry integration | optional package peer dependency |
|
|
24
24
|
| LangChain | `@agent-inspect/langchain` | implemented experimental | explicit callback | optional package peer dependency |
|
|
25
|
-
| OpenAI Agents JS | `@agent-inspect/openai-agents` | implemented experimental;
|
|
25
|
+
| OpenAI Agents JS | `@agent-inspect/openai-agents` | implemented experimental; public package, with v1.9 publication recovery tracked separately | `setTraceProcessors()` replacement | optional package peer dependency |
|
|
26
26
|
| LangGraph | `@agent-inspect/langchain` | fixture-backed through LangChain callback | explicit LangChain callback | existing LangChain adapter first |
|
|
27
27
|
|
|
28
28
|
## Required defaults
|
package/docs/ADAPTERS.md
CHANGED
|
@@ -128,8 +128,12 @@ await agent.invoke(input, { callbacks: [callback] });
|
|
|
128
128
|
npx agent-inspect list --dir ./.agent-inspect
|
|
129
129
|
npx agent-inspect view <run-id> --dir ./.agent-inspect
|
|
130
130
|
npx agent-inspect export <run-id> --format markdown --redaction-profile share
|
|
131
|
+
npx agent-inspect eval <run-id> --dir ./.agent-inspect --require-success --json
|
|
132
|
+
npx agent-inspect redact ./.agent-inspect/<trace-file>.jsonl --profile share --json
|
|
131
133
|
```
|
|
132
134
|
|
|
135
|
+
`eval` and `redact` read local adapter traces only. They do not call model providers, upload traces, or loosen the adapter metadata-only capture defaults.
|
|
136
|
+
|
|
133
137
|

|
|
134
138
|
|
|
135
139
|
*Synthetic demo — [examples/08-langchain-adapter](../../examples/08-langchain-adapter/README.md).*
|
package/docs/API.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# API
|
|
1
|
+
# API
|
|
2
2
|
|
|
3
3
|
This document describes the **public TypeScript API surface** of AgentInspect and classifies each area as **stable** or **experimental**.
|
|
4
4
|
|
|
@@ -6,13 +6,14 @@ AgentInspect is a **local-first execution-tree debugger**. It is not a SaaS, not
|
|
|
6
6
|
|
|
7
7
|
## 1. Stability policy
|
|
8
8
|
|
|
9
|
-
- **Stable**: intended to
|
|
10
|
-
- **Experimental**: available for adoption, but subject to refinement (including naming/shape changes) before a future stability declaration.
|
|
9
|
+
- **Stable**: intended to stay compatible within the current major version.
|
|
10
|
+
- **Experimental**: available for adoption, but subject to refinement (including naming/shape changes) before a future stability declaration.
|
|
11
11
|
|
|
12
12
|
Use the root import for stable beginner APIs. Use subpaths for advanced, experimental, or lower-level workflows.
|
|
13
13
|
|
|
14
14
|
```ts
|
|
15
15
|
import {
|
|
16
|
+
createInspector,
|
|
16
17
|
observe,
|
|
17
18
|
inspectRun,
|
|
18
19
|
maybeInspectRun,
|
|
@@ -21,7 +22,7 @@ import {
|
|
|
21
22
|
} from "agent-inspect";
|
|
22
23
|
```
|
|
23
24
|
|
|
24
|
-
**
|
|
25
|
+
**v2 root API contract:** do not add new root exports casually. Advanced, experimental, and lower-level helpers belong on the subpath where the API lives. The stable root value set is:
|
|
25
26
|
|
|
26
27
|
```ts
|
|
27
28
|
import {
|
|
@@ -34,7 +35,7 @@ import {
|
|
|
34
35
|
} from "agent-inspect";
|
|
35
36
|
```
|
|
36
37
|
|
|
37
|
-
**
|
|
38
|
+
**Subpath exports:** Subpaths (`/logs`, `/exporters`, `/persisted`, `/diff`, `/advanced`, `/writers`, `/readers`, `/checks`) narrow the import surface for experimental and advanced APIs. Design history: [API-BOUNDARY-V1.5.md](./implementation/API-BOUNDARY-V1.5.md).
|
|
38
39
|
|
|
39
40
|
```ts
|
|
40
41
|
import { parseLogsToTrees } from "agent-inspect/logs";
|
|
@@ -44,15 +45,15 @@ import { openTrace } from "agent-inspect/readers";
|
|
|
44
45
|
import { runTraceChecks } from "agent-inspect/checks";
|
|
45
46
|
import { diffTraceEvents } from "agent-inspect/diff";
|
|
46
47
|
import { traceEventsToPersistedInspectEvents } from "agent-inspect/persisted";
|
|
47
|
-
import {
|
|
48
|
+
import { createInspectorRuntime } from "agent-inspect/advanced";
|
|
48
49
|
```
|
|
49
50
|
|
|
50
51
|
Notes:
|
|
51
52
|
|
|
52
|
-
- The core guarantee
|
|
53
|
+
- The core guarantee is **stable local debugging**: manual tracing + CLI inspection.
|
|
53
54
|
- Export formats (OpenInference / OTLP JSON) are **local-only** and **compatibility-oriented**. They do **not** upload anywhere.
|
|
54
|
-
- There are **zero production sinks
|
|
55
|
-
- Advanced
|
|
55
|
+
- There are **zero production sinks**; sink/uploader APIs are not stable.
|
|
56
|
+
- Advanced APIs are available from `agent-inspect/advanced`, `agent-inspect/readers`, `agent-inspect/writers`, `agent-inspect/checks`, `agent-inspect/diff`, `agent-inspect/exporters`, `agent-inspect/logs`, and `agent-inspect/persisted`.
|
|
56
57
|
|
|
57
58
|
## 2. Stable core APIs (manual tracing)
|
|
58
59
|
|
|
@@ -78,14 +79,14 @@ import {
|
|
|
78
79
|
- **`maxEventBytes`**: max UTF-8 bytes per serialized JSONL event (default `65536`). Oversized events are truncated; instrumentation never throws into user code.
|
|
79
80
|
- **Correlation metadata (v1.3.0+):** optional `correlationId`, `requestId`, `decisionId`, and `groupId` strings. When set, they are written on `run_started.metadata` (not on every step). Top-level correlation options override the same keys in `options.metadata`. Useful for eval cases, CI job IDs, request tracing, and `stats --correlation-id` / `--group-id`. They are **metadata only** — they do not replace `runId`. Treat sensitive IDs as trace data before sharing exports.
|
|
80
81
|
- **`maybeInspectRun(name, fn, options?)`**: same as `inspectRun` when tracing is enabled; otherwise passthrough. Enablement: explicit **`options.enabled`** wins; when omitted, reads **`AGENT_INSPECT`** (`1`, `true`, `yes`, `on`, `enabled` — case-insensitive). Unset or other values disable tracing. Use in eval harnesses, CI, or jobs where tracing should be toggled by environment.
|
|
81
|
-
- **`isAgentInspectEnabled(value?)`**: returns whether a string (or `process.env.AGENT_INSPECT`) matches an enable token.
|
|
82
|
+
- **`isAgentInspectEnabled(value?)`**: advanced helper available from `agent-inspect/advanced`; returns whether a string (or `process.env.AGENT_INSPECT`) matches an enable token.
|
|
82
83
|
- **`step(name, fn, options?)`**: traces a named unit of work inside `inspectRun` (`step_started` / `step_completed`). Step `metadata` inherits the parent run's redaction and size-bound settings.
|
|
83
84
|
- **`step.llm(model, fn)`**: convenience wrapper (`type: "llm"`, `metadata.model`).
|
|
84
85
|
- **`step.tool(toolName, fn)`**: convenience wrapper (`type: "tool"`, `metadata.toolName`).
|
|
85
86
|
- **`observe(agent, options?)`**: proxy wrapper that traces top-level `run` / `execute` / `invoke` methods via `inspectRun`.
|
|
86
87
|
- **`getCurrentCorrelationMetadata()`**: returns active run correlation fields (`correlationId`, `requestId`, `decisionId`, `groupId`) inside `inspectRun` / `maybeInspectRun`; `undefined` outside a traced run or when none were set.
|
|
87
88
|
- **`RedactionProfile`**: `"local" | "share" | "strict"` — see `redactionProfile` on `InspectRunOptions` and `ExportOptions`.
|
|
88
|
-
- **`resolveRedactionProfile(profile?)`**: resolves profile extra keys and metadata caps
|
|
89
|
+
- **`resolveRedactionProfile(profile?)`**: advanced helper available from `agent-inspect/advanced`; resolves profile extra keys and metadata caps for integrations.
|
|
89
90
|
|
|
90
91
|
## 3. Stable local inspection APIs
|
|
91
92
|
|
|
@@ -328,29 +329,29 @@ The processor records local v0.2 persisted events for trace/run, agent, generati
|
|
|
328
329
|
|
|
329
330
|
## 15. Experimental persisted-event foundation (v1.2.0)
|
|
330
331
|
|
|
331
|
-
These helpers expose the **source-agnostic `PersistedInspectEvent` model
|
|
332
|
+
These helpers expose the **source-agnostic `PersistedInspectEvent` model**. They are **local-only** and support v0.2 compatibility plus the stable schema 1.0 persisted contract.
|
|
332
333
|
|
|
333
334
|
Import from `agent-inspect/persisted`:
|
|
334
335
|
|
|
335
336
|
| API | Role |
|
|
336
337
|
| --- | ---- |
|
|
337
|
-
| `isPersistedInspectEvent` | Runtime validator for
|
|
338
|
+
| `isPersistedInspectEvent` | Runtime validator for supported persisted events |
|
|
338
339
|
| `traceEventToPersistedInspectEvent` | Convert one v0.1 `TraceEvent` |
|
|
339
340
|
| `traceEventsToPersistedInspectEvents` | Batch v0.1 → v0.2 |
|
|
340
341
|
| `inspectEventToPersistedInspectEvent` | Convert one in-memory `InspectEvent` |
|
|
341
|
-
| `inspectEventsToPersistedInspectEvents` | Batch `InspectEvent` →
|
|
342
|
-
| `persistedInspectEventToInspectEvent` | Convert one
|
|
343
|
-
| `persistedInspectEventsToInspectEvents` | Batch
|
|
344
|
-
| `persistedInspectEventsToRunTrees` | Build `InspectRunTree[]` from
|
|
342
|
+
| `inspectEventsToPersistedInspectEvents` | Batch `InspectEvent` → persisted events |
|
|
343
|
+
| `persistedInspectEventToInspectEvent` | Convert one persisted event to `InspectEvent` |
|
|
344
|
+
| `persistedInspectEventsToInspectEvents` | Batch persisted events → `InspectEvent` |
|
|
345
|
+
| `persistedInspectEventsToRunTrees` | Build `InspectRunTree[]` from persisted events (via `TreeBuilder`) |
|
|
345
346
|
| `traceEventsToPersistedRunTrees` | v0.1 `TraceEvent[]` → persisted model → trees |
|
|
346
347
|
|
|
347
348
|
Related types: `PersistedInspectEvent`, `PersistedEventSourceType`, `PersistedEventStatus`, `TraceEventToPersistedOptions`, `InspectEventToPersistedOptions`, `PersistedToInspectEventOptions`, `PersistedTreeBridgeOptions`.
|
|
348
349
|
|
|
349
350
|
**Notes:**
|
|
350
351
|
|
|
351
|
-
- Manual trace **writing** remains `schemaVersion: "0.1"`.
|
|
352
|
-
-
|
|
353
|
-
- Inspection read paths normalize v0.1
|
|
352
|
+
- Manual global trace **writing** remains `schemaVersion: "0.1"`.
|
|
353
|
+
- `createInspector()` and built-in persisted writer paths target schema 1.0 rows.
|
|
354
|
+
- Inspection read paths normalize v0.1, v0.2, and v1.0 JSONL for local CLI/API use. v0.2 remains a compatibility foundation.
|
|
354
355
|
|
|
355
356
|
## 16. Local observability helpers (v1.4.0+)
|
|
356
357
|
|
|
@@ -373,7 +374,7 @@ Report redaction profiles are key-based safeguards applied to the complete rende
|
|
|
373
374
|
|
|
374
375
|
## 18. Experimental trace writers (v1.6)
|
|
375
376
|
|
|
376
|
-
Trace writers are the
|
|
377
|
+
Trace writers are the local persistence contract for tests, adapters, and `createInspector()` workflows.
|
|
377
378
|
|
|
378
379
|
Import from `agent-inspect/writers`:
|
|
379
380
|
|
|
@@ -395,7 +396,7 @@ import type {
|
|
|
395
396
|
```
|
|
396
397
|
|
|
397
398
|
- **`TraceWriter`**: async `write(event)`, optional `flush()`, optional `close()`, optional `getStats()`.
|
|
398
|
-
- **`fileWriter({ dir?, filePath? })`**: appends
|
|
399
|
+
- **`fileWriter({ dir?, filePath? })`**: appends `PersistedInspectEvent` JSONL rows to local disk. `createInspector()` emits schema 1.0 rows by default; compatibility adapters may still pass readable v0.2 rows. By default it derives one file per `event.runId`; `filePath` writes all events to an explicit local file. Filesystem and serialization failures are reflected in writer stats instead of being thrown into application code.
|
|
399
400
|
- **`bufferedFileWriter({ dir?, filePath?, maxQueueSize?, flushIntervalMs?, maxBatchSize?, overflow? })`**: buffers local JSONL writes with bounded queue behavior. Overflow supports `drop-oldest` and `drop-newest`; neither mode throws into application code.
|
|
400
401
|
- **`compositeWriter([...writers])`**: fans out events to multiple explicit local/custom writers. A failing child writer does not prevent other children from receiving events; failures are reflected in composite stats.
|
|
401
402
|
- **`memoryWriter()`**: stores cloned `PersistedInspectEvent` rows in memory for tests, adapter fixtures, and eval harnesses.
|
|
@@ -405,12 +406,12 @@ No network writer or vendor sink exists in this package.
|
|
|
405
406
|
|
|
406
407
|
## 19. Experimental inspector API/runtime (v1.6)
|
|
407
408
|
|
|
408
|
-
`createInspector()` is the
|
|
409
|
+
`createInspector()` is the public instance API for local-first tracing with explicit writers. It owns an instance-specific runtime context, records schema 1.0 persisted inspect events, preserves application return values/errors, and exposes diagnostics plus deterministic `flush()`/`close()` lifecycle hooks.
|
|
409
410
|
|
|
410
|
-
Import from `agent-inspect
|
|
411
|
+
Import from `agent-inspect`:
|
|
411
412
|
|
|
412
413
|
```ts
|
|
413
|
-
import { createInspector } from "agent-inspect
|
|
414
|
+
import { createInspector } from "agent-inspect";
|
|
414
415
|
import { memoryWriter } from "agent-inspect/writers";
|
|
415
416
|
|
|
416
417
|
const writer = memoryWriter();
|
|
@@ -441,9 +442,9 @@ Public methods:
|
|
|
441
442
|
|
|
442
443
|
`traceDir` and `silent` on `createInspector()` are context metadata for compatibility with existing helpers. They do not configure persistence or terminal output. Prefer writer-owned output configuration such as `fileWriter({ dir })` or `fileWriter({ filePath })`.
|
|
443
444
|
|
|
444
|
-
`createInspectorRuntime()` is
|
|
445
|
+
`createInspectorRuntime()` is available from `agent-inspect/advanced` as the low-level isolation primitive. Most users should prefer `createInspector()` and `inspector.getDiagnostics()`.
|
|
445
446
|
|
|
446
|
-
|
|
447
|
+
The low-level runtime helpers remain on `agent-inspect/advanced`. These APIs do not add a default network writer or vendor sink.
|
|
447
448
|
|
|
448
449
|
## 20. Experimental trace readers (v1.6)
|
|
449
450
|
|
|
@@ -496,14 +497,54 @@ The checks API is experimental in v1.x. The `agent-inspect check` CLI uses this
|
|
|
496
497
|
|
|
497
498
|
Recipes: [deterministic-ci-checks](../examples/recipes/deterministic-ci-checks/README.md) for check/baseline/artifact workflows, and [test-reporter-artifacts](../examples/recipes/test-reporter-artifacts/README.md) for Vitest/Jest reporter configuration patterns.
|
|
498
499
|
|
|
499
|
-
## 22. Experimental
|
|
500
|
+
## 22. Experimental `@agent-inspect/eval` APIs (v2.1)
|
|
501
|
+
|
|
502
|
+
`@agent-inspect/eval` is an optional package for deterministic local evals over existing traces. It consumes normalized reader output or local trace paths, returns stable JSON-compatible results, and does not call model providers, upload traces, replay agents, or create hosted datasets.
|
|
503
|
+
|
|
504
|
+
Import from `@agent-inspect/eval`:
|
|
505
|
+
|
|
506
|
+
```ts
|
|
507
|
+
import { checks, evalRun, renderEvalMarkdown } from "@agent-inspect/eval";
|
|
508
|
+
```
|
|
509
|
+
|
|
510
|
+
- **`evalRun(input, options?)`**: runs selected eval rules over a local trace path or `TraceReadResult`.
|
|
511
|
+
- **`checks`**: built-in deterministic rule factories for run status, tool usage, duration, depth, retries, token totals, failed steps, retrieval-before-generation, decision metadata, context overlap, quote overlap, citation presence, required source IDs, answer length bounds, and banned unsupported phrases.
|
|
512
|
+
- **`renderEvalMarkdown(result)`**: renders a deterministic Markdown summary suitable for local CI logs, PR text, or artifact files after review.
|
|
513
|
+
- Result types include **`EvalRunResult`**, **`EvalFinding`**, **`EvalDiagnostic`**, and **`EvalRule`**.
|
|
514
|
+
|
|
515
|
+
Findings are designed for CI output: they include rule IDs, expected/actual structural summaries, and evidence paths. They should not include raw prompt, answer, context, request/response, header, API key, secret, or full tool payload values.
|
|
516
|
+
|
|
517
|
+
CLI wrapper: `agent-inspect eval <trace-path-or-run-id> --require-success --json`.
|
|
518
|
+
|
|
519
|
+
Recipes: [eval-local-checks](../examples/recipes/eval-local-checks/README.md) and [eval-ci-artifacts](../examples/recipes/eval-ci-artifacts/README.md).
|
|
520
|
+
|
|
521
|
+
## 23. Experimental `@agent-inspect/redact` APIs (v2.1)
|
|
522
|
+
|
|
523
|
+
`@agent-inspect/redact` is an optional package for reusable local redaction. It powers the root CLI `redact` workflow and shared trace-safety integrations. Redaction operates on local values/files and returns a redacted copy; it does not mutate the source object, upload content, or claim compliance-grade DLP.
|
|
524
|
+
|
|
525
|
+
Import from `@agent-inspect/redact`:
|
|
526
|
+
|
|
527
|
+
```ts
|
|
528
|
+
import { createRedactor, redact } from "@agent-inspect/redact";
|
|
529
|
+
```
|
|
530
|
+
|
|
531
|
+
- **`redact(value, options?)`**: returns `{ value, findings, redacted, profile }` for a redacted copy.
|
|
532
|
+
- **`createRedactor(options?)`**: creates a reusable redactor with profile, custom detectors, and custom rules.
|
|
533
|
+
- **Profiles**: `local`, `share`, and `strict`.
|
|
534
|
+
- **Findings**: detector id, path, action, severity, and bounded preview metadata where applicable.
|
|
535
|
+
|
|
536
|
+
CLI wrapper: `agent-inspect redact <trace-or-file> --profile share --json`.
|
|
537
|
+
|
|
538
|
+
Recipe: [redact-share-safe-file](../examples/recipes/redact-share-safe-file/README.md).
|
|
539
|
+
|
|
540
|
+
## 24. Experimental local explain APIs (v1.9)
|
|
500
541
|
|
|
501
542
|
`buildLocalExplanation()` creates a deterministic local explanation payload from a reader-selected `InspectRunTree`. It performs no network I/O, does not call model providers, and separates observed facts from deterministic inference labels.
|
|
502
543
|
|
|
503
544
|
Import from `agent-inspect`:
|
|
504
545
|
|
|
505
546
|
```ts
|
|
506
|
-
import { buildLocalExplanation } from "agent-inspect";
|
|
547
|
+
import { buildLocalExplanation } from "agent-inspect/advanced";
|
|
507
548
|
```
|
|
508
549
|
|
|
509
550
|
- **`buildLocalExplanation(run, options?)`**:
|
|
@@ -521,7 +562,7 @@ Provider design gate:
|
|
|
521
562
|
- Provider prompts must use redacted facts only, label inferred claims, and must not request raw chain-of-thought.
|
|
522
563
|
- Provider packages or SDKs must not become root/core runtime dependencies.
|
|
523
564
|
|
|
524
|
-
##
|
|
565
|
+
## 25. Experimental `@agent-inspect/harness` APIs
|
|
525
566
|
|
|
526
567
|
`@agent-inspect/harness` is a private experimental workspace package during the v1.9 release train. It provides a no-framework fixture runner for local targets and recipes; first public package publication remains a manual maintainer gate.
|
|
527
568
|
|
|
@@ -547,23 +588,23 @@ The harness package does not add root/core dependencies, does not upload traces,
|
|
|
547
588
|
|
|
548
589
|
Recipes: [harness-basic](../examples/recipes/harness-basic/README.md) and [harness-adapter-local](../examples/recipes/harness-adapter-local/README.md).
|
|
549
590
|
|
|
550
|
-
##
|
|
591
|
+
## 26. Deprecated APIs
|
|
551
592
|
|
|
552
593
|
No deprecated APIs are declared as of 1.4.0.
|
|
553
594
|
|
|
554
|
-
##
|
|
595
|
+
## 27. Removal / deprecation policy
|
|
555
596
|
|
|
556
|
-
- Stable APIs are not removed
|
|
597
|
+
- Stable APIs are not removed within the current major version.
|
|
557
598
|
- If removal is necessary, the API should be **deprecated** first, documented, and kept for a reasonable window (target: at least one minor line) unless security requires faster action.
|
|
558
599
|
|
|
559
|
-
##
|
|
600
|
+
## 28. Backward compatibility policy
|
|
560
601
|
|
|
561
602
|
- Manual trace JSONL (`schemaVersion: "0.1"`) remains readable.
|
|
562
603
|
- Additive schema changes are allowed in minor versions.
|
|
563
604
|
- Breaking changes require a major version.
|
|
564
605
|
- Unknown fields should be ignored where safe.
|
|
565
606
|
|
|
566
|
-
##
|
|
607
|
+
## 29. Examples
|
|
567
608
|
|
|
568
609
|
### Minimal manual trace
|
|
569
610
|
|