agent-inspect 1.2.0 → 1.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +41 -4
- package/README.md +41 -38
- package/docs/ADAPTERS.md +122 -21
- package/docs/API.md +25 -7
- package/docs/ARCHITECTURE.md +1 -1
- package/docs/CLI.md +71 -2
- package/docs/EXPORTS.md +80 -8
- package/docs/GETTING-STARTED.md +29 -1
- package/docs/KNOWN-ISSUES.md +6 -0
- package/docs/LIMITATIONS.md +17 -0
- package/docs/SCHEMA.md +6 -4
- package/package.json +1 -1
- package/packages/cli/dist/index.cjs +1019 -14
- package/packages/cli/dist/index.cjs.map +1 -1
- package/packages/cli/dist/index.mjs +1019 -14
- package/packages/cli/dist/index.mjs.map +1 -1
- package/packages/core/dist/index.cjs +890 -14
- package/packages/core/dist/index.cjs.map +1 -1
- package/packages/core/dist/index.d.cts +176 -3
- package/packages/core/dist/index.d.ts +176 -3
- package/packages/core/dist/index.mjs +881 -15
- package/packages/core/dist/index.mjs.map +1 -1
package/docs/GETTING-STARTED.md
CHANGED
|
@@ -60,6 +60,24 @@ AGENT_INSPECT=1 node eval-runner.mjs
|
|
|
60
60
|
|
|
61
61
|
Enable tokens: `1`, `true`, `yes`, `on`, `enabled` (case-insensitive). Explicit `enabled: true | false` in options overrides the env var.
|
|
62
62
|
|
|
63
|
+
### Correlation metadata (v1.3.0+)
|
|
64
|
+
|
|
65
|
+
Attach optional correlation fields to `run_started` metadata — useful for eval cases, CI job IDs, and future stats views. They are **metadata only**, not run IDs. Review before sharing exports.
|
|
66
|
+
|
|
67
|
+
```ts
|
|
68
|
+
await inspectRun(
|
|
69
|
+
"support-agent",
|
|
70
|
+
async () => runAgent(),
|
|
71
|
+
{
|
|
72
|
+
correlationId: "eval-suite-2026-06",
|
|
73
|
+
requestId: "req-abc123",
|
|
74
|
+
groupId: "ci-job-42",
|
|
75
|
+
}
|
|
76
|
+
);
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
Inside a traced run, adapters can read active fields via `getCurrentCorrelationMetadata()`.
|
|
80
|
+
|
|
63
81
|
### Install compatibility
|
|
64
82
|
|
|
65
83
|
If `import` or `require` fails after install, see [KNOWN-ISSUES.md — Common install/runtime compatibility checks](./KNOWN-ISSUES.md#common-installruntime-compatibility-checks).
|
|
@@ -109,7 +127,14 @@ agent-inspect export minimal-success --dir fixtures/traces --format markdown
|
|
|
109
127
|
agent-inspect export minimal-success --dir fixtures/traces --format openinference --validate
|
|
110
128
|
```
|
|
111
129
|
|
|
112
|
-
|
|
130
|
+
Share-safe copy for a PR or issue (v1.3.0+):
|
|
131
|
+
|
|
132
|
+
```bash
|
|
133
|
+
agent-inspect export minimal-success --dir fixtures/traces \
|
|
134
|
+
--format markdown --redaction-profile share
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
Exports are **local-only** and do not upload anywhere. Review output before sharing — see [SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md).
|
|
113
138
|
|
|
114
139
|
## 8. Diff two runs
|
|
115
140
|
|
|
@@ -151,6 +176,9 @@ agent-inspect view <runId> --tui
|
|
|
151
176
|
- [docs/API.md](./API.md)
|
|
152
177
|
- [docs/CLI.md](./CLI.md)
|
|
153
178
|
- [docs/SCHEMA.md](./SCHEMA.md)
|
|
179
|
+
- [docs/EXPORTS.md](./EXPORTS.md)
|
|
180
|
+
- [docs/SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md)
|
|
181
|
+
- [docs/ADAPTERS.md](./ADAPTERS.md)
|
|
154
182
|
- [docs/LOGGING-PLAYBOOK.md](./LOGGING-PLAYBOOK.md)
|
|
155
183
|
- [docs/COMPARE.md](./COMPARE.md)
|
|
156
184
|
- [docs/LOG-TO-TREE-QUICKSTART.md](./LOG-TO-TREE-QUICKSTART.md)
|
package/docs/KNOWN-ISSUES.md
CHANGED
|
@@ -16,6 +16,12 @@ AgentInspect is **local-first** and **CLI-first**. These behaviors are intention
|
|
|
16
16
|
|
|
17
17
|
- **Vendor sinks** (hosted dashboards, Langfuse/Braintrust/New Relic/Datadog native uploads, OTLP gRPC streaming, etc.) are **not implemented** in the core packages described here.
|
|
18
18
|
- **LangChain adapter** captures **metadata-oriented** signals by default; it does not replace full framework observability.
|
|
19
|
+
- **LangChain `stream: true`** records chunk counts and timing only — not a full token replay. Per-token JSONL events are not emitted.
|
|
20
|
+
- **Correlation metadata** (`correlationId`, `requestId`, `decisionId`, `groupId`) is written on `run_started` but **CLI list/view does not filter by correlation fields** yet.
|
|
21
|
+
|
|
22
|
+
## Structured log parsing
|
|
23
|
+
|
|
24
|
+
- Ingest requires **line-delimited JSON** or a **recoverable JSON payload** in the log line (log4js-style is best-effort). Arbitrary printf-style text without JSON is not supported.
|
|
19
25
|
|
|
20
26
|
## UI / replay
|
|
21
27
|
|
package/docs/LIMITATIONS.md
CHANGED
|
@@ -9,12 +9,22 @@ This document states what AgentInspect **does not** provide today. It complement
|
|
|
9
9
|
- **No vendor upload pipeline**: no built-in Langfuse/Braintrust/New Relic/Datadog direct exporters as live sinks.
|
|
10
10
|
- **No automatic universal instrumentation** of every framework: integration is explicit (manual traces, log ingest, optional adapters).
|
|
11
11
|
|
|
12
|
+
## Correlation metadata (v1.3.0)
|
|
13
|
+
|
|
14
|
+
- **`list` / `view` / `export`** do not filter by correlation fields yet — `stats --correlation-id` / `--group-id` and `search` provide targeted read paths; full CLI filtering remains incremental.
|
|
15
|
+
|
|
12
16
|
## Persisted event model (v1.2.0 foundation)
|
|
13
17
|
|
|
14
18
|
- **v0.2 is not the default persisted trace file format.** `inspectRun()` / `step()` still write `schemaVersion: "0.1"` JSONL.
|
|
15
19
|
- **CLI commands** (`list`, `view`, `export`, `diff`, `logs`, `tail`) still primarily operate on current v0.1 trace and log paths.
|
|
16
20
|
- **v0.2 read/write integration** (dual-format storage, CLI consumption) is future work — v1.2.0 ships in-memory converters and canonical fixtures only.
|
|
17
21
|
|
|
22
|
+
## LangChain streaming (v1.3.0)
|
|
23
|
+
|
|
24
|
+
- **Metadata-focused only** — `stream: true` records chunk counts, timing, and optional bounded previews; it is **not** a replay/cassette system.
|
|
25
|
+
- **No full token stream storage by default** — even with `stream: true`, `capture: "metadata-only"` does not persist raw streamed text.
|
|
26
|
+
- **No per-token JSONL events** — streaming does not emit one trace line per token.
|
|
27
|
+
|
|
18
28
|
## Data fidelity
|
|
19
29
|
|
|
20
30
|
- **No full prompt/output capture by default** for manual traces (by design: safety and PII risk).
|
|
@@ -23,6 +33,7 @@ This document states what AgentInspect **does not** provide today. It complement
|
|
|
23
33
|
|
|
24
34
|
## Trace safety bounds
|
|
25
35
|
|
|
36
|
+
- **Redaction profiles** (`local`, `share`, `strict`) are key-based presets — not compliance-grade PII detection. Review exports before sharing even with `--redaction-profile strict`.
|
|
26
37
|
- **Default metadata redaction** covers common sensitive keys only (exact key match, case-insensitive). Custom secret field names are not redacted unless you add rules via `redact: { rules: [...] }`.
|
|
27
38
|
- **Metadata truncation** applies to string values and nested structures; very large metadata may be replaced with a truncation marker when `maxEventBytes` is exceeded (default 64 KiB per JSONL line).
|
|
28
39
|
- **Redaction is not encryption.** Local trace files remain readable on disk; treat `.agent-inspect-runs/` like any developer artifact that may contain operational data.
|
|
@@ -37,6 +48,12 @@ This document states what AgentInspect **does not** provide today. It complement
|
|
|
37
48
|
|
|
38
49
|
- **No cost engine**: no pricing tables, invoice-grade usage, or provider billing reconciliation.
|
|
39
50
|
|
|
51
|
+
## Local observability commands (v1.4.0)
|
|
52
|
+
|
|
53
|
+
- **`timeline`**, **`stats`**, and **`search`** scan local JSONL files — no database index; large directories may be slow.
|
|
54
|
+
- **`search`** is deterministic exact/contains matching only — no semantic or fuzzy search.
|
|
55
|
+
- **`stats`** is local file aggregation — not production fleet analytics.
|
|
56
|
+
|
|
40
57
|
## Scale
|
|
41
58
|
|
|
42
59
|
- Designed for **developer machines** and **inner-loop debugging**, not petabyte log warehouses.
|
package/docs/SCHEMA.md
CHANGED
|
@@ -1,4 +1,4 @@
|
|
|
1
|
-
# Schema (AgentInspect 1.
|
|
1
|
+
# Schema (AgentInspect 1.x)
|
|
2
2
|
|
|
3
3
|
This document describes the **persisted manual trace JSONL schema** and the **log-derived normalized model** used by AgentInspect.
|
|
4
4
|
|
|
@@ -25,7 +25,7 @@ Manual trace events use:
|
|
|
25
25
|
|
|
26
26
|
- **`schemaVersion: "0.1"`**
|
|
27
27
|
|
|
28
|
-
Existing `0.1` traces remain readable
|
|
28
|
+
Existing `0.1` traces remain readable across AgentInspect 1.x.
|
|
29
29
|
|
|
30
30
|
### 2.3 TraceEvent union
|
|
31
31
|
|
|
@@ -129,9 +129,11 @@ Unknown status must **not** be treated as success.
|
|
|
129
129
|
## 6. Metadata policy
|
|
130
130
|
|
|
131
131
|
- Runs may include `metadata` on `run_started`.
|
|
132
|
+
- **Correlation metadata (v1.3.0+):** optional `correlationId`, `requestId`, `decisionId`, and `groupId` on `run_started.metadata` when passed via `inspectRun` / `maybeInspectRun` options. Event names remain unchanged (`run_started`, `step_started`, `step_completed`, `run_completed`). CLI list/view does not filter by correlation fields yet.
|
|
132
133
|
- Steps may include `metadata` on `step_started`.
|
|
133
134
|
- Manual traces intentionally avoid full prompt/output capture by default.
|
|
134
135
|
- **Redaction (default on):** before disk, `inspectRun` / `step` redact sensitive keys using the shared `Redactor` defaults (`authorization`, `cookie`, `token`, `apiKey`, `password`, `secret`, `email`). Opt out with `redact: false`.
|
|
136
|
+
- **Redaction profiles (v1.3.0+):** optional `redactionProfile` (`local`, `share`, `strict`) adds preset extra keys and tighter metadata bounds for trace writing. Key-based only — not compliance-grade DLP. `export --redaction-profile` applies profiles to export copies without mutating source JSONL.
|
|
135
137
|
- **Size bounds:** long string values are truncated (`maxMetadataValueLength`, default 2000; preview-like keys use `maxPreviewLength`, default 500). Serialized events are capped at `maxEventBytes` (default 65536 UTF-8 bytes). If still too large, `metadata` may be replaced with `{ truncated: true, reason: "maxEventBytes", originalApproxBytes: number }`. Required event fields are never removed.
|
|
136
138
|
|
|
137
139
|
## 7. Additive fields and unknown fields
|
|
@@ -149,7 +151,7 @@ Manual trace reading:
|
|
|
149
151
|
|
|
150
152
|
## 9. Backward compatibility
|
|
151
153
|
|
|
152
|
-
- v0.1 JSONL traces remain readable
|
|
154
|
+
- v0.1 JSONL traces remain readable across AgentInspect 1.x.
|
|
153
155
|
- No automatic migrations or rewriting of old files.
|
|
154
156
|
|
|
155
157
|
## 10. Breaking change policy
|
|
@@ -234,7 +236,7 @@ Canonical samples: `fixtures/traces-v0.2/*.jsonl` (validated by `pnpm fixtures:c
|
|
|
234
236
|
| `tokenUsage` | no | `{ input?, output?, total? }` when known |
|
|
235
237
|
| `trace` | no | Optional `{ traceId?, spanId?, parentSpanId? }` for future OTel alignment |
|
|
236
238
|
|
|
237
|
-
Programmatic helpers: see [API.md](./API.md) §
|
|
239
|
+
Programmatic helpers: see [API.md](./API.md) §11 (experimental persisted-event foundation).
|
|
238
240
|
|
|
239
241
|
## 15. Migration notes
|
|
240
242
|
|