agent-inspect 3.5.3 → 3.5.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -1,5 +1,7 @@
1
1
  # Safe trace sharing checklist
2
2
 
3
+ **Docs site:** [https://agentinspect.vercel.app/docs/safe-sharing/](https://agentinspect.vercel.app/docs/safe-sharing/)
4
+
3
5
  AgentInspect traces, log-ingest outputs, and exports are local files. They may still contain sensitive metadata that you attached manually, collected from logs, or included through optional preview settings. Use this checklist before sharing an artifact in a GitHub issue, Discussion, PR, support thread, or public post.
4
6
 
5
7
  This guide is practical sharing guidance, not a guarantee that any artifact is safe to publish. Redaction profiles are **key-based safeguards**, not compliance-grade DLP.
@@ -4,7 +4,7 @@ Curated terminal recordings and static diagrams for AgentInspect **3.5.x**. They
4
4
 
5
5
  **Synthetic output only:** demos use committed [fixtures](../fixtures/README.md), [examples](../examples/README.md), and recipes. No external LLM calls or API keys.
6
6
 
7
- **npm note:** GIFs live in `docs/assets/demos/` for GitHub documentation. They are **not** shipped in the `agent-inspect` npm tarball.
7
+ **npm note:** GIFs live in `docs/assets/demos/` for GitHub documentation. They are **not** shipped in the `agent-inspect` npm tarball. Brand SVGs used by the root README **are** included in the npm package.
8
8
 
9
9
  **Re-record guide:** [assets/demos/RECORDING.md](assets/demos/RECORDING.md)
10
10
  **Maintainer audit:** [implementation/VISUAL-DEMO-AUDIT.md](implementation/VISUAL-DEMO-AUDIT.md)
@@ -12,6 +12,18 @@ Curated terminal recordings and static diagrams for AgentInspect **3.5.x**. They
12
12
 
13
13
  ---
14
14
 
15
+ ## README brand assets (v3.5.4)
16
+
17
+ | Asset | Use |
18
+ | ----- | --- |
19
+ | [agent-inspect-logo.svg](assets/agent-inspect-logo.svg) | Light-mode wordmark |
20
+ | [agent-inspect-logo-dark.svg](assets/agent-inspect-logo-dark.svg) | Dark-mode wordmark |
21
+ | [readme-product-loop.svg](assets/readme-product-loop.svg) | Capture → inspect → check → redact |
22
+
23
+ Root README embeds the logo and product-loop SVG only. Terminal GIFs stay below.
24
+
25
+ ---
26
+
15
27
  ## Adoption demo (v3.5.2)
16
28
 
17
29
  Deterministic broken-agent flow — recommended for live demos and Show HN rehearsal.
@@ -0,0 +1,47 @@
1
+ # Team workflows
2
+
3
+ How teams adopt AgentInspect without a hosted platform.
4
+
5
+ ## Solo developer (day 1)
6
+
7
+ 1. `npm install agent-inspect && npx agent-inspect init --yes`
8
+ 2. Run demo: `node examples/agent-inspect-demo.mjs`
9
+ 3. `npx agent-inspect view <run-id>`
10
+ 4. Before posting issues: `npx agent-inspect redact --profile share`
11
+
12
+ ## PR / code review
13
+
14
+ - Add `@agent-inspect/vitest` or `@agent-inspect/jest` reporter
15
+ - Upload `.agent-inspect/**/*.jsonl` as CI artifact on failure
16
+ - Reviewer downloads artifact and runs `agent-inspect report` locally
17
+
18
+ See [CI artifacts](./CI-ARTIFACTS.md).
19
+
20
+ ## Design partner sprint
21
+
22
+ | Week | Action |
23
+ | ---- | ------ |
24
+ | 1 | Pick one agent path (adapter or observe) |
25
+ | 2 | Add one deterministic check rule |
26
+ | 3 | Redact + share one artifact with the team |
27
+
28
+ Checklist: [DESIGN-PARTNER-GUIDE.md](./DESIGN-PARTNER-GUIDE.md).
29
+
30
+ ## Enterprise / no-cloud constraint
31
+
32
+ - All traces stay on disk under `AGENT_INSPECT_TRACE_DIR`
33
+ - No account, no default upload
34
+ - `verify-safe` before any export
35
+ - Redaction profiles: `local` → `share` → `strict`
36
+
37
+ ## Framework owners
38
+
39
+ - AI SDK: metadata-only telemetry; `recordInputs: false`, `recordOutputs: false`
40
+ - OpenAI Agents: local-only processor mode (see [OPENAI-AGENTS-LOCAL.md](./OPENAI-AGENTS-LOCAL.md))
41
+ - LangChain: callback with `persist: true`
42
+
43
+ ## What teams should not expect
44
+
45
+ - Org-wide trace search SaaS
46
+ - LLM-as-judge eval platform
47
+ - Compliance certification from redaction alone
@@ -0,0 +1,526 @@
1
+ # AgentInspect Technical Guide
2
+
3
+ **Version:** 3.5.3 · **Audience:** TypeScript agent developers, platform engineers, tech leads
4
+ **Purpose:** Source-of-truth technical overview for adoption content (blogs, articles, emails, demos).
5
+ **Status:** Grounded in shipped code and docs as of v3.5.3. Not a compliance or marketing guarantee.
6
+
7
+ ---
8
+
9
+ ## 1. Thesis
10
+
11
+ AgentInspect is a **local-first TypeScript toolkit** for AI agents. It helps you:
12
+
13
+ 1. **Trace** what happened — capture runs and nested steps as a local execution tree
14
+ 2. **Check** what should have happened — deterministic rules and eval heuristics in CI
15
+ 3. **Redact** what must not leave your machine — profiles before sharing artifacts
16
+
17
+ **North star:** install → one trace → one failure check → one share-safe artifact in under five minutes.
18
+
19
+ **Principles:** CLI-first · TypeScript-first · dependency-light at the root · safe by default · framework-aware but not framework-locked · **no account · no upload · no hosted dashboard** · metadata-only by default.
20
+
21
+ ---
22
+
23
+ ## 2. The problem
24
+
25
+ AI agents are not linear scripts. A single user request can fan out into parallel tool calls, nested LLM rounds, retries, handoffs, and MCP invocations. `console.log` gives you a **flat stream** — you see lines, not structure.
26
+
27
+ | Pain | Why flat logs fail |
28
+ | ---- | ------------------ |
29
+ | Wrong tool selected | You see a tool name, not sibling steps or parent run context |
30
+ | Eval/test failure | Assertion message only; no step-level evidence |
31
+ | Silent stall | No duration boundaries or step tree to spot hung work |
32
+ | Incident handoff | Raw logs may contain secrets; no share-safe workflow |
33
+ | Multi-agent flows | Handoffs span runs; hard to reconstruct causality |
34
+
35
+ **Hosted observability** (LangSmith, Langfuse, Braintrust, OTel backends) excels at retention, dashboards, and fleet-scale monitoring. AgentInspect targets the **inner loop**: local debugging, PR artifacts, and deterministic checks **before** you need a vendor account or collector setup.
36
+
37
+ AgentInspect is **complementary**, not a replacement. See [COMPARE.md](./COMPARE.md).
38
+
39
+ ---
40
+
41
+ ## 3. Core idea: the execution tree
42
+
43
+ AgentInspect models agent work as a **tree of runs and steps**, persisted as **JSONL files** on disk (default directory `.agent-inspect/` or `AGENT_INSPECT_TRACE_DIR`).
44
+
45
+ ### 3.1 Manual trace events (schema 0.1)
46
+
47
+ Written by `inspectRun()`, `step()`, and `observe()`:
48
+
49
+ | Event | Meaning |
50
+ | ----- | ------- |
51
+ | `run_started` | Top-level workflow begins |
52
+ | `step_started` | Nested unit of work begins |
53
+ | `step_completed` | Step ends (success or error) |
54
+ | `run_completed` | Run ends |
55
+
56
+ **Important:** There is no `step_failed` event. Failures use `step_completed` with `status: "error"`.
57
+
58
+ Step types include **tool**, **LLM**, and custom names. Convenience helpers: `step.tool(name, fn)` and `step.llm(model, fn)`.
59
+
60
+ ### 3.2 Persisted InspectEvent (schema 1.0)
61
+
62
+ Framework adapters and `createInspector()` / writers can emit **schema 1.0** persisted rows. Manual global tracing remains 0.1 for compatibility. Readers normalize multiple schema versions for CLI inspection.
63
+
64
+ ### 3.3 Log-derived trees
65
+
66
+ Structured JSON logs can be ingested via `agent-inspect logs` / `tail` into normalized in-memory trees with **confidence labels** (`explicit`, `correlated`, `heuristic`, `unknown`). Log-derived views are not identical to manual JSONL fidelity.
67
+
68
+ ### 3.4 Product loop
69
+
70
+ ```mermaid
71
+ flowchart LR
72
+ subgraph capture [Capture]
73
+ observe[observe]
74
+ manual[inspectRun_step]
75
+ adapters[Framework_adapters]
76
+ logs[Log_ingest]
77
+ end
78
+ subgraph persist [Local_persistence]
79
+ jsonl[".agent-inspect/*.jsonl"]
80
+ end
81
+ subgraph inspect [Inspect_and_quality]
82
+ cli[CLI_list_view_report]
83
+ check[check_eval_circuit]
84
+ end
85
+ subgraph safety [Safe_sharing]
86
+ redact[redact_profiles]
87
+ verify[verify-safe]
88
+ end
89
+ capture --> jsonl
90
+ jsonl --> cli
91
+ jsonl --> check
92
+ jsonl --> redact
93
+ redact --> verify
94
+ ```
95
+
96
+ ---
97
+
98
+ ## 4. System architecture
99
+
100
+ ### 4.1 Package layout
101
+
102
+ | Package | Published | Role |
103
+ | ------- | --------- | ---- |
104
+ | `agent-inspect` | Yes | Core tracing APIs + CLI binary |
105
+ | `@agent-inspect/core` | No (private) | Tracing, storage, readers, checks, export |
106
+ | `@agent-inspect/cli` | No (private) | Commander CLI implementation |
107
+
108
+ The root `agent-inspect` tarball ships core + CLI. Heavy optional deps stay in scoped packages.
109
+
110
+ ### 4.2 Public package ecosystem (v3.5.3)
111
+
112
+ Sixteen linked npm packages at **3.5.3**:
113
+
114
+ | Tier | Package | Purpose |
115
+ | ---- | ------- | ------- |
116
+ | **Core** | `agent-inspect` | `observe`, `inspectRun`, `step`, CLI |
117
+ | **Framework** | `@agent-inspect/ai-sdk` | Vercel AI SDK telemetry integration |
118
+ | | `@agent-inspect/openai-agents` | OpenAI Agents JS tracing processor |
119
+ | | `@agent-inspect/langchain` | LangChain callback adapter |
120
+ | **Real projects** | `@agent-inspect/harness` | Fixture runner (bootstrap → invoke → shutdown) |
121
+ | **Quality** | `@agent-inspect/eval` | Deterministic eval heuristics over traces |
122
+ | | `@agent-inspect/vitest` | Vitest reporter (artifacts on failure) |
123
+ | | `@agent-inspect/jest` | Jest reporter |
124
+ | | `@agent-inspect/guardrails` | Deterministic guardrail rules |
125
+ | | `@agent-inspect/circuit` | Loop/retry/stall analyzers |
126
+ | **Safety** | `@agent-inspect/redact` | JSON/trace redaction (`local` / `share` / `strict`) |
127
+ | **Inspect UX** | `@agent-inspect/viewer` | Localhost read-only viewer |
128
+ | | `@agent-inspect/tui` | Optional terminal UI (`view --tui`) |
129
+ | | `@agent-inspect/mcp-server` | Read-only MCP server for trace listing |
130
+ | **Extensibility** | `@agent-inspect/mcp` | MCP **client** tool-call tracing |
131
+ | | `@agent-inspect/adapter-sdk` | Third-party adapter authoring + conformance |
132
+
133
+ **In-repo (not on Marketplace yet):** `agent-inspect-vscode` — read-only trace explorer via F5 dev host. See [VSCODE.md](./VSCODE.md).
134
+
135
+ Each scoped package has its own README on npm.
136
+
137
+ ### 4.3 Instrumentation safety
138
+
139
+ Tracing, stepping, and observation **must not throw into user code** in ways that break the agent. Failures in instrumentation degrade gracefully (warn, skip event, continue). User errors inside `fn` are re-thrown unchanged.
140
+
141
+ ---
142
+
143
+ ## 5. Five capture paths
144
+
145
+ | Path | When | Entry |
146
+ | ---- | ---- | ----- |
147
+ | **Observe** | Existing class with `run` / `execute` / `invoke` | `observe(agent, { traceDir })` |
148
+ | **Manual** | Custom control flow, explicit step names | `inspectRun` + `step` / `step.tool` / `step.llm` |
149
+ | **AI SDK** | Vercel AI SDK `generateText` / `streamText` | `@agent-inspect/ai-sdk` + `recordInputs: false`, `recordOutputs: false` |
150
+ | **OpenAI Agents** | OpenAI Agents JS | `@agent-inspect/openai-agents` local-only processor mode |
151
+ | **LangChain** | Callback-based chains / LangGraph-via-LangChain | `@agent-inspect/langchain` with `persist: true` |
152
+ | **Logs** | Structured JSON logs already emitted | `agent-inspect logs` / `tail` |
153
+ | **Harness** | Real app (NestJS, etc.) with fixtures | `@agent-inspect/harness` + starters |
154
+
155
+ **Blessed starters** (no API keys): [examples/starters](../examples/starters/README.md)
156
+
157
+ **Quick bootstrap:**
158
+
159
+ ```bash
160
+ npm install agent-inspect
161
+ npx agent-inspect init --yes
162
+ node examples/agent-inspect-demo.mjs
163
+ ```
164
+
165
+ ---
166
+
167
+ ## 6. CLI workbench
168
+
169
+ All commands run locally. No network required.
170
+
171
+ ### 6.1 Onboarding
172
+
173
+ | Command | Purpose |
174
+ | ------- | ------- |
175
+ | `init` | Config, trace dir, deterministic demo |
176
+ | `doctor` | Node, permissions, optional packages |
177
+
178
+ ### 6.2 Browse and understand
179
+
180
+ | Command | Purpose |
181
+ | ------- | ------- |
182
+ | `list` | Runs in a trace directory |
183
+ | `view` | Execution tree in terminal (optional `--tui`, `--serve`) |
184
+ | `timeline` | Chronological view (e.g. `--focus slow`) |
185
+ | `what` | Concise run summary |
186
+ | `report` | Markdown/HTML report |
187
+ | `explain` | Local analysis dry-run |
188
+ | `sessions` / `session` | Multi-run session grouping |
189
+ | `search` | Filter by status, kind, etc. |
190
+ | `stats` | Aggregate directory metrics |
191
+ | `serve` | Localhost viewer |
192
+
193
+ ### 6.3 Compare and migrate
194
+
195
+ | Command | Purpose |
196
+ | ------- | ------- |
197
+ | `diff` | Compare two runs (baseline vs candidate) |
198
+ | `open` | Open trace with format detection |
199
+ | `migrate` | Explicit schema migration (non-destructive) |
200
+
201
+ ### 6.4 Quality gates
202
+
203
+ | Command | Purpose |
204
+ | ------- | ------- |
205
+ | `check` | Deterministic trace rules (duration, stalls, tool usage, etc.) |
206
+ | `eval` | Eval heuristics over traces |
207
+ | `scan` | Safety scan |
208
+ | `verify-safe` | Verify artifact is share-safe |
209
+ | `artifacts` / `ci-summary` | CI-oriented summaries |
210
+
211
+ **Example CI check:**
212
+
213
+ ```bash
214
+ npx agent-inspect check .agent-inspect/*.jsonl --require-completed --detect-stalls
215
+ ```
216
+
217
+ ### 6.5 Safe sharing
218
+
219
+ | Command | Purpose |
220
+ | ------- | ------- |
221
+ | `export` | Markdown, HTML, OpenInference, OTLP JSON (local files) |
222
+ | `redact` | Copy with redaction profile |
223
+
224
+ ### 6.6 Logs and scale
225
+
226
+ | Command | Purpose |
227
+ | ------- | ------- |
228
+ | `logs` / `tail` | Ingest structured logs |
229
+ | `index build` / `status` / `clean` | Optional metadata index for large directories |
230
+ | `clean` | Verified trace deletion |
231
+
232
+ ### 6.7 End-to-end debug flow
233
+
234
+ ```bash
235
+ cd examples/starters/broken-agent-debugging && pnpm install && pnpm start
236
+ npx agent-inspect report <run-id> --dir .agent-inspect
237
+ npx agent-inspect check .agent-inspect/*.jsonl
238
+ npx agent-inspect redact .agent-inspect/*.jsonl --profile share -o safe.jsonl
239
+ npx agent-inspect verify-safe safe.jsonl
240
+ ```
241
+
242
+ ---
243
+
244
+ ## 7. Programmatic API
245
+
246
+ ### 7.1 Stable root exports
247
+
248
+ ```ts
249
+ import {
250
+ createInspector,
251
+ observe,
252
+ inspectRun,
253
+ maybeInspectRun,
254
+ step,
255
+ getCurrentCorrelationMetadata,
256
+ } from "agent-inspect";
257
+ ```
258
+
259
+ | API | Use |
260
+ | --- | --- |
261
+ | `inspectRun(name, fn, opts?)` | Wrap workflow; writes JSONL |
262
+ | `maybeInspectRun` | Env-gated via `AGENT_INSPECT` |
263
+ | `step(name, fn)` | Named step inside a run |
264
+ | `step.tool` / `step.llm` | Typed step shortcuts |
265
+ | `observe(agent, opts?)` | Proxy `run`/`execute`/`invoke` |
266
+ | `getCurrentCorrelationMetadata()` | Active correlation IDs inside a run |
267
+
268
+ **Defaults:** metadata redaction before disk; size bounds on events; `enabled: false` skips tracing entirely.
269
+
270
+ ### 7.2 Subpath exports (advanced / experimental)
271
+
272
+ | Subpath | Examples |
273
+ | ------- | -------- |
274
+ | `agent-inspect/logs` | `parseLogsToTrees` |
275
+ | `agent-inspect/exporters` | Markdown, HTML, OpenInference, OTLP JSON export |
276
+ | `agent-inspect/writers` | `fileWriter`, custom persistence |
277
+ | `agent-inspect/readers` | `openTrace`, multi-format read |
278
+ | `agent-inspect/checks` | `runTraceChecks`, rule factories |
279
+ | `agent-inspect/diff` | `diffTraceEvents` |
280
+ | `agent-inspect/persisted` | Schema 1.0 types and converters |
281
+ | `agent-inspect/advanced` | Runtime helpers, redaction profiles |
282
+ | `agent-inspect/reporters` | Reporter utilities |
283
+
284
+ See [API.md](./API.md) for stability classification.
285
+
286
+ ### 7.3 Check rule families (deterministic)
287
+
288
+ Built-in check rules include (non-exhaustive): run status/duration, max step duration, stall detection, require-completed, tool usage/ordering/failures, LLM usage, structure (orphans, cycles, parallel width), retrieval, guardrails, decision metadata, safety (redaction, raw content, secret patterns, oversized attributes), baseline regression.
289
+
290
+ **No LLM-as-judge** in core checks or `@agent-inspect/eval`.
291
+
292
+ ---
293
+
294
+ ## 8. Framework adapters (high level)
295
+
296
+ ### 8.1 AI SDK (`@agent-inspect/ai-sdk`)
297
+
298
+ - Explicit `experimental_telemetry.integrations` wiring — no monkey-patching
299
+ - **Required:** `recordInputs: false`, `recordOutputs: false` on AI SDK calls
300
+ - Metadata-only: model, timing, token counts, tool names — not raw prompts/outputs
301
+ - Guide: [AI-SDK-ADOPTION.md](./AI-SDK-ADOPTION.md)
302
+
303
+ ### 8.2 OpenAI Agents (`@agent-inspect/openai-agents`)
304
+
305
+ - `TracingProcessor` for local JSONL
306
+ - **Local-only mode:** use `setTraceProcessors` replacement, not `addTraceProcessor`, to avoid default cloud export
307
+ - Maps agents, tools, handoffs, guardrails as metadata
308
+ - Guide: [OPENAI-AGENTS-LOCAL.md](./OPENAI-AGENTS-LOCAL.md)
309
+
310
+ ### 8.3 LangChain (`@agent-inspect/langchain`)
311
+
312
+ - `AgentInspectCallback` with `persist: true`
313
+ - LangGraph supported through LangChain callback surfaces
314
+ - Guide: [ADAPTERS.md](./ADAPTERS.md)
315
+
316
+ ### 8.4 MCP
317
+
318
+ - `@agent-inspect/mcp` — trace **client** tool list/call lifecycle only
319
+ - `@agent-inspect/mcp-server` — read-only trace access for IDE agents; no tool invocation, no mutation
320
+
321
+ ---
322
+
323
+ ## 9. CI and test integration
324
+
325
+ | Surface | Behavior |
326
+ | ------- | -------- |
327
+ | `@agent-inspect/vitest` | On test failure, write trace artifacts; passing tests stay quiet |
328
+ | `@agent-inspect/jest` | Same pattern for Jest |
329
+ | `agent-inspect check` | Gate merges on trace shape, completion, stalls |
330
+ | `agent-inspect eval` | Heuristic findings with evidence paths |
331
+ | CI artifacts | Upload `.agent-inspect/**/*.jsonl`; reviewer runs `report` locally |
332
+
333
+ Workflow: [CI-ARTIFACTS.md](./CI-ARTIFACTS.md) · Starter: [ci-eval-redact](../examples/starters/ci-eval-redact/)
334
+
335
+ ---
336
+
337
+ ## 10. Safety and privacy model
338
+
339
+ | Property | Behavior |
340
+ | -------- | -------- |
341
+ | Storage | Local JSONL only |
342
+ | Upload | **None** by default; AgentInspect does not phone home |
343
+ | Capture default | Metadata-only; no raw prompts/outputs unless you opt in |
344
+ | Redaction before disk | Common sensitive keys stripped on manual traces |
345
+ | Profiles | `local` → `share` → `strict` for export/redact |
346
+ | Verification | `scan`, `verify-safe` before attaching to issues |
347
+ | Instrumentation | Failures degrade gracefully; never break user agent |
348
+ | Compliance | **Not** a DLP or compliance engine — human review still required |
349
+
350
+ Details: [SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md) · [SECURITY.md](../SECURITY.md)
351
+
352
+ ---
353
+
354
+ ## 11. Performance and scale (honest limits)
355
+
356
+ AgentInspect optimizes for **developer workflows**, not production APM.
357
+
358
+ | Workload | Comfortable | Warning threshold |
359
+ | -------- | ----------- | ----------------- |
360
+ | Runs per directory | ≤ 1,000 | 1,000–10,000 (consider index/archive) |
361
+ | Events per run | ≤ 10,000 | 10,000–50,000 |
362
+ | Single trace file | ≤ 10 MB | 50 MB+ |
363
+
364
+ Optional: `agent-inspect index build` for large directories. See [PERFORMANCE.md](./PERFORMANCE.md) and [SCALE-LIMITS.md](./SCALE-LIMITS.md).
365
+
366
+ ---
367
+
368
+ ## 12. Positioning matrix
369
+
370
+ | Need | AgentInspect | Alternative |
371
+ | ---- | ------------ | ----------- |
372
+ | Local agent debugging | **Strong** | console.log (flat) |
373
+ | No-account CLI tracing | **Strong** | Hosted platforms |
374
+ | Deterministic CI checks | **Good** | Braintrust eval platform |
375
+ | Share-safe redacted copy | **Good** | Manual scrubbing |
376
+ | Production dashboards | Not the goal | Langfuse, LangSmith |
377
+ | Hosted eval datasets | Not the goal | Braintrust, LangSmith |
378
+ | Prompt registry | Not the goal | Langfuse, etc. |
379
+ | Fleet-wide OTel pipelines | Not the goal | OTel + collector + backend |
380
+ | Standards export | Partial (local OpenInference/OTLP JSON) | Phoenix, OTel |
381
+
382
+ Full comparison: [COMPARE.md](./COMPARE.md)
383
+
384
+ ---
385
+
386
+ ## 13. What AgentInspect is not
387
+
388
+ - Hosted SaaS or dashboard product
389
+ - Production APM replacement
390
+ - Eval dataset platform or LLM-as-judge service
391
+ - Prompt registry or pricing/cost engine
392
+ - Default telemetry uploader or replay/cassette engine
393
+ - Raw chain-of-thought recorder
394
+ - Universal monkey-patching framework
395
+ - Compliance certification from redaction alone
396
+
397
+ See [LIMITATIONS.md](./LIMITATIONS.md).
398
+
399
+ ---
400
+
401
+ ## 14. Real-world scenarios
402
+
403
+ | # | Problem | Mechanism | Starter / doc |
404
+ | - | ------- | --------- | ------------- |
405
+ | 1 | Wrong tool call | `view`, `report` | [broken-agent-debugging](../examples/starters/broken-agent-debugging/) |
406
+ | 2 | Eval failure — which step? | `check`, `@agent-inspect/vitest` | [ci-eval-redact](../examples/starters/ci-eval-redact/) |
407
+ | 3 | CI PR artifact | upload + `redact` + `verify-safe` | [CI-ARTIFACTS.md](./CI-ARTIFACTS.md) |
408
+ | 4 | Framework-native trace | adapter packages | [starters](../examples/starters/) |
409
+ | 5 | Safe incident handoff | `redact --profile share` | [SAFE-TRACE-SHARING.md](./SAFE-TRACE-SHARING.md) |
410
+ | 6 | Multi-agent / sessions | `sessions`, `search`, `diff` | [USE-CASES.md](./USE-CASES.md) |
411
+ | 7 | MCP tool tracing | `@agent-inspect/mcp` | [ADAPTERS.md](./ADAPTERS.md) |
412
+ | 8 | Team adoption sprint | one agent, one check, one artifact | [DESIGN-PARTNER-GUIDE.md](./DESIGN-PARTNER-GUIDE.md) |
413
+ | 9 | VS Code review | in-repo extension | [VSCODE.md](./VSCODE.md) |
414
+ | 10 | Existing logs only | `logs`, `tail` | [LOG-TO-TREE-QUICKSTART.md](./LOG-TO-TREE-QUICKSTART.md) |
415
+
416
+ Full index: [USE-CASES.md](./USE-CASES.md) · [TEAM-WORKFLOWS.md](./TEAM-WORKFLOWS.md)
417
+
418
+ ---
419
+
420
+ ## 15. Adoption personas
421
+
422
+ | Persona | Pain | AgentInspect answer |
423
+ | ------- | ---- | ------------------- |
424
+ | **Solo agent developer** | Flat logs, wrong tool | Local tree + `report` in 60 seconds |
425
+ | **CI maintainer** | Failed test, no evidence | Vitest/Jest reporter + `check` gates |
426
+ | **Platform / security** | No-cloud data constraint | Disk-only traces, explicit redact |
427
+ | **Framework owner** | Manual wrapping tedious | AI SDK / OpenAI Agents / LangChain adapters |
428
+ | **Design partner lead** | Team needs one win in a sprint | [DESIGN-PARTNER-GUIDE.md](./DESIGN-PARTNER-GUIDE.md) |
429
+
430
+ Daily workflow: capture → inspect → verify → share safely → scale. See [ADOPTION.md](./ADOPTION.md).
431
+
432
+ ---
433
+
434
+ ## 16. Content creation appendix
435
+
436
+ Reuse these blocks in blogs, emails, and comments. All statements are accurate to v3.5.3; adjust version numbers when publishing.
437
+
438
+ ### 16.1 Elevator pitches
439
+
440
+ **30 seconds:**
441
+ AgentInspect is a local-first TypeScript toolkit for AI agents. It writes execution trees to JSONL on your machine — no account, no upload. Inspect with the CLI, run deterministic checks in CI, and redact before sharing. It complements LangSmith and Langfuse for the inner loop.
442
+
443
+ **60 seconds:**
444
+ When an agent calls the wrong tool or a test fails, flat logs are not enough. AgentInspect captures runs and nested steps locally, lets you `report` and `diff` failures, and gates CI with deterministic `check` rules — no LLM judge. Framework adapters exist for AI SDK, OpenAI Agents, and LangChain. Metadata-only by default; redact before posting traces anywhere.
445
+
446
+ **120 seconds (technical):**
447
+ Add `observe()` or a framework adapter, get JSONL under `.agent-inspect/`, then use `agent-inspect list`, `view`, `report`, `check`, and `redact --profile share`. Sixteen npm packages cover adapters, eval, redaction, MCP tracing, harness fixtures, and test reporters. Schema 1.0 persisted events; readers handle 0.1/0.2/1.0. Not production APM — local debug and share-safe artifacts first.
448
+
449
+ ### 16.2 Pull quotes (factual)
450
+
451
+ - "Traces stay on your disk. AgentInspect does not upload them."
452
+ - "Metadata-only by default — not a chain-of-thought recorder."
453
+ - "Deterministic checks, not LLM-as-judge evals."
454
+ - "Wrong tool call? See the step tree, not just a log line."
455
+ - "Redact before Slack, GitHub, or email — `local`, `share`, `strict` profiles."
456
+ - "Works before you set up OpenTelemetry or a hosted dashboard."
457
+ - "Framework-aware: AI SDK, OpenAI Agents, LangChain — optional packages."
458
+ - "Instrumentation failures degrade gracefully; your agent keeps running."
459
+
460
+ ### 16.3 Blog post outlines
461
+
462
+ **A. First trace in 5 minutes**
463
+ Hook (flat logs) → `npm install` → `init` → demo → `list` / `report` → `check` → `redact` → link to starters.
464
+
465
+ **B. Debug the wrong tool call**
466
+ broken-agent-debugging starter → `report` highlights error step → fix → `diff` before/after → redact for issue attachment.
467
+
468
+ **C. CI trace artifacts without a vendor account**
469
+ Vitest reporter → upload artifact → `verify-safe` → reviewer workflow → contrast with hosted APM.
470
+
471
+ ### 16.4 FAQ (for comments and emails)
472
+
473
+ | Question | Answer |
474
+ | -------- | ------ |
475
+ | Does it upload traces? | No. Local files only. |
476
+ | Replace LangSmith? | No. Complementary for local inner loop. |
477
+ | Store full prompts? | Not by default. Metadata-only; opt-in is caller responsibility. |
478
+ | LLM judge evals? | No. Deterministic heuristics and rules only. |
479
+ | VS Code extension? | In-repo; F5 dev host. Marketplace not published yet. |
480
+ | Node version? | Node 20+ recommended; run `doctor`. |
481
+ | Monorepo? | `pnpm build` at root; starters use workspace packages. |
482
+ | OpenTelemetry? | Local OTLP JSON export possible; not a full OTel SDK. |
483
+ | Compliance? | Redaction helps; not certification or DLP. |
484
+
485
+ ### 16.5 Objection handlers (engineering managers)
486
+
487
+ | Objection | Response |
488
+ | --------- | -------- |
489
+ | "We already use Langfuse" | Use AgentInspect locally for PR/debug artifacts; Langfuse for retention and dashboards. |
490
+ | "Another dependency" | Root package is dependency-light; adapters are optional scoped packages. |
491
+ | "Security review" | No network, no account, metadata defaults, redact profiles, SECURITY.md policy. |
492
+ | "Will it slow our agents?" | Instrumentation is bounded; failures are swallowed; disable with `AGENT_INSPECT=0`. |
493
+
494
+ ### 16.6 Demo and launch assets
495
+
496
+ | Asset | Location |
497
+ | ----- | -------- |
498
+ | Live 3-min demo | [DEMO-SCRIPT.md](./DEMO-SCRIPT.md) |
499
+ | Video script | [VIDEO-WALKTHROUGH-SCRIPT.md](./VIDEO-WALKTHROUGH-SCRIPT.md) |
500
+ | Show HN draft | [SHOW-HN-DRAFT.md](./SHOW-HN-DRAFT.md) |
501
+ | GIF demos | [SCREENSHOTS.md](./SCREENSHOTS.md) |
502
+ | Pitch | [PITCH.md](./PITCH.md) |
503
+
504
+ ---
505
+
506
+ ## 17. References
507
+
508
+ | Topic | Doc |
509
+ | ----- | --- |
510
+ | Website | [https://agentinspect.vercel.app/](https://agentinspect.vercel.app/) |
511
+ | Docs site | [https://agentinspect.vercel.app/docs/](https://agentinspect.vercel.app/docs/) |
512
+ | Docs index | [README.md](./README.md) |
513
+ | Getting started | [GETTING-STARTED.md](./GETTING-STARTED.md) · [Web](https://agentinspect.vercel.app/docs/getting-started/) |
514
+ | 5-minute path | [FIRST-TRACE-IN-5-MINUTES.md](./FIRST-TRACE-IN-5-MINUTES.md) |
515
+ | API | [API.md](./API.md) |
516
+ | CLI | [CLI.md](./CLI.md) |
517
+ | Schema | [SCHEMA.md](./SCHEMA.md) |
518
+ | Adapters | [ADAPTERS.md](./ADAPTERS.md) |
519
+ | Examples | [examples/starters](../examples/starters/) · [recipes](../examples/recipes/) |
520
+ | Changelog | [CHANGELOG.md](../CHANGELOG.md) |
521
+ | Roadmap | [ROADMAP.md](../ROADMAP.md) |
522
+ | Discussions | [GitHub Discussions](https://github.com/rajudandigam/agent-inspect/discussions) |
523
+
524
+ ---
525
+
526
+ *AgentInspect v3.5.3 — local-first trace workbench for TypeScript AI agents. MIT license.*
package/docs/VSCODE.md ADDED
@@ -0,0 +1,21 @@
1
+ # AgentInspect VS Code extension
2
+
3
+ Read-only sidebar for local trace directories. The extension shells out to the published `agent-inspect` CLI (`list`, `view`, `timeline`, `report`, `check`, `doctor`, `verify-safe`).
4
+
5
+ ## Develop
6
+
7
+ ```bash
8
+ pnpm install
9
+ pnpm --filter agent-inspect-vscode run build
10
+ ```
11
+
12
+ Open `packages/vscode` in VS Code and press F5 (Extension Development Host).
13
+
14
+ ## Requirements
15
+
16
+ - Node 20+
17
+ - `agent-inspect` available via `npx` in the workspace (devDependency or global)
18
+
19
+ ## Manual gate
20
+
21
+ First VS Code Marketplace publish requires maintainer credentials. See [VSCODE-EXTENSION-RFC.md](./proposals/VSCODE-EXTENSION-RFC.md).
@@ -0,0 +1,13 @@
1
+ <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 280 72" role="img" aria-label="AgentInspect">
2
+ <defs>
3
+ <linearGradient id="ai-grad-d" x1="0%" y1="0%" x2="100%" y2="100%">
4
+ <stop offset="0%" stop-color="#818cf8"/>
5
+ <stop offset="100%" stop-color="#a5b4fc"/>
6
+ </linearGradient>
7
+ </defs>
8
+ <rect x="8" y="12" width="48" height="48" rx="12" fill="url(#ai-grad-d)"/>
9
+ <path d="M24 28h16M24 36h12M24 44h16" stroke="#0f172a" stroke-width="2.5" stroke-linecap="round"/>
10
+ <circle cx="44" cy="36" r="5" fill="none" stroke="#0f172a" stroke-width="2"/>
11
+ <path d="M47.5 39.5L52 44" stroke="#0f172a" stroke-width="2.5" stroke-linecap="round"/>
12
+ <text x="68" y="44" font-family="ui-sans-serif,system-ui,-apple-system,Segoe UI,sans-serif" font-size="22" font-weight="600" fill="#f8fafc">agent-inspect</text>
13
+ </svg>
@@ -0,0 +1,15 @@
1
+ <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 280 72" role="img" aria-label="AgentInspect">
2
+ <defs>
3
+ <linearGradient id="ai-grad" x1="0%" y1="0%" x2="100%" y2="100%">
4
+ <stop offset="0%" stop-color="#4f46e5"/>
5
+ <stop offset="100%" stop-color="#6366f1"/>
6
+ </linearGradient>
7
+ </defs>
8
+ <!-- Tree / inspect mark -->
9
+ <rect x="8" y="12" width="48" height="48" rx="12" fill="url(#ai-grad)"/>
10
+ <path d="M24 28h16M24 36h12M24 44h16" stroke="#fff" stroke-width="2.5" stroke-linecap="round"/>
11
+ <circle cx="44" cy="36" r="5" fill="none" stroke="#fff" stroke-width="2"/>
12
+ <path d="M47.5 39.5L52 44" stroke="#fff" stroke-width="2.5" stroke-linecap="round"/>
13
+ <!-- Wordmark -->
14
+ <text x="68" y="44" font-family="ui-sans-serif,system-ui,-apple-system,Segoe UI,sans-serif" font-size="22" font-weight="600" fill="#0f172a">agent-inspect</text>
15
+ </svg>