agent-inspect 0.1.1 → 1.0.0

package/README.md

@@ -26,6 +26,20 @@ agent-inspect is designed for inner-loop debugging, not as a replacement for pro
 npm install agent-inspect
 ```
 
+## Documentation (v1.0 stabilization)
+
+- [Getting started](docs/GETTING-STARTED.md)
+- [API reference](docs/API.md)
+- [CLI reference](docs/CLI.md)
+- [Schema reference](docs/SCHEMA.md)
+- [Security policy](SECURITY.md)
+- [Migration guide](docs/MIGRATION.md)
+- [Release checklist](docs/RELEASE-CHECKLIST.md)
+- [Changelog](CHANGELOG.md)
+- [Known issues](docs/KNOWN-ISSUES.md)
+- [Limitations](docs/LIMITATIONS.md)
+- [V1 readiness checklist (non-binding)](docs/V1-READINESS-CHECKLIST.md)
+
 ## See your first trace
 
 Run a traced workflow, then inspect it with the CLI.
@@ -46,6 +60,53 @@ npx agent-inspect view run_abc123
 
 Replace `run_abc123` with the run id printed by `agent-inspect list`.
 
+### Optional TUI viewer
+
+The core `agent-inspect` package stays lightweight and does not bundle Ink or React. For a keyboard-driven terminal UI over existing traces, install the optional package:
+
+```bash
+pnpm add agent-inspect @agent-inspect/tui
+
+npx agent-inspect view run_abc123 --tui
+```
+
+The plain CLI remains the default. `--tui` requires an interactive terminal; for scripts or CI, use `agent-inspect view` or `agent-inspect view --json`. There is no live tail TUI yet.
+
+### Export traces
+
+Export existing manual JSONL traces locally — **no upload**, **no vendor SDKs**. Markdown is handy for PRs and issues; HTML is a single offline file. OpenInference export is **OpenInference-compatible JSON** (not a guarantee for every backend). OTLP JSON uses **OTel GenAI-aligned attributes** where applicable and is **experimental** until validated against a specific collector.
+
+```bash
+npx agent-inspect export run_abc123 --format markdown
+npx agent-inspect export run_abc123 --format html -o run.html
+npx agent-inspect export run_abc123 --format openinference -o trace.openinference.json
+npx agent-inspect export run_abc123 --format otlp-json -o trace.otlp.json
+npx agent-inspect export run_abc123 --format openinference --validate
+```
+
+Review exported files for sensitive data before sharing. Attribute payloads are bounded and redacted by default; use `--include-attributes` only when you intend to share richer detail.
+
+### Compare runs
+
+Diff is **local** and **read-only**: it compares two existing AgentInspect JSONL traces and does **not** rerun agents, mutate trace files, or write output traces. It does **not** claim semantic equivalence and does **not** call an LLM.
+
+Finding differences does **not** change the exit code by default (exit code `1` is reserved for command errors such as a missing run).
+
+```bash
+npx agent-inspect diff run_a run_b
+npx agent-inspect diff run_a run_b --json
+npx agent-inspect diff run_a run_b --ignore-duration
+npx agent-inspect diff run_a run_b --duration-threshold 500ms
+npx agent-inspect diff run_a run_b --focus errors
+npx agent-inspect diff run_a run_b --check structure
+```
+
+Useful for comparing passing vs failing runs and spotting the **first divergence** in execution order.
+
+### Fixtures and hardening (v0.9)
+
+**v0.9** adds canonical [**fixtures/**](fixtures/README.md), validation (`pnpm fixtures:check`), **recipe examples** under [**examples/recipes/**](examples/recipes/README.md) (`pnpm recipes:check`), and docs aimed at adoption—not new tracing features. Recipes use mocks only and require no API keys or external services by default. Good starting points: **rag-pipeline**, **tool-failure-retry**, **proactive-agent-logs**. See [**Known issues**](docs/KNOWN-ISSUES.md), [**Limitations**](docs/LIMITATIONS.md), and the non-binding [**v1 readiness checklist**](docs/V1-READINESS-CHECKLIST.md).
+
 ## Minimal API
 
 ```ts
@@ -217,6 +278,41 @@ await agent.run("How do I reset my password?");
 
 `observe()` wraps top-level `run`, `execute`, and `invoke` methods. For internal detail, add manual `step()` calls inside the agent.
 
+## LangChain adapter (v0.5, experimental)
+
+Install:
+
+```bash
+pnpm add agent-inspect @agent-inspect/langchain @langchain/core
+```
+
+`@langchain/core` is a **peer dependency** of `@agent-inspect/langchain`. The adapter uses official LangChain.js **callbacks** only (extends `BaseCallbackHandler`): **no** monkey-patching, **no** `agent-inspect/auto`, **no** vendor observability sinks.
+
+```ts
+import { AgentInspectCallback } from "@agent-inspect/langchain";
+
+const callback = new AgentInspectCallback({
+  runName: "support-agent-eval",
+  capture: "metadata-only",
+});
+
+await agent.invoke(input, {
+  callbacks: [callback],
+});
+
+const events = callback.getEvents();
+```
+
+Behavior:
+
+- **Metadata-only** capture by default (model, tags, token usage when present, counts). **No** full prompt/output capture by default.
+- **Preview** mode is opt-in (`capture: "preview"`) with truncation via `maxPreviewChars` (default `200`).
+- **Parent** links use LangChain `parentRunId`, surfaced as `parentId` on `InspectEvent` with `confidence: "explicit"`.
+- **No** cost calculation; token fields are informational only.
+- In this pass, events are collected **in memory** only (`getEvents()` / `clear()`). **No trace-file persistence** for adapter events yet; they are **not** written into v0.1 JSONL manual traces.
+
+The API is **experimental** before v1.0. See [examples/08-langchain-adapter](examples/08-langchain-adapter).
+
 ## CLI
 
 List recent runs:
@@ -225,12 +321,83 @@ List recent runs:
 npx agent-inspect list
 ```
 
+Common filters:
+
+```bash
+npx agent-inspect list --status success
+npx agent-inspect list --status error
+npx agent-inspect list --status running
+npx agent-inspect list --status unknown
+npx agent-inspect list --name hotel
+npx agent-inspect list --since 24h
+npx agent-inspect list --json
+```
+
 View a run:
 
 ```bash
 npx agent-inspect view run_abc123
 ```
 
+Alternate view modes:
+
+```bash
+npx agent-inspect view run_abc123 --summary
+npx agent-inspect view run_abc123 --metadata
+npx agent-inspect view run_abc123 --errors-only
+npx agent-inspect view run_abc123 --json --summary
+```
+
+Safely clean up old traces (recommended: start with `--dry-run`):
+
+```bash
+npx agent-inspect clean --older-than 7d --dry-run
+npx agent-inspect clean --older-than 7d
+npx agent-inspect clean --keep 100 --dry-run
+npx agent-inspect clean --keep 100 --yes
+npx agent-inspect clean --dir ./traces --older-than 7d --dry-run
+```
+
+Safety notes:
+
+- `clean` **verifies each file** as an AgentInspect trace before deleting.
+- Arbitrary JSONL files are **not deleted**.
+- Malformed JSONL files are **not deleted**.
+- Without `--dry-run`, `clean` requires confirmation unless `--yes` is provided.
+- In non-interactive terminals, deletion requires `--yes`.
+
+Inspect structured logs:
+
+```bash
+npx agent-inspect logs ./agent.log --format json
+npx agent-inspect logs ./agent.log --format log4js
+npx agent-inspect logs ./agent.log --format auto
+npx agent-inspect logs ./agent.log --config agent-inspect.logs.json
+npx agent-inspect logs ./agent.log --json
+npx agent-inspect logs ./agent.log --summary
+npx agent-inspect logs ./agent.log --warnings all
+```
+
+Log ingestion notes:
+
+- JSON logs are first-class.
+- log4js text logs are best-effort: only embedded **valid JSON payloads** are supported.
+- JavaScript object-literal payloads are intentionally unsupported.
+- No eval is used.
+- Flat timeline is default (nesting only with explicit `parentId`).
+- Confidence labels explain attribution.
+- Redaction is applied to sensitive attributes (based on config).
+
+Live tail structured logs:
+
+```bash
+npx agent-inspect tail --file ./agent.log --format json
+npx agent-inspect tail --file ./agent.log --format log4js --config agent-inspect.logs.json
+npm run dev 2>&1 | npx agent-inspect tail --format log4js --config agent-inspect.logs.json
+npx agent-inspect tail --file ./agent.log --format auto --once
+npx agent-inspect tail --file ./agent.log --json --once
+```
+
 Use a custom trace directory:
 
 ```bash
@@ -240,6 +407,12 @@ npx agent-inspect view run_abc123 --dir ./traces
 
 By default, traces are stored under `~/.agent-inspect/runs`.
 
+You can also set a default trace directory with:
+
+```bash
+AGENT_INSPECT_TRACE_DIR=./traces npx agent-inspect list
+```
+
 For local repo development after `pnpm build`:
 
 ```bash
@@ -272,13 +445,15 @@ cat ~/.agent-inspect/runs/run_abc123.jsonl | jq
 
 ## Runnable examples
 
-The repo includes five runnable MVP examples:
+The repo includes five runnable MVP manual-tracing examples, the v0.3 structured log-to-tree example, and the v0.5 LangChain adapter example:
 
 - `examples/01-basic` — `inspectRun()` + `step()`
 - `examples/02-nested-steps` — nested execution tree hierarchy
 - `examples/03-parallel-steps` — `Promise.all` sibling isolation
 - `examples/04-error-handling` — failed steps and error traces
 - `examples/05-observe-wrapper` — `observe()` wrapper with internal steps
+- `examples/06-log-to-tree` — v0.3 structured log-to-tree example (includes historical spike prototype and production `agent-inspect logs` usage)
+- `examples/08-langchain-adapter` — v0.5 LangChain callback adapter (`@agent-inspect/langchain`), provider-free simulated lifecycle (install from repo root; see example README)
 
 Run one locally:
 
@@ -302,7 +477,7 @@ Supporting material:
 
 - [examples/README.md](examples/README.md)
 
-## MVP scope
+## Original MVP scope
 
 Included:
 
@@ -314,14 +489,31 @@ Included:
 - JSONL traces
 - CLI `list` and `view`
 
+Current scope also includes:
+
+- CLI `clean` (safe deletion with verification)
+- CLI `logs` (structured log-to-tree)
+- CLI `tail` (live log tailing into grouped timelines)
+- LangChain callback adapter via `@agent-inspect/langchain`
+- Optional TUI viewer via `@agent-inspect/tui`
+- Standards-aligned **local** exports (`export`: Markdown, HTML, OpenInference-compatible JSON, OTLP JSON mapping)
+- Run diff / compare (`diff`: two local traces, read-only)
+- Canonical **fixtures** under [`fixtures/`](fixtures/README.md) plus `pnpm fixtures:check` for deterministic samples
+
 Not included:
 
-- Framework adapters
-- Token or cost tracking
-- Replay
+- Live TUI / streaming trace updates in the TUI
+- Direct vendor sinks or uploads (Phoenix, Langfuse, Braintrust, New Relic, Datadog, …)
+- Live OTLP streaming / OTLP gRPC
+- Production monitoring platforms
+- Additional framework adapters beyond LangChain
+- Token cost calculation
+- Replay / fork execution
 - SQLite
 - Dashboards
-- OpenTelemetry
+- Multi-run statistical eval dashboards
+- Semantic / LLM-powered trace comparison
+- OpenTelemetry SDK instrumentation (exports are generated strings only)
 
 ## Development
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "agent-inspect",
-  "version": "0.1.1",
+  "version": "1.0.0",
   "license": "MIT",
   "type": "module",
   "description": "Local-first execution-tree debugger for TypeScript AI agents",
@@ -62,16 +62,19 @@
   },
   "scripts": {
     "clean": "pnpm -r exec -- rm -rf dist",
-    "build": "pnpm exec tsup --config tsup.core.config.ts && pnpm exec tsup --config tsup.cli.config.ts",
+    "build": "pnpm exec tsup --config tsup.core.config.ts && pnpm exec tsup --config tsup.cli.config.ts && pnpm exec tsup --config tsup.langchain.config.ts && pnpm exec tsup --config tsup.tui.config.ts",
     "typecheck": "tsc --noEmit",
     "test": "vitest run",
     "test:watch": "vitest",
     "test:coverage": "vitest run --coverage",
     "size": "size-limit --config size-limit.config.mjs",
     "test:all": "pnpm run typecheck && pnpm run test && pnpm run build && pnpm run size",
-    "prepublish:checks": "pnpm run typecheck && pnpm run test && pnpm run build && pnpm run size && pnpm run pack:smoke",
+    "prepublish:checks": "pnpm run typecheck && pnpm run test && pnpm run test:coverage && pnpm run build && pnpm run fixtures:check && pnpm run recipes:check && pnpm run size && pnpm run pack:smoke",
     "pack:dry-run": "pnpm run build && npm pack --dry-run",
     "pack:smoke": "pnpm run build && node scripts/package-smoke.mjs",
+    "fixtures:check": "node scripts/validate-fixtures.mjs",
+    "recipes:check": "node scripts/validate-recipes.mjs",
+    "perf:baseline": "node scripts/performance-baseline.mjs",
     "examples:check": "pnpm install && pnpm --filter agent-inspect-example-01-basic run start",
     "changeset": "changeset",
     "release": "changeset publish"