agent-inspect 1.0.0 → 1.0.1

package/docs/CLI.md

@@ -0,0 +1,212 @@
+# CLI (AgentInspect 1.0)
+
+This document describes the **stable CLI surface** of AgentInspect.
+
+AgentInspect is **local-first** and **read-only by default** where possible:
+
+- **No upload** (exports write local strings/files only)
+- **No vendor sinks**
+- **No external services required**
+- **No replay/fork execution**
+
+## 1. Overview
+
+The CLI command is:
+
+```bash
+agent-inspect <command> [options]
+```
+
+Core commands:
+
+- `list` — list local runs
+- `view` — render a single run
+- `clean` — safely delete old runs (verified traces only)
+- `logs` — parse structured logs into local execution trees
+- `tail` — live-tail logs into updating local trees
+- `export` — export manual traces to Markdown/HTML/OpenInference/OTLP JSON (local only)
+- `diff` — compare two manual traces (local, read-only)
+
+## 2. Environment variables
+
+- **`AGENT_INSPECT_TRACE_DIR`**: default directory for manual trace files (`.jsonl`) when not passed via `--dir` (or API options).
+- **`AGENT_INSPECT_SILENT`**: when `true`, suppresses live terminal tree output during manual tracing (`inspectRun` / `step`). Trace files are still written.
+
+## 3. Exit code policy
+
+- **0**: command succeeded (even if a diff reports “differences”)
+- **1**: command error (invalid args, missing files, missing runs, parse failures, validation failures, etc.)
+
+AgentInspect favors **human-readable errors without stack traces** for expected user mistakes.
+
+## 4. JSON output policy
+
+Many commands support `--json` for scripting. JSON output is intended to be:
+
+- machine-parseable
+- deterministic for the same input files
+- local-only (no network)
+
+## 5. Safety and redaction notes
+
+- Log-derived output includes **confidence** labels and avoids inventing parent-child relationships.
+- Redaction defaults are conservative (e.g. `authorization`, `cookie`, `token`, `apiKey`, `password`, `secret`, `email`).
+- Exported payloads are **redacted by default** unless explicitly configured otherwise.
+
+## 6. Command reference
+
+### 6.1 `list`
+
+List recent local runs (trace files).
+
+```bash
+agent-inspect list [options]
+```
+
+Options:
+
+- `--dir <path>`: trace directory
+- `--limit <number>`: max runs to show (default 20, max 100)
+- `--status <running|success|error|unknown>`: filter by status
+- `--name <query>`: substring match on run id/name
+- `--since <duration>`: only include recent runs (e.g. `30s`, `5m`, `2h`, `7d`)
+- `--json`: print list as JSON
+
+### 6.2 `view`
+
+Render a single manual trace by run id.
+
+```bash
+agent-inspect view <run-id> [options]
+```
+
+Options:
+
+- `--dir <path>`: trace directory
+- `--summary`: run summary (counts, max depth, longest step)
+- `--metadata`: file path/size + timestamps
+- `--errors-only`: only error events/failed steps
+- `--verbose`: include extra detail (types, metadata, stacks)
+- `--json`: print raw trace events as JSON
+- `--tui`: open optional interactive TUI viewer (requires `@agent-inspect/tui`)
+
+### 6.3 `clean`
+
+Safely delete old local trace files. This is safety-critical: the CLI verifies trace files before deletion.
+
+```bash
+agent-inspect clean --older-than <duration> [--dry-run] [--yes]
+agent-inspect clean --keep <count> [--dry-run] [--yes]
+```
+
+Options:
+
+- `--dir <path>`: trace directory
+- `--older-than <duration>`: delete runs older than a duration
+- `--keep <count>`: keep N most recent runs
+- `--dry-run`: show what would be deleted
+- `--yes`: skip confirmation prompt
+
+Recommendation: run with `--dry-run` first.
+
+### 6.4 `logs`
+
+Parse structured logs into local execution trees.
+
+```bash
+agent-inspect logs <file> [options]
+```
+
+Options:
+
+- `--format <auto|json|log4js>`
+- `--config <path>`: ingest config JSON (see `docs/SCHEMA.md` for config types)
+- `--run-id-key <keys>`: override runId keys (comma-separated)
+- `--event-key <key>`: override event key
+- `--timestamp-key <key>`: override timestamp key
+- `--message-key <key>`: override message key
+- `--level-key <key>`: override level key
+- `--parent-id-key <key>`: override parent id key
+- `--duration-key <key>`: override duration key
+- `--status-key <key>`: override status key
+- `--json`: emit JSON payload (events/trees/warnings/summary)
+- `--summary`: include summary section in human output
+- `--warnings <none|summary|all>`: warning output mode
+
+Example (fixtures):
+
+```bash
+agent-inspect logs fixtures/logs/proactive-json.log --format json --config fixtures/configs/proactive-agent-inspect.logs.json
+```
+
+### 6.5 `tail`
+
+Live-tail logs into updating execution trees in the terminal.
+
+```bash
+agent-inspect tail [options]
+```
+
+Options:
+
+- `--file <path>`: tail a file (otherwise reads stdin)
+- `--format <auto|json|log4js>`
+- `--config <path>`
+- `--once`: read once and exit (useful for CI/scripting with `--file`)
+- `--warnings <none|summary|all>`
+- `--refresh <ms>`: minimum time between renders
+- `--json`: newline-delimited JSON updates
+
+Important: `tail` is a local developer tool, not a production monitor.
+
+### 6.6 `export`
+
+Export a manual trace run to local formats. **No upload**.
+
+```bash
+agent-inspect export <run-id> --format <markdown|html|openinference|otlp-json> [options]
+```
+
+Options:
+
+- `--dir <path>`
+- `--format <format>`
+- `-o, --output <path>`: write file
+- `--json`: JSON wrapper output (includes content if not writing to file)
+- `--validate`: validate exported payload shape
+- `--include-attributes`: include bounded attributes (review before sharing)
+- `--no-metadata`: omit summary/metadata sections
+- `--no-errors`: omit error sections
+
+### 6.7 `diff`
+
+Compare two manual trace runs. Diff is **local** and **read-only** (does not rerun agents).
+
+```bash
+agent-inspect diff <left-run-id> <right-run-id> [options]
+```
+
+Options:
+
+- `--dir <path>`
+- `--json`
+- `--ignore-duration`
+- `--duration-threshold <duration>`
+- `--focus <all|errors|structure|outputs>`
+- `--check <all|structure|outputs|errors|timing>`
+
+## 7. Optional TUI behavior
+
+`view --tui` delegates to `@agent-inspect/tui` and requires an interactive terminal. If the package is not installed, the CLI prints a short install hint.
+
+## 8. Warnings behavior
+
+Log parsing emits warnings for malformed lines or missing required keys. `--warnings` controls whether warnings are hidden, summarized, or printed line-by-line.
+
+## 9. Limitations (reminder)
+
+See:
+
+- `docs/KNOWN-ISSUES.md`
+- `docs/LIMITATIONS.md`
+

package/docs/COMPARE.md

@@ -0,0 +1,69 @@
+## Compare AgentInspect
+
+AgentInspect is a local-first execution-tree debugger for TypeScript AI agents. It’s designed for inner-loop debugging and quick inspection — not as a replacement for hosted observability, evaluation, or production monitoring platforms.
+
+## AgentInspect vs console.log
+
+- **console.log is flat**: logs are a stream of lines without run grouping or step boundaries.
+- **AgentInspect adds structure**: runs, nested steps, step types (tool/LLM), durations, status summaries, and local trace files you can inspect later.
+- **console.log still matters**: use it for quick values or ad-hoc debugging inside a step.
+- **AgentInspect helps you understand flow**: especially when your agent fans out, retries, or has nested tool/LLM calls.
+
+## AgentInspect vs LangSmith
+
+LangSmith is a hosted/platform workflow for tracing, evaluation, and observability in the LangChain ecosystem.
+
+AgentInspect is local-first CLI debugging:
+
+- Use AgentInspect to debug locally before/alongside LangSmith when iterating on agent logic.
+- AgentInspect does not provide hosted dashboards, dataset/eval workflows, or production tracing pipelines.
+
+## AgentInspect vs Langfuse
+
+Langfuse provides broader LLM observability (tracing, prompt management, datasets/evals, dashboards).
+
+AgentInspect focuses on local execution trees and CLI workflows:
+
+- Complementary: use AgentInspect for quick local run understanding; use Langfuse for dashboards and longer-lived observability workflows.
+- Not a replacement.
+
+## AgentInspect vs Braintrust
+
+Braintrust is strong for evals, regressions, datasets, and production AI quality workflows.
+
+AgentInspect is lighter and local-first:
+
+- Use AgentInspect to understand a single run locally.
+- Use Braintrust when you want repeatable evals, comparisons at scale, and production quality workflows.
+
+## AgentInspect vs Phoenix / OpenInference
+
+Phoenix and the OpenInference ecosystem are standards-oriented and useful for trace visualization and interoperability.
+
+AgentInspect can export **OpenInference-compatible JSON** and **OTLP JSON** as local files:
+
+- Exports are **compatibility-oriented** and should be validated against your target backend/collector.
+- AgentInspect does not claim that every backend will accept these exports without configuration.
+
+## AgentInspect vs OpenTelemetry setup
+
+OpenTelemetry is powerful, but setup can be heavier (SDK configuration, exporters, collectors, backend).
+
+AgentInspect avoids SDK/collector setup for local debugging:
+
+- Use AgentInspect when you want quick, local execution trees with no collector required.
+- Use OpenTelemetry when you need organization-wide production telemetry pipelines.
+- Exports can help bridge later, but AgentInspect is not an OpenTelemetry SDK replacement.
+
+## Quick decision table
+
+| Need | AgentInspect fit |
+| --- | --- |
+| Local agent debugging | Strong fit |
+| No-account CLI tracing | Strong fit |
+| Production dashboards | Not the goal |
+| Hosted eval datasets | Not the goal |
+| Prompt management | Not the goal |
+| Standards-aligned local export | Partial (compatibility-oriented) |
+| Full observability platform | Use a dedicated platform |
+

package/docs/DIFF.md

@@ -0,0 +1,11 @@
+## Diff / compare
+
+AgentInspect can compare two runs (manual traces) locally and render a diff summary.
+
+- **CLI usage**: `docs/CLI.md` (`diff`)
+- **Schema expectations for diff inputs**: `docs/SCHEMA.md` (manual trace JSONL, `schemaVersion: "0.1"`)
+
+Notes:
+- Programmatic diff APIs are documented as **experimental** in `docs/API.md`.
+- `diff` is read-only and does **not** rerun agents; “differences” do not necessarily imply failure (command errors do).
+

package/docs/EXPORTS.md

@@ -0,0 +1,12 @@
+## Exports
+
+AgentInspect supports local-only exports from run trees and traces. Exports are intended for **compatibility and sharing**, but should be reviewed before sending to others.
+
+- **CLI usage**: `docs/CLI.md` (`export`)
+- **Schema + compatibility guarantees**: `docs/SCHEMA.md`
+- **Safety / redaction notes**: `docs/CLI.md`, `SECURITY.md`, and `docs-local/architecture/REDACTION.md`
+
+Notes:
+- Export formats (Markdown/HTML/OpenInference/OTLP JSON) are documented as **experimental / compatibility-oriented** in `docs/API.md`.
+- AgentInspect does **not** upload anywhere; exports write local strings/files only.
+

package/docs/GETTING-STARTED.md

@@ -0,0 +1,128 @@
+# Getting started
+
+AgentInspect is a **local-first execution-tree debugger** for TypeScript AI agents. It helps you produce and inspect an execution tree of steps, safely and deterministically, without uploading data anywhere.
+
+## 1. Install
+
+```bash
+pnpm add agent-inspect
+```
+
+The `agent-inspect` package includes the CLI binary via its `bin` field:
+
+```bash
+npx agent-inspect --help
+```
+
+For local repo development (this monorepo), build and run the CLI from `packages/cli`:
+
+```bash
+pnpm build
+node packages/cli/dist/index.cjs --help
+```
+
+## 2. Basic manual trace
+
+```ts
+import { inspectRun, step } from "agent-inspect";
+
+await inspectRun("demo-agent", async () => {
+  await step("plan", async () => "ok");
+  await step.tool("search", async () => ({ count: 2 }));
+  await step.llm("fixture-model", async () => "done");
+});
+```
+
+This writes a local JSONL trace with stable v1.0 event names:
+
+- `run_started`, `run_completed`
+- `step_started`, `step_completed`
+
+## 3. View runs
+
+```bash
+agent-inspect list
+agent-inspect view <runId>
+```
+
+## 4. Clean old runs (safely)
+
+Always start with `--dry-run`:
+
+```bash
+agent-inspect clean --older-than 7d --dry-run
+agent-inspect clean --older-than 7d --yes
+```
+
+## 5. Parse existing logs
+
+```bash
+agent-inspect logs fixtures/logs/proactive-json.log \
+  --format json \
+  --config fixtures/configs/proactive-agent-inspect.logs.json
+```
+
+## 6. Tail logs
+
+For scripting/CI-style usage, `--once` reads and exits:
+
+```bash
+agent-inspect tail \
+  --file fixtures/logs/proactive-json.log \
+  --format json \
+  --config fixtures/configs/proactive-agent-inspect.logs.json \
+  --once
+```
+
+## 7. Export a run
+
+```bash
+agent-inspect export minimal-success --dir fixtures/traces --format markdown
+agent-inspect export minimal-success --dir fixtures/traces --format openinference --validate
+```
+
+Exports are **local-only** and do not upload anywhere.
+
+## 8. Diff two runs
+
+```bash
+agent-inspect diff minimal-success minimal-error --dir fixtures/traces
+```
+
+## 9. Try recipes
+
+See `examples/recipes/README.md`.
+
+## 10. Optional LangChain adapter
+
+`@agent-inspect/langchain` is optional and **experimental**:
+
+```bash
+pnpm add @agent-inspect/langchain
+```
+
+## 11. Optional TUI
+
+`@agent-inspect/tui` is optional and **experimental**. The CLI can invoke it with:
+
+```bash
+agent-inspect view <runId> --tui
+```
+
+## 12. Safety notes
+
+- Redaction is on by default for log-derived attributes and exports.
+- Confidence labels are required to keep attribution honest.
+- AgentInspect is for local debugging, not production monitoring.
+
+## 13. Next docs
+
+- `docs/API.md`
+- `docs/CLI.md`
+- `docs/SCHEMA.md`
+- `docs/COMPARE.md`
+- `docs/LOG-TO-TREE-QUICKSTART.md`
+- `docs/KNOWN-ISSUES.md`
+- `docs/LIMITATIONS.md`
+- `docs-local/V1-READINESS-CHECKLIST.md`
+

package/docs/KNOWN-ISSUES.md

@@ -0,0 +1,34 @@
+# Known issues
+
+AgentInspect is **local-first** and **CLI-first**. These behaviors are intentional constraints or best-effort areas—not silent guarantees.
+
+## Logs
+
+- **log4js-style parsing** is **best-effort**: embedded JSON must be recoverable from the line; unusual layouts may lose fields or warn.
+- **JavaScript object-style log payloads** (e.g. `{ msg: '...', meta: ... }` printed without JSON) are **not** a supported interchange format for ingestion.
+
+## Exports
+
+- **OpenInference** and **OTLP JSON** exports are **compatibility-oriented** and **experimental**. Validate against your target collector or backend before relying on them.
+- Exports generate **strings/files locally** only—there is **no** automatic upload.
+
+## Integrations
+
+- **Vendor sinks** (hosted dashboards, Langfuse/Braintrust/New Relic/Datadog native uploads, OTLP gRPC streaming, etc.) are **not implemented** in the core packages described here.
+- **LangChain adapter** captures **metadata-oriented** signals by default; it does not replace full framework observability.
+
+## UI / replay
+
+- **Optional TUI** (`@agent-inspect/tui`) is separate from the default CLI and requires an interactive terminal where documented.
+- **Live streaming tree inside the TUI** is not the same as **live tail** over logs; product scope varies by version—see roadmap docs.
+- **Full replay / fork execution** of historical traces is **out of scope** for current MVP-style releases.
+
+## Cost / tokens
+
+- **Token counting** and **cost calculation** are **not** core features.
+
+## Confidence labels
+
+- Log-derived trees carry **confidence** (`explicit`, `correlated`, `heuristic`, `unknown`). Labels explain **how** relationships were inferred—they are not semantic proof.
+
+When in doubt, treat AgentInspect as a **debugging aid**, not a compliance or billing system of record.

package/docs/LIMITATIONS.md

@@ -0,0 +1,32 @@
+# Limitations
+
+This document states what AgentInspect **does not** provide today. It complements [KNOWN-ISSUES.md](./KNOWN-ISSUES.md).
+
+## Product boundaries
+
+- **No SaaS dashboard** or hosted multi-tenant product in the open-source core workflow.
+- **No production APM replacement**: no sampling agents, no fleet-wide aggregation, no uptime SLAs.
+- **No vendor upload pipeline**: no built-in Langfuse/Braintrust/New Relic/Datadog direct exporters as live sinks.
+- **No automatic universal instrumentation** of every framework: integration is explicit (manual traces, log ingest, optional adapters).
+
+## Data fidelity
+
+- **No full prompt/output capture by default** for manual traces (by design: safety and PII risk).
+- **Log-derived runs** and **manual JSONL traces** may differ in fidelity (timestamps, nesting, confidence).
+- **Confidence labels** qualify inferred relationships; they do not guarantee correctness.
+
+## Execution semantics
+
+- **No replay / fork** of past runs from traces alone.
+- **No time-travel debugging** across arbitrary runtime state.
+- **No multi-run statistical evaluation** built into core.
+
+## Economics
+
+- **No cost engine**: no pricing tables, invoice-grade usage, or provider billing reconciliation.
+
+## Scale
+
+- Designed for **developer machines** and **inner-loop debugging**, not petabyte log warehouses.
+
+For roadmap intent, see `docs-local/roadmap/VERSION-ROADMAP.md` and `docs-local/prd/v1.0-STABLE-LOCAL-INSPECTOR-PRD.md` (planning docs; not a promise of future features).

package/docs/LOG-TO-TREE-QUICKSTART.md

@@ -0,0 +1,54 @@
+## Log-to-tree quickstart
+
+AgentInspect can inspect **structured logs** and render a local execution tree without requiring you to wrap every function in manual tracing.
+
+## Example JSON log lines
+
+`agent.log` (JSONL):
+
+```json
+{"timestamp":"2026-05-08T10:00:00.000Z","requestId":"req_123","event":"agent.started","message":"Agent started"}
+{"timestamp":"2026-05-08T10:00:00.150Z","requestId":"req_123","event":"tool.search.started","tool":"searchDocs"}
+{"timestamp":"2026-05-08T10:00:00.420Z","requestId":"req_123","event":"tool.search.completed","tool":"searchDocs","durationMs":270}
+{"timestamp":"2026-05-08T10:00:00.500Z","requestId":"req_123","event":"llm.answer.started","model":"gpt-example"}
+{"timestamp":"2026-05-08T10:00:01.100Z","requestId":"req_123","event":"llm.answer.completed","model":"gpt-example","durationMs":600}
+{"timestamp":"2026-05-08T10:00:01.150Z","requestId":"req_123","event":"agent.completed","status":"ok"}
+```
+
+## Parse logs into trees
+
+```bash
+npx agent-inspect logs ./agent.log \
+  --format json \
+  --run-id-key requestId \
+  --event-key event \
+  --timestamp-key timestamp
+```
+
+Example output:
+
+```text
+Run req_123
+├─ agent.started
+├─ tool.search.started
+├─ tool.search.completed (270ms)
+├─ llm.answer.started
+├─ llm.answer.completed (600ms)
+└─ agent.completed (ok)
+```
+
+## Important notes
+
+- **JSON logs are first-class**. (Line-delimited JSON is the recommended input.)
+- **log4js text logs with embedded JSON are best-effort**.
+- AgentInspect does **not** evaluate JavaScript object strings (`eval` is not used).
+- **Flat timeline is the default** for log-derived events.
+- Nesting is conservative: **explicit parent/config wins**, and inferred relationships are labeled.
+- **Confidence labels** explain how relationships were inferred (`explicit`, `correlated`, `heuristic`, `unknown`).
+- **Redaction** is applied where configured/defaulted for log-derived attributes and exports.
+
+See also:
+- `docs/CLI.md` (`logs`, `tail`)
+- `docs/LOGS.md`
+- `docs/SCHEMA.md` (log ingest config types + confidence)
+

package/docs/LOGS.md

@@ -0,0 +1,13 @@
+## Logs (structured log → tree)
+
+AgentInspect can parse **existing logs** (line-delimited JSON, and best-effort log4js-style text with embedded JSON) into a local execution tree / grouped timeline.
+
+- **CLI usage**: `docs/CLI.md` (`logs`, `tail`)
+- **Quickstart**: `docs/LOG-TO-TREE-QUICKSTART.md`
+- **Log ingest config (field mapping + redaction)**: `docs-local/architecture/LOG-INGEST-CONFIG.md`
+- **Safety constraints** (no eval, no JS object parsing): `docs-local/architecture/ARCHITECTURE.md`
+
+Notes:
+- Log parsing APIs are documented as **experimental** in `docs/API.md`.
+- Log-derived events include **confidence labels**; AgentInspect is conservative about inferring parent/child structure.
+

package/docs/MIGRATION.md

@@ -0,0 +1,109 @@
+# Migration guide (to AgentInspect 1.0)
+
+This guide summarizes how to move from early AgentInspect MVP usage to **AgentInspect 1.0**.
+
+AgentInspect remains local-first: it does not introduce any network upload or vendor sink workflows.
+
+## Scope
+
+Covered:
+
+- manual tracing (`inspectRun`, `step`, `observe`)
+- trace directory behavior
+- CLI commands (`list`, `view`, `clean`, `logs`, `tail`, `export`, `diff`)
+- optional packages (`@agent-inspect/langchain`, `@agent-inspect/tui`)
+- schema compatibility guarantees
+
+Not covered:
+
+- publish/version bump workflows (see `docs-local/RELEASE-CHECKLIST.md`)
+- vendor sinks (not implemented)
+- replay (not implemented)
+- cost engine (not implemented)
+
+## Manual tracing API
+
+If you were using:
+
+```ts
+import { inspectRun, step } from "agent-inspect";
+```
+
+that remains the recommended stable path. AgentInspect 1.0 is specifically about keeping these entry points compatible.
+
+### Event names and failure representation
+
+Manual JSONL event names remain stable:
+
+- `run_started`
+- `run_completed`
+- `step_started`
+- `step_completed`
+
+There is **no `step_failed` event**. Step failures are represented as:
+
+- `step_completed` with `status: "error"`
+
+Existing `schemaVersion: "0.1"` traces remain readable. No migration command is required.
+
+## Trace directory behavior
+
+- `AGENT_INSPECT_TRACE_DIR` is supported.
+- When unset, AgentInspect uses its default local directory (see `docs/CLI.md` and `docs/API.md`).
+- Trace files are JSONL and are not automatically rewritten.
+
+## CLI changes and additive commands
+
+Manual inspection commands (`list`, `view`) are stable and local-only.
+
+The following commands are additive workflows that remain local-only:
+
+- `logs`: parse structured logs into normalized trees
+- `tail`: live-tail structured logs in the terminal
+- `export`: export a manual trace as Markdown/HTML/OpenInference-compatible JSON/OTLP JSON (local-only)
+- `diff`: compare two manual traces (local, read-only)
+
+### `clean` is safety-critical
+
+`clean` verifies traces before deletion using conservative heuristics. If a file cannot be verified as an AgentInspect trace, it is skipped.
+
+## Logs and tail
+
+- JSON logs are first-class.
+- log4js parsing is best-effort.
+- No JS object literal parsing or `eval`.
+- Redaction is applied to log-derived attributes based on config/default rules.
+
+## Export and share safety
+
+- Exports are **generated locally** and do not upload anywhere.
+- Exporters default to **redacted** and **bounded** attribute previews.
+- Always review exports before sharing.
+
+## Diff and compare
+
+- Diff compares two existing local traces.
+- Diff does not rerun agents, mutate trace files, or call an LLM.
+
+## Optional LangChain adapter
+
+`@agent-inspect/langchain` is optional and separate from core. It uses `@langchain/core` as a **peer dependency**.
+
+## Optional TUI
+
+`@agent-inspect/tui` is optional and separate from core. It contains `ink`/`react` dependencies so the main `agent-inspect` install remains lean.
+
+## Breaking changes
+
+This stabilization effort aims to avoid breaking changes. If a breaking change is ever required:
+
+- it requires a major version
+- it must preserve trace readability where possible
+- it must be explicitly documented
+
+## Known non-migrations
+
+- No vendor sink migration exists because there are **no vendor sinks** in core.
+- No replay migration exists because replay is **not implemented**.
+- No cost engine migration exists because cost calculation is **not implemented**.
+