agent-inspect 0.1.2 → 1.0.1

package/CHANGELOG.md

@@ -0,0 +1,99 @@
+# Changelog
+
+## 1.0.1
+
+### Patch Changes
+
+- 575b093: docs: onboarding polish
+
+## Unreleased
+
+### Documentation
+
+- Improved README onboarding for new users.
+- Added comparison guide.
+- Added log-to-tree quickstart.
+- Added a quickstart demo example.
+- Clarified docs organization between user-facing `docs/` and internal `docs-local/`.
+
+## 1.0.0
+
+### Stable local tracing
+
+- Stable manual tracing entry points: `inspectRun`, `step`, `step.llm`, `step.tool`, `observe`
+- v0.1 JSONL trace compatibility retained (schemaVersion `"0.1"`)
+
+### Local inspection CLI
+
+- Stable CLI workflows: `list`, `view`, `clean`
+- Safety-critical cleanup verifies traces before deletion
+
+### Structured logs and live tail
+
+- Local log-to-tree parsing and live tail workflows (`logs`, `tail`) with confidence labeling
+- Best-effort log4js parsing; JSON logs first-class; no unsafe object parsing
+
+### Optional LangChain adapter
+
+- `@agent-inspect/langchain` optional adapter package (experimental surface)
+
+### Optional TUI
+
+- `@agent-inspect/tui` optional Ink/React viewer (experimental programmatic surface)
+
+### Standards-aligned local export
+
+- Markdown/HTML exports for sharing traces locally
+- OpenInference-compatible JSON export (experimental; verify against backends)
+- OTLP JSON export (experimental; JSON mapping only, no OTLP gRPC)
+
+### Diff and compare
+
+- Local, read-only diff of two manual traces (`diff`)
+
+### Fixtures, recipes, and hardening (v0.9)
+
+- Canonical fixtures under `fixtures/` plus validation scripts
+- Runnable recipes under `examples/recipes/` with deterministic expected output markers
+- Package smoke checks and adoption hardening tests
+
+### Documentation and stability
+
+- Added/updated API/CLI/schema/getting-started docs for v1.0 stabilization
+- Added stability and compatibility tests to prevent accidental surface breaks
+
+### Known limitations
+
+- Local-first only; no SaaS/dashboard; no vendor sinks; no replay; no cost engine
+
+## Historical notes (v0.1–v0.9)
+
+AgentInspect started as a minimal manual tracing MVP (v0.1) and evolved through:
+
+- local inspection improvements (metadata, filtering, safety checks)
+- structured log ingestion (JSON first-class, log4js best-effort)
+- conservative tree building rules with confidence labels
+- incremental live tail rendering
+- standards-aligned local exports (experimental)
+- run diff and compare
+- fixtures, recipes, and hardening focused on adoption
+
+For detailed intent and sequencing (planning docs), see:
+
+- `docs-local/roadmap/VERSION-ROADMAP.md`
+- `docs-local/strategy/PRODUCT-PRINCIPLES.md`
+
+# agent-inspect
+
+## 0.1.2
+
+### Patch Changes
+
+- 62afb94: fix CI/publish smoke + vitest config
+
+## 0.1.1
+
+### Patch Changes
+
+- bd719ef: Prepare npm publishing (Trusted Publishing via GitHub Actions OIDC) and polish documentation.
+- 76791b8: Improve README, release docs, and npm publishing guidance.

package/README.md

@@ -1,24 +1,10 @@
 # agent-inspect
 
-agent-inspect is a local-first execution-tree debugger for TypeScript AI agents.
+Local execution trees for TypeScript AI agents.
 
-## Why
+AgentInspect helps you debug multi-step AI workflows locally by turning manual steps, structured logs, and agent callbacks into readable execution trees.
 
-AI agents are multi-step. Console logs are flat.
-
-agent-inspect turns runs into structured execution trees with JSONL traces and CLI inspection.
-
-agent-inspect is designed for inner-loop debugging, not as a replacement for production observability platforms.
-
-## What you get
-
-- Execution-tree tracing for TypeScript agent workflows
-- Nested `step()` support with parent-child relationships
-- `step.llm()` and `step.tool()` helpers for agent-aware traces
-- Local JSONL trace files
-- Real-time terminal output while the agent runs
-- CLI commands to inspect previous runs
-- No accounts, API keys, dashboards, or cloud ingestion
+No account. No cloud upload. No dashboard required.
 
 ## Install
 
@@ -26,25 +12,140 @@ agent-inspect is designed for inner-loop debugging, not as a replacement for pro
 npm install agent-inspect
 ```
 
-## See your first trace
+```bash
+pnpm add agent-inspect
+```
+
+## 60-second quickstart
 
-Run a traced workflow, then inspect it with the CLI.
+Create `demo.mjs`:
 
-```ts
+```js
 import { inspectRun, step } from "agent-inspect";
 
-await inspectRun("hello-agent", async () => {
-  const plan = await step("plan", async () => "search hotels");
-  return step("finalize", async () => ({ plan, status: "done" }));
-});
+const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
+
+await inspectRun(
+  "support-agent",
+  async () => {
+    const plan = await step("plan", async () => {
+      await delay(50);
+      return { query: "refund policy", intent: "support" };
+    });
+
+    const docs = await step.tool("search-docs", async () => {
+      await delay(75);
+      return ["Refunds are available within 30 days."];
+    });
+
+    return step.llm("answer", async () => {
+      await delay(100);
+      return `Based on ${docs.length} document(s), refunds are available within 30 days.`;
+    });
+  },
+  { traceDir: "./.agent-inspect" }
+);
 ```
 
+Run it, then inspect it:
+
 ```bash
-npx agent-inspect list
-npx agent-inspect view run_abc123
+node demo.mjs
+npx agent-inspect list --dir ./.agent-inspect
+npx agent-inspect view <run-id> --dir ./.agent-inspect
+npx agent-inspect view <run-id> --dir ./.agent-inspect --summary
+```
+
+Simplified example output:
+
+```text
+Run run_abc123 (support-agent)
+├─ ✔ plan (50ms)
+├─ ✔ tool:search-docs (75ms)
+└─ ✔ llm:answer (100ms)
+
+Summary:
+  Steps: 3 (0 error)
+  Duration: 225ms
+```
+
+Want a runnable demo folder? See [examples/00-quickstart-demo](examples/00-quickstart-demo/README.md).
+
+## Why not just console.log?
+
+Console logs are great for quick values, but they’re flat. AgentInspect gives you:
+
+- run grouping and local trace files
+- explicit step boundaries (including nesting)
+- step types (`tool:*`, `llm:*`)
+- status + duration summaries
+- a CLI to list/view/export/diff runs
+- log ingestion workflows (`logs`, `tail`) when you already have structured logs
+
+## Inspect existing structured logs
+
+```bash
+npx agent-inspect logs ./agent.log \
+  --format json \
+  --run-id-key requestId \
+  --event-key event \
+  --timestamp-key timestamp
 ```
 
-Replace `run_abc123` with the run id printed by `agent-inspect list`.
+See the log-to-tree guide: [docs/LOG-TO-TREE-QUICKSTART.md](docs/LOG-TO-TREE-QUICKSTART.md).
+
+## When to use AgentInspect
+
+Use AgentInspect when:
+
+- you are building TypeScript/Node.js AI agents
+- you want local debugging before a hosted observability setup
+- console logs are too flat for multi-step execution
+- you want to inspect tool calls, LLM calls, failures, and durations locally
+- you want a lightweight CLI workflow with no account and no cloud upload
+- you want to compare two local runs
+- you want to turn structured logs into readable execution trees
+
+## When not to use AgentInspect
+
+Do not use AgentInspect as a replacement for:
+
+- production monitoring or alerting
+- hosted observability dashboards
+- long-term trace storage
+- eval dataset management
+- prompt management
+- cost analytics
+- replay/fork execution
+- vendor telemetry pipelines
+
+AgentInspect can complement tools like LangSmith, Langfuse, Braintrust, Phoenix/OpenInference, OpenTelemetry, New Relic, Datadog, etc. It does not replace their production/eval/dashboard workflows.
+
+## Security and privacy posture
+
+- local files only by default (no upload)
+- no vendor sinks
+- no API keys required
+- small root dependency footprint
+- traces can include **user-provided metadata**; review exports before sharing
+
+See `SECURITY.md`.
+
+## Documentation
+
+- **Getting started**: [docs/GETTING-STARTED.md](docs/GETTING-STARTED.md)
+- **API**: [docs/API.md](docs/API.md)
+- **CLI**: [docs/CLI.md](docs/CLI.md)
+- **Schema**: [docs/SCHEMA.md](docs/SCHEMA.md)
+- **Logs**: [docs/LOGS.md](docs/LOGS.md) and [docs/LOG-TO-TREE-QUICKSTART.md](docs/LOG-TO-TREE-QUICKSTART.md)
+- **Exports**: [docs/EXPORTS.md](docs/EXPORTS.md)
+- **Diff**: [docs/DIFF.md](docs/DIFF.md)
+- **Adapters**: [docs/ADAPTERS.md](docs/ADAPTERS.md)
+- **Compare with other tools**: [docs/COMPARE.md](docs/COMPARE.md)
+- **Known issues**: [docs/KNOWN-ISSUES.md](docs/KNOWN-ISSUES.md)
+- **Limitations**: [docs/LIMITATIONS.md](docs/LIMITATIONS.md)
+
+Screenshots/GIFs are planned; see [docs/SCREENSHOTS.md](docs/SCREENSHOTS.md).
 
 ## Minimal API
 
@@ -217,6 +318,41 @@ await agent.run("How do I reset my password?");
 
 `observe()` wraps top-level `run`, `execute`, and `invoke` methods. For internal detail, add manual `step()` calls inside the agent.
 
+## LangChain adapter (experimental)
+
+Install:
+
+```bash
+pnpm add agent-inspect @agent-inspect/langchain @langchain/core
+```
+
+`@langchain/core` is a **peer dependency** of `@agent-inspect/langchain`. The adapter uses official LangChain.js **callbacks** only (extends `BaseCallbackHandler`): **no** monkey-patching, **no** `agent-inspect/auto`, **no** vendor observability sinks.
+
+```ts
+import { AgentInspectCallback } from "@agent-inspect/langchain";
+
+const callback = new AgentInspectCallback({
+  runName: "support-agent-eval",
+  capture: "metadata-only",
+});
+
+await agent.invoke(input, {
+  callbacks: [callback],
+});
+
+const events = callback.getEvents();
+```
+
+Behavior:
+
+- **Metadata-only** capture by default (model, tags, token usage when present, counts). **No** full prompt/output capture by default.
+- **Preview** mode is opt-in (`capture: "preview"`) with truncation via `maxPreviewChars` (default `200`).
+- **Parent** links use LangChain `parentRunId`, surfaced as `parentId` on `InspectEvent` with `confidence: "explicit"`.
+- **No** cost calculation; token fields are informational only.
+- Events are collected **in memory** only (`getEvents()` / `clear()`). **No trace-file persistence** for adapter events yet; they are **not** written into the manual JSONL trace format.
+
+The LangChain adapter API remains **experimental** even though the core AgentInspect tracing API is stable in 1.0. See [examples/08-langchain-adapter](examples/08-langchain-adapter).
+
 ## CLI
 
 List recent runs:
@@ -225,12 +361,83 @@ List recent runs:
 npx agent-inspect list
 ```
 
+Common filters:
+
+```bash
+npx agent-inspect list --status success
+npx agent-inspect list --status error
+npx agent-inspect list --status running
+npx agent-inspect list --status unknown
+npx agent-inspect list --name hotel
+npx agent-inspect list --since 24h
+npx agent-inspect list --json
+```
+
 View a run:
 
 ```bash
 npx agent-inspect view run_abc123
 ```
 
+Alternate view modes:
+
+```bash
+npx agent-inspect view run_abc123 --summary
+npx agent-inspect view run_abc123 --metadata
+npx agent-inspect view run_abc123 --errors-only
+npx agent-inspect view run_abc123 --json --summary
+```
+
+Safely clean up old traces (recommended: start with `--dry-run`):
+
+```bash
+npx agent-inspect clean --older-than 7d --dry-run
+npx agent-inspect clean --older-than 7d
+npx agent-inspect clean --keep 100 --dry-run
+npx agent-inspect clean --keep 100 --yes
+npx agent-inspect clean --dir ./traces --older-than 7d --dry-run
+```
+
+Safety notes:
+
+- `clean` **verifies each file** as an AgentInspect trace before deleting.
+- Arbitrary JSONL files are **not deleted**.
+- Malformed JSONL files are **not deleted**.
+- Without `--dry-run`, `clean` requires confirmation unless `--yes` is provided.
+- In non-interactive terminals, deletion requires `--yes`.
+
+Inspect structured logs:
+
+```bash
+npx agent-inspect logs ./agent.log --format json
+npx agent-inspect logs ./agent.log --format log4js
+npx agent-inspect logs ./agent.log --format auto
+npx agent-inspect logs ./agent.log --config agent-inspect.logs.json
+npx agent-inspect logs ./agent.log --json
+npx agent-inspect logs ./agent.log --summary
+npx agent-inspect logs ./agent.log --warnings all
+```
+
+Log ingestion notes:
+
+- JSON logs are first-class.
+- log4js text logs are best-effort: only embedded **valid JSON payloads** are supported.
+- JavaScript object-literal payloads are intentionally unsupported.
+- No eval is used.
+- Flat timeline is default (nesting only with explicit `parentId`).
+- Confidence labels explain attribution.
+- Redaction is applied to sensitive attributes (based on config).
+
+Live tail structured logs:
+
+```bash
+npx agent-inspect tail --file ./agent.log --format json
+npx agent-inspect tail --file ./agent.log --format log4js --config agent-inspect.logs.json
+npm run dev 2>&1 | npx agent-inspect tail --format log4js --config agent-inspect.logs.json
+npx agent-inspect tail --file ./agent.log --format auto --once
+npx agent-inspect tail --file ./agent.log --json --once
+```
+
 Use a custom trace directory:
 
 ```bash
@@ -240,6 +447,12 @@ npx agent-inspect view run_abc123 --dir ./traces
 
 By default, traces are stored under `~/.agent-inspect/runs`.
 
+You can also set a default trace directory with:
+
+```bash
+AGENT_INSPECT_TRACE_DIR=./traces npx agent-inspect list
+```
+
 For local repo development after `pnpm build`:
 
 ```bash
@@ -272,13 +485,16 @@ cat ~/.agent-inspect/runs/run_abc123.jsonl | jq
 
 ## Runnable examples
 
-The repo includes five runnable MVP examples:
+The repo includes runnable examples for manual tracing, log-to-tree, and the optional LangChain adapter:
 
+- `examples/00-quickstart-demo` — minimal install-and-try demo
 - `examples/01-basic` — `inspectRun()` + `step()`
 - `examples/02-nested-steps` — nested execution tree hierarchy
 - `examples/03-parallel-steps` — `Promise.all` sibling isolation
 - `examples/04-error-handling` — failed steps and error traces
 - `examples/05-observe-wrapper` — `observe()` wrapper with internal steps
+- `examples/06-log-to-tree` — structured log-to-tree example (`agent-inspect logs`, `tail`)
+- `examples/08-langchain-adapter` — optional LangChain callback adapter (`@agent-inspect/langchain`), provider-free simulated lifecycle (install from repo root; see example README)
 
 Run one locally:
 
@@ -302,26 +518,20 @@ Supporting material:
 
 - [examples/README.md](examples/README.md)
 
-## MVP scope
-
-Included:
-
-- `inspectRun()`
-- `step()`
-- `step.llm()`
-- `step.tool()`
-- `observe()`
-- JSONL traces
-- CLI `list` and `view`
-
 Not included:
 
-- Framework adapters
-- Token or cost tracking
-- Replay
+- Live TUI / streaming trace updates in the TUI
+- Direct vendor sinks or uploads (Phoenix, Langfuse, Braintrust, New Relic, Datadog, …)
+- Live OTLP streaming / OTLP gRPC
+- Production monitoring platforms
+- Additional framework adapters beyond LangChain
+- Token cost calculation
+- Replay / fork execution
 - SQLite
 - Dashboards
-- OpenTelemetry
+- Multi-run statistical eval dashboards
+- Semantic / LLM-powered trace comparison
+- OpenTelemetry SDK instrumentation (exports are generated strings only)
 
 ## Development
 

package/SECURITY.md

@@ -0,0 +1,77 @@
+# Security policy
+
+AgentInspect is a **local-first** debugging tool. It does **not** upload your traces or logs anywhere, and it does not include vendor sink integrations in the core package.
+
+This document describes how to report vulnerabilities and how to think about data safety when using AgentInspect.
+
+## Supported status
+
+AgentInspect is in active development. Security fixes are accepted and prioritized based on impact and exploitability.
+
+There is **no formal SLA**. If a fix is needed, it should land as a patch release once verified.
+
+## Reporting vulnerabilities
+
+- If **GitHub Security Advisories** are enabled for this repository, please report via a private advisory.
+- Otherwise, open a GitHub issue with **minimal sensitive detail** and request a private contact path for the full report.
+
+### What to include in a report
+
+- A clear description of the issue and expected impact
+- Steps to reproduce (prefer a small repro repo or minimized snippet)
+- Affected package(s) and version(s)
+- Platform info (Node version, OS)
+- Any mitigations/workarounds you found
+
+### Please do not include
+
+- real API keys or tokens
+- production log files
+- customer/user PII
+- full unredacted traces
+
+If you need to include sample data, use synthetic placeholders (for example, `example.test` emails and fake tokens).
+
+## Scope
+
+In scope:
+
+- vulnerabilities in trace/log parsing that could lead to code execution, path traversal, or data disclosure
+- unsafe redaction defaults (obvious secrets displayed where the product promises safety)
+- package supply-chain / dependency boundary issues that cause heavy or unsafe dependencies to be pulled into the main `agent-inspect` install
+
+## Out of scope
+
+- production monitoring SLAs or uptime guarantees (AgentInspect is not a production observability platform)
+- vendor sink behavior (not implemented in core)
+- network upload security (AgentInspect does not upload)
+- replay sandboxing (replay is not implemented)
+- cost calculation correctness (cost engine is not implemented)
+
+## Data handling model (local-first)
+
+- Manual tracing writes local JSONL files (see `docs/SCHEMA.md`).
+- The CLI reads and renders local files.
+- Exports generate local strings/files only (Markdown/HTML/OpenInference/OTLP JSON); they do not upload anywhere.
+
+## Redaction expectations
+
+AgentInspect aims to be safe-by-default for **log-derived attributes** and **exported payloads**:
+
+- Log ingestion applies redaction to parsed attributes using configured rules (with conservative defaults).
+- Exporters default to redacted output and bounded attribute previews.
+
+Important limitations:
+
+- **Manual trace metadata is user-controlled.** If you attach secrets to `inspectRun({ metadata })` or step metadata, those values may appear in local trace files and could appear in some views/exports depending on your settings and what you choose to include.
+- Always **review exported files** before sharing them externally.
+- Avoid committing trace directories (`.agent-inspect-runs/`) to source control.
+
+For redaction design details, see `docs-local/architecture/REDACTION.md`.
+
+## Dependency and security review process
+
+- Prefer Node.js built-ins over new dependencies.
+- Do not add vendor SDKs, OpenTelemetry SDKs, or framework dependencies to the main `agent-inspect` package.
+- Keep optional integrations (`@agent-inspect/langchain`, `@agent-inspect/tui`) separate so users do not pull them in by default.
+

package/docs/ADAPTERS.md

@@ -0,0 +1,15 @@
+## Adapters
+
+AgentInspect is framework-agnostic at its core, but can optionally integrate with frameworks via adapter packages.
+
+- **LangChain.js adapter** (optional): `@agent-inspect/langchain`
+  - Documented as **experimental** in `docs/API.md`
+  - Requires `@langchain/core` as a peer dependency
+- **Interactive TUI** (optional): `@agent-inspect/tui`
+  - Documented as **experimental** in `docs/API.md`
+  - Intended for CLI integration; programmatic TUI APIs may change
+
+See also:
+- `docs/MIGRATION.md` (what changed from early versions)
+- `docs-local/RELEASE-CHECKLIST.md` (maintainer-only release steps)
+