agentfootprint-lens 0.4.0 → 0.6.0

package/README.md

@@ -1,227 +1,260 @@
 # agentfootprint-lens
 
-> **See through your agent's decisions.**
+> **See the context engineering as it happens.**
 >
-> React components for debugging agents built on [`agentfootprint`](https://www.npmjs.com/package/agentfootprint): messages, prompt composition, tool calls, decision scope, and cost — in one scrub-able timeline.
+> React components for watching agents built on [`agentfootprint`](https://www.npmjs.com/package/agentfootprint). Every injection into the Agent's slots (RAG, Memory, Skills, Instructions, Tools) is tagged inline — students and engineers see exactly what was put into the prompt, by whom, on which iteration. No hidden abstractions.
+
+---
+
+### The pitch
+
+agentfootprint = **2 primitives (LLM, Agent) + 3 compositions (Sequence, Parallel, Conditional) + N patterns (ReAct, Reflexion, Tree-of-Thoughts...) + cross-cutting context engineering.** Lens is the surface that makes the context engineering visible — not as a "RAG view" or a "Memory view," but as tagged injections inside the ONE Agent card. That's the whole pedagogy.
 
 [![npm version](https://img.shields.io/npm/v/agentfootprint-lens.svg)](https://www.npmjs.com/package/agentfootprint-lens)
 [![license: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](./LICENSE)
 
 ---
 
-## Why this exists
+## 30-second quick start
 
-`footprint-explainable-ui` is the debugger for **pipelines** — stages, flowchart topology, commit log, data lineage. It's perfect for the person writing a footprintjs tool.
+```bash
+npm install agentfootprint agentfootprint-lens
+```
 
-But an **agent** isn't a pipeline — it's a loop of (LLM call → parse → tool calls → repeat), steered by a decision scope and gated by skills. The questions an agent developer asks are different:
+```tsx
+import { Agent, anthropic } from 'agentfootprint';
+import { Lens, useLens } from 'agentfootprint-lens';
 
-- "What prompt did the LLM actually see at iter 4?"
-- "Why did the agent pick **this** tool over that one?"
-- "Which skill was active when it went sideways?"
-- "How many tokens did this turn cost me?"
-- "What changed between turn 1 and the follow-up?"
+export function App() {
+  const agent = useLens(() =>
+    Agent.create({ provider: anthropic('claude-sonnet-4') })
+      .system('You are a helpful assistant.')
+      .build()
+  );
 
-**Lens** answers those. It reads `agent.getSnapshot()`, parses it into an agent-shaped timeline, and renders the agent-native surfaces:
+  return (
+    <>
+      <button onClick={() => agent.run('Hello!')}>Run</button>
+      <Lens for={agent} />
+    </>
+  );
+}
+```
 
-1. **Messages Panel** — chat bubbles with expandable tool-call cards, turn boundaries, iteration markers.
-2. **Iteration Strip** — horizontal ribbon of every LLM call in the run. Click to jump.
-3. **Tool Call Inspector** — flat sidebar of every tool invocation across all turns.
+That's it. Two lines — `useLens(...)` + `<Lens for={agent} />` — and you get:
 
-(phase-2: Prompt Composer, Decision Scope Ribbon, Cost & Latency Attribution.)
+- A live **Messages** view (everything the LLM saw and said, per turn)
+- An **Iteration Strip** (one cell per LLM call, tool call, or decision — scrubbable)
+- A **Tool Call Inspector** (args, result, timing for the currently selected step)
+- A **Decision Scope Ribbon** (which skill / decision rule was active)
+- An **Explainable Trace** tab (the full footprintjs stage-level view)
 
-For tool-internal debugging (what did this tool's footprintjs flowchart actually do?), Lens composes [`footprint-explainable-ui`](https://www.npmjs.com/package/footprint-explainable-ui) as a drill-in drawer. The two libraries are siblings, not competitors.
+No event wiring, no timeline prop, no snapshot prop. Lens figures it out by watching the runner directly.
 
 ---
 
-## Install
+## What you actually see
 
-```bash
-npm install agentfootprint-lens footprint-explainable-ui
-```
+As the agent runs, the three columns of Lens fill in live:
 
-Peer deps: React 18+, `footprint-explainable-ui@^0.18.0`.
+| Column | Shows |
+|---|---|
+| **Messages** | The conversation from the agent's perspective — system prompt, user turns, assistant replies, tool results |
+| **Iteration Strip** | One row per ReAct loop iteration. Each row lists the LLM call that ran it, the tool calls it picked, and the time each took |
+| **Context** | Whichever iteration or tool call is selected — shows the exact prompt the LLM saw, the tools it had available, and what it returned |
+
+When the run finishes, the second tab (**Explainable Trace**) lights up with the full stage-level flowchart — same surface `footprint-explainable-ui` ships, zero extra wiring.
 
 ---
 
-## Quick start
+## Multiple watchers, one agent
+
+`Lens` doesn't own the agent. Anything can observe it — a Lens, a Datadog exporter, a custom logger, or three of them at once.
 
 ```tsx
-import { AgentLens } from 'agentfootprint-lens';
-import { FootprintTheme, coolDark, coolLight } from 'footprint-explainable-ui';
-import { Agent, anthropic } from 'agentfootprint';
-import { useState, useEffect } from 'react';
+const agent = useLens(() => Agent.create(...).build());
+
+// Lens in the sidebar
+<Lens for={agent} />
+
+// At the same time — ship events to your telemetry backend
+useEffect(() => {
+  const stop = agent.observe((event) => {
+    if (event.type === 'llm_end') {
+      telemetry.record('llm.tokens', event.usage?.totalTokens);
+    }
+  });
+  return stop;   // auto-unsubscribe on unmount
+}, [agent]);
+```
 
-export function MyApp() {
-  const [dark, setDark] = useState(true);
-  const [snapshot, setSnapshot] = useState(null);
+`agent.observe(handler)` is the single subscribe primitive. It returns a `() => void` unsubscribe function. Add as many observers as you want.
 
-  useEffect(() => {
-    const agent = Agent.create({ provider: anthropic('claude-haiku-4-5') })
-      .system('You are a helpful agent.')
-      .build();
-    agent.run('What time is it?').then(() => setSnapshot(agent.getSnapshot()));
-  }, []);
+Event shape:
 
-  return (
-    <FootprintTheme tokens={dark ? coolDark : coolLight}>
-      <div style={{ height: '100vh' }}>
-        <AgentLens runtimeSnapshot={snapshot} />
-      </div>
-    </FootprintTheme>
-  );
-}
+```ts
+type AgentEvent =
+  | { type: 'turn_start';  userMessage: string }
+  | { type: 'llm_start';   iteration: number }
+  | { type: 'llm_end';     iteration: number; content: string; toolCallCount: number; usage?: TokenUsage; latencyMs: number }
+  | { type: 'tool_start';  toolName: string; args: Record<string, unknown> }
+  | { type: 'tool_end';    toolName: string; result: { content: string }; latencyMs: number }
+  | { type: 'token';       content: string }                        // streaming
+  | { type: 'turn_end';    content: string; iterations: number };
 ```
 
-That's it. Lens reads the same `FootprintTheme` context explainable-ui reads, so **the consumer owns theming** — flip `coolDark` ↔ `coolLight` at the app root and both the agent view (Lens) and any drill-in trace view (explainable-ui) follow together.
-
 ---
 
-## Composition: drill-in to a tool's flowchart
+## Works with every agentfootprint runner
 
-Each tool in agentfootprint is typically a `flowChart<State>` underneath. When a user wants to debug *what that tool did internally*, open an explainable-ui drawer:
+`<Lens for={...}>` accepts any agentfootprint runner — the same prop works for all of them, and they all light up Lens identically:
 
 ```tsx
-import { AgentLens, type AgentToolInvocation } from 'agentfootprint-lens';
-import { ExplainableShell } from 'footprint-explainable-ui';
+// Agent — a ReAct loop
+const agent = useLens(() => Agent.create(...).build());
 
-function Shell({ snapshot }) {
-  const [selected, setSelected] = useState<AgentToolInvocation | null>(null);
-  return (
-    <>
-      <AgentLens runtimeSnapshot={snapshot} onToolCallClick={setSelected} />
-      {selected && (
-        <Drawer onClose={() => setSelected(null)}>
-          <ExplainableShell
-            runtimeSnapshot={extractToolSubSnapshot(snapshot, selected.id)}
-            title={`${selected.name} · internals`}
-          />
-        </Drawer>
-      )}
-    </>
-  );
-}
+// LLMCall — a single prompt-in, response-out
+const caller = useLens(() => LLMCall.create(...).build());
+
+// RAG — retrieve + augment + answer
+const rag = useLens(() => RAG.create(...).retriever(...).build());
+
+// Swarm — LLM-routed specialists
+const swarm = useLens(() => Swarm.create(...).build());
+
+// ...same pattern for FlowChart, Parallel, Conditional
+
+<Lens for={caller} />   // pick whichever
 ```
 
-Lens says "this is what the agent did." explainable-ui says "this is what each tool's flowchart did." Clean separation, no duplication.
+One mental model. The runner does the work; Lens watches.
 
 ---
 
-## API surface
+## Theming
 
-### `<AgentLens>` — the one-stop shell
+Lens inherits from `footprint-explainable-ui`'s theme system. Two built-in presets, or bring your own:
 
 ```tsx
-<AgentLens
-  runtimeSnapshot={agent.getSnapshot()}
-  onToolCallClick={(invocation) => /* open drill-in */}
+import { coolDark, coolLight } from 'footprint-explainable-ui';
+import { Lens } from 'agentfootprint-lens';
+
+<Lens for={agent} theme={isDark ? coolDark : coolLight} />
+```
+
+Pass any `ThemeTokens` object. CSS vars work too — handy if your app already flips theme at the `:root` level:
+
+```tsx
+<Lens
+  for={agent}
+  theme={{
+    colors: {
+      bgPrimary: 'var(--my-bg)',
+      textPrimary: 'var(--my-fg)',
+      // …
+    },
+  }}
 />
 ```
 
-Props:
-- `runtimeSnapshot` (any | null) — raw output of `agent.getSnapshot()`. Null renders an empty state.
-- `timeline` (`AgentTimeline`) — pre-parsed timeline, overrides `runtimeSnapshot`. Useful for sharing across multiple Lens instances.
-- `systemPrompt` (string) — override the system-prompt preview in MessagesPanel. Auto-derived from snapshot otherwise.
-- `onToolCallClick` ((`AgentToolInvocation`) => void) — fires when any tool-call card is clicked.
+---
 
-### Individual panels (composable)
+## Responsive
 
-- `<MessagesPanel timeline onToolCallClick systemPrompt />`
-- `<IterationStrip timeline selectedKey onSelect />`
-- `<ToolCallInspector timeline selectedId onSelect />`
+Lens resizes to whatever space you give it. Below ~640px wide it stacks panels vertically (like `<ExplainableShell>` does). Drop it in a splitter, a drawer, or a full-screen tab — no config needed.
 
-### Adapter
+---
 
-- `fromAgentSnapshot(runtimeSnapshot) → AgentTimeline` — pure function. Useful for non-UI consumers (eval scripts, exporters).
+## Escape hatches
 
-### Theme
+If you want to manage the timeline yourself (custom ingestion, recording to a file, replaying a stored run), the explicit path is still available:
 
-Lens reads `useFootprintTheme()` (from explainable-ui). Wrap your tree in `<FootprintTheme tokens={...}>` and Lens follows. No Lens-specific theme API.
+```tsx
+import { Lens, useLiveTimeline } from 'agentfootprint-lens';
+
+const lens = useLiveTimeline();
+
+// You control ingestion
+for (const event of storedEvents) lens.ingest(event);
+
+<Lens
+  timeline={lens.timeline}
+  runtimeSnapshot={storedSnapshot}
+/>
+```
 
 ---
 
-## Data model
+## Recorder pattern (power users)
 
-`AgentTimeline` is what Lens renders against. `fromAgentSnapshot` derives it from the raw runtime snapshot:
+For advanced observability — multiple exporters, buffering, filtering before dispatch — agentfootprint's recorder system is still there:
 
 ```ts
-interface AgentTimeline {
-  turns: AgentTurn[];               // one per agent.run() call
-  messages: AgentMessage[];         // full flat conversation
-  tools: AgentToolInvocation[];     // every tool call, across turns
-  finalDecision: Record<string, unknown>;
-  rawSnapshot: unknown;             // escape hatch
-}
+import { createStreamEventRecorder } from 'agentfootprint';
 
-interface AgentTurn {
-  index: number;
-  userPrompt: string;
-  iterations: AgentIteration[];
-  finalContent: string;
-  totalInputTokens: number;
-  totalOutputTokens: number;
-  totalDurationMs: number;
-}
+const myRec = createStreamEventRecorder(myHandler, 'my-telemetry');
+const agent = Agent.create(...).recorder(myRec).build();
+```
 
-interface AgentIteration {
-  index: number;
-  model?: string;
-  inputTokens?: number;
-  outputTokens?: number;
-  durationMs?: number;
-  stopReason?: string;
-  assistantContent: string;
-  toolCalls: AgentToolInvocation[];
-  decisionAtStart: Record<string, unknown>;
-  matchedInstructions?: string[];
-  visibleTools: string[];            // tool names visible to the LLM that iter
-}
+`<Lens for={...}>` is just sugar over this internally — the recorder you'd write for Datadog is the same shape Lens uses.
 
-interface AgentToolInvocation {
-  id: string;
-  name: string;
-  arguments: Record<string, unknown>;
-  result: string;
-  error?: boolean;
-  decisionUpdate?: Record<string, unknown>;
-  iterationIndex: number;
-  turnIndex: number;
-  durationMs?: number;
-}
+---
+
+## API reference
+
+### `useLens(factory)`
+
+Memoizes a runner across renders. Call `factory` exactly once on mount; reuses the same instance forever. Works for any agentfootprint runner — `Agent`, `LLMCall`, `RAG`, `Swarm`, `FlowChart`, `Parallel`, `Conditional`.
+
+```ts
+const agent  = useLens(() => Agent.create(...).build());
+const caller = useLens(() => LLMCall.create(...).build());
+const rag    = useLens(() => RAG.create(...).build());
 ```
 
----
+### `<Lens for={runner} />`
 
-## Roadmap
+The one-prop integration. Subscribes to the runner's events, watches its snapshot, renders both tabs.
 
-### v0.1 (shipped)
-- `<AgentLens>` shell + MessagesPanel + IterationStrip + ToolCallInspector
-- Theme pass-through via FootprintTheme
-- `fromAgentSnapshot` adapter
+| Prop | Type | Description |
+|---|---|---|
+| `for` | `Runner` (any agentfootprint runner) | The agent / caller / swarm / etc. to watch. |
+| `theme` | `ThemeTokens?` | Optional — defaults to `coolDark`. |
+| `appName` | `string?` | Optional brand label in the tab strip. |
 
-### v0.2 — "why did the LLM decide that"
-- **Prompt Composer** — assembled system prompt per iteration with diff-across-iterations highlighting (base + skill body + instruction injections + message window + tool list).
-- **Decision Scope Ribbon** — horizontal pill timeline of decision scope fields; arrows back to the tool call that wrote each via `decisionUpdate`.
+### `runner.observe(handler)`
 
-### v0.3 — "is it shippable"
-- **Cost & Latency Attribution** — token burn curve, per-iter/per-tool breakdown, pluggable price table.
-- **Skill Dock** — loaded skills, activation state, tools-per-skill.
+Subscribe to live events. Returns `() => void` (unsubscribe).
 
-### v1.0 — "Lens v1"
-- Trace compare (two runs side-by-side)
-- Eval grid (batch runs → pass/fail matrix)
-- Exportable / importable traces via `TraceViewer` from explainable-ui
+```ts
+const stop = agent.observe((event) => { /* ... */ });
+// later:
+stop();
+```
+
+### `runner.getSnapshot()`, `runner.getNarrativeEntries()`, `runner.getSpec()`
+
+The standard agentfootprint introspection methods. `<Lens for={...}>` reads these automatically. You only call them yourself if you're building a custom UI.
+
+### `useLiveTimeline()` (escape hatch)
+
+Returns `{ timeline, ingest, startTurn, reset, builder }`. Use when you want to feed Lens from a non-runner source (replayed logs, server-sent events, etc.).
 
 ---
 
-## Design decisions
+## Why this design
+
+**The runner is the single source of truth.** Agents fire events as they work. Lens subscribes to those events. Telemetry exporters subscribe to those events. CLI loggers subscribe to those events. Nobody owns the runner; everyone can watch it.
+
+This is the observer pattern, applied consistently across every agentfootprint runner. The outcome:
 
-- **Separate package, not a fork of explainable-ui.** Different audience (agent devs vs. pipeline/tool devs), different mental model (conversation loop vs. data-flow graph), different failure modes. Overloading explainable-ui would muddy both.
-- **Theme via FootprintTheme context, not a Lens-specific prop.** One source of truth; automatic propagation to the drill-in drawer; consumers don't learn two theme APIs.
-- **Adapter at the boundary.** `fromAgentSnapshot` is the single data-shape conversion. Panels render against the derived `AgentTimeline`, so agentfootprint's internal snapshot shape can evolve without breaking Lens.
-- **No new agentfootprint library changes required to use Lens.** Every field Lens needs is already emitted today (messages, commitLog, recorder snapshots, emit events).
+- **One line to integrate** — `<Lens for={agent} />`
+- **Zero coupling** — the agent doesn't know Lens exists
+- **Composable** — Lens + your telemetry + your logger all watch the same agent with no conflict
+- **Uniform** — any runner works with any observer
 
 ---
 
 ## License
 
-MIT © Sanjay Krishna Anbalagan
+MIT