@respan/cli 0.14.2 → 0.15.0

package/dist/lib/skill-refs.generated.js

@@ -1,1585 +0,0 @@
-// AUTO-GENERATED by scripts/embed-skill-refs.mjs
-// Source: skills/SKILL.md and skills/references/{tracing,gateway,prompts,evals,monitors}.md
-// Do not edit directly — edit the markdown sources and rebuild.
-export const SKILL_MD = `# Respan
-
-Use the Respan CLI and SDK for LLM observability — tracing, evals, prompts, datasets, and gateway routing.
-
-## When To Use
-
-- **Set up tracing** (instrument your app to capture LLM calls — install + decorators, propagation, processors) → read [references/tracing.md](references/tracing.md)
-- **Set up gateway** (route LLM calls through the Respan proxy — wiring + model switching, fallbacks, caching) → read [references/gateway.md](references/gateway.md)
-- **Prompt management** (create, version, deploy) → read [references/prompts.md](references/prompts.md)
-- **Evals** (datasets, evaluators, experiments) → read [references/evals.md](references/evals.md)
-- **Monitors & automation** (alerts, online evals, webhooks) → read [references/monitors.md](references/monitors.md)
-
-Tracing and gateway are separate setups — configure one at a time, never both in the same pass.
-
-## Core Principles
-
-1. **Read the reference first.** Each reference file has the exact setup steps, API patterns, MCP tools, and CLI commands.
-2. **Use MCP tools** for platform operations (prompts, datasets, evaluators, experiments, traces, logs).
-3. **Use CLI** when MCP is not available: \`respan traces list\`, \`respan prompts list\`, etc.
-4. **Fetch docs** for integration-specific details not covered in references.
-
-## Quick Reference
-
-| Task | Reference / Command |
-|------|--------------------|
-| Set up tracing (steps + decorators, propagation) | [references/tracing.md](references/tracing.md) |
-| Set up gateway (steps + features) | [references/gateway.md](references/gateway.md) |
-| Prompt management | [references/prompts.md](references/prompts.md) |
-| Evals & experiments | [references/evals.md](references/evals.md) |
-| Monitors & automation | [references/monitors.md](references/monitors.md) |
-| List traces | \`respan traces list --limit 10\` |
-| View a trace | \`respan traces get <id>\` |
-| Check auth | \`respan auth status\` |
-
-## Documentation Access
-
-Any doc page can be fetched as markdown:
-\`https://respan.ai/docs/integrations/openai-sdk.md\`
-\`https://respan.ai/docs/sdks/typescript-sdk/overview.md\`
-
-Full docs index: \`https://www.respan.ai/docs/llms.txt\`
-
-Platform: \`https://platform.respan.ai\`
-`;
-export const TRACING_MD = `# Tracing
-
-Set up and configure Respan SDK tracing: instrument an app to capture LLM calls as
-structured traces, then refine with decorators, context propagation, and processors.
-
-This file covers **both** the initial setup steps and the advanced reference. Tracing
-is a separate setup from the gateway — do one at a time, never both in the same pass.
-
-Docs: \`https://respan.ai/docs/documentation/features/tracing/traces/tracing-sdk.md\`
-Full docs index: \`https://www.respan.ai/docs/llms.txt\`
-
----
-
-## Setup
-
-Use this section when the user asks to set up Respan tracing in their project.
-
-### Hard Rules
-
-- **Interactive mode:** Ask the user questions when you need input. Do not assume.
-- **Only add Respan code.** Do not refactor or modify unrelated code.
-- **Pin exact versions.** Never use \`latest\` or unpinned ranges.
-- **Do not guess APIs.** Use only the patterns from the integration docs linked below.
-- **If Respan is already installed/configured, do not duplicate work.** Check for existing \`respan\` imports first.
-- **Read the code before proposing changes.** Understand the actual workflow, not just the dependencies.
-
-The API key is stored in \`.env\` as \`RESPAN_API_KEY\`.
-
-### 1. Ask first: Auto or Full?
-
-**Before analyzing anything, ask the user which setup they want.** This is the first question — the two branches diverge immediately.
-
-1. **Auto** — the fastest working path. For a supported direct SDK, install the core SDK and add \`Respan()\` (one line) to capture flat LLM spans. The Vercel AI SDK is an explicit-only exception: Auto must also add \`@respan/instrumentation-vercel\` and \`VercelAIInstrumentor\`, but still adds no decorators or manual workflow wrappers. Best for a quick start or a live demo: traces flowing in under two minutes.
-2. **Full** — structured setup. Adds framework-specific instrumentation and/or workflow structure on top:
-   - **Explicit instrumentor** — for agent frameworks (LangChain, CrewAI, OpenAI Agents, Claude Agent SDK, LlamaIndex, Haystack, …) so the framework's agent / tool / chain structure is captured.
-   - **Decorators** — wrap the user's own functions with \`@workflow\` / \`@task\` for nested spans.
-
-Follow only the branch the user picks; skip the other one.
-
-> **Frameworks under Auto:** Auto normally does not install a framework instrumentor. If the app uses a framework (LangChain, CrewAI, an agents SDK), Auto captures only spans from a supported underlying direct SDK. **Direct Vercel AI SDK usage is different:** the Respan facade deliberately does not auto-discover it, and AI SDK calls are not covered by core-only setup. If the project directly calls the \`ai\` package and no higher-level framework owns its AI SDK telemetry, follow the Vercel branch below in both Auto and Full.
-
----
-
-### Auto path
-
-**A1. Detect language, package manager, and integration** — inspect \`package.json\` and source imports for JS/TS, or \`pyproject.toml\` / \`requirements.txt\` for Python. Before choosing core-only initialization, check whether the JS/TS project directly imports and calls the \`ai\` package (Vercel AI SDK). Apply the priority rule first: if a higher-level integration owns AI SDK telemetry, use that integration instead of also adding Vercel instrumentation. For example, an Eve app must use \`EveInstrumentor\`; do not activate \`VercelAIInstrumentor\` or another \`@ai-sdk/otel\` bridge alongside it.
-
-**A2. Install the core SDK** (pin the exact current version):
-\`\`\`bash
-# Python
-pip install respan-ai
-
-# TypeScript
-npm install @respan/respan
-\`\`\`
-
-If the project uses Vercel AI SDK directly, fetch and follow its [integration doc](https://respan.ai/docs/integrations/vercel-ai-sdk.md), and pin exact compatible versions of:
-
-- \`@respan/respan\`
-- \`@respan/instrumentation-vercel\`
-- \`@ai-sdk/otel\` when the installed \`ai\` major version is 7 (AI SDK 4 through 6 do not require this adapter)
-
-**A3. Add initialization code** at the top of the entrypoint, before any LLM client is created:
-\`\`\`python
-# Python
-from respan import Respan
-Respan()
-\`\`\`
-\`\`\`typescript
-// TypeScript
-import { Respan } from "@respan/respan";
-const respan = new Respan();
-await respan.initialize();
-\`\`\`
-
-For a Vercel AI SDK project, use its required explicit instrumentor instead of the core-only TypeScript initialization above:
-
-\`\`\`typescript
-import { Respan } from "@respan/respan";
-import { VercelAIInstrumentor } from "@respan/instrumentation-vercel";
-
-const respan = new Respan({
-  instrumentations: [new VercelAIInstrumentor()],
-});
-await respan.initialize();
-\`\`\`
-
-If the project already has a \`Respan\` instance, merge \`VercelAIInstrumentor\` into its existing \`instrumentations\` list instead of creating a second instance. Preserve other explicit instrumentors that cover genuinely separate direct-SDK calls; do not add a provider instrumentor solely for calls made through Vercel AI SDK.
-
-If the application already registers its own compatible \`@ai-sdk/otel\` integration, construct \`VercelAIInstrumentor({ autoRegisterAISDKTelemetry: false })\` so Respan translates the spans without claiming a second registration. For Next.js, follow the integration doc's root \`instrumentation.ts\` \`register()\` placement and \`serverExternalPackages\` configuration; keep tracing on the server runtime, not an Edge or browser path.
-
-Also enable Vercel telemetry on every AI SDK operation. Use the option that matches the installed AI SDK major version:
-
-\`\`\`typescript
-// AI SDK 7
-telemetry: { isEnabled: true }
-
-// AI SDK 4 through 6
-experimental_telemetry: { isEnabled: true }
-\`\`\`
-
-Do **not** add decorators. For non-Vercel Auto setups, do not add a framework instrumentor. For Vercel, keep the required \`VercelAIInstrumentor\` and go straight to **Verify**.
-
----
-
-### Full path
-
-The rest of this section is the **Full** path — follow it only if the user chose Full. It adds an explicit framework instrumentor and/or decorators on top of the core SDK.
-
-#### 1. Analyze the Project
-
-**1a. Detect language and package manager:**
-- Check \`package.json\` (JS/TS) or \`pyproject.toml\` / \`requirements.txt\` (Python)
-- Detect package manager from lock files
-
-**1b. Detect libraries in priority order:**
-
-Check higher-priority categories first. If a match is found, use that instrumentation — do NOT also add lower-level SDK instrumentation.
-
-**Priority 1 — Agent Frameworks & High-Level SDKs:**
-
-| Library | Python package | JS/TS package | Respan instrumentation (Python) | Respan instrumentation (JS/TS) | Docs |
-|---------|---------------|---------------|--------------------------------|-------------------------------|------|
-| Vercel AI SDK | — | \`ai\` | — | \`@respan/instrumentation-vercel\` | [docs](https://respan.ai/docs/integrations/vercel-ai-sdk.md) |
-| OpenAI Agents SDK | \`openai-agents\` | \`@openai/agents\` | \`respan-instrumentation-openai-agents\` | \`@respan/instrumentation-openai-agents\` | [docs](https://respan.ai/docs/integrations/openai-agents-sdk.md) |
-| Claude Agent SDK | \`claude-agent-sdk\` | — | \`respan-instrumentation-claude-agent-sdk\` | — | [docs](https://respan.ai/docs/integrations/claude-agents-sdk.md) |
-| Pi coding agent | — | \`@earendil-works/pi-coding-agent\` | — | \`@respan/instrumentation-pi\` | [README](https://github.com/RespanAI/respan/tree/main/javascript-sdks/instrumentations/respan-instrumentation-pi) |
-| Pydantic AI | \`pydantic-ai\` | — | \`respan-instrumentation-pydantic-ai\` | — | [docs](https://respan.ai/docs/integrations/pydantic-ai.md) |
-| LangChain | \`langchain\` | \`langchain\` | \`respan-instrumentation-langchain\` | \`@respan/instrumentation-langchain\` | [docs](https://respan.ai/docs/integrations/langchain.md) |
-| LangGraph | \`langgraph\` | \`@langchain/langgraph\` | \`respan-instrumentation-langchain\` | \`@respan/instrumentation-langchain\` | [docs](https://respan.ai/docs/integrations/langgraph.md) |
-| CrewAI | \`crewai\` | — | \`respan-instrumentation-crewai\` | — | [docs](https://respan.ai/docs/integrations/crewai.md) |
-| LlamaIndex | \`llama-index\` | — | \`respan-instrumentation-llama-index\` | — | [docs](https://respan.ai/docs/integrations/llama-index.md) |
-| Haystack | \`haystack-ai\` | — | \`respan-instrumentation-haystack\` | — | [docs](https://respan.ai/docs/integrations/haystack.md) |
-| Mastra | — | \`mastra\` | — | via OTEL | [docs](https://respan.ai/docs/integrations/mastra.md) |
-| Google ADK | \`google-adk\` | — | \`respan-instrumentation-google-adk\` | — | [docs](https://respan.ai/docs/integrations/google-adk.md) |
-
-If a Priority 1 framework is found, use its instrumentation. Do NOT also add Priority 2 instrumentation for the same provider.
-
-For direct Vercel AI SDK usage, the explicit \`VercelAIInstrumentor\` and per-operation telemetry option are required in both Auto and Full. Never replace them with core-only \`new Respan()\` setup, and never stack them with a higher-level integration that already owns AI SDK telemetry.
-
-**LangChain / LangGraph setup differs by language** (one package covers both frameworks: \`respan-instrumentation-langchain\` / \`@respan/instrumentation-langchain\`). In **Python**, pass \`LangChainInstrumentor()\` to \`Respan(instrumentations=[...])\` — it patches the LangChain and LangGraph callback managers globally on init, so every chain, graph, node, LLM, tool, and retriever run is traced **automatically with no per-call setup**. \`add_respan_callback(config=...)\` is optional and only labels a run with a name, tags, or metadata. In **TypeScript**, create a \`LangChainInstrumentor\`, pass it to \`new Respan({ instrumentations: [...] })\`, **and** attach \`instrumentor.addCallback(...)\` to the **outermost** chain or graph invocation (\`.invoke()\` / \`.stream()\`, not inside a node — LangChain/LangGraph propagate it to nested runs); the TypeScript instrumentor does **not** patch globally, so without the callback no spans are emitted. (The other Priority-1 frameworks use the standard \`Respan(instrumentations=[XInstrumentor()])\` plugin pattern.)
-
-**Pi coding agent (TypeScript only)** has no global patch point, so the instrumentor must be wired into the session. Create \`new PiInstrumentor()\` from \`@respan/instrumentation-pi\`, pass it to \`new Respan({ instrumentations: [...] })\`, and either pass \`instrumentor.extension\` via \`new DefaultResourceLoader({ cwd: process.cwd(), agentDir: getAgentDir(), extensionFactories: [instrumentor.extension] })\` (pi requires \`cwd\` and \`agentDir\`; \`getAgentDir\` is exported by \`@earendil-works/pi-coding-agent\`) before \`createAgentSession(...)\`, or call \`instrumentor.attach(session)\` on an existing \`AgentSession\`. By default every run of a pi session joins one multi-root trace whose id is derived from the pi session id (\`traceScope: "session"\`), so a session resumed across processes and pauses stays one trace; \`traceScope: "run"\` makes each agent run its own trace instead. All runs of a pi session share its thread id either way. For always-on agent fleets, pair it with a local OpenTelemetry Collector persistent queue (see Delivery Guarantees below) so spans survive Respan outages and restarts without a local journal. For the interactive \`pi\` CLI, run \`respan integrate pi\` instead — it installs the \`@respan/instrumentation-pi\` pi package (a pi extension) and writes a non-secret \`respan.json\` (\`--trace-scope run\` for one trace per prompt instead of one per session).
-
-**Priority 2 — Direct LLM SDKs** (only if no P1 framework covers this provider):
-
-The integrations listed for each language are **auto-instrumented** with \`Respan()\` / \`new Respan()\`. The application still provides the provider SDK; the matching first-party instrumentation adapter is bundled by the Respan facade. A dash means that language currently requires an explicit instrumentation package and instrumentor.
-
-| Library | Python SDK (auto) | JS/TS SDK (auto) | Docs |
-|---------|-------------------|------------------|------|
-| OpenAI SDK | \`openai\` | \`openai\` | [docs](https://respan.ai/docs/integrations/openai-sdk.md) |
-| Anthropic SDK | \`anthropic\` | \`@anthropic-ai/sdk\` | [docs](https://respan.ai/docs/integrations/anthropic.md) |
-| Azure OpenAI | \`openai\` (Azure config) | \`openai\` (Azure client) | [docs](https://respan.ai/docs/integrations/providers/azure.md) |
-| Google Vertex AI | \`google-cloud-aiplatform\` | \`@google-cloud/vertexai\` | [docs](https://respan.ai/docs/integrations/vertex-ai.md) |
-| Google GenAI | \`google-genai\` | — | [docs](https://respan.ai/docs/integrations/google-genai.md) |
-| AWS Bedrock | \`boto3\` | \`@aws-sdk/client-bedrock-runtime\` | [docs](https://respan.ai/docs/integrations/aws-bedrock.md) |
-| Cohere | — | \`cohere-ai\` | [docs](https://respan.ai/docs/integrations/providers/cohere.md) |
-| Together AI | \`together\` | \`together-ai\` | [docs](https://respan.ai/docs/integrations/together-ai.md) |
-| OpenRouter | — | \`@openrouter/sdk\` | [docs](https://respan.ai/docs/integrations/providers/openrouter.md) |
-| Writer | — | \`writer-sdk\` | [docs](https://respan.ai/docs/integrations/writer.md) |
-| Ollama | \`ollama\` | — | [docs](https://respan.ai/docs/integrations/ollama.md) |
-
-**Note:** LLM wrappers such as Python LiteLLM stay explicit-only to avoid overlapping provider spans. See the [LiteLLM guide](https://respan.ai/docs/integrations/litellm.md) for its explicit setup.
-
-**1c. Read the actual code and understand the workflow:**
-
-This is the most important step. Read the entrypoint and all files that make LLM calls. Map out:
-
-- What is the **overall workflow**? (e.g. "user sends question → retrieve context → generate answer → format response")
-- What are the **individual steps/tasks**? (e.g. "embed query", "search DB", "call GPT", "parse output")
-- Are there **agent loops**? (e.g. a loop that calls tools until done)
-- Are there **tool calls**? (e.g. functions the LLM invokes)
-
-#### 2. Propose an Implementation Plan
-
-Present the user with a concrete plan before making any changes. The plan should include:
-
-**a) Packages to install** — core SDK + instrumentation package (with exact versions)
-
-**b) Initialization code** — where to add it (which file, which line)
-
-**c) Workflow structure** — how to wrap the existing code:
-
-For **agent frameworks** (Priority 1): The framework instrumentation auto-captures the workflow structure. Usually just need init code, no manual wrapping needed. Fetch and follow the integration doc.
-
-For **direct LLM SDKs** (Priority 2): Individual LLM calls will be auto-traced as flat spans. Propose wrapping the logical workflow with Respan decorators/wrappers to get structured nested traces:
-
-TypeScript example:
-\`\`\`typescript
-// Before: flat traces — each LLM call is an isolated span
-const outline = await openai.chat.completions.create({...});
-const draft = await openai.chat.completions.create({...});
-
-// After: structured traces — nested spans showing the workflow
-const result = await withWorkflow({ name: "write_article" }, async () => {
-  const outline = await withTask({ name: "generate_outline" }, async () => {
-    return await openai.chat.completions.create({...});
-  });
-  const draft = await withTask({ name: "write_draft" }, async () => {
-    return await openai.chat.completions.create({...});
-  });
-  return draft;
-});
-\`\`\`
-
-Python example:
-\`\`\`python
-# Before: flat traces
-outline = client.chat.completions.create(...)
-draft = client.chat.completions.create(...)
-
-# After: structured traces
-@workflow(name="write_article")
-def write_article(topic):
-    outline = generate_outline(topic)
-    return write_draft(outline)
-
-@task(name="generate_outline")
-def generate_outline(topic):
-    return client.chat.completions.create(...)
-
-@task(name="write_draft")
-def write_draft(outline):
-    return client.chat.completions.create(...)
-\`\`\`
-
-Within Full, pick what the app needs (they can combine):
-- **Framework detected** → add its **explicit instrumentor** (or the LangChain callback). The framework auto-captures its agent / tool / chain structure — usually no manual wrapping needed.
-- **Direct LLM SDK, or you want structure over your own orchestration code** → add **decorators** (\`@workflow\` / \`@task\`) to wrap the logical workflow into nested spans.
-
-(If the user only wanted flat spans with zero code changes, that's the **Auto** path — go back to it.)
-
-When adding decorators:
-- **If multiple independent workflows are detected** (e.g. \`writeArticle()\`, \`summarizeDoc()\`, \`classifyEmail()\`), list them and ask which ones to instrument. Don't assume all of them.
-- **Show the user what the trace will look like** — describe the span hierarchy:
-\`\`\`
-workflow: write_article
-  ├── task: generate_outline
-  │     └── llm: openai.chat (auto-captured)
-  └── task: write_draft
-        └── llm: openai.chat (auto-captured)
-\`\`\`
-
-Wait for user confirmation before proceeding.
-
-#### 3. Implement
-
-**a) Install packages:**
-
-For direct LLM SDKs (Priority 2) — just the core SDK:
-\`\`\`bash
-# Python
-pip install respan-ai
-
-# TypeScript
-npm install @respan/respan
-\`\`\`
-
-For agent frameworks (Priority 1) — also install the instrumentor. Check the docs link in the table above for the exact packages.
-
-**b) Add initialization code** — at the top of the entrypoint, before any LLM client is created:
-
-For **direct LLM SDKs** (auto-instrumented):
-\`\`\`python
-# Python
-from respan import Respan
-Respan()
-\`\`\`
-\`\`\`typescript
-// TypeScript
-import { Respan } from "@respan/respan";
-const respan = new Respan();
-await respan.initialize();
-\`\`\`
-
-For **agent frameworks** (explicit instrumentor — fetch the docs URL from the table for the exact pattern):
-\`\`\`python
-# Python example (OpenAI Agents)
-from respan import Respan
-from respan_instrumentation_openai_agents import OpenAIAgentsInstrumentor
-Respan(instrumentations=[OpenAIAgentsInstrumentor()])
-\`\`\`
-\`\`\`typescript
-// TypeScript example (OpenAI Agents)
-import { Respan } from "@respan/respan";
-import { OpenAIAgentsInstrumentor } from "@respan/instrumentation-openai-agents";
-const respan = new Respan({ instrumentations: [new OpenAIAgentsInstrumentor()] });
-await respan.initialize();
-\`\`\`
-
-**c) Add workflow wrappers** — if the user chose structured traces in the plan.
-
-### Verify (final test — always run this, both paths)
-
-As the final step — for **both** Auto and Full — **run the user's program once with a small request** and confirm tracing works end to end:
-
-- The app runs without errors
-- A trace appears at https://platform.respan.ai or via \`respan traces list --limit 5\`
-- If wrappers were added, the trace shows the expected nested span hierarchy
-
-If no trace appears, the instrumentation isn't taking effect — debug (init placement, missing \`flush()\`, wrong entrypoint) and re-run before declaring setup done.
-
----
-
-## Reference
-
-Advanced tracing configuration: decorators, context propagation, processors, and span attributes.
-
-### Decorators
-
-Wrap functions to create structured span hierarchies. All decorators share the same signature.
-
-| Decorator | Purpose |
-|-----------|---------|
-| \`@workflow\` / \`withWorkflow\` | Root span — top-level pipeline |
-| \`@task\` / \`withTask\` | Step within a workflow |
-| \`@agent\` / \`withAgent\` | Agent loop span |
-| \`@tool\` / \`withTool\` | Tool/function call span |
-
-#### Python
-
-\`\`\`python
-from respan import Respan, workflow, task, agent, tool
-
-Respan()
-
-@workflow(name="write_article")
-def write_article(topic: str):
-    outline = generate_outline(topic)
-    return write_draft(outline)
-
-@task(name="generate_outline")
-def generate_outline(topic: str):
-    return client.chat.completions.create(...)
-
-@task(name="write_draft")
-def write_draft(outline: str):
-    return client.chat.completions.create(...)
-\`\`\`
-
-#### TypeScript
-
-\`\`\`typescript
-import { Respan, withWorkflow, withTask } from "@respan/respan";
-
-const respan = new Respan();
-await respan.initialize();
-
-const result = await withWorkflow({ name: "write_article" }, async () => {
-  const outline = await withTask({ name: "generate_outline" }, async () => {
-    return await client.chat.completions.create({...});
-  });
-  return await withTask({ name: "write_draft" }, async () => {
-    return await client.chat.completions.create({...});
-  });
-});
-\`\`\`
-
-#### Decorator Parameters
-
-\`\`\`python
-@workflow(
-    name="my_workflow",               # Display name (default: function name)
-    version=1,                        # Version number
-    processors="my_processor",        # Route to specific processor(s)
-    sample_rate=0.5,                  # 0.0-1.0, fraction of spans exported
-    export_filter={                   # Conditional export
-        "metadata.env": {"operator": "", "value": "production"}
-    },
-)
-\`\`\`
-
-All parameters are optional. Without any, the function name is used as the span name.
-
-#### Async Support
-
-Decorators work with \`async def\`, generators, and async generators automatically.
-
-### Context Propagation
-
-Attach attributes to **all spans** within a scope, including auto-instrumented LLM calls.
-
-#### propagate_attributes
-
-\`\`\`python
-from respan import Respan, propagate_attributes
-
-Respan()
-
-with propagate_attributes(
-    customer_identifier="user_123",
-    thread_identifier="conv_abc",
-    environment="production",
-    metadata={"plan": "pro", "team": "search"},
-):
-    # All spans here (including auto-instrumented OpenAI calls) get these attributes
-    result = run_pipeline()
-\`\`\`
-
-##### Available attributes
-
-| Attribute | Description |
-|-----------|-------------|
-| \`customer_identifier\` | User/customer ID — enables per-user analytics |
-| \`customer_email\` | Customer email |
-| \`customer_name\` | Customer display name |
-| \`thread_identifier\` | Conversation thread ID — groups related calls |
-| \`custom_identifier\` | Indexed custom identifier for fast queries |
-| \`group_identifier\` | Group related traces together |
-| \`environment\` | Environment name (e.g. \`"production"\`, \`"staging"\`) |
-| \`metadata\` | Dict of custom key-value pairs (merged in nested contexts) |
-| \`prompt\` | Dict with \`prompt_id\` and \`variables\` for prompt logging |
-
-##### TypeScript
-
-\`\`\`typescript
-const result = await respan.propagateAttributes(
-  {
-    customerIdentifier: "user_123",
-    threadIdentifier: "conv_abc",
-    metadata: { plan: "pro" },
-  },
-  async () => {
-    return await runPipeline();
-  }
-);
-\`\`\`
-
-##### Nesting behavior
-
-- Nested \`propagate_attributes\` calls merge with the outer context
-- \`metadata\` dicts are merged (not replaced)
-- For duplicate keys, inner values override outer
-- Async-safe via \`contextvars\`
-
-#### respan_span_attributes
-
-Attaches attributes to the **current active span only** (not auto-instrumented child spans):
-
-\`\`\`python
-from respan import respan_span_attributes
-
-with respan_span_attributes({
-    "customer_identifier": "user-123",
-    "metadata": {"priority": "high"},
-}):
-    pass
-\`\`\`
-
-Use \`propagate_attributes\` when you need attributes on all nested spans (most common). Use \`respan_span_attributes\` when you only want to tag the current span.
-
-### Imperative Span Creation
-
-For runtime-determined span names, use the client API instead of decorators:
-
-\`\`\`python
-from respan import get_client
-
-client = get_client()
-
-with client.start_span(name="dynamic_step", kind="task") as span:
-    result = do_work()
-    # span is automatically closed and exported
-\`\`\`
-
-### Updating the Current Span
-
-Add attributes or change status on the active span at runtime:
-
-\`\`\`python
-from respan import get_client
-
-client = get_client()
-
-# Inside a decorated function or start_span context:
-client.update_current_span(
-    respan_params={
-        "customer_identifier": "user_123",
-        "metadata": {"step": "final"},
-    },
-    name="renamed_span",
-)
-
-# Add events
-client.add_event("checkpoint_reached", attributes={"items": 42})
-
-# Record exceptions (span continues)
-try:
-    risky_operation()
-except Exception as e:
-    client.record_exception(e)
-\`\`\`
-
-### Custom Processors
-
-Route spans to multiple destinations with filtering:
-
-\`\`\`python
-from respan_tracing import RespanTelemetry
-
-telemetry = RespanTelemetry(api_key="...", base_url="...")
-
-# Add a second exporter for debug logging
-from opentelemetry.sdk.trace.export import ConsoleSpanExporter
-
-telemetry.add_processor(
-    exporter=ConsoleSpanExporter(),
-    name="debug",
-    filter_fn=lambda span: span.attributes.get("environment") == "debug",
-)
-\`\`\`
-
-Route specific spans to a processor using the \`processors\` parameter on decorators:
-
-\`\`\`python
-@task(name="sensitive_step", processors="debug")
-def sensitive_step():
-    ...
-\`\`\`
-
-### Flush
-
-Always call \`flush()\` before process exit to ensure all spans are exported:
-
-\`\`\`python
-respan = Respan()
-# ... your code ...
-respan.flush()
-\`\`\`
-
-In serverless/Lambda: call \`flush()\` at the end of every handler invocation.
-
-### Delivery Guarantees
-
-The SDK batches spans in memory and the OTLP exporter retries 429/502/503/504 with backoff; it never writes to disk. Spans still queued during a long outage or a process crash are lost — the same trade-off every LLM observability SDK makes. For long-running production agents where that is not acceptable, run the packaged Respan Collector next to the application: \`respan collector start\` (Docker if available, otherwise the pinned \`otelcol-contrib\` binary) or \`docker compose -f collector/docker-compose.yaml up -d\` from the monorepo's \`collector/\` directory, then set \`RESPAN_BASE_URL=http://127.0.0.1:4318\`. It is the upstream OpenTelemetry Collector with a persistent, bounded queue (\`file_storage\` write-ahead log capped by \`RESPAN_COLLECTOR_MAX_SIZE_BYTES\`, \`RESPAN_COLLECTOR_QUEUE_SIZE\`, retries with \`max_elapsed_time: 0\`, uncompressed JSON export because Respan's ingest does not inflate gzip) receiving on \`/api/v2/traces\`, the path the SDK posts to. Never build a local journal into the application itself; alert on \`otelcol_exporter_enqueue_failed_spans\` (drops) and watch \`otelcol_exporter_queue_size\`. \`respan collector status\` prints those counters.
-
-### Trace Hierarchy Example
-
-\`\`\`
-workflow: handle_request
-  +-- task: classify_intent
-  |     +-- llm: openai.chat (auto-captured)
-  +-- agent: support_agent
-  |     +-- tool: lookup_order
-  |     +-- llm: openai.chat (auto-captured)
-  |     +-- tool: process_refund
-  +-- task: generate_response
-        +-- llm: openai.chat (auto-captured)
-\`\`\`
-
-LLM calls within decorated functions are automatically nested as child spans.
-`;
-export const GATEWAY_MD = `# Gateway
-
-Set up and use the Respan **gateway** — routing an app's LLM calls through the Respan
-proxy for logging, caching, key management, fallbacks, and model switching.
-
-This file covers **both** the setup steps and the feature reference. Gateway is a
-separate setup from tracing — do one at a time, never both in the same pass. Gateway
-setup does **not** install instrumentation packages; it repoints the LLM client's base
-URL at the Respan proxy and applies the framework's specific wiring.
-
-Docs: \`https://respan.ai/docs/documentation/features/gateway/gateway-quickstart.md\`
-Full docs index: \`https://www.respan.ai/docs/llms.txt\`
-
----
-
-## Setup
-
-Use this section when the user asks to set up the Respan gateway.
-
-### Hard Rules
-
-- **Interactive mode:** Ask the user questions when you need input. Do not assume.
-- **Only add Respan wiring.** Do not refactor or modify unrelated code.
-- **The gateway serves only the completion / text-generation operation**, exposed per provider:
-  - OpenAI-compatible → \`https://api.respan.ai/api/chat/completions\`
-  - Anthropic Messages → \`https://api.respan.ai/api/anthropic/\`
-  - Google Gemini → \`https://api.respan.ai/api/google/gemini/...\`
-
-  **Use the exact base path from the per-SDK gateway doc you fetch in Step 4** — it is authoritative. Do **NOT** repoint clients used for any other operation — embeddings, moderation, images, audio/speech, assistants, batch, fine-tuning — they are not supported and will fail (e.g. 404). Leave those untouched.
-- **Use chat completions, not the Responses API.** The gateway serves chat completions (\`/api/chat/completions\`), not the OpenAI Responses API (\`/api/responses\`). SDKs that *default* to the Responses API are still fully supported — you just switch them to chat-completions mode, per their per-SDK doc:
-  - **OpenAI Agents SDK** — Python: \`set_default_openai_client(AsyncOpenAI(base_url=..., api_key=os.environ["RESPAN_API_KEY"]))\` + \`set_default_openai_api("chat_completions")\` (TS: \`setDefaultOpenAIClient(...)\` + \`setOpenAIAPI("chat_completions")\`).
-  - **Vercel AI SDK / \`@ai-sdk/openai\`** — use \`provider.chat(model)\`, not the default \`provider(model)\` (which targets the Responses API).
-- **Scope the wiring — do not blanket-rewrite.** Wire only the client(s) that make the completion call you are setting up. Do **NOT** repoint every client instance in the repo (especially in example/monorepo projects with many independent scripts).
-- **Do NOT install instrumentation packages.** Gateway routing does not use them. (That is tracing setup — see \`tracing.md\`.)
-- **Do not guess APIs or doc slugs.** Use only the patterns from the live integration doc you fetch, and the exact slug from the map below.
-- **Pin exact versions** if you install a framework SDK that is genuinely missing.
-- **If the gateway is already wired up, do not duplicate work.** Check for an existing \`base_url\`/\`baseURL\` pointing at \`api.respan.ai\` first.
-- **Read the code before proposing changes.** Find where the client is actually instantiated.
-
-The API key is stored in \`.env\` as \`RESPAN_API_KEY\`. Gateway base URL: \`https://api.respan.ai/api\` (the proxy completions path is \`https://api.respan.ai/api/chat/completions\`).
-
-### 1. Detect
-
-Identify:
-- **Language & package manager** — \`package.json\` (JS/TS), \`pyproject.toml\` / \`requirements.txt\` (Python), \`Gemfile\` (Ruby). Detect the package manager from lock files.
-- **LLM library / framework in use** — match the project's dependencies against the **Detection → slug map** below.
-
-### 2. Map to slug
-
-Match the detected package(s) against the map. **Use the slug from the map verbatim — never auto-derive a slug from a display name** (the kebab-casing is irregular).
-
-Apply the **priority rule:** a high-level framework wins over the raw SDK beneath it.
-For example, a project with both \`crewai\` and \`openai\` maps to \`crew-ai\` (not \`open-ai-sdk\`),
-because their gateway wiring differs. **If two frameworks match, ask the user which to configure.**
-
-#### Detection → slug map
-
-URL pattern: \`https://respan.ai/docs/integrations/gateway/<slug>.md\`
-
-| Detected package(s) | Slug |
-|---|---|
-| \`@openai/agents\` / \`openai-agents\` | \`open-ai-agents\` |
-| \`claude-agent-sdk\` | \`claude-agent-sdk\` |
-| \`ai\` (Vercel AI SDK) | \`vercel-ai-sdk\` |
-| \`pydantic-ai\` | \`pydantic-ai\` |
-| \`crewai\` | \`crew-ai\` |
-| \`haystack-ai\` | \`haystack\` |
-| \`langchain\` | \`lang-chain\` |
-| \`llama-index\` | \`llama-index\` |
-| \`autogen\` / \`autogen-agentchat\` | \`auto-gen\` |
-| \`dspy\` / \`dspy-ai\` | \`ds-py\` |
-| \`google-adk\` | \`google-adk\` |
-| \`openai\` | \`open-ai-sdk\` |
-| \`anthropic\` / \`@anthropic-ai/sdk\` | \`anthropic-sdk\` |
-| \`google-genai\` / \`@google/genai\` | \`google-gen-ai\` |
-| \`litellm\` | \`lite-llm\` |
-| \`ruby_llm\` (gem) | \`ruby-llm\` |
-| \`google-cloud-aiplatform\` | \`vertex-ai\` |
-
-> Namespace note: OpenAI, Anthropic, and Vertex each have both an SDK-level gateway page (\`open-ai-sdk\`, \`anthropic-sdk\`, \`vertex-ai\`) and a \`model-providers/*\` page. The SDK-level slugs above are the correct targets for this map.
-
-### 3. Confirm
-
-Show the user what was detected and which doc will be used. Wait for confirmation.
-
-> Detected:
-> - Language: [TypeScript/Python/Ruby]
-> - Framework / SDK: [e.g. OpenAI Agents]
-> - Gateway doc: https://respan.ai/docs/integrations/gateway/open-ai-agents.md
->
-> Set up the gateway for this? (yes/no)
-
-### 4. Fetch the live doc
-
-Fetch \`https://respan.ai/docs/integrations/gateway/<slug>.md\` for the chosen slug.
-Use the slug from the map verbatim.
-
-If a pinned URL fails to resolve, that signals the **map is stale** and the table should be
-updated — it does **not** mean you should guess an alternative slug.
-
-### 5. Read the project's client code
-
-Find where the LLM client / \`base_url\` is instantiated, and identify **which client(s) make
-completion / text-generation calls** (chat completions, Anthropic \`messages.create\`, Gemini
-\`generate_content\`) — that is the only path you repoint. This is the step that determines the
-correct wiring — do not skip it.
-
-If the project has multiple clients or many scripts (e.g. an examples repo), repoint **only**
-the completion ones. Leave clients that call embeddings, moderation, images, audio,
-assistants, batch, or fine-tuning exactly as they are — the gateway does not serve them.
-
-### 6. Apply the framework-specific wiring
-
-Apply the exact pattern from the fetched doc:
-
-- Ensure \`RESPAN_API_KEY\` is set (it is in \`.env\`).
-- Repoint \`base_url\` / \`baseURL\` to the gateway path from the fetched doc (e.g. \`https://api.respan.ai/api\` for OpenAI-compatible, \`…/api/anthropic/\` for Anthropic) — **only on the completion client(s)** identified in step 5, not on every client in the project.
-- Apply the framework's exact wiring — e.g. OpenAI Agents requires \`set_default_openai_client(AsyncOpenAI(base_url=...))\`, **not** env vars.
-- Install the framework SDK only if it is genuinely missing.
-- **Do NOT install instrumentation packages.**
-
-### 7. Verify (final test — always run this)
-
-As the final step, **run the user's program once with a small request** through the
-configured gateway and confirm it works:
-
-\`\`\`bash
-respan logs list --limit 5
-\`\`\`
-
-Confirm:
-- The app runs without errors
-- The request appears in the gateway logs (via the CLI above, or the Respan platform at https://platform.respan.ai)
-
-If no log entry appears, the request is not actually routing through the gateway — debug
-the \`base_url\`/wiring and re-run before declaring setup done.
-
----
-
-## Reference
-
-### Wiring Patterns
-
-Point the LLM client's \`base_url\` at the Respan gateway. The Respan API key authenticates both the proxy and tracing.
-
-#### Python (OpenAI SDK)
-
-\`\`\`python
-import os
-from openai import OpenAI
-
-client = OpenAI(
-    api_key=os.environ["RESPAN_API_KEY"],
-    base_url=os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api"),
-)
-
-response = client.chat.completions.create(
-    model="gpt-4o-mini",
-    messages=[{"role": "user", "content": "Hello!"}],
-)
-\`\`\`
-
-#### TypeScript (OpenAI SDK)
-
-\`\`\`typescript
-import OpenAI from "openai";
-
-const client = new OpenAI({
-  apiKey: process.env.RESPAN_API_KEY,
-  baseURL: process.env.RESPAN_BASE_URL || "https://api.respan.ai/api",
-});
-
-const response = await client.chat.completions.create({
-  model: "gpt-4o-mini",
-  messages: [{ role: "user", content: "Hello!" }],
-});
-\`\`\`
-
-#### Python (Anthropic SDK)
-
-\`\`\`python
-import os
-import anthropic
-
-client = anthropic.Anthropic(
-    api_key=os.environ["RESPAN_API_KEY"],
-    base_url=os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api"),
-)
-\`\`\`
-
-#### Environment variables
-
-Set once, works everywhere:
-
-\`\`\`bash
-export RESPAN_API_KEY="your-key"
-export OPENAI_API_KEY="$RESPAN_API_KEY"
-export OPENAI_BASE_URL="https://api.respan.ai/api"
-\`\`\`
-
-### Framework-Specific Gateway Setup
-
-#### CrewAI
-
-\`\`\`python
-import os
-os.environ["OPENAI_API_KEY"] = os.environ["RESPAN_API_KEY"]
-os.environ["OPENAI_BASE_URL"] = os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api")
-
-from crewai import Agent, Task, Crew
-# CrewAI uses the env vars automatically
-\`\`\`
-
-#### Haystack
-
-\`\`\`python
-import os
-os.environ["OPENAI_API_KEY"] = os.environ["RESPAN_API_KEY"]
-os.environ["OPENAI_BASE_URL"] = os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api")
-
-from haystack.components.generators import OpenAIGenerator
-# OpenAIGenerator picks up env vars automatically
-\`\`\`
-
-#### OpenAI Agents SDK
-
-\`\`\`python
-import os
-os.environ["OPENAI_API_KEY"] = os.environ["RESPAN_API_KEY"]
-os.environ["OPENAI_BASE_URL"] = os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api")
-\`\`\`
-
-#### Pydantic AI
-
-\`\`\`python
-import os
-os.environ["OPENAI_API_KEY"] = os.environ["RESPAN_API_KEY"]
-os.environ["OPENAI_BASE_URL"] = os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api")
-
-from pydantic_ai import Agent
-agent = Agent(model="openai:gpt-4o-mini")
-\`\`\`
-
-### Model Switching
-
-The gateway supports 250+ models across providers. Use any model name:
-
-\`\`\`python
-# OpenAI models
-response = client.chat.completions.create(model="gpt-4o", ...)
-
-# Anthropic models (via OpenAI SDK!)
-response = client.chat.completions.create(model="claude-sonnet-4-20250514", ...)
-
-# Google models
-response = client.chat.completions.create(model="gemini-2.0-flash", ...)
-
-# Open-source models
-response = client.chat.completions.create(model="llama-3.1-70b", ...)
-\`\`\`
-
-No code changes needed — just change the model string. View all models at \`https://platform.respan.ai/platform/models\`.
-
-### Advanced Features
-
-#### Fallback Models
-
-Automatic failover if the primary model fails:
-
-\`\`\`python
-response = client.chat.completions.create(
-    model="gpt-4o",
-    messages=[...],
-    extra_body={
-        "fallback_models": ["claude-sonnet-4-20250514", "gemini-2.0-flash"],
-    },
-)
-\`\`\`
-
-#### Retries
-
-\`\`\`python
-response = client.chat.completions.create(
-    model="gpt-4o",
-    messages=[...],
-    extra_body={
-        "retries": 3,
-    },
-)
-\`\`\`
-
-#### Caching
-
-Enable response caching to save cost on repeated calls:
-
-\`\`\`python
-response = client.chat.completions.create(
-    model="gpt-4o",
-    messages=[...],
-    extra_body={
-        "cache_enabled": True,
-        "cache_ttl": 3600,  # seconds
-    },
-)
-\`\`\`
-
-#### Streaming
-
-Streaming works normally through the gateway:
-
-\`\`\`python
-stream = client.chat.completions.create(
-    model="gpt-4o-mini",
-    messages=[{"role": "user", "content": "Tell me a story"}],
-    stream=True,
-)
-for chunk in stream:
-    print(chunk.choices[0].delta.content, end="")
-\`\`\`
-
-#### Disable Logging
-
-For sensitive data, disable logging on specific requests:
-
-\`\`\`python
-response = client.chat.completions.create(
-    model="gpt-4o",
-    messages=[...],
-    extra_body={
-        "disable_log": True,
-    },
-)
-\`\`\`
-
-### Key Benefits
-
-- **Unified key management** — one Respan API key for all providers
-- **Automatic logging** — every request logged as a span
-- **Cost tracking** — per-request cost tracking across providers
-- **Caching** — cache repeated LLM calls to reduce cost and latency
-- **Fallbacks** — automatic failover between providers
-- **Model switching** — swap models without code changes
-- **~50-150ms added latency** — minimal overhead
-`;
-export const PROMPTS_MD = `# Prompts Reference
-
-Create, version, deploy, and use prompt templates from the Respan platform.
-
-Docs: \`https://respan.ai/docs/documentation/features/prompt-management/prompt-management-quickstart.md\`
-
----
-
-## Overview
-
-Prompts are version-controlled templates stored on the Respan platform. The workflow is:
-
-1. **Create** a prompt with a name
-2. **Add versions** with messages, model config, and \`{{variables}}\`
-3. **Deploy** the version you want live
-4. **Fetch** the prompt in code using the API or MCP tools
-5. **Iterate** — commit new versions and deploy without code changes
-
----
-
-## Create a Prompt
-
-### Via MCP
-
-\`\`\`
-Use create_prompt to create a prompt:
-  name: "classify_intent"
-  description: "Classifies user messages into intent categories"
-
-Then use create_prompt_version to add content:
-  prompt_id: "<id from above>"
-  model: "gpt-4o-mini"
-  messages:
-    - role: "system"
-      content: "You are an intent classifier. Classify the user message into one of: question, complaint, request, feedback."
-    - role: "user"
-      content: "{{user_message}}"
-\`\`\`
-
-### Via CLI
-
-\`\`\`bash
-respan prompts create --name "classify_intent"
-respan prompts versions <prompt-id>
-\`\`\`
-
----
-
-## Variables
-
-Use \`{{variable_name}}\` in message content. Variables are filled at runtime.
-
-\`\`\`json
-{
-  "messages": [
-    {"role": "system", "content": "You are a {{role}} assistant."},
-    {"role": "user", "content": "{{user_input}}"}
-  ]
-}
-\`\`\`
-
-### Jinja templates
-
-Prompts support Jinja2 syntax:
-
-- **Conditionals:** \`{% if context %}Use this context: {{context}}{% endif %}\`
-- **JSON access:** \`{{ input.key }}\`
-- **Filters:** \`{{ var | upper }}\`
-- **Loops:** \`{% for item in items %}{{item}}{% endfor %}\`
-- **Default values:** Set via the API or platform UI
-
----
-
-## Versioning and Deployment
-
-Each prompt can have multiple versions. Only one version is **deployed** (live) at a time.
-
-### Workflow
-
-1. \`create_prompt_version\` — creates a new version (always starts as NOT deployed)
-2. Test the version in the playground or via API
-3. Deploy via platform UI when ready
-4. Code using \`prompt_id\` automatically gets the deployed version
-
-### Version pinning
-
-\`\`\`python
-# Always use the deployed version (default)
-prompt = get_prompt(prompt_id="abc123")
-
-# Pin to a specific version
-prompt = get_prompt(prompt_id="abc123", version=3)
-\`\`\`
-
----
-
-## Using Prompts in Code
-
-### Python (via Gateway)
-
-Fetch the prompt config and pass to the LLM:
-
-\`\`\`python
-import os
-from openai import OpenAI
-
-client = OpenAI(
-    api_key=os.environ["RESPAN_API_KEY"],
-    base_url=os.getenv("RESPAN_BASE_URL", "https://api.respan.ai/api"),
-)
-
-# The gateway resolves the prompt template and fills variables
-response = client.chat.completions.create(
-    model="gpt-4o-mini",
-    messages=[],  # Messages come from the prompt template
-    extra_body={
-        "prompt_id": "your-prompt-id",
-        "prompt_variables": {
-            "user_message": "I want to return my order",
-        },
-    },
-)
-\`\`\`
-
-### Linking traces to prompts
-
-Use \`propagate_attributes\` to tag traces with the prompt used:
-
-\`\`\`python
-from respan import propagate_attributes
-
-with propagate_attributes(
-    prompt={"prompt_id": "abc123", "variables": {"user_message": "Hello"}},
-):
-    response = client.chat.completions.create(...)
-\`\`\`
-
----
-
-## Prompt Composition
-
-Variables can reference other prompts (child prompts rendered first, injected as text).
-
-Configure in the platform UI by changing a variable's type from **Text** to **Prompt**.
-
-Max nesting depth: 2 (parent -> child).
-
----
-
-## Structured Output (JSON Schema)
-
-Attach a JSON schema to a prompt version for structured responses:
-
-\`\`\`json
-{
-  "response_format": {
-    "type": "json_schema",
-    "json_schema": {
-      "name": "extraction",
-      "strict": true,
-      "schema": {
-        "type": "object",
-        "properties": {
-          "intent": {"type": "string"},
-          "confidence": {"type": "number"}
-        },
-        "required": ["intent", "confidence"],
-        "additionalProperties": false
-      }
-    }
-  }
-}
-\`\`\`
-
----
-
-## MCP Tools Reference
-
-| Action | Tool |
-|--------|------|
-| List all prompts | \`list_prompts\` |
-| Get prompt details | \`get_prompt_detail(prompt_id)\` |
-| Create a prompt | \`create_prompt(name, description)\` |
-| Update name/description | \`update_prompt(prompt_id, name, description)\` |
-| Create a version | \`create_prompt_version(prompt_id, messages, model, ...)\` |
-| Update a version | \`update_prompt_version(prompt_id, version, ...)\` |
-| List versions | \`list_prompt_versions(prompt_id)\` |
-| Get version detail | \`get_prompt_version_detail(prompt_id, version)\` |
-
----
-
-## CLI Commands
-
-\`\`\`bash
-respan prompts list                    # List all prompts
-respan prompts get <id>                # Get prompt details
-respan prompts create --name <name>    # Create a prompt
-respan prompts update <id>             # Update a prompt
-respan prompts versions <id>           # List prompt versions
-\`\`\`
-`;
-export const EVALS_MD = `# Evals Reference
-
-Build evaluation pipelines: datasets, evaluators, and experiments.
-
-Docs: \`https://respan.ai/docs/documentation/features/evals/concepts.md\`
-
----
-
-## Overview
-
-Two evaluation modes:
-
-- **Offline** — build datasets, run experiments across prompt/model variants, compare scores
-- **Online** — automatically score live production traffic (see [monitors.md](monitors.md))
-
-Offline eval workflow:
-
-\`\`\`
-Create dataset -> Create evaluator -> Create experiment -> Run & compare
-\`\`\`
-
----
-
-## Datasets
-
-A dataset is a collection of test rows (input/output pairs) for experiments.
-
-### Create from scratch
-
-\`\`\`
-Use create_dataset:
-  name: "intent_classification_testset"
-  description: "Test cases for intent classifier"
-
-Then add rows with create_dataset_span:
-  dataset_id: "<id>"
-  input: '{"user_message": "I want a refund"}'
-  output: '{"intent": "refund_request"}'
-\`\`\`
-
-### Create from production traces
-
-Sample real spans from traced data:
-
-\`\`\`
-1. Use list_traces or list_logs to find relevant spans
-2. Use add_spans_to_dataset to add span IDs to a dataset:
-     dataset_id: "<id>"
-     span_ids: ["span_1", "span_2", "span_3"]
-\`\`\`
-
-Or use the platform UI: Datasets -> Insert by sampling (filter, set sampling %, preview, confirm).
-
-### CSV import
-
-Upload CSV via the platform UI. Columns map to prompt variables. Optional \`ideal_output\` column for expected outputs. Max 500 rows per import.
-
-### MCP tools
-
-| Action | Tool |
-|--------|------|
-| Create dataset | \`create_dataset(name, description)\` |
-| List datasets | \`list_datasets\` |
-| Get dataset | \`get_dataset(dataset_id)\` |
-| Update dataset | \`update_dataset(dataset_id, name, description)\` |
-| Add a row manually | \`create_dataset_span(dataset_id, input, output)\` |
-| Add existing spans | \`add_spans_to_dataset(dataset_id, span_ids)\` |
-| List rows | \`list_dataset_spans(dataset_id)\` |
-| Get a row | \`retrieve_dataset_span(dataset_id, log_id)\` |
-| Update a row | \`update_dataset_span(dataset_id, log_id, ...)\` |
-
----
-
-## Evaluators
-
-An evaluator scores LLM outputs. It is a workflow built from **graders** connected with conditions.
-
-### Grader types
-
-| Type | How it works |
-|------|-------------|
-| **LLM grader** | A language model judges the output. Uses variables: \`{{output}}\`, \`{{input}}\`, \`{{expected_output}}\`, \`{{metadata}}\`, \`{{metrics}}\` |
-| **Code grader** | A Python function returns a score. Signature: \`main(eval_inputs)\` where \`eval_inputs["output"]\` is required |
-| **Human grader** | Team members review and score outputs manually |
-
-### Score value types
-
-| Type | Use case |
-|------|----------|
-| \`numerical\` | Rating scales (e.g. 0-5) |
-| \`boolean\` | Pass/fail checks |
-| \`categorical\` | Multi-choice classifications |
-| \`comment\` | Qualitative text feedback |
-
-### Create an evaluator
-
-\`\`\`
-Use create_evaluator:
-  name: "response_quality"
-  score_value_type: "numerical"
-  type: "llm"
-  description: "Scores response quality 1-5"
-  evaluator_slug: "response-quality"
-\`\`\`
-
-For complex evaluators with grader workflows, conditions, and compute blocks — use the platform UI (Evaluators -> Create).
-
-### Evaluator workflow blocks
-
-- **Markers:** \`Original input\`, \`Final result\` (entry/exit)
-- **Graders:** LLM, Code, or Human scoring
-- **Conditions:** \`If...Then...Else\` branching based on scores
-- **Compute:** Average/weighted average of multiple grader scores
-- **Metrics:** Built-in: completion tokens, cost, latency, model, prompt tokens, total tokens
-- **Constants:** Fixed threshold values
-
-### MCP tools
-
-| Action | Tool |
-|--------|------|
-| List evaluators | \`list_evaluators\` |
-| Get evaluator | \`get_evaluator(evaluator_id)\` |
-| Create evaluator | \`create_evaluator(name, score_value_type, ...)\` |
-| Update evaluator | \`update_evaluator(evaluator_id, ...)\` |
-| Run evaluator | \`run_evaluator(evaluator_id, dataset_id or log_ids)\` |
-
----
-
-## Experiments
-
-An experiment runs evaluators across a dataset to compare prompt/model configurations.
-
-### Workflow
-
-1. Create a dataset with test rows
-2. Create evaluator(s) to score outputs
-3. Create an experiment linking dataset + evaluators
-4. Run the experiment — it generates outputs for each row and scores them
-5. Compare results across configurations
-
-### Create an experiment
-
-\`\`\`
-Use create_experiment:
-  name: "v2_vs_v3_comparison"
-  dataset_id: "<dataset-id>"
-  evaluator_slugs: ["response-quality", "hallucination-check"]
-  workflows:
-    - type: "prompt"
-      config: { prompt_id: "<prompt-v2-id>" }
-    - type: "prompt"
-      config: { prompt_id: "<prompt-v3-id>" }
-\`\`\`
-
-### Workflow types
-
-| Type | Description |
-|------|-------------|
-| \`prompt\` | Run a saved prompt template against dataset rows |
-| \`completion\` | Run a model completion with config |
-| \`custom\` | User-defined logic |
-
-### MCP tools
-
-| Action | Tool |
-|--------|------|
-| List experiments | \`list_experiments\` |
-| Get experiment | \`get_experiment(experiment_id)\` |
-| Create experiment | \`create_experiment(name, dataset_id, ...)\` |
-| List experiment spans | \`list_experiment_spans(experiment_id)\` |
-| Get experiment span | \`get_experiment_span(experiment_id, log_id)\` |
-| Update experiment span | \`update_experiment_span(experiment_id, log_id, ...)\` |
-| Get summary stats | \`get_experiment_spans_summary(experiment_id, start_time, end_time)\` |
-
----
-
-## End-to-End Example
-
-\`\`\`
-Step 1: Create dataset
-  create_dataset(name="qa_testset")
-  create_dataset_span(dataset_id="...", input="What is Python?", output="A programming language")
-  create_dataset_span(dataset_id="...", input="Who made Rust?", output="Mozilla Research")
-
-Step 2: Create evaluator
-  create_evaluator(
-    name="accuracy_check",
-    score_value_type="boolean",
-    type="llm",
-    evaluator_slug="accuracy-check"
-  )
-
-Step 3: Create experiment
-  create_experiment(
-    name="gpt4o_vs_claude",
-    dataset_id="...",
-    evaluator_slugs=["accuracy-check"]
-  )
-
-Step 4: Review results
-  list_experiment_spans(experiment_id="...")
-  get_experiment_spans_summary(experiment_id="...", start_time="...", end_time="...")
-\`\`\`
-
----
-
-## CLI Commands
-
-\`\`\`bash
-respan datasets list                   # List datasets
-respan datasets create --name <name>   # Create a dataset
-respan datasets get <id>               # Get dataset details
-respan datasets spans <id>             # List dataset spans
-respan evaluators list                 # List evaluators
-respan evaluators get <id>             # Get evaluator details
-respan evaluators create               # Create an evaluator
-respan evaluators run <id>             # Run an evaluator
-respan experiments list                # List experiments
-respan experiments get <id>            # Get experiment details
-respan experiments create              # Create an experiment
-respan eval <file>                     # Run a one-shot eval defined in a JSON file
-\`\`\`
-
-## One-Shot Evals via JSON
-
-\`respan eval <file.json>\` runs a complete eval pipeline from a single JSON spec —
-creates the prompt, dataset, evaluators, and experiment in one command.
-IDs are written back into the file so reruns reuse existing resources.
-
-### Minimal example
-
-\`\`\`json
-{
-  "name": "Movie matcher",
-  "prompt": {
-    "name": "movie-matcher",
-    "model": "gpt-5-mini",
-    "messages": [
-      { "role": "system", "content": "Identify the movie. Reply with only the title." },
-      { "role": "user", "content": "{{description}}" }
-    ]
-  },
-  "dataset": {
-    "name": "movie-descriptions",
-    "rows": [
-      { "description": "A detective investigates seven deadly sins.", "expected_output": "Se7en" },
-      { "description": "A hacker learns reality is a simulation.", "expected_output": "The Matrix" }
-    ]
-  },
-  "experiment": {
-    "evaluators": [
-      {
-        "name": "Title match",
-        "type": "llm",
-        "score_value_type": "boolean",
-        "llm_config": {
-          "model": "openai/gpt-5-mini",
-          "evaluator_definition": "Expected: {{expected_output}}. Got: {{output}}. Correct? Reply true or false."
-        }
-      }
-    ]
-  }
-}
-\`\`\`
-
-Run:
-
-\`\`\`bash
-respan eval movie-matcher.eval.json
-\`\`\`
-
-The runner creates the prompt (and deploys version 1), creates the dataset (and bulk-loads rows),
-creates each evaluator + its workflow (and deploys), then creates the experiment linking them.
-Each step writes the resulting IDs back to the JSON file so subsequent runs are idempotent —
-re-running with the same file reuses the existing resources.
-`;
-export const MONITORS_MD = `# Monitors & Automation Reference
-
-Set up monitoring alerts, online evaluations, and automated workflows.
-
-Docs: \`https://respan.ai/docs/documentation/features/monitoring/monitors.md\`
-
----
-
-## Monitors
-
-A monitor watches a metric over time and sends a notification when a threshold is crossed.
-
-### Examples
-
-- Error rate > 5% over 10 minutes
-- Total cost > $1.00 over 5 minutes
-- Average latency > 3 seconds over 15 minutes
-- P95 latency > 5 seconds over 1 hour
-
-### Setup
-
-Monitors are configured in the platform UI: **Monitoring -> Monitors -> Create**.
-
-A monitor has two parts:
-
-#### 1. Trigger
-
-Template: \`When [metric] of [source] is [threshold] over [time window]\`
-
-- **Metric:** cost, latency, error rate, token usage, request count, etc.
-- **Source:** all requests, specific model, specific customer, etc.
-- **Threshold:** condition (greater than, less than, equals)
-- **Time window:** evaluation period (1 min, 5 min, 10 min, 1 hour, etc.)
-
-Add **Where** conditions to narrow scope:
-- Filter by model, project, environment, customer, or custom metadata
-
-#### 2. Notifications
-
-Destinations:
-
-| Destination | Details |
-|-------------|---------|
-| **Email** | Customizable subject and body |
-| **Slack** | Customizable message |
-| **Webhook** | Raw URL — receives JSON payload |
-
-A single monitor can send to multiple destinations.
-
-### Manage
-
-- **Deploy** to make a monitor live
-- **Pause/Resume** deployed monitors
-- **Send test alert** to validate before deploying
-- **View versions** and restore earlier versions as drafts
-- **Create from chart** — select a metric on the dashboard, click to create a monitor with metric/source/interval prefilled
-
----
-
-## Online Evaluations (Automation)
-
-Automatically score production traffic in real time using deployed evaluators.
-
-### Setup
-
-Configure in the platform UI: **Evaluators -> Automations -> Create**.
-
-1. **Create a condition** — filter rules for which logs to evaluate:
-   - **Single log** — evaluate individual spans matching criteria
-   - **Aggregation** — evaluate aggregated metrics over a window
-
-2. **Create an automation** — connect:
-   - Condition (which logs)
-   - Evaluator (which scoring workflow)
-   - Sampling rate (0.0-1.0, fraction of matching logs to evaluate)
-
-3. **Check results** — view eval scores in the automation table
-
-### Use case
-
-> "Score 10% of production chat responses for hallucination using my LLM evaluator"
-
-\`\`\`
-Condition: log_type = "chat" AND model = "gpt-4o"
-Evaluator: hallucination-check
-Sampling: 0.1
-\`\`\`
-
----
-
-## Webhooks
-
-Receive event notifications via HTTP webhooks.
-
-### Setup
-
-1. Go to platform: **Settings -> Webhooks -> Create**
-2. Enter your endpoint URL
-3. Select event types to subscribe to
-4. Optionally set a webhook secret for HMAC-SHA256 verification
-
-### Verification
-
-Respan signs webhook payloads with HMAC-SHA256. Verify the signature:
-
-\`\`\`python
-import hashlib
-import hmac
-
-def verify_webhook(payload: bytes, signature: str, secret: str) -> bool:
-    expected = hmac.new(
-        secret.encode(),
-        payload,
-        hashlib.sha256,
-    ).hexdigest()
-    return hmac.compare_digest(expected, signature)
-\`\`\`
-
-The signature is sent in the request headers.
-
----
-
-## Metrics Dashboard
-
-Track key indicators in the platform: **Dashboard -> Metrics**.
-
-### Available metrics
-
-- **Requests:** total count, requests per model/customer/API key
-- **Tokens:** prompt tokens, completion tokens, total tokens
-- **Cost:** per-request, per-model, per-customer
-- **Latency:** average, P50, P95, P99
-- **Errors:** error count, error rate
-- **Time to first token (TTFT):** streaming latency
-
-### Breakdowns
-
-Slice metrics by:
-- Model
-- Customer (\`customer_identifier\`)
-- API key
-- Prompt template
-- Environment
-- Custom metadata keys
-
-### Per-user analytics
-
-Use \`customer_identifier\` in spans to enable:
-- Per-user cost tracking and budgets
-- Per-user rate limiting
-- User-level quality metrics
-
----
-
-## CLI Commands
-
-\`\`\`bash
-respan traces list --limit 10          # List recent traces
-respan traces get <trace-id>           # Get trace details
-respan traces summary                  # Trace summary stats
-respan logs list --limit 10            # List recent logs
-respan logs get <log-id>               # Get log details
-respan logs summary                    # Log summary stats
-\`\`\`
-`;