elasticdash-sdk 0.2.0

package/README.md

@@ -0,0 +1,775 @@
+# ElasticDash SDK
+
+An AI-native test runner for ElasticDash workflow testing. Built for async AI pipelines — not a general-purpose test runner.
+
+## Quick Links
+
+### Jump to Key Sections
+- [Quick Start](#quick-start)
+- [Documentation](#documentation)
+- [Tool Recording](#tool-recording)
+- [AI Call Recording](#ai-call-recording)
+- [Workflow Tracing](#workflow-tracing)
+- [HTTP Workflow Mode](#http-workflow-mode)
+- [CI/CD Runner](#cicd-runner)
+- [Configuration](#configuration)
+
+### Open Detailed Docs
+- **[Quick Start Guide](docs/quickstart.md)** ← Start here to set up your first workflow
+- [Test Writing Guidelines](docs/test-writing-guidelines.md)
+- [Test Matchers](docs/matchers.md)
+- [Tool Recording and Replay](docs/tools.md)
+- [Workflows Dashboard](docs/dashboard.md)
+- [Agent Mid-Trace Replay](docs/agents.md)
+- [Deno Support](docs/deno.md)
+- [Instrumentation Guide](docs/instrumentation.md) — how to write `ed_tools.ts`, `ed_workflows.ts`, `ed_agents.ts`
+- [Langfuse Trace Structure](docs/langfuse-trace-structure.md) — span structure required for dashboard replay
+
+## Features
+
+- 🎯 **Trace-first testing** — every test gets a `trace` context to record and assert on LLM calls and tool invocations
+- 🔍 **Automatic AI interception** — captures OpenAI, Gemini, and Grok calls without code changes
+- 🧪 **AI-specific matchers** — semantic output matching, LLM-judged evaluations, prompt assertions
+- 🛠️ **Tool & LLM recording & replay** — automatically trace tool and AI calls with checkpoint-based replay and mock support
+- 📊 **Interactive dashboard** — browse workflows, debug traces, validate fixes visually
+- 🤖 **Agent mid-trace replay** — resume long-running agents from any task without re-execution
+- 🌐 **HTTP workflow mode** — run workflows against your live dev server for framework-heavy apps (Next.js, Remix, etc.) with full AI and tool call observability
+- 🚀 **CI/CD runner** — fetch test groups from your ElasticDash project, execute tests, submit results, and fail the build on regressions
+
+---
+
+## Installation
+
+```bash
+npm install elasticdash-sdk
+```
+
+**Requirements:** Node 20+. For Deno projects, see [Using elasticdash-sdk in Deno](docs/deno.md).
+
+### Setup with a Coding Agent
+
+If you use a coding agent (Claude Code, Cursor, Copilot, Codex, Windsurf, etc.), tell your agent:
+
+```
+Integrate elasticdash-sdk into this project.
+Read node_modules/elasticdash-sdk/docs/agent-coding-instructions.md for how to proceed,
+and node_modules/elasticdash-sdk/docs/agent-integration-guide.md for technical reference.
+```
+
+Your agent will read both docs and handle the full setup — creating `ed_tools.ts`, `ed_workflows.ts`, updating source files, and validating the connection.
+
+**Optional:** To copy the agent instructions into your project for easier access:
+
+```bash
+npx elasticdash init-guide                              # creates AGENTS.md
+npx elasticdash init-guide --target CLAUDE.md            # for Claude Code
+npx elasticdash init-guide --target .cursor/rules/elasticdash.md  # for Cursor
+npx elasticdash init-guide --target .github/copilot-instructions.md  # for Copilot
+```
+
+If the target file already exists, the guide is appended (not overwritten). Use `--force` to replace the file entirely.
+
+### Cloud Setup
+
+Add these to your `.env` (or CI secrets):
+
+```bash
+ELASTICDASH_API_URL=https://server.elasticdash.com
+ELASTICDASH_API_KEY=ed_your_api_key_here
+```
+
+- **`ELASTICDASH_API_URL`** — The ElasticDash cloud backend URL. For cloud users this is always `https://server.elasticdash.com`. For self-hosted instances, use your own backend URL.
+- **`ELASTICDASH_API_KEY`** — Your project API key. Find it in the [ElasticDash dashboard](https://app.elasticdash.com) under project settings.
+
+> **Note:** `ELASTICDASH_SERVER` is an alias for `ELASTICDASH_API_URL`. Both work — the SDK checks `ELASTICDASH_API_URL` first, then falls back to `ELASTICDASH_SERVER`.
+
+**Git ignore:** ElasticDash writes temporary runtime artifacts under `.temp/`. Add this to your `.gitignore`:
+
+```gitignore
+.temp/
+```
+
+**Running CLI commands:** Use `npx` to run commands with your locally installed version (recommended to avoid version drift):
+
+```bash
+npx elasticdash test
+npx elasticdash dashboard
+```
+
+Alternatively, install globally if you prefer shorter commands:
+
+```bash
+npm install -g elasticdash-sdk
+elasticdash test
+elasticdash dashboard
+```
+
+---
+
+## Quick Start
+
+**1. Write a test file** (`my-flow.ai.test.ts`):
+
+```ts
+import '../node_modules/elasticdash-sdk/dist/test-setup.js'
+import { expect } from 'expect'
+
+aiTest('checkout flow', async (ctx) => {
+  await runCheckout(ctx)
+
+  expect(ctx.trace).toHaveLLMStep({ model: 'gpt-4', contains: 'order confirmed' })
+  expect(ctx.trace).toCallTool('chargeCard')
+})
+```
+
+**2. Run it:**
+
+```bash
+npx elasticdash test              # discover all * *.ai.test.ts files
+npx elasticdash test ./ai-tests   # discover in a specific directory
+npx elasticdash run my-flow.ai.test.ts  # run a single file
+```
+
+**3. Read the output:**
+
+```
+  ✓ checkout flow (1.2s)
+  ✗ refund flow (0.8s)
+    → Expected tool "chargeCard" to be called, but no tool calls were recorded
+
+2 passed
+1 failed
+Total: 3
+Duration: 3.4s
+```
+
+**Workflow export requirements (subprocess mode):**
+
+- Export plain callable functions from `ed_workflows.ts/js`.
+- Use JSON-serializable inputs/outputs (object or array) so dashboard replay can pass args and read results.
+- Do not export framework-bound handlers directly (for example Next.js `NextRequest`/`NextResponse` route handlers) — use [HTTP workflow mode](#http-workflow-mode) instead.
+
+---
+
+## Documentation
+
+### Core Concepts
+- **[Test Writing Guidelines](docs/test-writing-guidelines.md)** — comprehensive guide to writing AI workflow tests
+- **[Test Matchers](docs/matchers.md)** — all available matchers with examples
+- **[Tool Recording & Replay](docs/tools.md)** — automatic tool tracing and checkpoint-based replay
+
+### Advanced Features
+- **[Workflows Dashboard](docs/dashboard.md)** — interactive workflow browser, debugger, and fetching traces from Langfuse
+- **[Agent Mid-Trace Replay](docs/agents.md)** — resume long-running agents from any task
+- **[Deno Support](docs/deno.md)** — using ElasticDash SDK in Deno projects
+
+### Integration & Reference
+- **[Instrumentation Guide](docs/instrumentation.md)** — how to write `ed_tools.ts`, `ed_workflows.ts`, and `ed_agents.ts` to connect your production code to ElasticDash
+- **[Integration Guide](docs/agent-integration-guide.md)** — step-by-step SDK integration reference (templates, patterns, decision trees)
+- **[Agent Coding Instructions](docs/agent-coding-instructions.md)** — behavioral instructions for AI coding agents performing the integration
+- **[Langfuse Trace Structure](docs/langfuse-trace-structure.md)** — Langfuse span structure required for dashboard replay and tool-level diffing
+
+---
+
+## Quick Reference
+
+### Test Globals
+
+| Global | Description |
+|---|---|
+| `aiTest(name, fn)` | Register a test |
+| `beforeAll(fn)` | Run once before all tests in the file |
+| `beforeEach(fn)` | Run before every test in the file |
+| `afterEach(fn)` | Run after every test in the file (runs even if test fails) |
+| `afterAll(fn)` | Run once after all tests in the file |
+
+### Recording Trace Data
+
+**Automatic (recommended):** Workflow code making real API calls to OpenAI, Gemini, or Grok is automatically intercepted and recorded.
+
+**Manual (for custom providers or mocks):**
+
+```ts
+ctx.trace.recordLLMStep({
+  model: 'gpt-4',
+  prompt: 'What is the order status?',
+  completion: 'The order has been confirmed.',
+})
+
+ctx.trace.recordToolCall({
+  name: 'chargeCard',
+  args: { amount: 99.99 },
+})
+
+ctx.trace.recordCustomStep({
+  kind: 'rag',
+  name: 'pokemon-search',
+  payload: { query: 'pikachu' },
+  result: { ids: [25] },
+})
+```
+
+### Common Matchers
+
+```ts
+// Assert LLM calls
+expect(ctx.trace).toHaveLLMStep({ model: 'gpt-4' })
+expect(ctx.trace).toHaveLLMStep({ promptContains: 'order status' })
+
+// Assert tool calls
+expect(ctx.trace).toCallTool('chargeCard')
+
+// Semantic output matching (LLM-judged)
+expect(ctx.trace).toMatchSemanticOutput('order confirmed')
+
+// Custom steps (RAG, code, fixed)
+expect(ctx.trace).toHaveCustomStep({ kind: 'rag', name: 'pokemon-search' })
+```
+
+**→ See [Test Matchers](docs/matchers.md) for complete documentation**
+
+---
+
+## Automatic AI & Tool Tracing
+
+### AI Interception
+
+The runner automatically intercepts and records calls to:
+- OpenAI (`api.openai.com`)
+- Gemini (`generativelanguage.googleapis.com`)
+- Grok/xAI (`api.x.ai`)
+
+No code changes needed — just run your workflow and assertions work automatically.
+
+### Tool Recording
+
+**Recommended: `wrapTool`** wraps a tool function and automatically records its name, input, output, duration, and any streaming output. Works in both subprocess mode and HTTP mode:
+
+```ts
+import { wrapTool } from 'elasticdash-sdk'
+import { runSelectQuery } from './services/dataService'
+
+export const dataService = wrapTool('dataService', async (input: { query: string }) => {
+  return await runSelectQuery(input.query)
+})
+```
+
+**Manual pattern (legacy):** isolate tracing in the service `.then/.catch` path so tracing failures never block business logic:
+
+```ts
+import { runSelectQuery } from './services/dataService'
+
+export const dataService = async (input: any) => {
+  const { query } = input as { query: string }
+  return await runSelectQuery(query)
+    .then(async (res: any) => {
+      try {
+        const { recordToolCall } = await import('elasticdash-sdk')
+        recordToolCall('dataService', input, res)
+      } catch {
+        // tracing must never block the main service path
+      }
+      return res
+    })
+    .catch(async (err: any) => {
+      try {
+        const { recordToolCall } = await import('elasticdash-sdk')
+        recordToolCall('dataService', input, err)
+      } catch {
+        // tracing must never block the main service path
+      }
+      throw err
+    })
+}
+```
+
+In manual mode, always isolate tracing in a separate `try/catch` so trace logging errors cannot interrupt core service execution.
+
+**→ See [Tool Recording & Replay](docs/tools.md) for checkpoint-based replay and freezing**
+
+### AI Call Recording
+
+**`wrapAI`** wraps any AI call function and records its name, input, output, duration, and token usage (auto-detected for Anthropic, OpenAI, and Gemini SDK responses):
+
+```ts
+import { wrapAI } from 'elasticdash-sdk'
+import Anthropic from '@anthropic-ai/sdk'
+
+const client = new Anthropic()
+
+export const callClaude = wrapAI('claude-sonnet-4-5', async (messages: Anthropic.MessageParam[]) => {
+  return await client.messages.create({
+    model: 'claude-sonnet-4-5-20250929',
+    max_tokens: 1024,
+    messages,
+  })
+})
+```
+
+Use `wrapAI` when you have a custom AI wrapper or a provider not covered by automatic interception. For direct OpenAI/Anthropic/Gemini SDK calls inside a subprocess workflow, automatic interception via `installAIInterceptor` already handles recording without any code changes.
+
+**AI mocking (subprocess / test runner mode):** `wrapAI` also checks `resolveAIMock` at call time, so the dashboard can mock LLM responses the same way it mocks tool calls — without modifying your server code. Configure an `AIMockConfig` in the dashboard UI or pass it programmatically via the `aiMockConfig` option when running a workflow.
+
+### HTTP Streaming Capture and Replay
+
+ElasticDash also captures non-AI `fetch` responses that stream over HTTP (for example SSE and NDJSON endpoints) in the HTTP interceptor.
+
+Currently detected as streaming when response `content-type` includes:
+- `text/event-stream`
+- `application/x-ndjson`
+- `application/stream+json`
+- `application/jsonl`
+
+How it behaves today:
+- During live execution, ElasticDash tees the response stream and returns a real stream to your app code.
+- In parallel, ElasticDash buffers the recorder side of the stream as raw text for trace replay.
+- During replay, ElasticDash reconstructs a stream from that captured raw payload and restores status, status text, and response headers.
+
+Replay fidelity note:
+- Replay preserves stream payload content, but not original chunk boundaries or timing cadence.
+
+Minimal stream consumption example:
+
+```ts
+const res = await fetch('https://example.com/events')
+if (!res.body) throw new Error('Expected a streaming response body')
+
+const reader = res.body.getReader()
+const decoder = new TextDecoder()
+let buffer = ''
+
+for (;;) {
+  const { done, value } = await reader.read()
+  if (done) break
+  buffer += decoder.decode(value, { stream: true })
+}
+
+buffer += decoder.decode()
+```
+
+**→ See [Quick Start Guide](docs/quickstart.md#capture-streaming-flows) for end-to-end setup guidance**
+
+---
+
+## Workflow Tracing
+
+`wrapTool` and `wrapAI` record individual steps, but to group them into a single named workflow trace you need the **trace lifecycle**: call `edStartTrace` at the start of your handler and `edEndTrace` when it finishes. Every `wrapTool`/`wrapAI` call in between is automatically associated with that trace.
+
+### 1. Create `ed_workflows.ts`
+
+This file exports the trace lifecycle functions. It holds a reference to the `elasticdash-sdk` module (set from `ed_tools.ts`) so it can call `startTrace`/`endTrace`:
+
+```ts
+// ed_workflows.ts
+let _ed: any = null;
+
+/** Called from ed_tools.ts to share the SDK module instance. */
+export function setElasticDashModule(mod: any): void {
+  _ed = mod;
+}
+
+/** Call at the start of your request handler to begin a named trace. */
+export const edStartTrace = async (workflowName: string): Promise<void> => {
+  if (!_ed) return;
+  try {
+    await _ed.tryAutoInitHttpContext();
+    _ed.startTrace(workflowName);
+  } catch (err) {
+    console.error('[ed_workflows] edStartTrace error:', err);
+  }
+};
+
+/** Call in a finally block to end the trace and flush captured events. */
+export const edEndTrace = (): void => {
+  if (!_ed) return;
+  try {
+    _ed.endTrace();
+  } catch (err) {
+    console.error('[ed_workflows] edEndTrace error:', err);
+  }
+};
+```
+
+### 2. Create `ed_tools.ts`
+
+This file loads the SDK, shares the module instance with `ed_workflows.ts`, and exports your wrapped tools:
+
+```ts
+// ed_tools.ts
+import { setElasticDashModule } from './ed_workflows';
+
+let wrapTool: <T extends (...args: any[]) => any>(name: string, fn: T) => T = (_name, fn) => fn;
+
+try {
+  // For Next.js / Turbopack: eval('require') bypasses static analysis.
+  // For plain Node.js projects you can use a normal require() or import.
+  const _edModule = (eval('require') as (id: string) => any)('elasticdash-sdk');
+  wrapTool = _edModule.wrapTool ?? wrapTool;
+  setElasticDashModule(_edModule);
+} catch {
+  // elasticdash-sdk not available — tools run without tracing
+}
+
+export const myTool = wrapTool('myTool', async (input: { query: string }) => {
+  // ... your tool logic
+});
+```
+
+> **Why `setElasticDashModule`?** The SDK uses Node.js AsyncLocalStorage (ALS) to correlate events. Both `ed_tools.ts` and `ed_workflows.ts` must share the same CJS module instance so `wrapTool`/`wrapAI` calls write to the same ALS store that `startTrace`/`endTrace` reads from.
+
+### 3. Instrument Your Route Handler
+
+Call `edStartTrace` at handler entry and `edEndTrace` in a `finally` block. All `wrapTool`/`wrapAI` calls in between are grouped under the trace:
+
+```ts
+// app/api/chat/route.ts (Next.js example)
+import { edStartTrace, edEndTrace } from './ed_workflows';
+import { myTool } from './ed_tools';
+
+export async function POST(req: Request) {
+  await edStartTrace('chatHandler');
+  try {
+    const body = await req.json();
+    const result = await myTool({ query: body.message });
+    return Response.json(result);
+  } finally {
+    edEndTrace();
+  }
+}
+```
+
+### 4. Capture Traces to Disk
+
+Set the `ELASTICDASH_CAPTURE_TRACE` environment variable to save each workflow trace as a JSON file under `.ed_traces/`:
+
+```bash
+ELASTICDASH_CAPTURE_TRACE=1 npm run dev
+```
+
+Captured traces can be used for offline replay, benchmark tests (`defineTest` in `ed_tests.ts`), and debugging in the dashboard.
+
+---
+
+## HTTP Workflow Mode
+
+For apps where subprocess import fails (Next.js, Remix, SvelteKit, etc.), configure workflows to call your running dev server directly instead of importing the handler:
+
+```ts
+// elasticdash.config.ts
+export default {
+  testMatch: ['**/*.ai.test.ts'],
+  workflows: {
+    runChat: {
+      mode: 'http',
+      url: 'http://localhost:3001/api/chat',
+      method: 'POST',
+      headers: {
+        'Content-Type': 'application/json',
+        'x-user-id': '{{env.DEV_USER_ID}}',
+      },
+      bodyTemplate: {
+        messages: [{ role: 'user', content: '{{input.message}}' }],
+        selectedModel: 'claude-sonnet-4-5-20250929',
+      },
+      responseFormat: 'vercel-ai-stream',
+    },
+  },
+}
+```
+
+To enable full AI and tool call observability in HTTP mode, install `elasticdash-sdk` in your app:
+
+```ts
+// app/api/chat/route.ts
+import { initHttpRunContext, wrapTool, wrapAI } from 'elasticdash-sdk'
+
+export async function POST(req: Request) {
+  const runId = req.headers.get('x-elasticdash-run-id')
+  const serverUrl = req.headers.get('x-elasticdash-server')
+  if (runId && serverUrl) {
+    await initHttpRunContext(runId, serverUrl)
+  }
+  // ... rest of handler
+}
+```
+
+The dashboard injects `x-elasticdash-run-id` and `x-elasticdash-server` headers automatically when triggering a run. `initHttpRunContext` fetches any frozen steps from the dashboard before execution begins — this is what enables step freezing (replaying historical results for specific steps). Every `wrapAI` and `wrapTool` call downstream pushes telemetry events back to the dashboard in real time.
+
+> **Note:** Use `setHttpRunContext` (synchronous) if you only need observability and do not need step freezing. `initHttpRunContext` is required for the dashboard's breakpoint/replay functionality to work.
+
+### Dashboard Auto-Detection (env var mode)
+
+As an alternative to calling `initHttpRunContext` in your request handler, you can set two environment variables before starting your server or script. Every `wrapTool` and `wrapAI` call will then connect to the dashboard automatically — no code changes needed:
+
+```bash
+# Required: URL of the running ElasticDash dashboard
+ELASTICDASH_SERVER=http://localhost:4573
+
+# Optional: pre-registered run ID to fetch frozen steps for
+ELASTICDASH_RUN_ID=<run-id-from-dashboard>
+```
+
+- If only `ELASTICDASH_SERVER` is set, a fresh run ID is generated and all calls push live telemetry to the dashboard (observability only, no step freezing).
+- If both variables are set, frozen steps are fetched from the dashboard at startup and replayed as configured.
+- If the dashboard is unreachable the SDK falls through to live execution silently.
+- The initialization runs **once per process** — subsequent `wrapTool`/`wrapAI` calls reuse the cached context.
+
+This mode is intended for local development and testing scenarios. For production HTTP servers with concurrent requests, continue using `initHttpRunContext` inside your request handler.
+
+**Subprocess vs HTTP mode comparison:**
+
+| | Subprocess (default) | HTTP mode |
+|---|---|---|
+| Works with simple apps | Yes | Yes |
+| Works with Next.js / Remix | No | Yes |
+| Requires dev server running | No | Yes |
+| App code changes needed | Extract handler to `ed_workflows.ts` | Add `initHttpRunContext` to request handler (or use env vars for auto-detect) |
+| AI / tool call observability | Automatic via interceptors | Via `wrapAI` / `wrapTool` push |
+| Step freezing / breakpoints | Yes | Yes (`initHttpRunContext`, or `ELASTICDASH_SERVER` + `ELASTICDASH_RUN_ID` env vars) |
+| LLM response mocking | Yes (via `aiMockConfig`) | Yes (via frozen AI events) |
+
+---
+
+## CI/CD Runner
+
+Run your ElasticDash test groups directly from CI pipelines. The `ci` command fetches active test groups from your project via API key, executes each test locally, submits results back to the backend, and exits with code 1 if any test fails.
+
+### How It Works
+
+```
+┌──────────────┐     GET /testgroups/by-project     ┌──────────────────┐
+│   CI Runner  │ ──────────────────────────────────→ │  ElasticDash API │
+│  (SDK side)  │ ←────────────────────────────────── │   (your backend) │
+│              │   test groups + tests + expectations │                  │
+│              │                                      │                  │
+│  execute     │     POST /testgroups/:id/runs        │                  │
+│  each test   │ ──────────────────────────────────→ │  stores results  │
+│  locally     │                                      │                  │
+│              │     POST /testgroups/batches          │                  │
+│              │ ──────────────────────────────────→ │  groups the runs │
+└──────────────┘                                      └──────────────────┘
+```
+
+1. **Fetch** — Calls `GET /testgroups/by-project` with the API key (scoped to project). Returns all active test groups with their tests and expectations.
+2. **Execute** — For each test, runs it locally using existing SDK infrastructure:
+   - **Single-step tests** — replays a specific tool or AI step with `mock_input` and `frozen_events`
+   - **Full-flow tests** — runs the entire workflow from `ed_workflows.ts` with `workflow_input`
+3. **Evaluate** — Checks all expectations (token-budget, latency-budget, output-contains, output-schema, tool-called, determinism, llm-judge). Respects `run_count` and `pass_threshold`.
+4. **Submit** — POSTs each result to `POST /testgroups/:id/runs` with single run data, expectation results, and git metadata.
+5. **Batch** — Creates a batch grouping all run IDs for dashboard viewing.
+
+### CLI Usage
+
+```bash
+# Basic — uses env vars (set in .env or CI secrets)
+npx elasticdash ci
+
+# Explicit flags (if not using env vars)
+npx elasticdash ci --server https://server.elasticdash.com --api-key ed_xxx
+
+# Filter by workflow or tags
+npx elasticdash ci --server $ELASTICDASH_API_URL --api-key $ELASTICDASH_API_KEY \
+  --workflow checkout --tags payment,critical
+
+# Pass git metadata (auto-detected in GitHub Actions / GitLab CI)
+npx elasticdash ci --server $ELASTICDASH_API_URL --api-key $ELASTICDASH_API_KEY \
+  --git-branch main --git-commit abc123
+```
+
+**All flags:**
+
+| Flag | Env Var | Description |
+|------|---------|-------------|
+| `--server <url>` | `ELASTICDASH_API_URL` | Backend API URL (required) |
+| `--api-key <key>` | `ELASTICDASH_API_KEY` | Project API key (required) |
+| `--workflow <name>` | — | Filter test groups by workflow name |
+| `--tags <t1,t2>` | — | Filter test groups by tags (comma-separated) |
+| `--triggered-by <src>` | — | Trigger source label (default: `ci`) |
+| `--git-branch <branch>` | Auto-detected | Git branch name |
+| `--git-commit <sha>` | Auto-detected | Git commit SHA |
+| `--git-commit-message <msg>` | Auto-detected | Commit message |
+| `--git-pr-number <n>` | Auto-detected | PR number |
+| `--git-pr-url <url>` | Auto-detected | PR URL |
+
+### GitHub Actions Example
+
+```yaml
+name: AI Tests
+on: [push, pull_request]
+
+jobs:
+  test:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-node@v4
+        with:
+          node-version: 20
+
+      - run: npm ci
+
+      - name: Run ElasticDash CI tests
+        run: npx elasticdash ci
+        env:
+          ELASTICDASH_API_URL: ${{ secrets.ELASTICDASH_API_URL }}
+          ELASTICDASH_API_KEY: ${{ secrets.ELASTICDASH_API_KEY }}
+          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}  # if tests use OpenAI
+```
+
+Git branch, commit SHA, PR number, and PR URL are auto-detected from GitHub Actions environment variables — no extra flags needed.
+
+### GitLab CI Example
+
+```yaml
+ai-tests:
+  stage: test
+  image: node:20
+  script:
+    - npm ci
+    - npx elasticdash ci
+  variables:
+    ELASTICDASH_API_URL: $ELASTICDASH_API_URL
+    ELASTICDASH_API_KEY: $ELASTICDASH_API_KEY
+```
+
+### Programmatic Usage
+
+```ts
+import { runCI } from 'elasticdash-sdk'
+
+const summary = await runCI({
+  serverUrl: 'https://your-api.com',
+  apiKey: 'ed_xxx',
+  workflowName: 'checkout',       // optional filter
+  tags: ['payment', 'critical'],  // optional filter
+})
+
+console.log(`${summary.passed}/${summary.total} passed`)
+process.exit(summary.failed > 0 ? 1 : 0)
+```
+
+### Output
+
+```
+[elasticdash ci] Fetching test groups...
+[elasticdash ci] Found 2 test group(s), 5 test(s) total.
+
+  Checkout Flow (3 tests)
+    validate-input ... PASS (234ms)
+    charge-card ... PASS (1823ms)
+    send-confirmation ... FAIL (945ms)
+      [output-contains] Output text check failed.
+
+  Refund Flow (2 tests)
+    check-eligibility ... PASS (412ms)
+    process-refund ... PASS (1567ms)
+
+──────────────────────────────────────────────────
+Summary
+──────────────────────────────────────────────────
+  Total:    5
+  Passed:   4
+  Failed:   1
+  Duration: 5.0s
+  Batch ID: 42
+──────────────────────────────────────────────────
+
+[elasticdash ci] 1 test(s) failed.
+```
+
+### Prerequisites
+
+- An ElasticDash project with an API key (create one in the dashboard under Settings → API Keys)
+- Active test groups with tests and expectations configured in the dashboard
+- `ed_tools.ts` and/or `ed_workflows.ts` in your project root (for the executor to discover tools and workflows)
+- AI provider API keys in the environment if tests use LLM calls (e.g., `OPENAI_API_KEY`)
+
+---
+
+## Configuration
+
+Optional `elasticdash.config.ts` at project root:
+
+```ts
+export default {
+  testMatch: ['**/*.ai.test.ts'],
+  traceMode: 'local' as const,
+}
+```
+
+**Dashboard port:** defaults to `4573`. Override via CLI flag or `.env`:
+
+```bash
+# .env
+ELASTICDASH_PORT=5000
+```
+
+```bash
+# or CLI flag
+npx elasticdash dashboard --port 5000
+```
+
+Optional project file: `ed_workers.ts` can be used by your app architecture (for example, exporting worker handlers), but it is not required or discovered by the ElasticDash CLI/dashboard.
+
+## TypeScript Setup
+
+For typed globals and matchers, extend your test directory's `tsconfig.json`:
+
+```json
+{
+  "extends": "../tsconfig.json",
+  "include": ["../src/**/*", "./**/*"]
+}
+```
+
+---
+
+## Programmatic API
+
+```ts
+import { runFiles, reportResults, registerMatchers, installAIInterceptor } from 'elasticdash-sdk'
+
+registerMatchers()
+installAIInterceptor()
+
+const results = await runFiles(['./tests/flow.ai.test.ts'])
+reportResults(results)
+```
+
+**HTTP mode context (call inside your request handler):**
+
+```ts
+import { initHttpRunContext, setHttpRunContext } from 'elasticdash-sdk'
+
+// Async — fetches frozen steps from dashboard to enable step freezing/breakpoints
+await initHttpRunContext(runId, dashboardUrl)
+
+// Synchronous alternative — observability only, no step freezing
+setHttpRunContext(runId, dashboardUrl)
+```
+
+**Dashboard auto-detection (env var mode — no code changes needed):**
+
+```bash
+# Set before starting your server or script
+ELASTICDASH_API_URL=https://server.elasticdash.com  # cloud (or http://localhost:4573 for local dashboard)
+ELASTICDASH_API_KEY=ed_your_api_key_here             # your project API key
+ELASTICDASH_RUN_ID=<run-id-from-dashboard>           # optional, enables step freezing
+```
+
+`wrapTool` and `wrapAI` will auto-connect on their first call. See [Dashboard Auto-Detection](#dashboard-auto-detection-env-var-mode) for details.
+
+**CI runner (execute test groups from your project):**
+
+```ts
+import { runCI } from 'elasticdash-sdk'
+
+const summary = await runCI({ serverUrl: 'https://server.elasticdash.com', apiKey: 'ed_xxx' })
+// summary.total, summary.passed, summary.failed, summary.batchId, summary.results
+```
+
+---
+
+## License
+
+MIT