elasticdash-sdk 0.2.0

package/docs/agents.md

@@ -0,0 +1,140 @@
+# Agent Mid-Trace Replay
+
+ElasticDash supports resuming long-running agents from any task in their plan — without re-executing already-completed steps.
+
+## Use Cases
+
+- **Resuming after failures**: If task 3 of 5 fails, fix the issue and re-run from task 3 only
+- **Pausing for approval**: Capture state after task 2, get human sign-off, then continue
+- **Debugging in isolation**: Re-run a single task with modified input to diagnose a problem
+
+## How It Works
+
+Agents are structured as an **AgentPlan** — an ordered list of **AgentTask** objects. When serialized with captured trace events, this forms an **AgentState** that can be saved and replayed later.
+
+## Quick Start
+
+```ts
+import { plannerAgent, executorAgent, resumeAgentFromTrace } from './ed_agents'
+import { serializeAgentState, deserializeAgentState } from 'elasticdash-sdk'
+import fs from 'node:fs'
+
+// 1. Generate a plan
+const plan = await plannerAgent('Show me sales for Q1', { userToken: 'tok-abc' })
+
+// 2. Execute the plan (runs all tasks sequentially)
+const completedPlan = await executorAgent(plan)
+
+// 3. Serialize and save state (e.g., after partial execution)
+const state = serializeAgentState(completedPlan, [] /* pass recorder.events in worker context */)
+fs.writeFileSync('agent-state.json', JSON.stringify(state, null, 2))
+
+// 4. Later: load saved state and resume from task 2 (0-based index 1)
+const savedState = JSON.parse(fs.readFileSync('agent-state.json', 'utf8'))
+const stateToResume = deserializeAgentState({ ...savedState, resumeFromTaskIndex: 1 })
+const resumedPlan = await resumeAgentFromTrace(stateToResume)
+
+console.log('Resumed plan status:', resumedPlan.status)
+console.log('Task outputs:', resumedPlan.tasks.map((t) => ({ id: t.id, status: t.status })))
+```
+
+## Data Structures
+
+### AgentState
+
+```ts
+interface AgentState {
+  plan: AgentPlan               // Full plan with all tasks (completed and pending)
+  trace: WorkflowEvent[]        // Captured trace events from previous execution
+  resumeFromTaskIndex: number   // Zero-based index — tasks before this are loaded from cache
+}
+```
+
+### AgentPlan
+
+```ts
+interface AgentPlan {
+  id: string
+  tasks: AgentTask[]
+  status: 'planning' | 'executing' | 'completed' | 'failed' | 'paused'
+  currentTaskIndex: number
+  context: Record<string, unknown>
+  metadata: Record<string, unknown>
+}
+```
+
+### AgentTask
+
+```ts
+interface AgentTask {
+  id: string
+  status: 'pending' | 'in-progress' | 'completed' | 'failed'
+  description: string
+  tool: string          // Name of the tool function to invoke
+  input: unknown        // May contain { $ref: "task-N.output.fieldName" } placeholders
+  output?: unknown      // Populated after execution
+  error?: string
+  startedAt?: number
+  completedAt?: number
+}
+```
+
+## Task Input Placeholders
+
+Task inputs can reference previous task outputs using `{ $ref: "taskId.output.fieldPath" }`:
+
+```ts
+// task-2 uses the embedding produced by task-1
+{
+  id: 'task-2',
+  tool: 'taskSelectorService',
+  input: {
+    queryEmbedding: { $ref: 'task-1.output.embedding' },
+    topK: 3,
+  }
+}
+```
+
+Placeholders are resolved at execution time by `resolveTaskInput()`.
+
+## Dashboard Integration
+
+When running an agent workflow through the dashboard:
+
+1. **Agent task observations** are visually highlighted with a purple background and left border
+2. Each observation shows a **T1 / T2 / T3** badge indicating which task it belongs to
+3. In the observation detail panel, a **"Resume from Task N"** button appears (agent steps only)
+4. Clicking it calls `/api/resume-agent-from-task` with the serialized `AgentState` and chosen `taskIndex`
+5. The resumed run is added as a new trace in the comparison table
+
+## Best Practices
+
+- **Keep tasks idempotent** where possible — if a task must be re-run, ensure it produces the same result
+- **Store minimal outputs** — only record what downstream tasks need, not full API responses
+- **Version your state schema** — if tool interfaces change, old states may need migration
+- **Use sequential tasks** — the current implementation runs tasks one-by-one; parallel task support is planned
+
+## Example: Debugging a Failed Task
+
+```ts
+// 1. Original execution fails at task 3
+const plan = await plannerAgent('Process refund for order-123')
+const result = await executorAgent(plan)
+// Error: task 3 (calculateRefundAmount) failed
+
+// 2. Save the state
+const state = serializeAgentState(result, recorder.events)
+fs.writeFileSync('failed-run.json', JSON.stringify(state))
+
+// 3. Fix the issue in your tool/code
+
+// 4. Resume from task 3 with corrected state
+const savedState = JSON.parse(fs.readFileSync('failed-run.json'))
+const fixed = await resumeAgentFromTrace({
+  ...deserializeAgentState(savedState),
+  resumeFromTaskIndex: 2  // 0-based: task 3 = index 2
+})
+
+// Tasks 1-2 use cached results; task 3+ execute with fixes
+console.log('Fixed plan:', fixed.status)
+```

package/docs/dashboard.md

@@ -0,0 +1,394 @@
+# Workflows Dashboard
+
+Browse, search, and run all available workflow and tool functions in your project.
+
+## Starting the Dashboard
+
+```bash
+npx elasticdash dashboard         # open dashboard at http://localhost:4573
+npx elasticdash dashboard --port 4572  # use custom port
+npx elasticdash dashboard --no-open    # skip auto-opening browser
+```
+
+### Framework Runtime Compatibility (Next.js, TS Path Aliases)
+
+If your project uses **TypeScript path aliases** (e.g., `@/lib/...`) or requires runtime TypeScript loading, you may need to configure Node.js loaders.
+
+**Standard command (works for most projects):**
+
+```bash
+npx elasticdash dashboard
+```
+
+**Advanced command (Next.js, mixed ESM/CJS, path aliases):**
+
+```bash
+NODE_OPTIONS='--import tsx/esm --require tsx/cjs --require tsconfig-paths/register' npx elasticdash dashboard
+```
+
+Or add to `package.json`:
+
+```json
+{
+  "scripts": {
+    "dashboard:ai": "NODE_OPTIONS='--import tsx/esm --require tsx/cjs --require tsconfig-paths/register' elasticdash dashboard"
+  }
+}
+```
+
+**Why this works:**
+- `tsx/esm` and `tsx/cjs` handle mixed ESM/CJS module loading at runtime
+- `tsconfig-paths/register` resolves path mappings from `tsconfig.json` (e.g., `@/services/*` → `./src/services/*`)
+
+**How to check if you're using ESM or CJS:**
+
+Look at your `package.json`:
+- `"type": "module"` → ESM (uses `import`/`export`)
+- No `type` field or `"type": "commonjs"` → CJS (uses `require`/`module.exports`)
+
+**When you need the advanced setup:**
+
+- **Any file in your project uses path aliases** (e.g., `@/lib/*`, `@/services/*`) instead of relative imports (`../`) - **this is the main reason**
+- Your project mixes ESM (`import`) and CJS (`require`) in the same execution path
+- You see errors like `Cannot find module '@/...'` or `ERR_UNKNOWN_FILE_EXTENSION`
+
+**Important:** Path alias resolution is transitive. If your workflows import files that use `@/` aliases, you need the advanced setup even if `ed_workflows.ts` itself uses only relative imports.
+
+**When you DON'T need it:**
+- Plain JavaScript projects (`.js` files only)
+- **Basic TypeScript projects using only relative imports** (`./`, `../`) - the dashboard handles standard TS transpilation automatically
+- **Pure ESM or pure CJS projects without path aliases**
+- Pre-built projects where workflows are already compiled to `.js`
+
+## File Resolution
+
+The dashboard automatically handles TypeScript transpilation:
+
+| Scenario | File Used |
+|---|---|
+| Only `ed_workflows.ts` | Transpiled `.ts` on the fly |
+| Only `ed_workflows.js` | `.js` directly |
+| Both `.ts` and `.js` exist | `.ts` (preferred) |
+
+**No manual build step required** — write your workflows in TypeScript, and the dashboard handles the rest.
+
+## What the Dashboard Shows
+
+### Workflow Functions
+- **Function names** — all exported functions from `ed_workflows.ts`
+- **Signatures** — function parameters and return types
+- **Async indicator** — marks async vs sync functions
+- **Source module** — where the function is imported from (if re-exported)
+- **File path** — location in your codebase
+
+### Tool Functions
+- All exported functions from `ed_tools.ts`
+- Same display format as workflows
+- Used for agent task execution
+
+## Search and Filter
+
+Use the search field to filter by:
+- **Name** — find workflow by function name (e.g., `checkoutFlow`)
+- **Source module** — find all workflows from a specific module (e.g., `app.workflows`)
+- **File path** — filter by location in your codebase
+
+## Running Workflows
+
+The dashboard provides an interactive workflow debugger:
+
+### Step 1: Select Workflow
+Choose a workflow function to run
+
+### Step 2: Import Failed Trace
+Upload a JSON trace file from a failed workflow run
+
+### Step 3: Mark Broken Step
+Select which observations (AI calls, tool calls) need fixing
+
+### Step 4: Validate Fixes
+Edit inputs, re-run individual observations, and compare outputs
+
+### Step 5: Validate with Live Data
+Run the entire workflow with live API calls to verify your fixes work end-to-end
+
+## Fetching Traces from Langfuse
+
+To import a trace into the dashboard, you need to fetch observation data from Langfuse's API using the correct trace ID.
+
+### API Endpoint
+
+```
+https://cloud.langfuse.com/api/public/v2/observations?traceId=<TRACE_ID>&fields=time,io,basic,model
+```
+
+### Required Parameters
+
+| Parameter | Description |
+|---|---|
+| `traceId` | The unique identifier for the trace (required) |
+| `fields` | Comma-separated list of fields to include: `time,io,basic,model` (required) |
+
+### Example Request
+
+```bash
+curl "https://cloud.langfuse.com/api/public/v2/observations?traceId=a1b2af2bc594eb816ab55a1d0c94fd47&fields=time,io,basic,model" \
+  -H "Authorization: Basic <BASE64_ENCODED_CREDENTIALS>"
+```
+
+### Authentication
+
+Use HTTP Basic Auth with a base64-encoded credential string:
+
+1. **Create the credentials string:**
+   ```
+   <LANGFUSE_PUBLIC_KEY>:<LANGFUSE_SECRET_KEY>
+   ```
+
+2. **Encode to base64:**
+   ```bash
+   # On macOS/Linux:
+   echo -n "<LANGFUSE_PUBLIC_KEY>:<LANGFUSE_SECRET_KEY>" | base64
+   
+   # Result: abc123def456...  (the base64 string)
+   ```
+
+3. **Use in the Authorization header:**
+   ```bash
+   -H "Authorization: Basic abc123def456..."
+   ```
+
+Bearer token auth is not accepted for this API usage.
+
+### Using Postman
+
+To fetch traces using Postman:
+
+1. **Create a new GET request**
+   - Set request type to `GET`
+   - URL: `https://cloud.langfuse.com/api/public/v2/observations`
+
+2. **Add query parameters:**
+   - Click the "Params" tab
+   - Add `traceId` parameter: `a1b2af2bc594eb816ab55a1d0c94fd47`
+   - Add `fields` parameter: `time,io,basic,model`
+
+3. **Add Authorization header:**
+   - Click the "Auth" tab
+   - Select `Basic Auth` from the Type dropdown
+   - Username: `<LANGFUSE_PUBLIC_KEY>`
+   - Password: `<LANGFUSE_SECRET_KEY>`
+   - Postman will automatically base64-encode your credentials
+
+4. **Send the request**
+   - Click "Send"
+   - The response will show the array of observations for your trace
+
+**Alternatively, manually add the header:**
+- Go to "Headers" tab
+- Add header: `Authorization`
+- Value: `Basic <BASE64_ENCODED_CREDENTIALS>`
+
+### Example Response
+
+The API returns an array of observation objects. Note that `input` and `output` fields are **stringified JSON** (not objects):
+
+```json
+[
+  {
+    "id": "1e31d1efb1aae8be",
+    "traceId": "a1b2af2bc594eb816ab55a1d0c94fd47",
+    "type": "GENERATION",
+    "name": "OpenAI.chat",
+    "startTime": "2026-03-01T10:02:06.132Z",
+    "endTime": "2026-03-01T10:02:07.212Z",
+    "model": "gpt-4o-2024-08-06",
+    "input": "{\"messages\":[{\"role\":\"user\",\"content\":\"What is the order status?\"}]}",
+    "output": "{\"role\":\"assistant\",\"content\":\"The order has been confirmed.\"}",
+    "modelParameters": {
+      "temperature": 0.7,
+      "maxTokens": 500
+    },
+    "latency": 1.08,
+    "level": "DEFAULT",
+    "statusMessage": "",
+    "parentObservationId": "40e8ae4d0f708886"
+  },
+  {
+    "id": "dfe582e2934f570d",
+    "traceId": "a1b2af2bc594eb816ab55a1d0c94fd47",
+    "type": "TOOL",
+    "name": "chargeCard",
+    "startTime": "2026-03-01T10:02:07.215Z",
+    "endTime": "2026-03-01T10:02:07.457Z",
+    "model": "",
+    "input": "{\"amount\":99.99,\"cardToken\":\"tok_visa1234\"}",
+    "output": "{\"success\":true,\"transactionId\":\"txn-123\"}",
+    "modelParameters": {},
+    "latency": 0.242,
+    "level": "DEFAULT",
+    "statusMessage": "",
+    "parentObservationId": "40e8ae4d0f708886"
+  }
+]
+```
+
+**Tip:** If you prefer not to manually encode, curl's `-u username:password` flag will automatically base64-encode your credentials, but the header format shown above is the official recommended approach.
+
+**Reference:** See the [Langfuse Observations API documentation](https://langfuse.com/docs/api-and-data-platform/features/observations-api) for complete API details.
+
+### Important Notes
+
+**User Responsibility:** You must ensure that Langfuse correctly records the following fields for each observation:
+
+- **`input`** — the input data for the observation (stringified JSON: messages for LLM calls, args for tool calls)
+- **`output`** — the output/result from the observation (stringified JSON)
+- **`model`** — the model name for AI calls (e.g., `gpt-4o-2024-08-06`, `gemini-pro`). Can be empty string for non-AI observations
+- **`type`** — the observation type (`GENERATION` for LLM calls, `TOOL` or `SPAN` for tool calls)
+
+**If these fields are missing or incorrect, the SDK dashboard may not function properly.** Verify your Langfuse instrumentation captures these fields before attempting to import traces.
+
+**Note:** The `input` and `output` fields in Langfuse's API response are **stringified JSON strings**, not objects. The dashboard will parse these automatically.
+
+### Uploading to Dashboard
+
+Once you've fetched the JSON response from Langfuse:
+
+1. Save the response to a `.json` file (e.g., `trace.json`)
+2. Open the dashboard: `npx elasticdash dashboard`
+3. Select a workflow to debug
+4. Click "Import Failed Trace" and upload your JSON file
+5. The dashboard will parse the observations and let you mark broken steps
+
+## Configuration Files
+
+Dashboard discovery uses `ed_workflows.ts` (workflow list) and `ed_tools.ts` (tool list). `ed_agents.ts` is used for agent-related dashboard flows. A project-specific `ed_workers.ts` file is optional and not consumed by dashboard discovery.
+
+### `ed_workflows.ts`
+
+Re-export workflow functions from your application:
+
+```ts
+// ed_workflows.ts
+export { orderWorkflow, refundWorkflow } from './src/workflows'
+export { userLookupFlow } from './src/user-flows'
+```
+
+**Important: Workflow Function Compatibility**
+
+Dashboard replay executes exported workflow functions directly in a subprocess. Exported functions in `ed_workflows.ts/js` should:
+
+- Be directly callable functions (not framework request handlers)
+- Accept JSON-serializable input (object or array)
+- Return JSON-serializable output (object or array)
+
+Not directly compatible as workflow exports:
+
+```ts
+// Next.js route handler style
+export async function POST(req: NextRequest): Promise<NextResponse> {
+  return NextResponse.json({ ok: true })
+}
+```
+
+If your app uses framework handlers, create a plain workflow function and call it from the handler.
+
+### `ed_tools.ts`
+
+Re-export tool functions that agents or workflows can invoke:
+
+```ts
+// ed_tools.ts
+import { runSelectQuery } from './src/services/dataService'
+
+export const dataService = async (input: any) => {
+  const { query } = input as { query: string }
+  return await runSelectQuery(query)
+    .then(async (res: any) => {
+      try {
+        const { recordToolCall } = await import('elasticdash-sdk')
+        recordToolCall('dataService', input, res)
+      } catch {
+        // tracing must never block the main service path
+      }
+      return res
+    })
+    .catch(async (err: any) => {
+      try {
+        const { recordToolCall } = await import('elasticdash-sdk')
+        recordToolCall('dataService', input, err)
+      } catch {
+        // tracing must never block the main service path
+      }
+      throw err
+    })
+}
+```
+
+**Important: Tool Function Compatibility**
+
+Dashboard tool reruns execute exported tool functions directly in a subprocess. Exported functions in `ed_tools.ts/js` should:
+
+- Be directly callable functions (not framework request handlers)
+- Accept JSON-serializable input (object or array)
+- Return JSON-serializable output (object, array, or primitive)
+
+Not directly compatible as tool exports:
+
+```ts
+// Next.js route handler style
+export async function POST(req: NextRequest): Promise<NextResponse> {
+  return NextResponse.json({ ok: true })
+}
+```
+
+If your app uses framework handlers, create a plain tool function and call it from the handler.
+
+#### Important: Tool Name and Input Matching
+
+**Function names in `ed_tools.ts` must exactly match the tool `name` field in your Langfuse traces.** The dashboard uses this matching for step-level testing and trace replay.
+
+Example:
+- If Langfuse records a tool observation with `"name": "chargeCard"`
+- Then your `ed_tools.ts` must export a function named `chargeCard`:
+  ```ts
+  export const chargeCard = async (input: any) => { ... }
+  ```
+
+**Function input must match exactly what was recorded in Langfuse:**
+- The `input` parameter structure must match the `input` field in the Langfuse trace
+- When using `recordToolCall`, pass the function's `input` parameter directly as the second argument:
+  ```ts
+  recordToolCall('chargeCard', input, result)  // ✅ Correct
+  recordToolCall('chargeCard', input.amount, result)  // ❌ Wrong - partial input
+  ```
+
+If names or inputs don't match, step-level test assertions and dashboard replay features will not work correctly.
+
+### `ed_agents.ts`
+
+Re-export agent functions:
+
+```ts
+// ed_agents.ts
+export { checkoutAgent, paymentAgent } from './src/agents'
+
+// Or as a config object:
+export const agents = {
+  checkout: checkoutAgent,
+  payment: paymentAgent,
+}
+```
+
+## Usage in Tests
+
+Access workflows from tests:
+
+```ts
+import { orderWorkflow } from './ed_workflows'
+
+aiTest('full order workflow', async (ctx) => {
+  const result = await orderWorkflow('order-123', 'cust-456')
+  expect(ctx.trace).toCallTool('chargeCard')
+})
+```

package/docs/deno.md

@@ -0,0 +1,69 @@
+# Using elasticdash-sdk in Deno
+
+This guide shows how to add, pin, and run `elasticdash-sdk` inside a Deno project for AI workflow tests.
+
+> **Setting up instrumentation files?** See the [Instrumentation Guide](instrumentation.md) for how to create `ed_tools.ts`, `ed_workflows.ts`, and `ed_agents.ts` in your project. The examples there are written from a Deno project perspective.
+
+## 1) Add the dependency via import map
+
+In your package `deno.json` (e.g., `packages/api/deno.json`), add an import mapping and a test task. Example pinned to 0.2.0:
+
+```json
+{
+  "tasks": {
+    "test": "deno run -A --env-file=../../.env npm:elasticdash-sdk@0.2.0 test tests"
+  },
+  "imports": {
+    "elasticdash-sdk": "npm:elasticdash-sdk@0.2.0"
+  }
+}
+```
+
+If registry integrity issues block 0.2.0, use the last known good build, e.g. `npm:elasticdash-sdk@0.1.6-alpha-2`, and mirror that in the task command.
+
+## 2) Cache (optional, for reproducibility/offline)
+
+From the package directory (e.g., `packages/api`):
+
+```bash
+deno cache --reload npm:elasticdash-sdk@0.2.0
+```
+
+If you hit a checksum mismatch, switch to the pinned fallback version (e.g., `0.1.6-alpha-2`) and cache that instead.
+
+## 3) Write a test
+
+Create a test file under `tests/` ending with `.ai.test.ts` or `.ai.test.js` and import the setup:
+
+```javascript
+import "npm:elasticdash-sdk/dist/test-setup.js"
+import { expect } from "npm:elasticdash-sdk"
+
+aiTest("example", async () => {
+  expect(1 + 1).toBe(2)
+})
+```
+
+For API router testing with optional live OpenAI calls, see the existing example at `packages/api/tests/router.ai.test.js`.
+
+## 4) Run tests
+
+From your package directory:
+
+- Stubbed/deterministic mode (default):
+  ```bash
+  deno task test
+  ```
+
+- Live mode (hits OpenAI):
+  ```bash
+  RUN_LIVE_OPENAI=1 deno task test
+  ```
+
+Ensure `OPENAI_API_KEY` is available (e.g., via `.env` loaded by `--env-file`).
+
+## 5) Troubleshooting
+
+- **Checksum mismatch when caching**: use the fallback version that matches your import map (e.g., `0.1.6-alpha-2`) and rerun `deno cache --reload npm:elasticdash-sdk@...`.
+- **Command not found**: ensure you run tasks from the package folder where `deno.json` defines `test`.
+- **Missing env vars**: confirm `.env` is loaded via `--env-file` or exported in your shell.