@pinecall/skills 0.1.0

package/skills/pinecall-guides/references/guides/sse-streaming.md

@@ -0,0 +1,207 @@
+---
+title: "SSE Event Streaming"
+description: "Stream agent events to your frontend in real time with Server-Sent Events."
+---
+
+# SSE Event Streaming
+
+The SDK can stream every agent event (calls, speech, tools, metrics) to any HTTP client as Server-Sent Events. Build live dashboards, call monitors, or analytics pipelines with a single endpoint.
+
+## The basics
+
+```typescript
+import express from "express";
+import { Pinecall } from "@pinecall/sdk";
+
+const app = express();
+const pc = new Pinecall({ apiKey: process.env.PINECALL_API_KEY! });
+
+// Stream all events from all agents
+app.get("/events", (req, res) => pc.stream(res));
+
+app.listen(3000);
+```
+
+Open `http://localhost:3000/events` in a browser or connect with `EventSource` — you'll see every event as it happens.
+
+## Two ways to stream
+
+### `pc.stream()` — all agents
+
+Streams events from every agent on this `Pinecall` instance:
+
+```typescript
+// Express / Node.js ServerResponse
+app.get("/events", (req, res) => pc.stream(res));
+
+// Hono / Web API Response (no res argument)
+app.get("/events", () => pc.stream());
+```
+
+### `agent.stream()` — one agent
+
+Streams events scoped to a single agent:
+
+```typescript
+const mara = pc.agent("mara", { /* ... */ });
+
+app.get("/events/mara", (req, res) => mara.stream(res));
+app.get("/events/mara", () => mara.stream());
+```
+
+### Filtering by agent name
+
+If you have multiple agents but only want events from specific ones:
+
+```typescript
+app.get("/events", (req, res) => pc.stream(res, { agents: ["mara", "support"] }));
+```
+
+## Event format
+
+Each SSE event has an `event:` type and a JSON `data:` body:
+
+```
+event: call.started
+data: {"callId":"CA_abc","from":"+34611111111","agent":"mara","direction":"inbound"}
+
+event: user.message
+data: {"callId":"CA_abc","text":"Hola, quiero reservar","messageId":"msg_1","agent":"mara"}
+
+event: bot.speaking
+data: {"callId":"CA_abc","text":"¡Claro! ¿Para cuándo?","messageId":"msg_2","agent":"mara"}
+
+event: call.ended
+data: {"callId":"CA_abc","reason":"hangup","duration":45,"agent":"mara"}
+```
+
+A `:ping` comment is sent every 30 seconds as a keepalive to prevent proxy timeouts.
+
+## Connecting from the browser
+
+Use the native `EventSource` API:
+
+```typescript
+const events = new EventSource("/events");
+
+events.addEventListener("call.started", (e) => {
+  const data = JSON.parse(e.data);
+  console.log(`New call from ${data.from} on agent ${data.agent}`);
+});
+
+events.addEventListener("user.message", (e) => {
+  const data = JSON.parse(e.data);
+  addToTranscript("user", data.text);
+});
+
+events.addEventListener("bot.speaking", (e) => {
+  const data = JSON.parse(e.data);
+  addToTranscript("bot", data.text);
+});
+
+events.addEventListener("call.ended", (e) => {
+  const data = JSON.parse(e.data);
+  console.log(`Call ended: ${data.reason} (${data.duration}s)`);
+});
+```
+
+`EventSource` auto-reconnects on disconnect — no manual retry logic needed.
+
+## Building a live call monitor
+
+A minimal React component that shows active calls:
+
+```tsx
+function CallMonitor() {
+  const [calls, setCalls] = useState(new Map());
+
+  useEffect(() => {
+    const es = new EventSource("/events");
+
+    es.addEventListener("call.started", (e) => {
+      const d = JSON.parse(e.data);
+      setCalls((prev) => new Map(prev).set(d.callId, {
+        from: d.from, agent: d.agent, started: Date.now(), transcript: [],
+      }));
+    });
+
+    es.addEventListener("user.message", (e) => {
+      const d = JSON.parse(e.data);
+      setCalls((prev) => {
+        const next = new Map(prev);
+        const call = next.get(d.callId);
+        if (call) call.transcript.push({ role: "user", text: d.text });
+        return next;
+      });
+    });
+
+    es.addEventListener("call.ended", (e) => {
+      const d = JSON.parse(e.data);
+      setCalls((prev) => { const next = new Map(prev); next.delete(d.callId); return next; });
+    });
+
+    return () => es.close();
+  }, []);
+
+  return (
+    <div>
+      <h2>Active Calls ({calls.size})</h2>
+      {[...calls.entries()].map(([id, call]) => (
+        <div key={id}>
+          <strong>{call.agent}</strong> — {call.from}
+          {call.transcript.map((t, i) => <p key={i}>{t.role}: {t.text}</p>)}
+        </div>
+      ))}
+    </div>
+  );
+}
+```
+
+## When SSE works (and when it doesn't)
+
+SSE streaming requires the agent and the HTTP endpoint to run in the **same process**. This is the key architectural constraint:
+
+| Topology | SSE works? | Why |
+|---|---|---|
+| Agent + web server in one process | ✅ | `pc.stream()` has direct access to events |
+| Agent in a separate process (Docker, serverless) | ❌ | Events never reach the web server process |
+
+If you run agents in separate containers (the "separated" topology), you'd need a shared event bus (Redis Pub/Sub, NATS) between the agent and the dashboard. See [Deployment Topologies](/concepts/deployment-topologies).
+
+## Framework examples
+
+### Express
+
+```typescript
+app.get("/events", (req, res) => pc.stream(res));
+```
+
+### Hono
+
+```typescript
+app.get("/events", () => pc.stream());
+```
+
+### Next.js API Route
+
+```typescript
+// app/api/events/route.ts
+export async function GET() {
+  return pc.stream();
+}
+```
+
+### Fastify
+
+```typescript
+fastify.get("/events", (req, reply) => {
+  pc.stream(reply.raw);
+});
+```
+
+## What's next
+
+- [WebSocket Streaming](/guides/ws-streaming) — bidirectional alternative with session scoping
+- [Events reference](/reference/events) — every event with payload shapes
+- [Deployment Topologies](/concepts/deployment-topologies) — when SSE is available
+- [Multi-tenant](/guides/multi-tenant) — scope streams per customer

package/skills/pinecall-guides/references/guides/testing-agents.md

@@ -0,0 +1,272 @@
+---
+title: "Testing Agents"
+description: "Automated QA for voice agents using YAML specs and LLM judges."
+---
+
+Test your agents with `pinecall test` — define conversation workflows in YAML, and a judge LLM evaluates your agent's behavior automatically.
+
+## Quick Start
+
+### 1. Create a spec file
+
+Create `agent/specs/greeting.spec.yaml` in your agent project:
+
+```yaml
+agent: florencia
+description: "Verify the agent greets correctly"
+
+workflow: |
+  1. Say "Hola"
+  2. Verify the agent responds warmly with a greeting
+  3. Verify it mentions the business name
+  4. PASS if greeting is correct, FAIL if not
+```
+
+### 2. Run the test
+
+```bash
+pinecall test agent/specs/
+```
+
+That's it. The judge LLM will converse with your agent, follow the workflow, and report pass/fail.
+
+## How It Works
+
+`pinecall test` uses a **judge LLM** (Claude Haiku by default) to test your agent:
+
+![AI agent testing loop](/assets/diagrams/test-flow.png)
+
+The judge has two tools:
+- **`test_passed(summary)`** — marks the workflow as passed
+- **`test_failed(reason)`** — marks the workflow as failed
+
+The judge's text messages are sent directly to your agent as user messages. It acts like a real customer following a script.
+
+## Voice Mode
+
+By default `pinecall test` talks to your agent over **text** (fast and cheap). To exercise the **real voice pipeline** — STT, turn detection, TTS, barge-in — run the spec as an actual voice call.
+
+In voice mode the judge is a **real Pinecall agent** (server-rendered voice) **bridged** to your agent: two agents, two WebSockets, one call. Each side runs the full STT → LLM → TTS → turn-detection pipeline, and neither side knows the other is a bot — to your agent it's a normal inbound call. The CLI records both voices to a WAV, opens a live browser player, and resolves the verdict when the judge calls `test_passed()` / `test_failed()`.
+
+### Run it
+
+```bash
+# Force voice mode with the flag…
+pinecall test agent/specs/greeting.spec.yaml --voice
+
+# …or declare `mode: voice` in the spec and run it normally
+pinecall test agent/specs/greeting.spec.yaml
+```
+
+**Requirements:** `PINECALL_API_KEY` + the judge LLM key (e.g. `ANTHROPIC_API_KEY`). No TTS/STT key needed — the judge's voice is rendered server-side.
+
+### Voice spec fields
+
+```yaml
+agent: pines
+description: "Pines greets and answers a question (voice)"
+mode: voice                          # run as a real voice call
+voice: elevenlabs/professional-male  # judge's TTS voice (default)
+stt: flux                            # judge's STT (default: deepgram-flux)
+language: en                         # optional language override
+greeting: "Hi!"                      # optional: judge opens the call;
+                                     # omit → your agent greets first, judge waits
+timeout: 45s                         # voice turns are slower than text
+
+workflow: |
+  1. Greet the agent and ask what it can help with.
+  2. Verify it responds coherently and offers to help.
+  3. Ask "What is Pinecall?" and verify the answer is on-topic.
+  4. PASS if the conversation was coherent and helpful.
+```
+
+### Voice CLI flags
+
+| Flag | Description |
+|------|-------------|
+| `--voice <p/v>` | Judge's TTS voice (e.g. `elevenlabs/professional-male`). Passing it also forces voice mode. |
+| `--stt <prov>` | Judge's STT (e.g. `flux`). Default: `deepgram-flux`. |
+| `--record <file>` | WAV output path. Default: `<spec>.wav` next to the spec. |
+| `--no-listen` | Don't auto-open the live browser player. |
+| `--lang <code>` | Language override (e.g. `es`). |
+
+The judge speaks one short sentence per turn (it behaves like a real caller), the mixed call is recorded to a WAV next to the spec, and the transcript shows both sides per utterance.
+
+## Writing Specs
+
+### Spec format
+
+Specs are YAML files ending in `.spec.yaml` or `.spec.yml`.
+
+```yaml
+# Required fields
+agent: florencia                     # Agent name (as registered with pc.agent())
+workflow: |                          # Natural language workflow for the judge
+  1. Do something
+  2. Verify something
+  3. PASS or FAIL
+
+# Optional fields
+description: "Human-readable title"  # Shown in test output
+timeout: 45s                         # Per-turn timeout (default: 30s)
+
+# Optional: override judge model
+judge:
+  provider: anthropic                # anthropic | openai | google | deepseek | ollama
+  model: claude-haiku-4-5            # Model name
+  maxTurns: 10                       # Max conversation turns (default: 20)
+```
+
+### Workflow tips
+
+The `workflow` field is natural language — the judge LLM interprets it. Write it like instructions for a QA tester:
+
+```yaml
+# ✅ Good — clear, actionable steps
+workflow: |
+  1. Ask what services are available
+  2. Verify the agent lists at least 3 services with prices
+  3. Ask to book one of them
+  4. Verify the agent calls the booking tool
+  5. PASS if booking flow works, FAIL if anything breaks
+
+# ❌ Bad — too vague
+workflow: |
+  Test if the agent works correctly
+```
+
+**Best practices:**
+- Number your steps for clarity
+- Be specific about what "correct" means (dates, tool calls, content)
+- Always end with explicit PASS/FAIL criteria
+- Write in the same language your agent speaks
+
+### Verifying tool calls
+
+The judge sees your agent's tool calls, so you can verify them:
+
+```yaml
+workflow: |
+  1. Ask to book a haircut for tomorrow
+  2. Verify the agent calls checkAvailability
+  3. Verify the date argument is tomorrow's date in YYYY-MM-DD format
+  4. PASS if the tool was called with the correct date
+```
+
+### Verifying behavior
+
+Test what your agent says (or doesn't say):
+
+```yaml
+workflow: |
+  1. Ask to book on a Sunday
+  2. Verify the agent says the business is CLOSED on Sundays
+  3. Verify the agent does NOT call checkAvailability
+  4. Verify the agent suggests an alternative day
+  5. PASS if all conditions met
+```
+
+## Judge Providers
+
+The judge is the LLM that evaluates your agent. Choose based on cost and reliability:
+
+| Provider | Model | Cost (per 1M tokens) | Notes |
+|----------|-------|---------------------|-------|
+| `anthropic` | `claude-haiku-4-5-20251001` | $0.80 in / $4.00 out | Default. Most reliable. |
+| `openai` | `gpt-4.1-nano` | $0.10 in / $0.40 out | **Recommended.** 10x cheaper than Haiku. |
+| `google` | `gemini-2.5-flash` | — | Fast, low cost. |
+| `deepseek` | `deepseek-v4-flash` | $0.14 in / $0.28 out | Cheapest cloud option. |
+| `ollama` | `gemma3:4b` | Free | Local. Requires Ollama running. |
+
+Set the judge in the spec file or override with CLI:
+
+```bash
+# Override all specs to use OpenAI
+pinecall test agent/specs/ --judge anthropic/claude-haiku-4-5
+```
+
+### Environment variables
+
+Each provider needs its API key:
+
+| Variable | Provider |
+|----------|----------|
+| `ANTHROPIC_API_KEY` | Anthropic (default) |
+| `OPENAI_API_KEY` | OpenAI |
+| `GOOGLE_API_KEY` | Google |
+| `DEEPSEEK_API_KEY` | DeepSeek |
+| `OLLAMA_HOST` | Ollama (default: `http://localhost:11434`) |
+
+## CLI Reference
+
+```bash
+# Run all specs in a directory
+pinecall test agent/specs/
+
+# Run a single spec
+pinecall test agent/specs/date-handling.spec.yaml
+
+# Override judge model
+pinecall test agent/specs/ --judge anthropic/claude-haiku-4-5
+
+# Override agent name
+pinecall test agent/specs/ --agent dev-berna-florencia
+
+# Filter specs by name
+pinecall test agent/specs/ --grep "date"
+
+# JSON output (for CI pipelines)
+pinecall test agent/specs/ --json
+
+# List specs without running
+pinecall test agent/specs/ --list
+
+# Verbose mode (full agent responses)
+pinecall test agent/specs/ --verbose
+```
+
+## CI Integration
+
+`pinecall test` exits with code `1` when any spec fails, and supports `--json` for machine-readable output:
+
+```bash
+# In your CI pipeline
+export PINECALL_API_KEY="pk_..."
+export OPENAI_API_KEY="sk-..."
+
+pinecall test agent/specs/ --judge anthropic/claude-haiku-4-5 --json
+```
+
+JSON output structure:
+
+```json
+{
+  "passed": 2,
+  "failed": 0,
+  "results": [
+    {
+      "file": "agent/specs/date-handling.spec.yaml",
+      "agent": "florencia",
+      "passed": true,
+      "summary": "All dates verified correctly",
+      "turns": [...],
+      "durationMs": 4300
+    }
+  ]
+}
+```
+
+## Project Structure
+
+Recommended layout for your agent project:
+
+```
+my-agent/
+├── agent/
+│   ├── index.js          # Agent code
+│   └── specs/            # Test specs
+│       ├── greeting.spec.yaml
+│       ├── booking.spec.yaml
+│       └── edge-cases.spec.yaml
+└── package.json
+```