memory-journal-mcp 7.7.0 → 8.0.0

package/skills/agents-sdk/references/mcp/references/testing-reference.md

@@ -0,0 +1,171 @@
+# Testing Strategy Reference
+
+4-layer testing model with structural invariants for MCP servers.
+
+> Read this reference when setting up tests or reviewing test coverage for an MCP server.
+
+---
+
+## 4-Layer Testing Model
+
+MCP servers require four complementary testing layers plus structural invariant tests.
+
+| Layer                      | Tool           | Scope                                                       | Transport     | Speed    |
+| -------------------------- | -------------- | ----------------------------------------------------------- | ------------- | -------- |
+| **1. Unit Tests**          | Vitest         | Handler logic, schemas, validation, error paths             | None (mocked) | Fast     |
+| **1b. Invariant Tests**    | Vitest         | Structural enforcement (annotations, output schemas)        | None          | Fast     |
+| **1c. Benchmarks**         | Vitest bench   | Handler dispatch, schema parsing, logger sanitization       | None          | Fast     |
+| **2. E2E Tests**           | Playwright     | Live server process, HTTP lifecycle, security headers       | HTTP (dual)   | Medium   |
+| **3. Integration Scripts** | Node.js `.mjs` | Protocol compliance, scheduler, transport-specific features | stdio + HTTP  | Variable |
+| **4. Agent-Driven Tests**  | MCP tool calls | Functional verification of all tools, resources, prompts    | stdio (live)  | Slow     |
+
+---
+
+## Layer 1: Unit Tests (Vitest)
+
+Core handler logic, input validation, error formatting, database adapters. Mock external dependencies. Target ≥90% line coverage.
+
+```
+tests/
+├── handlers/          # Tool handler logic
+├── database/          # Adapter + query tests
+├── server/            # Resource + prompt tests
+├── auth/              # OAuth module (8 files)
+├── security/          # Input sanitization
+├── filtering/         # Tool filter expressions
+├── codemode/          # Sandbox execution
+└── e2e/               # Playwright (Layer 2)
+```
+
+---
+
+## Layer 1b: Invariant Tests (Vitest)
+
+Structural enforcement tests that verify every tool meets minimum standards. Run as part of the unit test suite.
+
+| Test File                     | Enforces                                                                                               |
+| ----------------------------- | ------------------------------------------------------------------------------------------------------ |
+| `tool-annotations.test.ts`    | Every tool has `annotations` with explicit `readOnlyHint`, per-group checks, title validation          |
+| `tool-output-schemas.test.ts` | Every tool has `outputSchema`, error response acceptance, centralized schema wiring, no orphan schemas |
+
+---
+
+## Layer 2: E2E Tests (Playwright)
+
+Spawn real `dist/cli.js` process + MCP SDK client. Test the system as a black box.
+
+**Canonical spec inventory (~50 spec files, ~300+ tests):**
+
+| Category              | Spec Files                                                           | Coverage                                                                       |
+| --------------------- | -------------------------------------------------------------------- | ------------------------------------------------------------------------------ |
+| **Zod Sweeps**        | `zod-sweep.spec.ts`, `zod-sweep-native.spec.ts`                      | Every tool called with `{}` — must return structured error, never raw `-32602` |
+| **Numeric Coercion**  | `numeric-coercion.spec.ts`, `numeric-coercion-native.spec.ts`        | String-typed numeric param coercion                                            |
+| **Boundary**          | `boundary.spec.ts`                                                   | Empty tables, NULLs, idempotency, edge cases                                   |
+| **Aliases**           | `aliases.spec.ts`                                                    | Backward-compatible parameter aliases                                          |
+| **Payloads**          | `payloads-{group}.spec.ts`                                           | Per-group payload correctness (one spec per tool group)                        |
+| **Errors**            | `errors.spec.ts`, `errors-extended.spec.ts`, `errors-native.spec.ts` | Error path coverage                                                            |
+| **Code Mode**         | `codemode.spec.ts`, `codemode-groups.spec.ts`                        | Sandbox lifecycle + all groups                                                 |
+| **Help Resources**    | `help-resources.spec.ts`                                             | Root + per-group help resources listed, readable, non-empty                    |
+| **Resources/Prompts** | `resources.spec.ts`, `prompts.spec.ts`                               | MCP resource/prompt verification                                               |
+| **Auth**              | `auth.spec.ts`, `oauth-discovery.spec.ts`                            | OAuth 2.1 and RFC 9728 metadata                                                |
+| **Security**          | `security.spec.ts`                                                   | Security header assertions, HSTS                                               |
+| **Rate Limiting**     | `rate-limiting.spec.ts`                                              | 429 burst, Retry-After header, health exemption                                |
+| **Sessions**          | `sessions.spec.ts`, `session-advanced.spec.ts`                       | Session management                                                             |
+| **Transport**         | `streaming.spec.ts`, `streamable-http.spec.ts`, `protocols.spec.ts`  | Transport protocol tests                                                       |
+
+**Testing cadence per group:** invariant (vitest) → zod sweep (E2E) → payload correctness (E2E) → error paths (E2E).
+
+**Dual transport parity:** Run the same assertion suite against both stateful and stateless HTTP modes.
+
+---
+
+## Layer 3: Integration Scripts (Node.js `.mjs`)
+
+For features that require **separate server processes** — can't be tested via vitest mocks or MCP tool calls. Zero external dependencies (just Node.js `child_process` + `fetch`).
+
+**Standard scripts:**
+
+| Script                        | Purpose                                                                                    | Transport |
+| ----------------------------- | ------------------------------------------------------------------------------------------ | --------- |
+| `test-help-resources.mjs`     | Verify slim instructions + help resource filtering by group                                | stdio     |
+| `test-tool-annotations.mjs`   | Parse `tools/list` response, verify `openWorldHint` counts and 0 missing                   | stdio     |
+| `test-scheduler.mjs`          | Init HTTP session, read `memory://health`, wait for jobs, verify `runCount` + `lastResult` | HTTP      |
+| `test-instruction-levels.mjs` | Instruction level generation (`essential`, `standard`, `full`) — verify token budgets      | stdio     |
+
+**Script conventions:**
+
+- Node.js ESM (`.mjs`), no dependencies — run with `node test-server/script.mjs`
+- Exit code 0 = pass, 1 = fail
+- Handle SSE response parsing for HTTP transport (`text/event-stream` → JSON-RPC)
+- Support env vars for configuration (e.g., `MCP_URL`, `WAIT_SECONDS`)
+- Always `npm run build` before running
+
+**Dual transport testing:** The scheduler activates in both HTTP stateful and stateless modes. Integration scripts should verify behavior in both when applicable:
+
+| Mode           | CLI Flags                      | Sessions                 | Scheduler |
+| -------------- | ------------------------------ | ------------------------ | --------- |
+| stdio          | `--transport stdio` (default)  | N/A                      | ❌        |
+| HTTP stateful  | `--transport http`             | `mcp-session-id` header  | ✅        |
+| HTTP stateless | `--transport http --stateless` | No sessions (serverless) | ✅        |
+
+---
+
+## Layer 4: Agent-Driven Tests (MCP Tool Calls)
+
+Exhaustive functional testing of all tools, resources, and prompts via direct MCP calls from an AI agent. Organized in phased test plans.
+
+**Orchestration pattern (`test-server/` directory):**
+
+| File                       | Purpose                                                                                            |
+| -------------------------- | -------------------------------------------------------------------------------------------------- |
+| `README.md`                | Agent-optimized orchestration doc — file inventory, conventions, script reference, troubleshooting |
+| `code-map.md`              | Comprehensive architecture reference (directory layout, module responsibilities, cross-references) |
+| `tool-reference.md`        | Tool inventory taxonomy (all tools by group with scope, annotations, schema status)                |
+| `prompt-template.md`       | Extracted testing prompt boilerplate for consistent formatting across all test prompts             |
+| `test-tool-groups/*.md`    | ~20 self-contained prompts at sub-group granularity (e.g., `test-core.md`, `test-json-get.md`)     |
+| `test-codemode/*.md`       | ~12 prompts for Code Mode groups + sandbox security + WASM degradation                             |
+| `test-advanced/*.md`       | ~10 stress test prompts per tool group (Pattern P401: Agent-First Stress Testing)                  |
+| `test-resources.md`        | Dedicated resource verification (all data + help resources)                                        |
+| `test-prompts.md`          | Dedicated prompt verification                                                                      |
+| `test-agent-experience.md` | ~20 open-ended scenarios validating help resource sufficiency                                      |
+| `*.mjs`                    | Integration scripts (Layer 3)                                                                      |
+
+**Test count taxonomy:** Document tool counts using the formal taxonomy: Group tools (per-group subtotal) / Built-in tools (server_info, server_health, list_adapters) / Audit tools (when audit enabled) / Inventory tools (when applicable) / MCP total (sum visible to clients via `tools/list`).
+
+**Test plan conventions:**
+
+- Phases numbered 0-N, each with a table of `| Test | Command | Expected |`
+- Error-path tests marked with 🔴 prefix
+- Reporting: ❌ Fail, ⚠️ Issue, ✅ inline only (omit from summary)
+- Mermaid-producing resources must specify `text/plain` MIME in expected results
+- Script-based phases (help resources, scheduler) reference scripts, not inline commands
+
+**Standardized Prompt Format:** Use `prompt-template.md` boilerplate for all test prompts. This ensures consistent structure (Setup, Prerequisites, Test Matrix, Reporting Format) across 40+ test files. Generate or update prompts programmatically when adding new tool groups.
+
+**OAuth Tests:** 8 test files in `src/auth/tests/`. See [`oauth-reference.md`](../oauth-reference.md) §Testing Patterns for file list and mocking strategies.
+
+---
+
+## Code Quality & Supply Chain
+
+- No duplicated code, consistent error handling, full type coverage, <600 line modules
+- **SHA-pinned CI Actions:** Pin all GitHub Actions by SHA digest (not version tag) to prevent supply chain injection
+- **Version SSoT:** Create `src/version.ts` that reads `version` from `package.json` at runtime (via `createRequire`). **Both adapters** (if dual-backend) `import { VERSION }` — never hardcode version strings. On a bump, only `package.json` needs updating
+- **Lockfile integrity:** Verify lockfile SHA-256 checksum + `git diff --exit-code` before `npm ci` in CI pipelines. Prevents supply chain attacks via lockfile manipulation
+- **Dockerfile patch drift:** Weekly CI workflow to detect stale transitive dependency patches (e.g., `package.json` overrides) against upstream versions. Alert when patches diverge from upstream releases
+- Regular dependency scanning and artifact integrity
+
+**Compact JSON Serialization:** Use `JSON.stringify(result)` (no pretty-print) for tool responses — ~15-20% payload reduction on large results. Retain `JSON.stringify(result, null, 2)` only for error responses where human readability matters.
+
+**WASM/Native Backend Parity:** For servers with dual backends (e.g., WASM + Native), maintain a parity checklist:
+
+- Features registered but always failing in one backend (e.g., FTS5 on WASM) must return structured "Feature Not Available" errors, not raw crashes
+- Test each backend independently — Code Mode can mask registration-only failures
+- Document backend-specific capabilities in help resources
+
+**WASM Degradation Testing:** For dual-backend servers, add dedicated degradation test suites:
+
+- Verify WASM-unsupported features return structured `{ success: false, code: 'FEATURE_NOT_AVAILABLE' }` errors
+- Test Code Mode gracefully degrades (e.g., `codemode-wasm-degradation.md` prompt)
+- Confirm tool count differences between backends match expected parity (`test-tool-annotations.mjs` validates this)
+- Document expected tool counts per backend in `test-server/README.md`

package/skills/agents-sdk/references/mcp.md

@@ -0,0 +1,157 @@
+# MCP Server Integration
+
+Fetch `docs/mcp-client.md` and `docs/mcp-servers.md` from `https://github.com/cloudflare/agents/tree/main/docs` for complete documentation.
+
+Agents include a multi-server MCP client for connecting to external MCP servers.
+
+## Add an MCP Server
+
+```typescript
+import { Agent, callable } from 'agents'
+
+export class MyAgent extends Agent<Env, State> {
+  @callable()
+  async addServer(name: string, url: string) {
+    // Options-based API (recommended)
+    const result = await this.addMcpServer(name, url, {
+      callbackHost: 'https://my-worker.workers.dev',
+      transport: { headers: { Authorization: 'Bearer ...' } },
+    })
+
+    if (result.state === 'authenticating') {
+      // OAuth required - redirect user to result.authUrl
+      return { needsAuth: true, authUrl: result.authUrl }
+    }
+
+    return { ready: true, id: result.id }
+  }
+}
+```
+
+## Use MCP Tools
+
+```typescript
+async onChatMessage() {
+  // Get AI-compatible tools from all connected MCP servers
+  const mcpTools = this.mcp.getAITools();
+
+  const allTools = {
+    ...localTools,
+    ...mcpTools
+  };
+
+  const result = streamText({
+    model: openai("gpt-4o"),
+    messages: await convertToModelMessages(this.messages),
+    tools: allTools
+  });
+
+  return result.toUIMessageStreamResponse();
+}
+```
+
+## List MCP Resources
+
+```typescript
+// List all registered servers
+const servers = this.mcp.listServers()
+
+// List tools from all servers
+const tools = this.mcp.listTools()
+
+// List resources
+const resources = this.mcp.listResources()
+
+// List prompts
+const prompts = this.mcp.listPrompts()
+```
+
+## Remove Server
+
+```typescript
+await this.removeMcpServer(serverId)
+```
+
+## Building an MCP Server
+
+Use `McpAgent` from the SDK to create an MCP server.
+
+**Install dependencies:**
+
+```bash
+npm install @modelcontextprotocol/sdk zod
+```
+
+**Wrangler config:**
+
+```jsonc
+{
+  "durable_objects": {
+    "bindings": [{ "name": "MyMCP", "class_name": "MyMCP" }],
+  },
+  "migrations": [{ "tag": "v1", "new_sqlite_classes": ["MyMCP"] }],
+}
+```
+
+**Server implementation:**
+
+```typescript
+import { McpServer } from '@modelcontextprotocol/sdk/server/mcp.js'
+import { McpAgent } from 'agents/mcp'
+import { z } from 'zod'
+
+type State = { counter: number }
+
+export class MyMCP extends McpAgent<Env, State, {}> {
+  server = new McpServer({
+    name: 'MyMCPServer',
+    version: '1.0.0',
+  })
+
+  initialState = { counter: 0 }
+
+  async init() {
+    // Register a resource
+    this.server.resource('counter', 'mcp://resource/counter', (uri) => ({
+      contents: [{ text: String(this.state.counter), uri: uri.href }],
+    }))
+
+    // Register a tool
+    this.server.registerTool(
+      'increment',
+      {
+        description: 'Increment the counter',
+        inputSchema: { amount: z.number().default(1) },
+      },
+      async ({ amount }) => {
+        this.setState({ counter: this.state.counter + amount })
+        return {
+          content: [{ text: `Counter: ${this.state.counter}`, type: 'text' }],
+        }
+      }
+    )
+  }
+}
+```
+
+## Serve MCP Server
+
+```typescript
+export default {
+  fetch(request: Request, env: Env, ctx: ExecutionContext) {
+    const url = new URL(request.url)
+
+    // SSE transport (legacy)
+    if (url.pathname.startsWith('/sse')) {
+      return MyMCP.serveSSE('/sse', { binding: 'MyMCP' }).fetch(request, env, ctx)
+    }
+
+    // Streamable HTTP transport (recommended)
+    if (url.pathname.startsWith('/mcp')) {
+      return MyMCP.serve('/mcp', { binding: 'MyMCP' }).fetch(request, env, ctx)
+    }
+
+    return new Response('Not found', { status: 404 })
+  },
+}
+```

package/skills/agents-sdk/references/state-scheduling.md

@@ -0,0 +1,164 @@
+# State & Scheduling
+
+Fetch `docs/state.md` and `docs/scheduling.md` from `https://github.com/cloudflare/agents/tree/main/docs` for complete documentation.
+
+## State Management
+
+State persists to SQLite and broadcasts to connected clients automatically.
+
+### Define Typed State
+
+```typescript
+type State = {
+  count: number
+  items: string[]
+}
+
+export class MyAgent extends Agent<Env, State> {
+  initialState: State = { count: 0, items: [] }
+}
+```
+
+### Read and Update
+
+```typescript
+// Read (lazy-loaded from SQLite)
+const count = this.state.count
+
+// Write (sync, persists, broadcasts)
+this.setState({ count: this.state.count + 1 })
+```
+
+### Validation Hook
+
+`validateStateChange()` runs synchronously before state persists. Throw to reject the update.
+
+```typescript
+validateStateChange(nextState: State, source: Connection | "server") {
+  if (nextState.count < 0) {
+    throw new Error("Count cannot be negative");
+  }
+}
+```
+
+### Execution Order
+
+1. `validateStateChange(nextState, source)` - sync, gating
+2. State persisted to SQLite
+3. State broadcast to connected clients
+4. `onStateUpdate(nextState, source)` - async via `ctx.waitUntil`, non-gating
+
+### Client-Side Sync (React)
+
+```tsx
+import { useAgent } from 'agents/react'
+
+function App() {
+  const [state, setLocalState] = useState<State>({ count: 0 })
+
+  const agent = useAgent<State>({
+    agent: 'MyAgent',
+    name: 'instance-1',
+    onStateUpdate: (newState) => setLocalState(newState),
+  })
+
+  return (
+    <button onClick={() => agent.setState({ count: state.count + 1 })}>Count: {state.count}</button>
+  )
+}
+```
+
+## SQL API
+
+Direct SQLite access for custom queries:
+
+```typescript
+// Create table
+this.sql`
+  CREATE TABLE IF NOT EXISTS items (
+    id TEXT PRIMARY KEY,
+    name TEXT,
+    created_at INTEGER DEFAULT (unixepoch())
+  )
+`
+
+// Insert
+this.sql`INSERT INTO items (id, name) VALUES (${id}, ${name})`
+
+// Query with types
+const items = this.sql<{ id: string; name: string }>`
+  SELECT * FROM items WHERE name LIKE ${`%${search}%`}
+`
+```
+
+## Scheduling
+
+### Schedule Types
+
+| Mode     | Syntax                              | Use Case                   |
+| -------- | ----------------------------------- | -------------------------- |
+| Delay    | `this.schedule(60, ...)`            | Run in 60 seconds          |
+| Date     | `this.schedule(new Date(...), ...)` | Run at specific time       |
+| Cron     | `this.schedule("0 8 * * *", ...)`   | Recurring schedule         |
+| Interval | `this.scheduleEvery(30, ...)`       | Fixed interval (every 30s) |
+
+### Examples
+
+```typescript
+// Delay (seconds)
+await this.schedule(60, 'checkStatus', { id: 'abc123' })
+
+// Specific date
+await this.schedule(new Date('2025-12-25T00:00:00Z'), 'sendGreeting', { to: 'user' })
+
+// Cron (recurring)
+await this.schedule('0 9 * * 1-5', 'weekdayReport', {})
+
+// Fixed interval (every 30 seconds, overlap prevention built-in)
+await this.scheduleEvery(30, 'pollUpdates')
+await this.scheduleEvery(300, 'syncData', { source: 'api' })
+```
+
+### Handler
+
+```typescript
+async sendGreeting(payload: { to: string }, schedule: Schedule) {
+  console.log(`Sending greeting to ${payload.to}`);
+  // Cron schedules auto-reschedule; one-time schedules are deleted
+}
+```
+
+### Manage Schedules
+
+```typescript
+const schedules = this.getSchedules()
+const crons = this.getSchedules({ type: 'cron' })
+await this.cancelSchedule(schedule.id)
+```
+
+## Lifecycle Callbacks
+
+```typescript
+export class MyAgent extends Agent<Env, State> {
+  async onStart() {
+    // Agent started or woke from hibernation
+  }
+
+  onConnect(conn: Connection, ctx: ConnectionContext) {
+    // WebSocket connected
+  }
+
+  onMessage(conn: Connection, message: WSMessage) {
+    // WebSocket message (non-RPC)
+  }
+
+  onStateUpdate(state: State, source: Connection | 'server') {
+    // State changed (async, non-blocking)
+  }
+
+  onError(error: unknown) {
+    // Error handler
+    throw error // Re-throw to propagate
+  }
+}
+```

package/skills/agents-sdk/references/streaming-chat.md

@@ -0,0 +1,168 @@
+# Streaming Chat with AIChatAgent
+
+Fetch `docs/resumable-streaming.md` and `docs/client-sdk.md` from `https://github.com/cloudflare/agents/tree/main/docs` for complete documentation.
+
+`AIChatAgent` provides streaming chat with automatic message persistence and resumable streams.
+
+## Basic Chat Agent
+
+```typescript
+import { AIChatAgent } from '@cloudflare/ai-chat'
+import { streamText, convertToModelMessages } from 'ai'
+import { openai } from '@ai-sdk/openai'
+
+export class Chat extends AIChatAgent<Env> {
+  async onChatMessage(onFinish) {
+    const result = streamText({
+      model: openai('gpt-4o'),
+      messages: await convertToModelMessages(this.messages),
+      onFinish,
+    })
+    return result.toUIMessageStreamResponse()
+  }
+}
+```
+
+## With Custom System Prompt
+
+```typescript
+export class Chat extends AIChatAgent<Env> {
+  async onChatMessage(onFinish) {
+    const result = streamText({
+      model: openai('gpt-4o'),
+      system: 'You are a helpful assistant specializing in...',
+      messages: await convertToModelMessages(this.messages),
+      onFinish,
+    })
+    return result.toUIMessageStreamResponse()
+  }
+}
+```
+
+## With Tools
+
+```typescript
+import { tool } from 'ai'
+import { z } from 'zod'
+
+const tools = {
+  getWeather: tool({
+    description: 'Get weather for a location',
+    parameters: z.object({ location: z.string() }),
+    execute: async ({ location }) => `Weather in ${location}: 72°F, sunny`,
+  }),
+}
+
+export class Chat extends AIChatAgent<Env> {
+  async onChatMessage(onFinish) {
+    const result = streamText({
+      model: openai('gpt-4o'),
+      messages: await convertToModelMessages(this.messages),
+      tools,
+      onFinish,
+    })
+    return result.toUIMessageStreamResponse()
+  }
+}
+```
+
+## Custom UI Message Stream
+
+For more control, use `createUIMessageStream`:
+
+```typescript
+import { createUIMessageStream, createUIMessageStreamResponse } from 'ai'
+
+export class Chat extends AIChatAgent<Env> {
+  async onChatMessage(onFinish) {
+    const stream = createUIMessageStream({
+      execute: async ({ writer }) => {
+        const result = streamText({
+          model: openai('gpt-4o'),
+          messages: await convertToModelMessages(this.messages),
+          onFinish,
+        })
+        writer.merge(result.toUIMessageStream())
+      },
+    })
+    return createUIMessageStreamResponse({ stream })
+  }
+}
+```
+
+## Resumable Streaming
+
+Streams automatically resume if client disconnects and reconnects:
+
+1. Chunks buffered to SQLite during streaming
+2. On reconnect, buffered chunks sent immediately
+3. Live streaming continues from where it left off
+
+**Enabled by default.** To disable:
+
+```tsx
+const { messages } = useAgentChat({ agent, resume: false })
+```
+
+## React Client
+
+```tsx
+import { useAgent } from 'agents/react'
+import { useAgentChat } from '@cloudflare/ai-chat/react'
+
+function ChatUI() {
+  const agent = useAgent({
+    agent: 'Chat',
+    name: 'my-chat-session',
+  })
+
+  const { messages, input, handleInputChange, handleSubmit, status } = useAgentChat({ agent })
+
+  return (
+    <div>
+      {messages.map((m) => (
+        <div key={m.id}>
+          <strong>{m.role}:</strong> {m.content}
+        </div>
+      ))}
+
+      <form onSubmit={handleSubmit}>
+        <input value={input} onChange={handleInputChange} disabled={status === 'streaming'} />
+        <button type="submit">Send</button>
+      </form>
+    </div>
+  )
+}
+```
+
+## Streaming RPC Methods
+
+For non-chat streaming, use `@callable({ streaming: true })`:
+
+```typescript
+import { Agent, callable, StreamingResponse } from 'agents'
+
+export class MyAgent extends Agent<Env> {
+  @callable({ streaming: true })
+  async streamData(stream: StreamingResponse, query: string) {
+    for (let i = 0; i < 10; i++) {
+      stream.send(`Result ${i}: ${query}`)
+      await sleep(100)
+    }
+    stream.close()
+  }
+}
+```
+
+Client receives streamed messages via WebSocket RPC.
+
+## Status Values
+
+`useAgentChat` status:
+
+| Status      | Meaning               |
+| ----------- | --------------------- |
+| `ready`     | Idle, ready for input |
+| `streaming` | Response streaming    |
+| `submitted` | Request sent, waiting |
+| `error`     | Error occurred        |