@rune-kit/rune 2.10.0 → 2.12.0

package/extensions/ai-ml/PACK.md

@@ -1,88 +1,88 @@
----
-name: "@rune/ai-ml"
-description: AI/ML integration patterns — LLM integration, RAG pipelines, embeddings, fine-tuning workflows, stateful AI agents, code execution sandboxes, web extraction, and deep research loops.
-metadata:
-  author: runedev
-  version: "0.4.0"
-  layer: L4
-  price: "$15"
-  target: AI engineers
-  format: split
----
-
-# @rune/ai-ml
-
-## Purpose
-
-AI-powered features fail in predictable ways: LLM calls without retry logic that crash on rate limits, RAG pipelines that retrieve irrelevant chunks because the chunking strategy ignores document structure, embedding search that returns semantic matches with zero keyword overlap, fine-tuning runs that overfit because the eval set leaked into training data, AI agents that leak state across requests or lose progress on crashes, and code interpreters that execute untrusted LLM output without isolation. This pack codifies production patterns for each — from API client resilience to retrieval quality to model evaluation to agent state management to secure sandboxed execution — so AI features ship with the reliability of traditional software.
-
-## Triggers
-
-- Auto-trigger: when `openai`, `anthropic`, `@langchain`, `pinecone`, `pgvector`, `embedding`, `llm` detected in dependencies or code
-- `/rune llm-integration` — audit or improve LLM API usage
-- `/rune rag-patterns` — build or audit RAG pipeline
-- `/rune embedding-search` — implement or optimize semantic search
-- `/rune fine-tuning-guide` — prepare and execute fine-tuning workflow
-- `/rune ai-agents` — design and build stateful AI agents
-- `/rune code-sandbox` — set up secure code execution for AI
-- `/rune web-extraction` — build structured data extraction from web pages
-- `/rune deep-research` — implement iterative AI research loops with convergence
-- Called by `cook` (L1) when AI/ML task detected
-- Called by `plan` (L2) when AI architecture decisions needed
-
-## Skills Included
-
-| Skill | Model | Description |
-|-------|-------|-------------|
-| [llm-integration](skills/llm-integration.md) | sonnet | API client wrappers, streaming, structured output, retry + fallback chain, prompt versioning |
-| [rag-patterns](skills/rag-patterns.md) | sonnet | Document chunking, embedding generation, vector store setup, retrieval, reranking |
-| [embedding-search](skills/embedding-search.md) | sonnet | Semantic search, hybrid BM25 + vector, similarity thresholds, index optimization |
-| [fine-tuning-guide](skills/fine-tuning-guide.md) | sonnet | Dataset preparation, training config, evaluation metrics, deployment, A/B testing |
-| [llm-architect](skills/llm-architect.md) | opus | Model selection, prompt engineering, evaluation frameworks, cost optimization, guardrails |
-| [prompt-patterns](skills/prompt-patterns.md) | sonnet | Structured output, chain-of-thought, self-critique, ReAct, multi-turn memory management |
-| [ai-agents](skills/ai-agents.md) | sonnet | Stateful agents, RPC methods, scheduling, multi-agent coordination, MCP integration, HITL |
-| [code-sandbox](skills/code-sandbox.md) | sonnet | Container isolation, resource limits, timeout enforcement, stateful sessions, output capture |
-| [web-extraction](skills/web-extraction.md) | sonnet | Schema-driven extraction, anti-bot handling, prompt injection defense, multi-entity dedup |
-| [deep-research](skills/deep-research.md) | sonnet | Iterative research loop with convergence, source attribution, confidence scoring |
-
-## Connections
-
-```
-Calls → research (L3): lookup model documentation and best practices
-Calls → docs-seeker (L3): API reference for LLM providers
-Calls → verification (L3): validate pipeline correctness
-Calls → @rune/devops (L4): ai-agents → edge-serverless for agent deployment (Workers, Lambda)
-Calls → @rune/backend (L4): ai-agents → API patterns for agent endpoints and WebSocket handlers
-Calls → sentinel (L2): code-sandbox security audit on container isolation
-Called By ← cook (L1): when AI/ML task detected
-Called By ← plan (L2): when AI architecture decisions needed
-Called By ← review (L2): when AI code under review
-Called By ← mcp-builder (L2): ai-agents feeds MCP server patterns for agent-based MCP
-ai-agents → code-sandbox: agents use sandboxes for executing LLM-generated code safely
-code-sandbox → ai-agents: sandbox results feed back into agent state and conversation
-web-extraction → rag-patterns: extracted structured data feeds into RAG ingestion pipeline
-deep-research → web-extraction: research loop uses extraction for each discovered URL
-deep-research → embedding-search: relevance scoring uses embeddings for semantic similarity
-```
-
-## Sharp Edges
-
-- **Rate limits**: MUST implement exponential backoff retry on all LLM API calls — guaranteed at scale.
-- **Schema validation**: MUST validate LLM output with Zod/Pydantic — never trust raw text parsing.
-- **Eval leakage**: MUST separate training and evaluation datasets — leakage invalidates all metrics.
-- **Similarity thresholds**: MUST set thresholds on vector search — unrestricted results degrade quality.
-- **PII in embeddings**: MUST NOT embed sensitive data without consent — not easily deletable from vector stores.
-- **Embedding model pinning**: Pin model version in index metadata — dimension mismatch on upgrade is CRITICAL.
-- **Prompt injection**: Web pages may contain adversarial content targeting extraction LLMs — system prompt must block.
-- **Sandbox escape**: Use rootless Docker or gVisor for high-security code execution environments.
-
-## Done When
-
-- LLM API client implemented with retry logic, exponential backoff, and structured output validation via Zod/Pydantic
-- RAG pipeline operational: chunking, embedding, vector store, retrieval, and reranking all configured and tested
-- Embedding index metadata includes pinned model version and dimension count to prevent upgrade mismatches
-- AI agent state persists across requests with no cross-session leakage and graceful crash recovery
-
-## Cost Profile
-
-~24,000–40,000 tokens per full pack run (all 10 skills). Individual skill: ~2,500–5,000 tokens. Sonnet default. Use haiku for code detection scans; escalate to sonnet for pipeline design, extraction strategy, and research loop orchestration.
+---
+name: "@rune/ai-ml"
+description: AI/ML integration patterns — LLM integration, RAG pipelines, embeddings, fine-tuning workflows, stateful AI agents, code execution sandboxes, web extraction, and deep research loops.
+metadata:
+  author: runedev
+  version: "0.4.0"
+  layer: L4
+  price: "$15"
+  target: AI engineers
+  format: split
+---
+
+# @rune/ai-ml
+
+## Purpose
+
+AI-powered features fail in predictable ways: LLM calls without retry logic that crash on rate limits, RAG pipelines that retrieve irrelevant chunks because the chunking strategy ignores document structure, embedding search that returns semantic matches with zero keyword overlap, fine-tuning runs that overfit because the eval set leaked into training data, AI agents that leak state across requests or lose progress on crashes, and code interpreters that execute untrusted LLM output without isolation. This pack codifies production patterns for each — from API client resilience to retrieval quality to model evaluation to agent state management to secure sandboxed execution — so AI features ship with the reliability of traditional software.
+
+## Triggers
+
+- Auto-trigger: when `openai`, `anthropic`, `@langchain`, `pinecone`, `pgvector`, `embedding`, `llm` detected in dependencies or code
+- `/rune llm-integration` — audit or improve LLM API usage
+- `/rune rag-patterns` — build or audit RAG pipeline
+- `/rune embedding-search` — implement or optimize semantic search
+- `/rune fine-tuning-guide` — prepare and execute fine-tuning workflow
+- `/rune ai-agents` — design and build stateful AI agents
+- `/rune code-sandbox` — set up secure code execution for AI
+- `/rune web-extraction` — build structured data extraction from web pages
+- `/rune deep-research` — implement iterative AI research loops with convergence
+- Called by `cook` (L1) when AI/ML task detected
+- Called by `plan` (L2) when AI architecture decisions needed
+
+## Skills Included
+
+| Skill | Model | Description |
+|-------|-------|-------------|
+| [llm-integration](skills/llm-integration.md) | sonnet | API client wrappers, streaming, structured output, retry + fallback chain, prompt versioning |
+| [rag-patterns](skills/rag-patterns.md) | sonnet | Document chunking, embedding generation, vector store setup, retrieval, reranking |
+| [embedding-search](skills/embedding-search.md) | sonnet | Semantic search, hybrid BM25 + vector, similarity thresholds, index optimization |
+| [fine-tuning-guide](skills/fine-tuning-guide.md) | sonnet | Dataset preparation, training config, evaluation metrics, deployment, A/B testing |
+| [llm-architect](skills/llm-architect.md) | opus | Model selection, prompt engineering, evaluation frameworks, cost optimization, guardrails |
+| [prompt-patterns](skills/prompt-patterns.md) | sonnet | Structured output, chain-of-thought, self-critique, ReAct, multi-turn memory management |
+| [ai-agents](skills/ai-agents.md) | sonnet | Stateful agents, RPC methods, scheduling, multi-agent coordination, MCP integration, HITL |
+| [code-sandbox](skills/code-sandbox.md) | sonnet | Container isolation, resource limits, timeout enforcement, stateful sessions, output capture |
+| [web-extraction](skills/web-extraction.md) | sonnet | Schema-driven extraction, anti-bot handling, prompt injection defense, multi-entity dedup |
+| [deep-research](skills/deep-research.md) | sonnet | Iterative research loop with convergence, source attribution, confidence scoring |
+
+## Connections
+
+```
+Calls → research (L3): lookup model documentation and best practices
+Calls → docs-seeker (L3): API reference for LLM providers
+Calls → verification (L3): validate pipeline correctness
+Calls → @rune/devops (L4): ai-agents → edge-serverless for agent deployment (Workers, Lambda)
+Calls → @rune/backend (L4): ai-agents → API patterns for agent endpoints and WebSocket handlers
+Calls → sentinel (L2): code-sandbox security audit on container isolation
+Called By ← cook (L1): when AI/ML task detected
+Called By ← plan (L2): when AI architecture decisions needed
+Called By ← review (L2): when AI code under review
+Called By ← mcp-builder (L2): ai-agents feeds MCP server patterns for agent-based MCP
+ai-agents → code-sandbox: agents use sandboxes for executing LLM-generated code safely
+code-sandbox → ai-agents: sandbox results feed back into agent state and conversation
+web-extraction → rag-patterns: extracted structured data feeds into RAG ingestion pipeline
+deep-research → web-extraction: research loop uses extraction for each discovered URL
+deep-research → embedding-search: relevance scoring uses embeddings for semantic similarity
+```
+
+## Sharp Edges
+
+- **Rate limits**: MUST implement exponential backoff retry on all LLM API calls — guaranteed at scale.
+- **Schema validation**: MUST validate LLM output with Zod/Pydantic — never trust raw text parsing.
+- **Eval leakage**: MUST separate training and evaluation datasets — leakage invalidates all metrics.
+- **Similarity thresholds**: MUST set thresholds on vector search — unrestricted results degrade quality.
+- **PII in embeddings**: MUST NOT embed sensitive data without consent — not easily deletable from vector stores.
+- **Embedding model pinning**: Pin model version in index metadata — dimension mismatch on upgrade is CRITICAL.
+- **Prompt injection**: Web pages may contain adversarial content targeting extraction LLMs — system prompt must block.
+- **Sandbox escape**: Use rootless Docker or gVisor for high-security code execution environments.
+
+## Done When
+
+- LLM API client implemented with retry logic, exponential backoff, and structured output validation via Zod/Pydantic
+- RAG pipeline operational: chunking, embedding, vector store, retrieval, and reranking all configured and tested
+- Embedding index metadata includes pinned model version and dimension count to prevent upgrade mismatches
+- AI agent state persists across requests with no cross-session leakage and graceful crash recovery
+
+## Cost Profile
+
+~24,000–40,000 tokens per full pack run (all 10 skills). Individual skill: ~2,500–5,000 tokens. Sonnet default. Use haiku for code detection scans; escalate to sonnet for pipeline design, extraction strategy, and research loop orchestration.

package/extensions/ai-ml/skills/ai-agents.md

@@ -1,172 +1,172 @@
----
-name: "ai-agents"
-pack: "@rune/ai-ml"
-description: "Stateful AI agent architecture — persistent state, callable RPC methods, scheduling, multi-agent coordination, MCP server integration, and real-time client communication via WebSocket."
-model: sonnet
-tools: [Read, Edit, Write, Grep, Glob, Bash]
----
-
-# ai-agents
-
-Stateful AI agent architecture — persistent state, callable RPC methods, scheduling, multi-agent coordination, MCP server integration, and real-time client communication via WebSocket. Covers agent lifecycle, state management patterns, tool registration, human-in-the-loop approval flows, and durable workflow orchestration for long-running agent tasks.
-
-#### Workflow
-
-**Step 1 — Classify agent type**
-Identify what the agent needs to do and map to an architecture:
-
-| Agent Type | Key Characteristics | Platform Options |
-|---|---|---|
-| Stateless tool-caller | Single request → tool calls → response. No memory between requests. | Any LLM API + function calling |
-| Conversational with memory | Multi-turn dialogue. Needs chat history persistence. | Session store (Redis, KV) + LLM |
-| Stateful autonomous | Persistent state, scheduled tasks, reacts to events. Long-lived. | Cloudflare Agents SDK, LangGraph, CrewAI |
-| Multi-agent coordinator | Multiple specialized agents collaborating on a task. | LangGraph, AutoGen, custom orchestrator |
-| MCP server | Exposes tools/resources to any MCP-compatible client. | Cloudflare McpAgent, custom MCP server |
-
-**Step 2 — Design state management**
-For stateful agents, define the state contract:
-
-```typescript
-// State must be serializable (JSON-safe) — no functions, no circular refs
-interface AgentState {
-  // Domain state
-  conversations: ConversationEntry[];
-  preferences: Record<string, string>;
-  taskQueue: ScheduledTask[];
-
-  // Metadata
-  createdAt: string;
-  lastActiveAt: string;
-  version: number;
-}
-
-// State validation — reject invalid transitions
-function validateStateChange(current: AgentState, next: AgentState): void {
-  if (next.version < current.version) {
-    throw new Error('State version cannot decrease — concurrent modification detected');
-  }
-  if (next.conversations.length > 10_000) {
-    throw new Error('Conversation limit exceeded — archive old entries first');
-  }
-}
-```
-
-**Step 3 — Implement tool registration**
-Define agent capabilities as typed, callable methods:
-
-```typescript
-// Tools as typed RPC methods (Cloudflare Agents SDK pattern)
-import { Agent, callable } from 'agents';
-
-export class ResearchAgent extends Agent<Env, ResearchState> {
-  initialState: ResearchState = { findings: [], status: 'idle' };
-
-  @callable()
-  async search(query: string): Promise<SearchResult[]> {
-    this.setState({ ...this.state, status: 'searching' });
-    const results = await this.env.AI.run('@cf/meta/llama-3-8b-instruct', {
-      prompt: `Search for: ${query}`,
-    });
-    const findings = parseResults(results);
-    this.setState({
-      ...this.state,
-      findings: [...this.state.findings, ...findings],
-      status: 'idle',
-    });
-    return findings;
-  }
-
-  @callable()
-  async summarize(): Promise<string> {
-    if (this.state.findings.length === 0) {
-      throw new Error('No findings to summarize — run search first');
-    }
-    return generateSummary(this.state.findings);
-  }
-}
-```
-
-**Step 4 — Add scheduling and durability**
-For agents that need to perform work on a schedule or survive restarts:
-
-```typescript
-// Scheduled tasks — one-time, recurring, and cron
-@callable()
-async scheduleDigest(userId: string) {
-  // Daily digest at 9 AM
-  await this.schedule('0 9 * * *', 'sendDigest', { userId });
-
-  // One-time reminder in 1 hour
-  await this.schedule(3600, 'sendReminder', { userId, message: 'Check results' });
-
-  // Recurring every 30 minutes
-  await this.scheduleEvery(1800, 'pollDataSource');
-}
-
-// Handler runs when scheduled time arrives — even if agent was hibernated
-async onScheduledTask(task: ScheduledTask) {
-  switch (task.type) {
-    case 'sendDigest':
-      await this.compileAndSendDigest(task.payload.userId);
-      break;
-    case 'pollDataSource':
-      const newData = await fetchLatest();
-      if (newData.length > 0) {
-        this.setState({ ...this.state, lastPoll: Date.now(), data: newData });
-      }
-      break;
-  }
-}
-```
-
-**Step 5 — Human-in-the-loop patterns**
-For agents that need approval before taking high-impact actions:
-
-```typescript
-// Approval flow — agent pauses, human approves, agent resumes
-interface PendingApproval {
-  id: string;
-  action: string;
-  params: Record<string, unknown>;
-  requestedAt: string;
-  status: 'pending' | 'approved' | 'rejected';
-}
-
-@callable()
-async requestApproval(action: string, params: Record<string, unknown>): Promise<string> {
-  const approval: PendingApproval = {
-    id: crypto.randomUUID(),
-    action,
-    params,
-    requestedAt: new Date().toISOString(),
-    status: 'pending',
-  };
-  this.setState({
-    ...this.state,
-    pendingApprovals: [...this.state.pendingApprovals, approval],
-  });
-  // Client receives state update via WebSocket → shows approval UI
-  return approval.id;
-}
-
-@callable()
-async resolveApproval(id: string, decision: 'approved' | 'rejected') {
-  const updated = this.state.pendingApprovals.map(a =>
-    a.id === id ? { ...a, status: decision } : a
-  );
-  this.setState({ ...this.state, pendingApprovals: updated });
-  if (decision === 'approved') {
-    const approval = updated.find(a => a.id === id)!;
-    await this.executeAction(approval.action, approval.params);
-  }
-}
-```
-
-#### Sharp Edges
-
-| Failure Mode | Mitigation |
-|---|---|
-| State grows unbounded (conversation history, logs) | Implement max size limits with archival; prune old entries on state update |
-| Concurrent state mutations from multiple clients | Use version counter in state; reject updates with stale version |
-| Agent crashes mid-workflow, loses progress | Use durable workflows (Cloudflare Workflows, Temporal) for multi-step tasks — each step is persisted |
-| Scheduled tasks pile up during agent hibernation | Deduplicate on wake-up; use idempotency keys for task handlers |
+---
+name: "ai-agents"
+pack: "@rune/ai-ml"
+description: "Stateful AI agent architecture — persistent state, callable RPC methods, scheduling, multi-agent coordination, MCP server integration, and real-time client communication via WebSocket."
+model: sonnet
+tools: [Read, Edit, Write, Grep, Glob, Bash]
+---
+
+# ai-agents
+
+Stateful AI agent architecture — persistent state, callable RPC methods, scheduling, multi-agent coordination, MCP server integration, and real-time client communication via WebSocket. Covers agent lifecycle, state management patterns, tool registration, human-in-the-loop approval flows, and durable workflow orchestration for long-running agent tasks.
+
+#### Workflow
+
+**Step 1 — Classify agent type**
+Identify what the agent needs to do and map to an architecture:
+
+| Agent Type | Key Characteristics | Platform Options |
+|---|---|---|
+| Stateless tool-caller | Single request → tool calls → response. No memory between requests. | Any LLM API + function calling |
+| Conversational with memory | Multi-turn dialogue. Needs chat history persistence. | Session store (Redis, KV) + LLM |
+| Stateful autonomous | Persistent state, scheduled tasks, reacts to events. Long-lived. | Cloudflare Agents SDK, LangGraph, CrewAI |
+| Multi-agent coordinator | Multiple specialized agents collaborating on a task. | LangGraph, AutoGen, custom orchestrator |
+| MCP server | Exposes tools/resources to any MCP-compatible client. | Cloudflare McpAgent, custom MCP server |
+
+**Step 2 — Design state management**
+For stateful agents, define the state contract:
+
+```typescript
+// State must be serializable (JSON-safe) — no functions, no circular refs
+interface AgentState {
+  // Domain state
+  conversations: ConversationEntry[];
+  preferences: Record<string, string>;
+  taskQueue: ScheduledTask[];
+
+  // Metadata
+  createdAt: string;
+  lastActiveAt: string;
+  version: number;
+}
+
+// State validation — reject invalid transitions
+function validateStateChange(current: AgentState, next: AgentState): void {
+  if (next.version < current.version) {
+    throw new Error('State version cannot decrease — concurrent modification detected');
+  }
+  if (next.conversations.length > 10_000) {
+    throw new Error('Conversation limit exceeded — archive old entries first');
+  }
+}
+```
+
+**Step 3 — Implement tool registration**
+Define agent capabilities as typed, callable methods:
+
+```typescript
+// Tools as typed RPC methods (Cloudflare Agents SDK pattern)
+import { Agent, callable } from 'agents';
+
+export class ResearchAgent extends Agent<Env, ResearchState> {
+  initialState: ResearchState = { findings: [], status: 'idle' };
+
+  @callable()
+  async search(query: string): Promise<SearchResult[]> {
+    this.setState({ ...this.state, status: 'searching' });
+    const results = await this.env.AI.run('@cf/meta/llama-3-8b-instruct', {
+      prompt: `Search for: ${query}`,
+    });
+    const findings = parseResults(results);
+    this.setState({
+      ...this.state,
+      findings: [...this.state.findings, ...findings],
+      status: 'idle',
+    });
+    return findings;
+  }
+
+  @callable()
+  async summarize(): Promise<string> {
+    if (this.state.findings.length === 0) {
+      throw new Error('No findings to summarize — run search first');
+    }
+    return generateSummary(this.state.findings);
+  }
+}
+```
+
+**Step 4 — Add scheduling and durability**
+For agents that need to perform work on a schedule or survive restarts:
+
+```typescript
+// Scheduled tasks — one-time, recurring, and cron
+@callable()
+async scheduleDigest(userId: string) {
+  // Daily digest at 9 AM
+  await this.schedule('0 9 * * *', 'sendDigest', { userId });
+
+  // One-time reminder in 1 hour
+  await this.schedule(3600, 'sendReminder', { userId, message: 'Check results' });
+
+  // Recurring every 30 minutes
+  await this.scheduleEvery(1800, 'pollDataSource');
+}
+
+// Handler runs when scheduled time arrives — even if agent was hibernated
+async onScheduledTask(task: ScheduledTask) {
+  switch (task.type) {
+    case 'sendDigest':
+      await this.compileAndSendDigest(task.payload.userId);
+      break;
+    case 'pollDataSource':
+      const newData = await fetchLatest();
+      if (newData.length > 0) {
+        this.setState({ ...this.state, lastPoll: Date.now(), data: newData });
+      }
+      break;
+  }
+}
+```
+
+**Step 5 — Human-in-the-loop patterns**
+For agents that need approval before taking high-impact actions:
+
+```typescript
+// Approval flow — agent pauses, human approves, agent resumes
+interface PendingApproval {
+  id: string;
+  action: string;
+  params: Record<string, unknown>;
+  requestedAt: string;
+  status: 'pending' | 'approved' | 'rejected';
+}
+
+@callable()
+async requestApproval(action: string, params: Record<string, unknown>): Promise<string> {
+  const approval: PendingApproval = {
+    id: crypto.randomUUID(),
+    action,
+    params,
+    requestedAt: new Date().toISOString(),
+    status: 'pending',
+  };
+  this.setState({
+    ...this.state,
+    pendingApprovals: [...this.state.pendingApprovals, approval],
+  });
+  // Client receives state update via WebSocket → shows approval UI
+  return approval.id;
+}
+
+@callable()
+async resolveApproval(id: string, decision: 'approved' | 'rejected') {
+  const updated = this.state.pendingApprovals.map(a =>
+    a.id === id ? { ...a, status: decision } : a
+  );
+  this.setState({ ...this.state, pendingApprovals: updated });
+  if (decision === 'approved') {
+    const approval = updated.find(a => a.id === id)!;
+    await this.executeAction(approval.action, approval.params);
+  }
+}
+```
+
+#### Sharp Edges
+
+| Failure Mode | Mitigation |
+|---|---|
+| State grows unbounded (conversation history, logs) | Implement max size limits with archival; prune old entries on state update |
+| Concurrent state mutations from multiple clients | Use version counter in state; reject updates with stale version |
+| Agent crashes mid-workflow, loses progress | Use durable workflows (Cloudflare Workflows, Temporal) for multi-step tasks — each step is persisted |
+| Scheduled tasks pile up during agent hibernation | Deduplicate on wake-up; use idempotency keys for task handlers |