@warlock.js/ai 4.2.10 → 4.3.0

package/llms.txt

@@ -6,15 +6,20 @@
 
 ## Skills
 
-- [ai-basics](@warlock.js/ai/ai-basics/SKILL.md): Start with @warlock.js/ai — provider-agnostic core for agents / tools / workflows / supervisors. 4-primitive ladder (agent → workflow → supervisor → orchestrator v2). Every primitive returns {data, error, usage, report}. Triggers: `ai.agent`, `ai.tool`, `ai.workflow`, `ai.supervisor`, `ai.systemPrompt`, `ExecuteResult`, `BaseReport`, `AIError`; 'which AI primitive do I use', 'what is warlock ai', 'pick an AI skill'; typical import `import { ai } from "@warlock.js/ai"`. Skip: agent details — `@warlock.js/ai/run-ai-agent/SKILL.md`; competing libs `langchain`, `llamaindex`, `ai` (Vercel SDK); raw `openai` / `@anthropic-ai/sdk`.
-- [attach-ai-middleware](@warlock.js/ai/attach-ai-middleware/SKILL.md): Wire agent middleware — ai.middleware.budget (token / USD caps), ai.middleware.guardrail (pre / post content checks), ai.middleware.semanticCache (exact + vector cache), plus authoring custom hooks (execute / trip / tool). Triggers: `ai.middleware.budget`, `ai.middleware.guardrail`, `ai.middleware.semanticCache`, `ai.middleware.compose`, `ai.middleware.forTool`, `AgentMiddleware`, `BudgetExceededError`, `GuardrailViolationError`; 'cap token cost', 'block pii in prompts', 'semantic cache before LLM', 'write custom hook'; typical import `import { ai } from "@warlock.js/ai"`. Skip: agent lifecycle — `@warlock.js/ai/run-ai-agent/SKILL.md`; cache drivers — `@warlock.js/ai/persist-ai-data/SKILL.md`; competing libs `langchain` callbacks.
+- [ai-basics](@warlock.js/ai/ai-basics/SKILL.md): Start with @warlock.js/ai — provider-agnostic core for agents / tools / workflows / supervisors / orchestrators. 4-primitive ladder (agent → workflow → supervisor → orchestrator, all shipped) plus planner, memory, stores, DX helpers, and the optional @warlock.js/ai-panoptic observability sidecar. Every primitive returns {data, error, usage, report}. Triggers: `ai.agent`, `ai.tool`, `ai.workflow`, `ai.supervisor`, `ai.orchestrator`, `ai.planner`, `ai.memory`, `ai.systemPrompt`, `ExecuteResult`, `BaseReport`, `AIError`, `panoptic`; 'which AI primitive do I use', 'what is warlock ai', 'pick an AI skill', 'how do I observe / trace AI runs'; typical import `import { ai } from "@warlock.js/ai"`. Skip: agent details — `@warlock.js/ai/run-ai-agent/SKILL.md`; competing libs `langchain`, `llamaindex`, `ai` (Vercel SDK); raw `openai` / `@anthropic-ai/sdk`.
+- [ai-dx-helpers](@warlock.js/ai/ai-dx-helpers/SKILL.md): Developer-experience helpers across @warlock.js/ai — ai.batch (fan-out an executable over a dataset w/ concurrency + per-item retry), ai.fallbackModel (ordered model failover), agent.eval + ai.eval scorers + Vitest matchers (registerAiMatchers / toRouteTo / toConverge / toPassStep / toOutputShape) + ai.mockRouter, SLO/cost budget contracts (ai.middleware.budget({contract}) + readBudgetFallbackSignal), supervisor-level middleware, ai.systemPrompt.fromFile, and auto-adapt executables in tools:[]. Triggers: `ai.batch`, `BatchResult`, `ai.fallbackModel`, `FallbackModelContract`, `agent.eval`, `ai.eval`, `EvalReport`, `EvalScorer`, `ai.eval.judge`, `registerAiMatchers`, `toRouteTo`, `toConverge`, `toPassStep`, `toOutputShape`, `ai.mockRouter`, `MockSDK`, `mockAgent`, `budget({contract})`, `BudgetContract`, `maxLatencyMs`, `onViolation`, `readBudgetFallbackSignal`, `supervisor middleware`, `systemPrompt.fromFile`; 'run an agent over a list', 'fail over to a backup model', 'evaluate / score an agent', 'SLO budget', 'test a supervisor without an LLM', 'prompt from a file'; typical import `import { ai } from "@warlock.js/ai"`. Skip: core agent lifecycle — `@warlock.js/ai/run-ai-agent/SKILL.md`; the budget/guardrail/semanticCache basics — `@warlock.js/ai/attach-ai-middleware/SKILL.md`; competing libs `promptfoo`, `langsmith`.
+- [attach-ai-middleware](@warlock.js/ai/attach-ai-middleware/SKILL.md): Wire agent middleware — ai.middleware.budget (token / USD caps + SLO/cost contract w/ maxLatencyMs + onViolation fallback), ai.middleware.guardrail (pre / post content checks), ai.middleware.semanticCache (exact + vector cache), supervisor-level middleware, plus authoring custom hooks (execute / trip / tool). Triggers: `ai.middleware.budget`, `ai.middleware.guardrail`, `ai.middleware.semanticCache`, `ai.middleware.compose`, `ai.middleware.forTool`, `AgentMiddleware`, `BudgetExceededError`, `GuardrailViolationError`, `BudgetContract`, `maxLatencyMs`, `onViolation`, `readBudgetFallbackSignal`, `supervisor middleware`; 'cap token cost', 'SLO budget', 'block pii in prompts', 'semantic cache before LLM', 'supervisor-level middleware', 'write custom hook'; typical import `import { ai } from "@warlock.js/ai"`. Skip: agent lifecycle — `@warlock.js/ai/run-ai-agent/SKILL.md`; cache drivers — `@warlock.js/ai/persist-ai-data/SKILL.md`; competing libs `langchain` callbacks.
 - [define-ai-tool](@warlock.js/ai/define-ai-tool/SKILL.md): Define tools with ai.tool({...}) — typed validated async functions the model can call. Covers name / description / action / mode (feedback / silent) / input / execute, `ctx.artifacts` side-channel, `ToolExecutionError`. Triggers: `ai.tool`, `ToolContract`, `ToolContext`, `ToolCall`, `ToolExecutionError`, `artifactsSchema`, `mode: "silent"`, `workflow.asTool`; 'define a tool', 'wire tool into agent', 'tool input validation', 'side-channel artifacts'; typical import `import { ai } from "@warlock.js/ai"`. Skip: agent loop — `@warlock.js/ai/run-ai-agent/SKILL.md`; supervisor artifacts — `@warlock.js/ai/run-supervisor/SKILL.md`; competing libs `langchain` tools, raw `openai` function-calling.
 - [embed-text](@warlock.js/ai/embed-text/SKILL.md): Text-to-vector via sdk.embedder({...}) — embed(string) for single, embedMany(string[]) for batch. Peer primitive on the SDK adapter, not wired into agents. Compose into RAG tools, workflow run steps, or ai.middleware.semanticCache. Triggers: `sdk.embedder`, `EmbedderContract`, `embedder.embed`, `embedder.embedMany`, `EmbeddingResult`, `EmbeddingBatchResult`, `dimensions`; 'embed text', 'build RAG tool', 'populate vector store', 'embedding batch'; typical import `import { OpenAISDK } from "@warlock.js/ai-openai"`. Skip: cache similarity — `@warlock.js/cache/use-cache-similarity/SKILL.md`; pgvector queries — `@warlock.js/cascade/search-by-vector/SKILL.md`; competing libs `langchain` embeddings, raw `openai.embeddings.create`.
-- [handle-ai-errors](@warlock.js/ai/handle-ai-errors/SKILL.md): Typed AIError hierarchy with stable code strings + coarse category for retry-policy dispatch. execute() never throws — errors surface via result.error. Triggers: `AIError`, `ProviderRateLimitError`, `ProviderAuthError`, `ContextLengthExceededError`, `ContentFilterError`, `SchemaValidationError`, `ToolExecutionError`, `WorkflowDriftError`, `BudgetExceededError`, `GuardrailViolationError`, `error.code`, `error.category`; 'handle ai error', 'retry on rate limit', 'branch on error code', 'build fallback ladder'; typical import `import { AIError } from "@warlock.js/ai"`. Skip: log surfacing — `@warlock.js/ai/log-ai-calls/SKILL.md`; native `try / catch` on raw `openai`.
+- [handle-ai-errors](@warlock.js/ai/handle-ai-errors/SKILL.md): Typed AIError hierarchy with stable code strings + coarse category for retry-policy dispatch. execute() never throws — errors surface via result.error (the sole exception: OrchestratorConfigError throws at construction). Triggers: `AIError`, `ProviderRateLimitError`, `ProviderAuthError`, `ContextLengthExceededError`, `ContentFilterError`, `SchemaValidationError`, `ToolExecutionError`, `WorkflowDriftError`, `SupervisorDriftError`, `SupervisorFailedError`, `SupervisorRoutingError`, `OrchestratorFailedError`, `OrchestratorDriftError`, `OrchestratorConfigError`, `OrchestratorCancelledError`, `PlannerFailedError`, `PlannerPlanInvalidError`, `PlannerCancelledError`, `BudgetExceededError`, `GuardrailViolationError`, `error.code`, `error.category`; 'handle ai error', 'retry on rate limit', 'branch on error code', 'ORCHESTRATOR_DRIFT', 'PLANNER_PLAN_INVALID', 'build fallback ladder'; typical import `import { AIError } from "@warlock.js/ai"`. Skip: log surfacing — `@warlock.js/ai/log-ai-calls/SKILL.md`; native `try / catch` on raw `openai`.
 - [log-ai-calls](@warlock.js/ai/log-ai-calls/SKILL.md): Framework logging delegated to @warlock.js/logger — every primitive emits via the log singleton, configure channels / levels / redaction once at boot. Four-arg call convention (module, action, message, context). Triggers: `log.configure`, `log.setMinLevel`, `log.setChannels`, `ConsoleLog`, `FileLog`, `LogChannel`, `redact.paths`, `ai.agent.<name>` / `ai.workflow.<name>` / `ai.supervisor.<name>` modules; 'configure ai logging', 'mask prompts in logs', 'silence logs in tests', 'capture log entries'; typical import `import { log } from "@warlock.js/logger"`. Skip: error hierarchy — `@warlock.js/ai/handle-ai-errors/SKILL.md`; competing libs `pino`, `winston`, `console.log`.
-- [persist-ai-data](@warlock.js/ai/persist-ai-data/SKILL.md): Persistence delegated to @warlock.js/cache — workflow + supervisor snapshot resume via snapshotStore, semantic cache via vector-capable CacheDriver, global default via ai.config({defaultStore}). Covers drift detection + three recovery paths. Triggers: `ai.config`, `defaultStore`, `snapshotStore`, `wf.resume`, `supervisor.resume`, `WorkflowSnapshot`, `SupervisorSnapshot`, `WorkflowDriftError`, `SupervisorDriftError`, `force: true`; 'resume a workflow run', 'configure snapshot store', 'handle signature drift', 'wire pg vector cache'; typical import `import { ai } from "@warlock.js/ai"`. Skip: cache driver catalog — `@warlock.js/cache/cache-basics/SKILL.md`; competing libs `temporal`, `inngest`.
-- [pick-ai-provider](@warlock.js/ai/pick-ai-provider/SKILL.md): Choose an AI provider adapter — @warlock.js/ai-openai (shipped, also handles OpenRouter / Azure via baseURL), @warlock.js/ai-anthropic, @warlock.js/ai-bedrock, @warlock.js/ai-google, @warlock.js/ai-ollama. Triggers: `OpenAISDK`, `SDKAdapterContract`, `ModelContract`, `sdk.model`, `sdk.embedder`, `capabilities.vision`, `capabilities.structuredOutput`, `pricing`, `baseURL`, `provider: "openrouter"`; 'pick a provider', 'openai vs openrouter', 'does this model support vision', 'configure pricing'; typical import `import { OpenAISDK } from "@warlock.js/ai-openai"`. Skip: agent factory — `@warlock.js/ai/run-ai-agent/SKILL.md`; competing libs raw `openai`, `@anthropic-ai/sdk`, `@aws-sdk/client-bedrock-runtime`.
-- [run-ai-agent](@warlock.js/ai/run-ai-agent/SKILL.md): Build agents with ai.agent({...}) — the single-LLM-turn primitive. Covers execute / stream, attachments, structured output, placeholders, events. Triggers: `ai.agent`, `agent.execute`, `agent.stream`, `AgentResult`, `AgentReport`, `streamingToolGuard`, `attachments`, `repair`, `maxTrips`, `sessionId`; 'run an agent', 'stream an agent response', 'structured output schema', 'pass image to agent', 'cancel an agent run'; typical import `import { ai } from "@warlock.js/ai"`. Skip: tool definition — `@warlock.js/ai/define-ai-tool/SKILL.md`; workflows — `@warlock.js/ai/run-ai-workflow/SKILL.md`; competing libs `langchain`, `ai` (Vercel), raw `openai`.
+- [manage-ai-stores](@warlock.js/ai/manage-ai-stores/SKILL.md): Durable orchestrator stores — ai.checkpoint.{memory,pg,redis}() for cross-turn SESSION STATE and ai.snapshot.{memory,pg,redis}() for in-flight SUPERVISOR/WORKFLOW run state. Two distinct contracts (CheckpointStore vs SnapshotStore), dev-owned pg/redis clients (no peer dep), never-auto-migrated schema(), global defaults via ai.config({defaultCheckpointStore, defaultSnapshotStore}). Triggers: `ai.checkpoint`, `ai.snapshot`, `checkpointStore`, `snapshotStore`, `CheckpointStore`, `SnapshotStore`, `CheckpointRecord`, `checkpoint.pg`, `checkpoint.redis`, `snapshot.pg`, `snapshot.redis`, `store.schema()`, `keepSnapshots`, `defaultCheckpointStore`, `defaultSnapshotStore`, `PgClientLike`, `RedisClientLike`; 'persist orchestrator sessions', 'wire a pg checkpoint store', 'run the store DDL', 'checkpoint vs snapshot'; typical import `import { ai } from "@warlock.js/ai"`. Skip: orchestrator lifecycle — `@warlock.js/ai/run-orchestrator/SKILL.md`; cache-backed snapshot resume / semanticCache store — `@warlock.js/ai/persist-ai-data/SKILL.md`; competing libs `temporal`, `inngest`.
+- [persist-ai-data](@warlock.js/ai/persist-ai-data/SKILL.md): Persistence delegated to @warlock.js/cache — workflow + supervisor snapshot resume via snapshotStore (4.3.0: now a SnapshotStore from ai.snapshot.*, ⚠ moved off raw CacheDriver), semantic cache + memory via vector-capable CacheDriver, global defaults via ai.config({defaultStore}) + ai.config({defaultSnapshotStore}). Covers drift detection + three recovery paths. Triggers: `ai.config`, `defaultStore`, `defaultSnapshotStore`, `snapshotStore`, `ai.snapshot`, `wf.resume`, `supervisor.resume`, `WorkflowSnapshot`, `SupervisorSnapshot`, `WorkflowDriftError`, `SupervisorDriftError`, `force: true`; 'resume a workflow run', 'configure snapshot store', 'handle signature drift', 'wire pg vector cache'; typical import `import { ai } from "@warlock.js/ai"`. Skip: orchestrator checkpoint/snapshot store factories — `@warlock.js/ai/manage-ai-stores/SKILL.md`; cache driver catalog — `@warlock.js/cache/cache-basics/SKILL.md`; competing libs `temporal`, `inngest`.
+- [pick-ai-provider](@warlock.js/ai/pick-ai-provider/SKILL.md): Choose an AI provider adapter — @warlock.js/ai-openai (shipped, also handles OpenRouter / Azure via baseURL), @warlock.js/ai-anthropic, @warlock.js/ai-bedrock, @warlock.js/ai-google, @warlock.js/ai-ollama — plus cost truth: ModelPricing (per-1M tokens), Usage cost breakdown, the cachedTokens / cacheWriteTokens / reasoningTokens channels, and capability flags. Triggers: `OpenAISDK`, `SDKAdapterContract`, `ModelContract`, `ModelPricing`, `ModelCapabilities`, `sdk.model`, `sdk.embedder`, `capabilities.vision`, `capabilities.structuredOutput`, `capabilities.reasoning`, `capabilities.promptCaching`, `pricing`, `Usage.cost`, `cachedTokens`, `cacheWriteTokens`, `reasoningTokens`, `reasoning.effort`, `cacheControl`, `baseURL`, `provider: "openrouter"`; 'pick a provider', 'openai vs openrouter', 'does this model support vision/reasoning', 'configure pricing', 'how much did reasoning cost', 'prompt cache tokens'; typical import `import { OpenAISDK } from "@warlock.js/ai-openai"`. Skip: agent factory — `@warlock.js/ai/run-ai-agent/SKILL.md`; competing libs raw `openai`, `@anthropic-ai/sdk`, `@aws-sdk/client-bedrock-runtime`.
+- [run-ai-agent](@warlock.js/ai/run-ai-agent/SKILL.md): Build agents with ai.agent({...}) — the single-LLM-turn primitive. Covers execute / stream, attachments, structured output, placeholders, events, agent.eval scoring, and auto-adapting raw executables in tools:[]. Triggers: `ai.agent`, `agent.execute`, `agent.stream`, `agent.eval`, `AgentResult`, `AgentReport`, `AgentToolEntry`, `streamingToolGuard`, `attachments`, `repair`, `maxTrips`, `sessionId`; 'run an agent', 'stream an agent response', 'structured output schema', 'pass image to agent', 'evaluate an agent', 'put a supervisor in tools', 'cancel an agent run'; typical import `import { ai } from "@warlock.js/ai"`. Skip: tool definition — `@warlock.js/ai/define-ai-tool/SKILL.md`; workflows — `@warlock.js/ai/run-ai-workflow/SKILL.md`; eval matchers / batch / fallback detail — `@warlock.js/ai/ai-dx-helpers/SKILL.md`; competing libs `langchain`, `ai` (Vercel), raw `openai`.
 - [run-ai-workflow](@warlock.js/ai/run-ai-workflow/SKILL.md): Build durable resumable pipelines with ai.workflow({...}) + ai.step({...}) — lifecycle (skip / before / run|agent|parallel / output / after / nextStep), retry, parallel groups, snapshot resume. Triggers: `ai.workflow`, `ai.step`, `wf.execute`, `wf.resume`, `WorkflowContext`, `WorkflowResult`, `StepSnapshot`, `nextStep`, `onFailure`, `WorkflowDriftError`; 'build a workflow', 'define a step', 'resume after crash', 'parallel steps', 'retry with backoff'; typical import `import { ai } from "@warlock.js/ai"`. Skip: agent — `@warlock.js/ai/run-ai-agent/SKILL.md`; supervisor — `@warlock.js/ai/run-supervisor/SKILL.md`; competing libs `temporal`, `inngest`, `bullmq`.
-- [run-supervisor](@warlock.js/ai/run-supervisor/SKILL.md): Multi-intent routing with ai.supervisor({...}) — classifier (iter-0 dispatch), router agent OR route callback, intents as agents / workflows / callbacks, fan-out, evaluate quality loop, ack receptionist. Triggers: `ai.supervisor`, `supervisor.execute`, `supervisor.resume`, `intents`, `router`, `route`, `classifier`, `evaluate`, `ack`, `artifactsSchema`, `END`, `ctx.intents.X.execute`; 'route one input across specialists', 'multi-intent dispatch', 'fan-out then evaluate', 'classifier then router'; typical import `import { ai } from "@warlock.js/ai"`. Skip: fixed pipelines — `@warlock.js/ai/run-ai-workflow/SKILL.md`; single agent — `@warlock.js/ai/run-ai-agent/SKILL.md`; competing libs `langgraph`, `crewai`.
-- [write-system-prompt](@warlock.js/ai/write-system-prompt/SKILL.md): Compose system prompts via ai.systemPrompt() / ai.persona() / ai.instruction() — immutable builders with {{placeholder}} substitution. Triggers: `ai.systemPrompt`, `ai.persona`, `ai.instruction`, `SystemPromptBlockContract`, `PersonaContract`, `InstructionContract`, `placeholders`, `{{placeholder|default}}`; 'write a system prompt', 'compose persona + instructions', 'per-call prompt override', 'mustache placeholder'; typical import `import { ai } from "@warlock.js/ai"`. Skip: agent factory wiring — `@warlock.js/ai/run-ai-agent/SKILL.md`; competing libs `langchain` `PromptTemplate`, raw f-strings.
+- [run-orchestrator](@warlock.js/ai/run-orchestrator/SKILL.md): Durable stateful sessions with ai.orchestrator({...}) — the capstone of the 4-primitive ladder. Wraps a supervisor with cross-turn session state (checkpointStore), per-turn windowing, drift detection, post-turn compaction, mid-turn resume (iterate: true + snapshotStore), per-turn memory, typed commands, asTool, and a 3-tier event model. Triggers: `ai.orchestrator`, `orchestrator.execute`, `orchestrator.resume`, `orchestrator.command`, `orchestrator.stream`, `OrchestratorConfig`, `OrchestratorResult`, `OrchestratorReport`, `OrchestratorContract`, `CheckpointStore`, `OrchestratorDriftError`, `sessionId`, `iterate`, `historyWindow`, `summarize`, `keepSnapshots`, `awaiting-input`, `turns[]`, `TurnSnapshot`, `CompactionResult`, `initialAgent`, `checkpointStore`; 'multi-turn conversation that persists', 'durable session across calls', 'resume an interrupted turn', 'compact session history', 'per-session memory'; typical import `import { ai } from "@warlock.js/ai"`. Skip: a single routing turn with no session — `@warlock.js/ai/run-supervisor/SKILL.md`; a fixed pipeline — `@warlock.js/ai/run-ai-workflow/SKILL.md`; the store factories themselves — `@warlock.js/ai/manage-ai-stores/SKILL.md`; competing libs `langgraph`, `crewai`.
+- [run-planner](@warlock.js/ai/run-planner/SKILL.md): Goal-driven planning with ai.planner({...}) — an LLM GENERATES an ordered execution plan over your registered capabilities (agents / workflows / supervisors / tools), then the planner EXECUTES that plan step-by-step, threading each step output into the next, and returns the unified {data, report, usage, error} envelope with report.type "planner". A plan step may delegate via ai.spawnSubAgent({...}) — a general one-shot-agent helper (a fresh agent + optional per-task budget) covered in `@warlock.js/ai/run-ai-agent/SKILL.md`, not planner-specific. Triggers: `ai.planner`, `planner.execute`, `spawnSubAgent`, `PlannerConfig`, `PlannerCapability`, `PlannerResult`, `PlannerReport`, `PlannerPlan`, `PlannerStep`, `maxSteps`, `report.plan`, `report.executedSteps`; 'let the model plan the steps', 'dynamic plan from a goal', 'decompose a goal into capability calls', 'spawn a sub-agent for a subtask'; typical import `import { ai } from "@warlock.js/ai"`. Skip: a FIXED known pipeline — `@warlock.js/ai/run-ai-workflow/SKILL.md`; routing one input to a specialist each turn — `@warlock.js/ai/run-supervisor/SKILL.md`; a single model + tools call — `@warlock.js/ai/run-ai-agent/SKILL.md`; competing libs `langgraph`, `crewai`.
+- [run-supervisor](@warlock.js/ai/run-supervisor/SKILL.md): Multi-intent routing with ai.supervisor({...}) — classifier (iter-0 dispatch), router agent OR route callback, intents as agents / workflows / callbacks, fan-out, evaluate quality loop, ack receptionist, supervisor-level middleware. Triggers: `ai.supervisor`, `ai.router`, `ai.fanOut`, `supervisor.execute`, `supervisor.resume`, `intents`, `router`, `route`, `classifier`, `evaluate`, `ack`, `artifactsSchema`, `middleware`, `END`, `ctx.intents.X.execute`; 'route one input across specialists', 'multi-intent dispatch', 'fan-out then evaluate', 'classifier then router', 'supervisor middleware', 'self-consistency / voting'; typical import `import { ai } from "@warlock.js/ai"`. Skip: durable multi-turn sessions — `@warlock.js/ai/run-orchestrator/SKILL.md`; fixed pipelines — `@warlock.js/ai/run-ai-workflow/SKILL.md`; single agent — `@warlock.js/ai/run-ai-agent/SKILL.md`; competing libs `langgraph`, `crewai`.
+- [use-ai-memory](@warlock.js/ai/use-ai-memory/SKILL.md): Agent memory with ai.memory({...}) — a provider-neutral store with FOUR tiers: WORKING (in-run scratch, recalled by recency), SEMANTIC (durable facts by cosine similarity over a @warlock.js/cache vector driver via .similar()), EPISODIC (durable events, similarity blended with recency), and PROCEDURAL (durable how-tos, similarity blended with reinforcement). remember() / recall() / clear(); wire it into ai.orchestrator({ memory }). Triggers: `ai.memory`, `memory.remember`, `memory.recall`, `memory.clear`, `MemoryContract`, `MemoryConfig`, `MemoryItem`, `RecalledMemory`, `MemoryTier`, `SemanticMemoryConfig`, `EpisodicMemoryConfig`, `ProceduralMemoryConfig`, `working`, `semantic`, `episodic`, `procedural`, `defaultTier`, `threshold`, `recencyWeight`, `halfLifeMs`, `reinforcementWeight`, `injectKey`; 'give the agent memory', 'remember user preferences', 'semantic recall', 'per-session working memory', 'episodic / event memory', 'procedural / how-to memory', 'recency-weighted recall', 'reinforce a procedure'; typical import `import { ai } from "@warlock.js/ai"`. Skip: orchestrator wiring of the memory — `@warlock.js/ai/run-orchestrator/SKILL.md`; the vector cache driver itself — `@warlock.js/cache/cache-basics/SKILL.md`; embeddings primitive — `@warlock.js/ai/embed-text/SKILL.md`; competing libs `mem0`, `langchain` memory.
+- [write-system-prompt](@warlock.js/ai/write-system-prompt/SKILL.md): Compose system prompts via ai.systemPrompt() / ai.persona() / ai.instruction() — immutable builders with {{placeholder}} substitution, plus ai.systemPrompt.fromFile(path) to seed from a file read once at construction. Triggers: `ai.systemPrompt`, `ai.systemPrompt.fromFile`, `ai.persona`, `ai.instruction`, `SystemPromptBlockContract`, `PersonaContract`, `InstructionContract`, `placeholders`, `{{placeholder|default}}`, `InvalidRequestError`; 'write a system prompt', 'compose persona + instructions', 'prompt from a file', 'per-call prompt override', 'mustache placeholder'; typical import `import { ai } from "@warlock.js/ai"`. Skip: agent factory wiring — `@warlock.js/ai/run-ai-agent/SKILL.md`; competing libs `langchain` `PromptTemplate`, raw f-strings.

package/package.json

@@ -15,10 +15,10 @@
     "@standard-schema/spec": "^1.0.0"
   },
   "peerDependencies": {
-    "@warlock.js/cache": "4.2.10",
-    "@warlock.js/logger": "4.2.10"
+    "@warlock.js/cache": "4.3.0",
+    "@warlock.js/logger": "4.3.0"
   },
-  "version": "4.2.10",
+  "version": "4.3.0",
   "main": "./cjs/index.cjs",
   "module": "./esm/index.mjs",
   "types": "./esm/index.d.mts",

package/skills/README.md

@@ -6,11 +6,15 @@ Per-task skills. All cross-references use the form `@warlock.js/<pkg>/<skill>/SK
 
 ### [`ai-basics/`](./ai-basics/SKILL.md)
 
-Start with @warlock.js/ai — provider-agnostic core for agents / tools / workflows / supervisors. 4-primitive ladder (agent → workflow → supervisor → orchestrator v2). Every primitive returns {data, error, usage, report}. Persistence + logging delegated. Load when importing @warlock.js/ai, picking a primitive, or choosing which AI skill to load.
+Start with @warlock.js/ai — provider-agnostic core for agents / tools / workflows / supervisors / orchestrators. 4-primitive ladder (agent → workflow → supervisor → orchestrator, all shipped) plus planner, memory, stores, and DX helpers. Every primitive returns {data, error, usage, report}. Persistence + logging delegated. Load when importing @warlock.js/ai, picking a primitive, or choosing which AI skill to load.
+
+### [`ai-dx-helpers/`](./ai-dx-helpers/SKILL.md)
+
+Developer-experience helpers — ai.batch (fan-out over a dataset w/ concurrency + retry), ai.fallbackModel (ordered model failover), agent.eval + ai.eval scorers + Vitest matchers (registerAiMatchers / toRouteTo / toConverge / toPassStep / toOutputShape) + ai.mockRouter, SLO/cost budget contracts (ai.middleware.budget({contract}) + readBudgetFallbackSignal), supervisor-level middleware, ai.systemPrompt.fromFile, auto-adapt executables in tools:[]. Load when running an executable over a list, failing over models, evaluating / testing an agent or supervisor, declaring an SLO budget, or seeding a prompt from a file.
 
 ### [`attach-ai-middleware/`](./attach-ai-middleware/SKILL.md)
 
-Wire agent middleware — ai.middleware.budget (token / USD caps), ai.middleware.guardrail (pre / post content checks), ai.middleware.semanticCache (exact + vector cache via @warlock.js/cache), plus authoring custom hooks (execute / trip / tool). Load when capping cost / tokens, gating input or output, hitting semantic cache before the LLM, or writing custom hooks.
+Wire agent + supervisor middleware — ai.middleware.budget (token / USD caps + SLO/cost contract), ai.middleware.guardrail (pre / post content checks), ai.middleware.semanticCache (exact + vector cache via @warlock.js/cache), plus authoring custom hooks (execute / trip / tool). Load when capping cost / tokens, gating input or output, hitting semantic cache before the LLM, attaching supervisor-level middleware, or writing custom hooks.
 
 ### [`define-ai-tool/`](./define-ai-tool/SKILL.md)
 
@@ -28,9 +32,13 @@ Typed AIError hierarchy with stable code strings + coarse category for retry-pol
 
 Framework logging delegated to @warlock.js/logger — every primitive emits via the log singleton, configure channels / levels / redaction once at boot. Four-arg call convention (module, action, message, context). Load when configuring AI logging, masking prompts / API keys / PII, picking which events surface, or filtering by module / action.
 
+### [`manage-ai-stores/`](./manage-ai-stores/SKILL.md)
+
+Durable orchestrator stores — ai.checkpoint.{memory,pg,redis}() for cross-turn SESSION STATE and ai.snapshot.{memory,pg,redis}() for in-flight SUPERVISOR/WORKFLOW run state. Two distinct contracts (CheckpointStore vs SnapshotStore), dev-owned pg/redis clients (no peer dep), never-auto-migrated schema(), global defaults via ai.config({defaultCheckpointStore, defaultSnapshotStore}). Load when persisting orchestrator sessions, wiring a pg/redis store, running the store DDL, or untangling checkpoint vs snapshot.
+
 ### [`persist-ai-data/`](./persist-ai-data/SKILL.md)
 
-Persistence delegated to @warlock.js/cache — workflow + supervisor snapshot resume via snapshotStore, semantic cache via vector-capable CacheDriver, global default via ai.config({defaultStore}). Covers drift detection + three recovery paths. Load when configuring snapshotStore, calling resume(runId), or handling WorkflowDriftError / SupervisorDriftError.
+Persistence delegated to @warlock.js/cache — workflow + supervisor snapshot resume via snapshotStore (4.3.0: now a SnapshotStore from ai.snapshot.*, ⚠ moved off raw CacheDriver), semantic cache + memory via vector-capable CacheDriver, global defaults via ai.config({defaultStore}) + ai.config({defaultSnapshotStore}). Covers drift detection + three recovery paths. Load when configuring snapshotStore, calling resume(runId), or handling WorkflowDriftError / SupervisorDriftError.
 
 ### [`pick-ai-provider/`](./pick-ai-provider/SKILL.md)
 
@@ -38,16 +46,28 @@ Choose an AI provider adapter — @warlock.js/ai-openai (shipped, also handles O
 
 ### [`run-ai-agent/`](./run-ai-agent/SKILL.md)
 
-Build agents with ai.agent({...}) — the single-LLM-turn primitive. Covers execute / stream, attachments, structured output, placeholders, events, AgentResult envelope, streamingToolGuard. Load when calling ai.agent(...), reading AgentResult, wiring options.output / attachments / repair, or streaming.
+Build agents with ai.agent({...}) — the single-LLM-turn primitive. Covers execute / stream, attachments, structured output, placeholders, events, AgentResult envelope, streamingToolGuard, and ai.spawnSubAgent({...}) (a thin one-shot-agent wrapper with a per-task budget — a general primitive, not planner-specific). Load when calling ai.agent(...), reading AgentResult, wiring options.output / attachments / repair, streaming, or spawning a one-shot sub-agent.
 
 ### [`run-ai-workflow/`](./run-ai-workflow/SKILL.md)
 
 Build durable resumable pipelines with ai.workflow({...}) + ai.step({...}) — lifecycle (skip / before / run|agent|parallel / output / after / nextStep), routing on success / failure, retry with backoff, parallel groups, cancel via AbortSignal, snapshot resume. Load when authoring ai.workflow, defining steps, handling WorkflowDriftError, or resuming a run.
 
+### [`run-orchestrator/`](./run-orchestrator/SKILL.md)
+
+Durable stateful sessions with ai.orchestrator({...}) — the capstone of the 4-primitive ladder. Wraps a supervisor with cross-turn session state (checkpointStore), per-turn history windowing, drift detection, post-turn compaction, mid-turn resume (iterate: true + snapshotStore), per-turn memory, typed commands, asTool, and a 3-tier event model. Load when a conversation must persist across calls, resume an interrupted turn, compact session history, or wire per-session memory; report.type "orchestrator", status may be "awaiting-input".
+
+### [`run-planner/`](./run-planner/SKILL.md)
+
+Goal-driven planning with ai.planner({...}) — an LLM GENERATES an ordered plan over your registered capabilities (agents / workflows / supervisors / tools), then the planner EXECUTES it step-by-step, threading each step output into the next. A plan step may delegate via ai.spawnSubAgent({...}) — a general one-shot-agent helper covered in run-ai-agent (not planner-specific). Load when the steps are NOT known up front and the model should plan them; report.type "planner".
+
 ### [`run-supervisor/`](./run-supervisor/SKILL.md)
 
 Multi-intent routing with ai.supervisor({...}) — classifier (iter-0 dispatch), router agent OR route callback (iter 1+), intents as agents / workflows / callbacks, fan-out, evaluate quality loop, ack receptionist, ctx.intents.X.execute composition. Load when routing one user input across a fixed roster of specialists.
 
+### [`use-ai-memory/`](./use-ai-memory/SKILL.md)
+
+Agent memory with ai.memory({...}) — a provider-neutral store with two v1 tiers: WORKING (in-run scratch, volatile, recalled by recency) and SEMANTIC (durable cosine-similarity recall over a @warlock.js/cache vector driver via .similar()). remember() / recall() / clear(); wire it into ai.orchestrator({ memory }). Episodic/procedural + decay deferred to 4.4. Load when giving an agent memory, remembering user preferences, doing semantic recall, or wiring per-session working memory.
+
 ### [`write-system-prompt/`](./write-system-prompt/SKILL.md)
 
-Compose system prompts via ai.systemPrompt() / ai.persona() / ai.instruction() — immutable builders with {{placeholder}} substitution. Load when building or chaining system prompts, mixing persona + instruction blocks, using {{placeholder}}, or doing per-call override via agent.execute(input, {systemPrompt}).
+Compose system prompts via ai.systemPrompt() / ai.persona() / ai.instruction() — immutable builders with {{placeholder}} substitution, plus ai.systemPrompt.fromFile(path) to seed from a file read once at construction. Load when building or chaining system prompts, mixing persona + instruction blocks, using {{placeholder}}, seeding a prompt from a file, or doing per-call override via agent.execute(input, {systemPrompt}).

package/skills/ai-basics/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: ai-basics
-description: 'Start with @warlock.js/ai — provider-agnostic core for agents / tools / workflows / supervisors. 4-primitive ladder (agent → workflow → supervisor → orchestrator v2). Every primitive returns {data, error, usage, report}. Triggers: `ai.agent`, `ai.tool`, `ai.workflow`, `ai.supervisor`, `ai.systemPrompt`, `ExecuteResult`, `BaseReport`, `AIError`; ''which AI primitive do I use'', ''what is warlock ai'', ''pick an AI skill''; typical import `import { ai } from "@warlock.js/ai"`. Skip: agent details — `@warlock.js/ai/run-ai-agent/SKILL.md`; competing libs `langchain`, `llamaindex`, `ai` (Vercel SDK); raw `openai` / `@anthropic-ai/sdk`.'
+description: 'Start with @warlock.js/ai — provider-agnostic core for agents / tools / workflows / supervisors / orchestrators. 4-primitive ladder (agent → workflow → supervisor → orchestrator, all shipped) plus planner, memory, stores, DX helpers, and the optional @warlock.js/ai-panoptic observability sidecar. Every primitive returns {data, error, usage, report}. Triggers: `ai.agent`, `ai.tool`, `ai.workflow`, `ai.supervisor`, `ai.orchestrator`, `ai.planner`, `ai.memory`, `ai.systemPrompt`, `ExecuteResult`, `BaseReport`, `AIError`, `panoptic`; ''which AI primitive do I use'', ''what is warlock ai'', ''pick an AI skill'', ''how do I observe / trace AI runs''; typical import `import { ai } from "@warlock.js/ai"`. Skip: agent details — `@warlock.js/ai/run-ai-agent/SKILL.md`; competing libs `langchain`, `llamaindex`, `ai` (Vercel SDK); raw `openai` / `@anthropic-ai/sdk`.'
 ---
 
 # AI foundations
@@ -15,10 +15,12 @@ Provider-agnostic core for building AI primitives in TypeScript. Adapters live i
 ai.agent()        →  single task, stateless                    [shipped]
 ai.workflow()     →  static predefined steps, resumable        [shipped]
 ai.supervisor()   →  multi-agent dynamic routing, resumable    [shipped]
-ai.orchestrator() →  stateful — owns session/history/ctx       (v2)
+ai.orchestrator() →  durable session — state/history/resume    [shipped]
 ```
 
-Each primitive is an escape hatch to the next level of complexity. Users start low, graduate upward only when needed. Every primitive returns the same result envelope — canonical destructure `{ data, error, usage, report }` (the shared `BaseResult` guarantees `usage` + optional `error`; each primitive adds `data` + `report`). Workflows and supervisors expose `.asTool()` so an agent can call them inside its tool loop; compose freely.
+Each primitive is an escape hatch to the next level of complexity. Users start low, graduate upward only when needed. Every primitive returns the same result envelope — canonical destructure `{ data, error, usage, report }` (the shared `BaseResult` guarantees `usage` + optional `error`; each primitive adds `data` + `report`). Workflows, supervisors, and orchestrators expose `.asTool()` so an agent can call them inside its tool loop; raw executables also auto-adapt when dropped into an agent's `tools: []`. Compose freely.
+
+Beyond the ladder: `ai.planner()` (LLM-generated plans), `ai.memory()` (working + semantic recall), `ai.batch()` / `ai.fallbackModel()` / `ai.router()` / `ai.fanOut()` (DX helpers), `agent.eval()` (scoring), and the `ai.checkpoint.*` / `ai.snapshot.*` orchestrator stores.
 
 ## Foundations
 
@@ -56,17 +58,23 @@ console.log(text, usage.total, report.duration);
 
 | If the task is about… | Load |
 | --- | --- |
-| `ai.agent({...})` — single-LLM-turn primitive, structured output, streaming, attachments | [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) |
+| `ai.agent({...})` — single-LLM-turn primitive, structured output, streaming, attachments, `spawnSubAgent` | [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) |
 | `ai.tool({...})` — typed validated functions the model can call | [`@warlock.js/ai/define-ai-tool/SKILL.md`](@warlock.js/ai/define-ai-tool/SKILL.md) |
 | `ai.systemPrompt()` / `ai.persona()` / `ai.instruction()` — composable prompts with placeholders | [`@warlock.js/ai/write-system-prompt/SKILL.md`](@warlock.js/ai/write-system-prompt/SKILL.md) |
 | `ai.workflow({...})` — durable resumable pipelines with steps, routing, retry | [`@warlock.js/ai/run-ai-workflow/SKILL.md`](@warlock.js/ai/run-ai-workflow/SKILL.md) |
 | `ai.supervisor({...})` — multi-intent routing, fan-out, evaluate loops | [`@warlock.js/ai/run-supervisor/SKILL.md`](@warlock.js/ai/run-supervisor/SKILL.md) |
+| `ai.orchestrator({...})` — durable stateful sessions, drift, compaction, resume | [`@warlock.js/ai/run-orchestrator/SKILL.md`](@warlock.js/ai/run-orchestrator/SKILL.md) |
+| `ai.planner({...})` — LLM-generated plans over registered capabilities | [`@warlock.js/ai/run-planner/SKILL.md`](@warlock.js/ai/run-planner/SKILL.md) |
+| `ai.memory({...})` — working + semantic recall for agents / sessions | [`@warlock.js/ai/use-ai-memory/SKILL.md`](@warlock.js/ai/use-ai-memory/SKILL.md) |
+| `ai.checkpoint.*` / `ai.snapshot.*` — orchestrator session + run stores | [`@warlock.js/ai/manage-ai-stores/SKILL.md`](@warlock.js/ai/manage-ai-stores/SKILL.md) |
+| DX helpers — `batch` / `fallbackModel` / `eval` + matchers / SLO contracts / `fromFile` | [`@warlock.js/ai/ai-dx-helpers/SKILL.md`](@warlock.js/ai/ai-dx-helpers/SKILL.md) |
 | `sdk.embedder({...})` — text-to-vector for RAG tools, vector ingest | [`@warlock.js/ai/embed-text/SKILL.md`](@warlock.js/ai/embed-text/SKILL.md) |
-| Agent middleware — `budget` / `guardrail` / `semanticCache` + custom hooks | [`@warlock.js/ai/attach-ai-middleware/SKILL.md`](@warlock.js/ai/attach-ai-middleware/SKILL.md) |
+| Agent + supervisor middleware — `budget` / `guardrail` / `semanticCache` + custom hooks | [`@warlock.js/ai/attach-ai-middleware/SKILL.md`](@warlock.js/ai/attach-ai-middleware/SKILL.md) |
 | Snapshot resume + semantic cache via `@warlock.js/cache` | [`@warlock.js/ai/persist-ai-data/SKILL.md`](@warlock.js/ai/persist-ai-data/SKILL.md) |
 | Configuring framework logging | [`@warlock.js/ai/log-ai-calls/SKILL.md`](@warlock.js/ai/log-ai-calls/SKILL.md) |
-| `AIError` hierarchy, `error.code` / `error.category`, retry patterns | [`@warlock.js/ai/handle-ai-errors/SKILL.md`](@warlock.js/ai/handle-ai-errors/SKILL.md) |
-| Choosing a provider adapter (OpenAI / OpenRouter / Anthropic / Bedrock / Ollama) | [`@warlock.js/ai/pick-ai-provider/SKILL.md`](@warlock.js/ai/pick-ai-provider/SKILL.md) |
+| `AIError` hierarchy, `error.code` / `error.category`, retry patterns (incl. `ORCHESTRATOR_*` / `PLANNER_*` families) | [`@warlock.js/ai/handle-ai-errors/SKILL.md`](@warlock.js/ai/handle-ai-errors/SKILL.md) |
+| Provider adapters + cost truth (pricing / cache + reasoning tokens / capabilities) | [`@warlock.js/ai/pick-ai-provider/SKILL.md`](@warlock.js/ai/pick-ai-provider/SKILL.md) |
+| Observability — `panoptic()` subscriber, queryable trace store, OTEL / Langfuse / console / file exporters | [`@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md`](@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md) |
 
 ## Package layout
 
@@ -77,8 +85,11 @@ console.log(text, usage.total, report.duration);
 @warlock.js/ai-bedrock       — AWS Bedrock adapter (Converse API + Titan embeddings)
 @warlock.js/ai-google        — Google / Gemini adapter (@google/genai + batch embeddings)
 @warlock.js/ai-ollama        — Ollama adapter for local models
+@warlock.js/ai-panoptic      — observability sidecar: panoptic() subscriber → collector → queryable trace store + console / file / OTEL / Langfuse exporters
 ```
 
+The observability sidecar is OPTIONAL and lives in its own package — it subscribes to the report tree every primitive already emits, so you wire `panoptic(...)` once and never touch primitive code. Load [`@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md`](@warlock.js/ai-panoptic/observe-with-panoptic/SKILL.md) for collecting / querying traces and [`@warlock.js/ai-panoptic/export-traces/SKILL.md`](@warlock.js/ai-panoptic/export-traces/SKILL.md) for OTEL / Langfuse / console / file exporters.
+
 Runtime deps: `@warlock.js/cache` (persistence), `@warlock.js/logger` (logging), `@warlock.js/seal` (recommended schema lib).
 
 ## When NOT to use this skill

package/skills/ai-dx-helpers/SKILL.md

@@ -0,0 +1,180 @@
+---
+name: ai-dx-helpers
+description: 'Developer-experience helpers across @warlock.js/ai — ai.batch (fan-out an executable over a dataset w/ concurrency + per-item retry), ai.fallbackModel (ordered model failover), agent.eval + ai.eval scorers + Vitest matchers (registerAiMatchers / toRouteTo / toConverge / toPassStep / toOutputShape) + ai.mockRouter, SLO/cost budget contracts (ai.middleware.budget({contract}) + readBudgetFallbackSignal), supervisor-level middleware, ai.systemPrompt.fromFile, and auto-adapt executables in tools:[]. Triggers: `ai.batch`, `BatchResult`, `ai.fallbackModel`, `FallbackModelContract`, `agent.eval`, `ai.eval`, `EvalReport`, `EvalScorer`, `ai.eval.judge`, `registerAiMatchers`, `toRouteTo`, `toConverge`, `toPassStep`, `toOutputShape`, `ai.mockRouter`, `MockSDK`, `mockAgent`, `budget({contract})`, `BudgetContract`, `maxLatencyMs`, `onViolation`, `readBudgetFallbackSignal`, `supervisor middleware`, `systemPrompt.fromFile`; ''run an agent over a list'', ''fail over to a backup model'', ''evaluate / score an agent'', ''SLO budget'', ''test a supervisor without an LLM'', ''prompt from a file''; typical import `import { ai } from "@warlock.js/ai"`. Skip: core agent lifecycle — `@warlock.js/ai/run-ai-agent/SKILL.md`; the budget/guardrail/semanticCache basics — `@warlock.js/ai/attach-ai-middleware/SKILL.md`; competing libs `promptfoo`, `langsmith`.'
+---
+
+# DX helpers — batch, fallback, eval, SLO, supervisor middleware
+
+A grab-bag of additive 4.3.0 helpers. Each is independent — load the section you need.
+
+## `ai.batch(executable, items, options?)` — fan-out a dataset
+
+Runs the SAME executable (agent / workflow / supervisor / tool — anything `ExecutableContract`) N times, once per item, with bounded concurrency and per-item retry. Aggregates into the unified `ExecuteResult` envelope so a batch slots into cost dashboards exactly like a single run.
+
+```ts
+const result = await ai.batch(summarizer, articles, {
+  concurrency: 4,                                   // default = items.length (all at once); <=0 → serial
+  retry: { attempts: 3, backoff: "exponential" },   // workflow RetryConfig, applied per item
+  onItem: (item) => log.info("batch", "item", "settled", { index: item.index }),
+  signal: AbortSignal.timeout(120_000),
+  sessionId: "ingest-2026-06-19",                   // lineage onto every child report
+  name: "summarize-articles",
+});
+
+console.log(`${result.report.succeeded}/${result.report.total} ok`);
+console.log(`${result.usage.total} tokens total`);
+
+for (const item of result.items) {
+  if (item.status === "completed") console.log(item.index, item.result?.data);
+  else console.warn(item.index, item.error?.code, "after", item.attempts, "attempts");
+}
+```
+
+**Isolation.** Items are independent — one item's failure (after its retries) never cancels a sibling, and **the batch never rejects as a whole** (`result.error` stays undefined). Failures live on each `BatchItemResult` (`status: "completed" | "failed" | "cancelled"`, `error`, `attempts`). `result.data` is the positional array of successful items' `.data` with `undefined` in failed/cancelled slots. Usage rolls up bottom-up (batch has zero own cost); each item's report attaches under `report.children[]` in original order. An `onItem` throw is swallowed — a progress hook never breaks the batch.
+
+## `ai.fallbackModel(models, options?)` — ordered model failover
+
+A drop-in `ModelContract` that wraps an ordered list and advances to the next model only on a **transient** provider error.
+
+```ts
+const model = ai.fallbackModel([
+  ai.openai.model({ name: "gpt-4o" }),
+  ai.anthropic.model({ name: "claude-3-5-sonnet" }),
+]);
+
+const agent = ai.agent({ model });   // hand it anywhere a model goes
+
+// custom retry predicate or code list:
+ai.fallbackModel([primary, backup], { retryOn: ["PROVIDER_RATE_LIMIT", "PROVIDER_TIMEOUT"] });
+ai.fallbackModel([primary, backup], { retryOn: (error) => error instanceof ProviderError });
+```
+
+Default retryable codes: `PROVIDER_RATE_LIMIT`, `PROVIDER_TIMEOUT`, `PROVIDER_ERROR`. Auth / invalid-request / context-length / content-filter re-throw immediately (they'd fail identically downstream — retrying only burns budget). Identity/capabilities/pricing front the primary model. Usage aggregates across attempted models. Inspect `model.lastAttempts` for the failed models of the most recent call.
+
+**Streaming caveat:** `stream()` can only fail over while no chunk has been emitted yet — once the first `delta` / `tool-call` reaches the consumer, a mid-stream failure propagates instead of restarting. It advances *instantly* (no backoff) — pair with a backoff middleware if you want delay.
+
+## `agent.eval(options)` + `ai.eval.*` scorers — evaluate an agent
+
+Run a suite of cases through `agent.execute()` and score each.
+
+```ts
+const report = await myAgent.eval({
+  cases: [
+    { name: "capital", input: "Capital of Egypt?", expected: "Cairo" },
+    { name: "tone", input: "Comfort an upset user." },          // judge-scored
+  ],
+  scorers: [ai.eval.contains()],                                 // default scorers for cases w/o their own
+  judge: { agent: judgeAgent, rubric: "Score 1.0 only if empathetic." },  // LLM-as-judge fallback
+  passThreshold: 0.5,                                            // default
+  onFailure: (caseResult) => snapshot(caseResult),
+});
+
+expect(report.passed).toBe(true);   // true only when EVERY case passed
+report.passRate; report.meanScore; report.cases;   // drill-down
+```
+
+Built-in scorers on `ai.eval.*`: `exact()` (trimmed, case-insensitive; structured compared by canonical JSON), `contains()` (substring), `predicate(fn)` (arbitrary boolean assertion), `judge(config)` (LLM-as-judge). Scorer precedence per case: the case's own `scorers` → suite `scorers` → synthesized judge. A case with NONE throws at author time. A case passes only when the agent did not error AND every scorer passed.
+
+## Vitest matchers + `ai.mockRouter` — test report trees
+
+```ts
+import { registerAiMatchers } from "@warlock.js/ai";
+registerAiMatchers();   // once per test file (idempotent)
+
+expect(await supervisor.execute(input)).toRouteTo("critic");        // dispatched the named intent
+expect(await supervisor.execute(input)).toConverge();               // terminated cleanly on own decision
+expect(await workflow.execute(input)).toPassStep("draft");          // named step completed
+expect(await agent.execute(input, { output: schema })).toOutputShape(schema);  // data validates
+```
+
+The pure verdict functions (`matchConverge`, `matchOutputShape`, `matchPassStep`, `matchRouteTo`) and `AiMatchers` ship eagerly with no `vitest` coupling; only `registerAiMatchers` lazily imports `vitest` (a devDependency), so importing `@warlock.js/ai` in production never pulls in `vitest`.
+
+`ai.mockRouter(decisions, options?)` builds a deterministic `route` callback that replays a canned sequence — one decision per supervisor iteration — for testing supervisors without an LLM router:
+
+```ts
+import { END } from "@warlock.js/ai";
+
+ai.supervisor({
+  name: "draft-then-review",
+  intents: { writer, critic },
+  route: ai.mockRouter(["writer", "critic", END]),
+});
+
+// branch on state, repeat the last decision until done:
+ai.mockRouter(["research", (ctx) => (ctx.state.summary ? END : "research")], { onExhausted: "repeat" });
+```
+
+A decision is a literal `Next` (intent name / fan-out array / `END`) or a predicate over the live `RouteContext`. On exhaustion: `"end"` (default — terminate), `"throw"` (test failure), `"repeat"` (replay last). For a scripted LLM, use `MockSDK` (script the model output) and `mockAgent({ name, responses })` for fixed-response capabilities.
+
+## SLO / cost budget contracts — `ai.middleware.budget({ contract })`
+
+On top of the legacy `maxTokens` / `maxCostUSD` caps, declare a run-level SLO as data, with one global reaction:
+
+```ts
+const guard = ai.middleware.budget({
+  pricing: { "gpt-4o": { inputPer1K: 0.005, outputPer1K: 0.015 } },
+  contract: {
+    maxCostUSD: 0.05,
+    maxLatencyMs: 8_000,        // wall-clock from first execute.before to each trip.after
+    maxTokens: 40_000,
+    onViolation: "fallback",    // "abort" (default) hard-stops; "fallback" records a signal + continues
+    fallback: (violation) => routeToCheaperModel(violation.dimension),
+  },
+});
+```
+
+Every clause is optional (a contract with no caps is inert). `onViolation: "abort"` throws `BudgetExceededError` at the next trip boundary; `"fallback"` does NOT abort — it records a typed `BudgetContractViolation` and fires `fallback`, letting the run continue (the middleware can't itself swap models). A latency breach has no `BudgetUnit` — its numbers surface via the error's `context.dimension`. `maxCostUSD` still needs a `pricing` entry for the running model or it degrades silently.
+
+Read a recorded fallback signal in an outer middleware's `execute.after`:
+
+```ts
+import { readBudgetFallbackSignal } from "@warlock.js/ai";
+
+const signal = readBudgetFallbackSignal(ctx.state);   // pass the middleware name as 2nd arg if non-default
+if (signal?.dimension === "cost") await rerunOnCheaperModel();
+```
+
+## Supervisor-level middleware
+
+The `middleware: AgentMiddleware[]` array on `ai.supervisor({...})` fires each middleware's optional `supervisor` hook map (`before` / `after` / `onError`) ONCE around the entire `execute()` / `stream()` / `resume()` run — the supervisor-level peer of an agent's `execute`-level middleware.
+
+```ts
+ai.supervisor({ name: "support", router, intents, middleware: [auditTrail] });
+```
+
+Same onion semantics as the agent pipeline: `before` runs top-down (return a `SupervisorResult` to short-circuit, throw to abort), `after` / `onError` run bottom-up. A middleware without a `supervisor` hook map is skipped — so the same builtin objects (budget, guardrail, …) can be registered on agents AND on the supervisor, each declaring whichever level applies. Each needs a unique `name` (its `ctx.state` namespace).
+
+## `ai.systemPrompt.fromFile(path)`
+
+Build a system prompt by reading a file **once, synchronously, at construction** — the file's UTF-8 contents seed one instruction block, so placeholders inside resolve at `resolve()` time and the result forks with further `.persona()` / `.instruction()` calls.
+
+```ts
+const prompt = ai.systemPrompt.fromFile("./prompts/support-agent.md");
+const localized = prompt.instruction("Respond in {{language|English}}.");
+localized.resolve({ language: "Arabic" });
+```
+
+One-shot by design (never re-read on `resolve()`). Throws `InvalidRequestError` when the file can't be read — a typo in the path fails loudly at construction instead of producing an empty prompt. `SystemPrompt.fromFile(path)` and `ai.systemPrompt.fromFile(path)` are identical.
+
+## Auto-adapt executables in `tools: []`
+
+An agent's `tools` array accepts a raw executable primitive (`AgentContract` / `WorkflowInstance` / `SupervisorContract` / orchestrator) directly — it is auto-adapted into a `ToolContract` at factory time. The tool manifest is derived from the executable's `name` + `description` + (optional) `inputSchema`; dispatch flows through its `execute()`.
+
+```ts
+const support = ai.supervisor({ name: "support", inputSchema: v.object({ message: v.string() }), router, intents });
+
+const concierge = ai.agent({
+  model,
+  tools: [support, billingWorkflow, lookupTool],   // no .asTool() needed
+});
+```
+
+`.asTool()` still works and takes precedence when you need a custom name / schema per use. For a supervisor/orchestrator, declaring `inputSchema` on the config is what lets it drop straight into `tools: []`.
+
+## See also
+
+- [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — `agent.eval`, `tools: []`, the agent the helpers wrap
+- [`@warlock.js/ai/run-supervisor/SKILL.md`](@warlock.js/ai/run-supervisor/SKILL.md) — `ai.router` / `ai.fanOut` / supervisor `middleware` / `mockRouter`
+- [`@warlock.js/ai/attach-ai-middleware/SKILL.md`](@warlock.js/ai/attach-ai-middleware/SKILL.md) — budget / guardrail / semanticCache basics
+- [`@warlock.js/ai/write-system-prompt/SKILL.md`](@warlock.js/ai/write-system-prompt/SKILL.md) — `systemPrompt.fromFile` in context
+- [`@warlock.js/ai/pick-ai-provider/SKILL.md`](@warlock.js/ai/pick-ai-provider/SKILL.md) — `fallbackModel` wraps these adapters; cost-truth tokens

package/skills/attach-ai-middleware/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: attach-ai-middleware
-description: 'Wire agent middleware — ai.middleware.budget (token / USD caps), ai.middleware.guardrail (pre / post content checks), ai.middleware.semanticCache (exact + vector cache), plus authoring custom hooks (execute / trip / tool). Triggers: `ai.middleware.budget`, `ai.middleware.guardrail`, `ai.middleware.semanticCache`, `ai.middleware.compose`, `ai.middleware.forTool`, `AgentMiddleware`, `BudgetExceededError`, `GuardrailViolationError`; ''cap token cost'', ''block pii in prompts'', ''semantic cache before LLM'', ''write custom hook''; typical import `import { ai } from "@warlock.js/ai"`. Skip: agent lifecycle — `@warlock.js/ai/run-ai-agent/SKILL.md`; cache drivers — `@warlock.js/ai/persist-ai-data/SKILL.md`; competing libs `langchain` callbacks.'
+description: 'Wire agent middleware — ai.middleware.budget (token / USD caps + SLO/cost contract w/ maxLatencyMs + onViolation fallback), ai.middleware.guardrail (pre / post content checks), ai.middleware.semanticCache (exact + vector cache), supervisor-level middleware, plus authoring custom hooks (execute / trip / tool). Triggers: `ai.middleware.budget`, `ai.middleware.guardrail`, `ai.middleware.semanticCache`, `ai.middleware.compose`, `ai.middleware.forTool`, `AgentMiddleware`, `BudgetExceededError`, `GuardrailViolationError`, `BudgetContract`, `maxLatencyMs`, `onViolation`, `readBudgetFallbackSignal`, `supervisor middleware`; ''cap token cost'', ''SLO budget'', ''block pii in prompts'', ''semantic cache before LLM'', ''supervisor-level middleware'', ''write custom hook''; typical import `import { ai } from "@warlock.js/ai"`. Skip: agent lifecycle — `@warlock.js/ai/run-ai-agent/SKILL.md`; cache drivers — `@warlock.js/ai/persist-ai-data/SKILL.md`; competing libs `langchain` callbacks.'
 ---
 
 # Middleware — agent-level pipeline
@@ -53,6 +53,25 @@ Breach → `BudgetExceededError` on `result.error`. Inspect `error.unit` (`"toke
 
 USD only fires when both `maxCostUSD` AND a matching `pricing[modelName]` entry exist.
 
+### SLO / cost contract — `budget({ contract })`
+
+On top of the legacy caps, declare a run-level SLO as data with one global reaction. Adds a wall-clock `maxLatencyMs` dimension:
+
+```ts
+ai.middleware.budget({
+  pricing: { "gpt-4o": { inputPer1K: 0.005, outputPer1K: 0.015 } },
+  contract: {
+    maxTokens: 40_000,
+    maxCostUSD: 0.05,
+    maxLatencyMs: 8_000,        // wall-clock, first execute.before → each trip.after
+    onViolation: "fallback",    // "abort" (default) throws; "fallback" records a signal + continues
+    fallback: (violation) => routeToCheaperModel(violation.dimension),
+  },
+});
+```
+
+Every clause optional (no caps = inert). `"fallback"` can't itself swap models — it records a typed `BudgetContractViolation` and fires `fallback`; an outer layer reads it via `readBudgetFallbackSignal(ctx.state)` and degrades the next run. A latency breach has no `BudgetUnit` — read its numbers from the thrown error's `context.dimension`. Full coverage in [`@warlock.js/ai/ai-dx-helpers/SKILL.md`](@warlock.js/ai/ai-dx-helpers/SKILL.md).
+
 ## `ai.middleware.guardrail(options)`
 
 Pre / post content checks.
@@ -171,11 +190,21 @@ const scoped = ai.middleware.forTool(["paid_api", "expensive_db"], toolRateLimit
 - **`name` must be unique** across an agent's middleware array.
 - **Middleware state does NOT cross `agent.execute()` boundaries.** One execute → one fresh `ctx.state`.
 
-## Workflow + middleware — what works today (v1)
+## Supervisor-level middleware
+
+`ai.supervisor({ middleware: [...] })` fires each middleware's optional `supervisor` hook map (`before` / `after` / `onError`) ONCE around the whole `execute()` / `stream()` / `resume()` run — the supervisor-level peer of an agent's `execute`-level middleware:
+
+```ts
+ai.supervisor({ name: "support", router, intents, middleware: [auditTrail] });
+```
+
+Same onion semantics: `before` top-down (return a `SupervisorResult` to short-circuit, throw to abort), `after` / `onError` bottom-up. A middleware WITHOUT a `supervisor` hook map is skipped — so the SAME builtin objects (budget, guardrail, …) can be registered on agents AND on the supervisor, each declaring whichever level applies. Each needs a unique `name`. See [`@warlock.js/ai/ai-dx-helpers/SKILL.md`](@warlock.js/ai/ai-dx-helpers/SKILL.md).
+
+## Workflow + middleware — what works today
 
 - Inside a workflow step with `agent: myAgent` — the agent's own middleware fires normally.
 - `workflow.asTool()` called from an agent — the calling agent's `tool`-level middleware wraps the workflow.
-- Step-level / workflow-level / supervisor-level middleware does NOT exist yet.
+- Step-level / workflow-level middleware does NOT exist yet (supervisor-level DOES — see above).
 
 ## See also
 

package/skills/handle-ai-errors/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: handle-ai-errors
-description: 'Typed AIError hierarchy with stable code strings + coarse category for retry-policy dispatch. execute() never throws — errors surface via result.error. Triggers: `AIError`, `ProviderRateLimitError`, `ProviderAuthError`, `ContextLengthExceededError`, `ContentFilterError`, `SchemaValidationError`, `ToolExecutionError`, `WorkflowDriftError`, `BudgetExceededError`, `GuardrailViolationError`, `error.code`, `error.category`; ''handle ai error'', ''retry on rate limit'', ''branch on error code'', ''build fallback ladder''; typical import `import { AIError } from "@warlock.js/ai"`. Skip: log surfacing — `@warlock.js/ai/log-ai-calls/SKILL.md`; native `try / catch` on raw `openai`.'
+description: 'Typed AIError hierarchy with stable code strings + coarse category for retry-policy dispatch. execute() never throws — errors surface via result.error (the sole exception: OrchestratorConfigError throws at construction). Triggers: `AIError`, `ProviderRateLimitError`, `ProviderAuthError`, `ContextLengthExceededError`, `ContentFilterError`, `SchemaValidationError`, `ToolExecutionError`, `WorkflowDriftError`, `SupervisorDriftError`, `SupervisorFailedError`, `SupervisorRoutingError`, `OrchestratorFailedError`, `OrchestratorDriftError`, `OrchestratorConfigError`, `OrchestratorCancelledError`, `PlannerFailedError`, `PlannerPlanInvalidError`, `PlannerCancelledError`, `BudgetExceededError`, `GuardrailViolationError`, `error.code`, `error.category`; ''handle ai error'', ''retry on rate limit'', ''branch on error code'', ''ORCHESTRATOR_DRIFT'', ''PLANNER_PLAN_INVALID'', ''build fallback ladder''; typical import `import { AIError } from "@warlock.js/ai"`. Skip: log surfacing — `@warlock.js/ai/log-ai-calls/SKILL.md`; native `try / catch` on raw `openai`.'
 ---
 
 # Typed errors — `AIError` hierarchy
@@ -91,9 +91,16 @@ AIError  (base — code, category, message, cause?, context?)
 │   └── RoutingError             WORKFLOW_INVALID_GOTO      { stepName, targetName }
 ├── SupervisorFailedError        SUPERVISOR_FAILED  (base + authoring/runtime)
 │   ├── MaxIterationsError       SUPERVISOR_MAX_ITERATIONS  { maxIterations }
-│   ├── SupervisorRoutingError   SUPERVISOR_INVALID_ROUTE
+│   ├── SupervisorRoutingError   SUPERVISOR_INVALID_ROUTE   { returned, availableKeys }
 │   ├── SupervisorCancelledError SUPERVISOR_CANCELLED       { cancelledAt, reason }
 │   └── SupervisorDriftError     SUPERVISOR_DRIFT           { savedSignature, currentSignature, runId }
+├── OrchestratorFailedError      ORCHESTRATOR_FAILED  (base — durable-session turn)
+│   ├── OrchestratorConfigError    ORCHESTRATOR_CONFIG     authoring-time, THROWS  (validation) — bad ai.orchestrator(config)
+│   ├── OrchestratorDriftError     ORCHESTRATOR_DRIFT      { savedSignature, currentSignature, sessionId } (drift) — checkpoint ≠ definition
+│   └── OrchestratorCancelledError ORCHESTRATOR_CANCELLED  { cancelledAt, sessionId, reason } (cancelled) — mid-turn abort
+├── PlannerFailedError           PLANNER_FAILED  (base — plan generation/execution)
+│   ├── PlannerPlanInvalidError    PLANNER_PLAN_INVALID    (schema) — LLM plan unparseable or names an unregistered capability
+│   └── PlannerCancelledError      PLANNER_CANCELLED       { cancelledAt, reason } (cancelled) — mid-plan abort
 ├── ProviderError                PROVIDER_ERROR  (base + catch-all)
 │   ├── ProviderRateLimitError   PROVIDER_RATE_LIMIT        { retryAfter? } — transient
 │   ├── QuotaExceededError       PROVIDER_QUOTA_EXCEEDED    — NOT retryable (billing cap)
@@ -106,7 +113,9 @@ AIError  (base — code, category, message, cause?, context?)
 └── GuardrailViolationError      GUARDRAIL_VIOLATION        { phase, reason } — from ai.middleware.guardrail
 ```
 
-> `SupervisorFailedError` doubles as the base for the supervisor family **and** the authoring-time error for bad config (e.g. `route` + `router` both set). It carries extra `SUPERVISOR_INTENT_*` / `SUPERVISOR_DISPATCH_CYCLE` codes for specific intent-validation failures.
+> `SupervisorFailedError` doubles as the base for the supervisor family **and** the authoring-time error for bad config (e.g. `route` + `router` both set). It carries extra `SUPERVISOR_INTENT_*` / `SUPERVISOR_DISPATCH_CYCLE` codes for specific intent-validation failures (`SUPERVISOR_INTENT_DESCRIPTION_REQUIRED`, `SUPERVISOR_INTENT_MIXED_DISPATCH`, `SUPERVISOR_INTENT_STREAM_AND_OUTPUT`, `SUPERVISOR_INTENT_STREAM_TO_REQUIRED`, `SUPERVISOR_INTENT_STREAM_ON_WORKFLOW`, `SUPERVISOR_DISPATCH_CYCLE`).
+
+> **Orchestrator + planner families** anchor on `OrchestratorFailedError` / `PlannerFailedError` (the `ORCHESTRATOR_*` / `PLANNER_*` code families). Both follow the never-throw rule: `orchestrator.execute()` / `resume()` / `command()` and `planner.execute()` surface failures on `result.error` with `report.status` `"failed"` / `"cancelled"`. The **one exception** is `OrchestratorConfigError` (`ORCHESTRATOR_CONFIG`) — an authoring-time misconfiguration (`iterate: true` with no resolvable `snapshotStore`, no `checkpointStore`, both `route` and `router` set, `initialAgent` absent from `intents`) that **throws synchronously at construction** so a bad definition fails fast at boot. Child-execution errors (agent / tool / provider / supervisor / workflow) flow through both primitives **unchanged** — captured on the step / turn report and surfaced on `result.error` directly, never re-wrapped into a `PLANNER_*` / `ORCHESTRATOR_*` code. On an `iterate: true` mid-turn cancel the underlying `SupervisorCancelledError` rides on `OrchestratorCancelledError.cause`.
 
 ## Error fields
 
@@ -131,9 +140,11 @@ Typed fields (`retryAfter`, `toolName`, `issues`, `stepName`, …) are first-cla
 | `ContentFilterError` | Usually **no** — the prompt itself is the issue |
 | `SchemaValidationError` | Use agent `repair: { maxAttempts }` instead |
 | `ToolExecutionError` | Depends on `cause` |
-| `WorkflowDriftError` | **No** — manual migration or `force: true` |
-| `WorkflowCancelledError` | **No** — caller-driven cancel |
-| `MaxStepsExceededError` / `RoutingError` | **No** — programmer error |
+| `WorkflowDriftError` / `SupervisorDriftError` / `OrchestratorDriftError` | **No** — manual migration or `force: true` |
+| `WorkflowCancelledError` / `SupervisorCancelledError` / `OrchestratorCancelledError` / `PlannerCancelledError` | **No** — caller-driven cancel |
+| `MaxStepsExceededError` / `RoutingError` / `SupervisorRoutingError` | **No** — programmer error |
+| `OrchestratorConfigError` | **No** — authoring-time config bug; thrown at construction |
+| `PlannerPlanInvalidError` | **No** — bad LLM plan / unregistered capability; re-prompt or fix the capability roster |
 | `BudgetExceededError` | **No** — raise the cap, split the workload |
 | `GuardrailViolationError` (`phase: "input"`) | **No** — block / sanitize at product layer |
 | `GuardrailViolationError` (`phase: "output"`) | Sometimes — re-prompt with adjusted system message |
@@ -189,6 +200,9 @@ async function runWithFallbacks(input: string) {
 
 - [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — `AgentResult.error`
 - [`@warlock.js/ai/run-ai-workflow/SKILL.md`](@warlock.js/ai/run-ai-workflow/SKILL.md) — `WorkflowError` subclasses
+- [`@warlock.js/ai/run-supervisor/SKILL.md`](@warlock.js/ai/run-supervisor/SKILL.md) — `SupervisorFailedError` family + intent-validation codes
+- [`@warlock.js/ai/run-orchestrator/SKILL.md`](@warlock.js/ai/run-orchestrator/SKILL.md) — `OrchestratorFailedError` family + `ORCHESTRATOR_CONFIG` boot-time throw
+- [`@warlock.js/ai/run-planner/SKILL.md`](@warlock.js/ai/run-planner/SKILL.md) — `PlannerFailedError` family + `PLANNER_PLAN_INVALID`
 - [`@warlock.js/ai/define-ai-tool/SKILL.md`](@warlock.js/ai/define-ai-tool/SKILL.md) — `ToolExecutionError` wrapping
 - [`@warlock.js/ai/log-ai-calls/SKILL.md`](@warlock.js/ai/log-ai-calls/SKILL.md) — error logging
 - `domains/ai/conventions/errors.md` — framework vs app error convention