@warlock.js/ai 4.3.0 → 4.5.0

package/skills/observe-ai-flows/SKILL.md

@@ -0,0 +1,94 @@
+---
+name: observe-ai-flows
+description: 'The core Observer seam — a generic, tool-agnostic observability hook every flow routes its completed ExecutionReport through. Covers the per-flow `observe?: boolean | Observer` option on ai.agent / workflow / supervisor / team, the global registry (registerObserver / getObservers / setObserveAll / isObserveAll / clearObservers), resolveObservers / notifyObservers resolution, the opt-in AgentConfig.captureMessages → AgentReport.messages full-history capture, the onConfigApplied dependency-inversion seam, and that @warlock.js/ai-panoptic is the batteries-included Observer. Triggers: `Observer`, `observe`, `registerObserver`, `getObservers`, `setObserveAll`, `isObserveAll`, `clearObservers`, `resolveObservers`, `notifyObservers`, `FlowObserveOption`, `ExecutionReport`, `captureMessages`, `AgentReport.messages`, `CapturedMessage`, `onConfigApplied`, `observeAll`; ''observe an agent run'', ''send finished reports to a collector'', ''capture the full message history'', ''observe every flow by default'', ''wire panoptic / tracing''; typical import `import { ai, registerObserver } from "@warlock.js/ai"`. Skip: structured logging of events — `@warlock.js/ai/log-ai-calls/SKILL.md`; reading the report tree shape (trips / children) — `@warlock.js/ai/run-ai-agent/SKILL.md`; per-call cost / usage rollup — `@warlock.js/ai/handle-ai-errors/SKILL.md`. The batteries-included Observer is the `@warlock.js/ai-panoptic` package.'
+---
+
+# The `Observer` seam — generic, tool-agnostic observability
+
+Core defines a structural `Observer` and a tiny registry; it never imports any observability package (panoptic, OTel, Langfuse, …). A flow that resolves to "observed" hands its completed `ExecutionReport` to every registered observer. An observability tool **implements `Observer` and registers itself**, so `observe: true` / observe-all route reports without coupling core to the tool — the dependency inversion that keeps the two sides decoupled.
+
+```ts
+export interface Observer {
+  collect(report: ExecutionReport): void | Promise<void>;
+}
+```
+
+`collect` may be sync or async — the flow awaits it. A throw is **swallowed** by the flow (never breaks the run), mirroring the existing `onUsage` / `onComplete` hooks.
+
+## Per-flow `observe` option
+
+`observe?: boolean | Observer` (`FlowObserveOption`) is accepted on **`ai.agent`, `ai.workflow`, `ai.supervisor`, and `ai.team`** (a team forwards it verbatim to the supervisor it desugars into):
+
+```ts
+const collector: Observer = { collect(report) { exporter.send(report); } };
+
+ai.agent({ model, observe: true });       // → the globally registered observers, even if observe-all is off
+ai.agent({ model, observe: false });      // → opt out entirely, even when observe-all is on
+ai.agent({ model, observe: collector });  // → a flow-LOCAL collector; only this flow's report, only to it
+ai.agent({ model });                      // → undefined: follow the global observe-all flag
+```
+
+Resolution (`resolveObservers(observe)`):
+
+- `false` → `[]` (opted out).
+- `true` → the globally registered observers.
+- an `Observer` object → just that one (flow-local; the global observers are skipped).
+- `undefined` → the global observers when observe-all is on, otherwise `[]`.
+
+`notifyObservers(observe, report)` routes a completed report to each resolved observer, awaiting each `collect` (so async exporters finish before the flow returns) and swallowing any throw. The object form is typed as the structural `Observer` (NOT a panoptic-specific type), so a panoptic flow-local collector — which implements `Observer` — can be passed directly.
+
+## The global registry
+
+```ts
+import {
+  registerObserver, getObservers, setObserveAll, isObserveAll, clearObservers,
+} from "@warlock.js/ai";
+
+registerObserver(collector);  // an observability tool registers ONE collector when its config is applied
+getObservers();               // read-only snapshot of the registered observers (do not mutate)
+
+setObserveAll(true);          // "observe every flow by default" — flows without their own `observe` get observed
+isObserveAll();               // read the flag (default false — opt-in observability)
+
+clearObservers();             // test-only: reset observers + the observe-all flag for spec isolation
+```
+
+`observeAll` defaults to `false` (opt-in). A flow that never sets `observe` is observed **only** when observe-all is on; individual flows still opt out with `observe: false`.
+
+## Full-history capture — `captureMessages` → `AgentReport.messages`
+
+Off by default. When `ai.agent({ captureMessages: true })` is set, the agent normalizes the real assembled turn array onto `AgentReport.messages` as a `CapturedMessage[]`:
+
+```ts
+const { report } = await ai.agent({ model, tools, captureMessages: true }).execute("Go");
+report.messages; // CapturedMessage[] — every role (system/user/assistant/tool), every trip
+```
+
+A `CapturedMessage` is a JSON-safe projection: `{ role, content, toolCalls?, toolCallId? }` — `content` is always a string (tool results stringified), assistant turns that triggered tools carry `toolCalls`, tool-result turns carry the `toolCallId` they answer. Unlike `trips[].input` (which stubs non-first trips with `"[tool results]"`), this preserves the **real** turn array. Omitted ⇒ the field is **absent** and the report is byte-for-byte as before. Opt-in because messages can be large and sensitive (full prompts, tool inputs/outputs) — and **required for panoptic full-history capture**.
+
+## Callback sub-agents nest in the report tree
+
+The `ExecutionReport` an observer receives reflects **full** lineage: a supervisor / team / orchestrator callback that calls `agent.execute()` directly auto-nests `callback → agent → tool` (via an ambient `RunFrame`), so usage / cost roll up and panoptic renders the sub-agent under its callback instead of as a lone `$0` span. No observer-side change is needed — the tree arrives already nested. A team's root span carries `type: "team"` (a first-class `ReportType`, not `"supervisor"`), so observers can distinguish, group, and label team runs as their own type. See [`@warlock.js/ai/run-supervisor/SKILL.md`](@warlock.js/ai/run-supervisor/SKILL.md).
+
+## The config seam — `onConfigApplied`
+
+An observability tool reacts to its own augmented config slot without core importing it. Core lets tools attach an opaque slot (e.g. `panoptic?`) via declaration merging on `AIConfig`, then fires registered listeners after each `ai.config(...)` merge:
+
+```ts
+import { onConfigApplied, getAIConfig } from "@warlock.js/ai";
+
+onConfigApplied((config) => applyPanopticConfig(config.panoptic)); // react on every config merge
+applyPanopticConfig(getAIConfig().panoptic);                        // catch a pre-set config
+```
+
+A misbehaving listener's throw is swallowed (same swallow-on-throw discipline as the observer hooks). This mirrors the `Observer` registry's dependency inversion: a tool flips `setObserveAll(true)` and calls `registerObserver(...)` from inside its `onConfigApplied` listener.
+
+## The batteries-included Observer
+
+`@warlock.js/ai-panoptic` is the shipped, full-featured `Observer` — install it, configure it via `ai.config({ panoptic })`, and it registers its collector + (optionally) flips observe-all for you. Core stays dependency-free; this skill documents the seam panoptic plugs into.
+
+## See also
+
+- [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — the `AgentReport` / `ExecutionReport` tree (`trips`, `children`) an observer receives
+- [`@warlock.js/ai/log-ai-calls/SKILL.md`](@warlock.js/ai/log-ai-calls/SKILL.md) — event-level structured logging (vs. report-level observation)
+- [`@warlock.js/ai/run-ai-team/SKILL.md`](@warlock.js/ai/run-ai-team/SKILL.md) — a team inherits `observe` through the supervisor it forwards to

package/skills/record-replay-llm/SKILL.md

@@ -0,0 +1,92 @@
+---
+name: record-replay-llm
+description: 'Deterministic, offline LLM tests with ai.vcr(model,{path,mode}) — a record/replay decorator over ANY ModelContract that intercepts only complete()/stream(), delegates name/provider/capabilities/pricing to the inner model, and hashes each request against a JSON cassette on disk. Covers the three modes (record / replay / auto), the cassette format, save(), VcrCassetteMissError, streaming round-trip, hashOptions, and composing below fallbackModel. Triggers: `ai.vcr`, `vcr`, `VcrModel`, `VcrOptions`, `VcrMode`, `Cassette`, `CassetteEntry`, `VcrCassetteMissError`, `hashRequest`, `DEFAULT_HASH_OPTIONS`, `mode`, `path`, `hashOptions`, `save`, `cassette`, record, replay, cassette; ''record LLM responses for tests'', ''replay model calls offline in CI'', ''deterministic agent test without hitting the provider'', ''cassette for model calls''; typical import `import { ai } from "@warlock.js/ai"`. Skip: eval scoring + regression gating — `@warlock.js/ai/eval-datasets-and-ci/SKILL.md`; the Vitest matchers + mockRouter — `@warlock.js/ai/ai-dx-helpers/SKILL.md`; choosing a provider adapter — `@warlock.js/ai/pick-ai-provider/SKILL.md`; competing libs `nock`, `polly.js`.'
+---
+
+# `ai.vcr()` — record / replay any model
+
+`ai.vcr(model, { path, mode })` wraps any `ModelContract` in a record/replay decorator backed by a JSON cassette on disk. It intercepts only `complete()` / `stream()` — the single seam every agent trip funnels through — and delegates `name`, `provider`, `capabilities`, and `pricing` to the inner model untouched, so cost accounting and capability detection are unchanged. Depends only on `ModelContract`, so it works with **any** adapter.
+
+## Shape
+
+```ts
+import { ai } from "@warlock.js/ai";
+import { OpenAISDK } from "@warlock.js/ai-openai";
+
+const openai = new OpenAISDK({ apiKey: process.env.OPENAI_API_KEY! });
+const live = openai.model({ name: "gpt-4o-mini" });
+
+const model = ai.vcr(live, {
+  path: "./cassettes/support.json",  // cassette file (JSON); read on construct, written on save()
+  mode: "auto",                      // "record" | "replay" | "auto" (default)
+});
+
+const agent = ai.agent({ model, systemPrompt: "..." });
+const result = await agent.execute("hi");
+
+await model.save(); // first run records; later runs replay deterministically
+```
+
+The returned `VcrModel` is a `ModelContract` plus `save(): Promise<void>` and a readonly `cassette` (exposed for assertions).
+
+## Modes — `VcrMode`
+
+- **`record`** — always calls the inner model and appends a cassette entry. Never replays (the in-memory cassette starts empty, so a record run never accidentally replays a stale entry). Use to (re)capture a fresh cassette.
+- **`replay`** — never calls the inner model. A cassette hit returns the stored response / re-yields its chunks / re-throws its error; a **miss throws `VcrCassetteMissError`** — never a silent live call. Use in CI for deterministic, offline tests.
+- **`auto`** (default) — replay on a hit, record on a miss. The friendliest mode for local dev: records once, replays thereafter.
+
+## Request hashing — `hashOptions`
+
+On each call VCR computes a stable hash over `{ messages, picked options }` and looks for a matching `CassetteEntry`. The hashed option fields default to:
+
+```ts
+["temperature", "maxTokens", "responseSchema", "tools", "reasoning"]
+```
+
+(`DEFAULT_HASH_OPTIONS`). `signal` and unknown provider keys are **always excluded**, so an otherwise-identical logical call still matches. `tools` are hashed by name + description + input-schema shape, not object identity. Override `hashOptions` to widen / narrow what counts as "the same request". `hashRequest(messages, options, hashOptions)` is exported for direct use.
+
+## Cassette format
+
+A `Cassette` is `{ version: 1, model, provider, entries: CassetteEntry[] }`. Each `CassetteEntry` is `{ requestHash, request: { messages, options? }, ... }` where **exactly one** of `response` / `chunks` / `error` is populated — mirroring the three outcomes of a model call (non-streaming reply, streamed chunk list, or a thrown provider error). The full `request` is stored verbatim for human readability and so the cassette can be re-hashed if the hashing format ever changes.
+
+## Streaming round-trip
+
+```ts
+for await (const chunk of model.stream(messages)) {
+  // record mode: buffers each chunk into entry.chunks[] while re-emitting
+  // replay mode: re-yields the stored chunks in order (delta / tool-call / done sequence)
+}
+await model.save();
+```
+
+Recorded chunks reproduce the exact `delta` / `tool-call` / `done` sequence on replay; a recorded error is re-thrown.
+
+## `VcrCassetteMissError`
+
+```ts
+import { VcrCassetteMissError } from "@warlock.js/ai";
+
+try {
+  await vcrModel.complete(messages);
+} catch (error) {
+  if (error instanceof VcrCassetteMissError) {
+    console.error("Re-record the cassette:", error.path, error.requestHash);
+  }
+}
+```
+
+Thrown only in `replay` mode on a miss (code `"VCR_CASSETTE_MISS"`). It carries the looked-up `requestHash` and the cassette `path` so a failing CI run names exactly which call was not recorded. Extends `AIError` directly (not `ProviderError`) — a miss is a harness/config failure, not a provider failure. **The whole point:** `replay` never falls back to a live call, which would silently re-introduce non-determinism into a test that asked for the opposite. Re-record by running once in `record` / `auto`.
+
+## `save()` — flush new entries
+
+`save()` writes newly recorded entries to `path`. It is a **no-op when nothing was recorded** (pure replay, or a record/auto run that only hit cached entries), so calling it unconditionally is safe.
+
+## Composition
+
+VCR composes **below** `ai.fallbackModel` and works with any adapter. Wrap the live model in `vcr(...)`, then pass it anywhere a `ModelContract` is accepted (agent, planner, reranker, judge).
+
+## See also
+
+- [`@warlock.js/ai/eval-datasets-and-ci/SKILL.md`](@warlock.js/ai/eval-datasets-and-ci/SKILL.md) — pair a cassette with a dataset for fully offline eval CI
+- [`@warlock.js/ai/ai-dx-helpers/SKILL.md`](@warlock.js/ai/ai-dx-helpers/SKILL.md) — `ai.mockRouter` + Vitest matchers for the rest of the test surface
+- [`@warlock.js/ai/pick-ai-provider/SKILL.md`](@warlock.js/ai/pick-ai-provider/SKILL.md) — the adapters whose models VCR wraps

package/skills/run-ai-agent/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: run-ai-agent
-description: 'Build agents with ai.agent({...}) — the single-LLM-turn primitive. Covers execute / stream, attachments, structured output, placeholders, events, agent.eval scoring, and auto-adapting raw executables in tools:[]. Triggers: `ai.agent`, `agent.execute`, `agent.stream`, `agent.eval`, `AgentResult`, `AgentReport`, `AgentToolEntry`, `streamingToolGuard`, `attachments`, `repair`, `maxTrips`, `sessionId`, `spawnSubAgent`, `SpawnSubAgentSpec`; ''run an agent'', ''stream an agent response'', ''structured output schema'', ''pass image to agent'', ''evaluate an agent'', ''put a supervisor in tools'', ''cancel an agent run'', ''spawn a one-shot sub-agent with a per-task budget''; typical import `import { ai } from "@warlock.js/ai"`. Skip: tool definition — `@warlock.js/ai/define-ai-tool/SKILL.md`; workflows — `@warlock.js/ai/run-ai-workflow/SKILL.md`; eval matchers / batch / fallback detail — `@warlock.js/ai/ai-dx-helpers/SKILL.md`; competing libs `langchain`, `ai` (Vercel), raw `openai`.'
+description: 'Build agents with ai.agent({...}) — the single-LLM-turn primitive. Covers execute / stream, attachments, structured output, placeholders, events, agent.eval scoring, the judge-safe preset for resilient LLM-as-judge / verdict classifiers (ai.agent.judge / judge: true — lenient JSON parse + repair + never-throw, for Nova-class models), and auto-adapting raw executables in tools:[]. Triggers: `ai.agent`, `ai.agent.judge`, `agent.execute`, `agent.stream`, `agent.eval`, `AgentResult`, `AgentReport`, `AgentToolEntry`, `JudgeConfig`, `JudgeAgentConfig`, `judge`, `repairAttempts`, `streamingToolGuard`, `attachments`, `repair`, `maxTrips`, `sessionId`, `spawnSubAgent`, `SpawnSubAgentSpec`; ''run an agent'', ''stream an agent response'', ''structured output schema'', ''pass image to agent'', ''evaluate an agent'', ''LLM-as-judge that survives malformed JSON'', ''grade with a Nova model without crashing'', ''put a supervisor in tools'', ''cancel an agent run'', ''spawn a one-shot sub-agent with a per-task budget''; typical import `import { ai } from "@warlock.js/ai"`. Skip: tool definition — `@warlock.js/ai/define-ai-tool/SKILL.md`; workflows — `@warlock.js/ai/run-ai-workflow/SKILL.md`; eval matchers / batch / fallback detail — `@warlock.js/ai/ai-dx-helpers/SKILL.md`; competing libs `langchain`, `ai` (Vercel), raw `openai`.'
 ---
 
 # `ai.agent()` — single-turn primitive
@@ -183,6 +183,31 @@ await myAgent.execute(input, {
 
 Disabled by default. Each repair attempt counts against `maxTrips`.
 
+## `judge` preset — resilient LLM-as-judge / verdict classifiers
+
+For graders and verdict classifiers running on models that emit **corrupted** structured output — notably the Amazon Nova family, which wraps verdicts in fenced ` ```json ` blocks, prepends prose, or trails commentary — set `judge: true` (or a `JudgeConfig`). It turns on three behaviors at once:
+
+```ts
+const grader = ai.agent.judge({
+  model: nova.model({ name: "amazon.nova-pro-v1:0" }),
+  systemPrompt: "Grade the answer. Respond with JSON only.",
+  output: verdictSchema,
+});
+
+const result = await grader.execute(prompt);
+if (result.error) {
+  // graceful default — the judge couldn't produce a clean verdict
+}
+```
+
+1. **Repair** — a couple of re-ask attempts by default (`repairAttempts`, defaults to `2`; bounded by `maxTrips`) when the verdict fails to parse / validate. The caller's per-call `options.repair` still wins.
+2. **Lenient verdict parsing** — extracts the first balanced JSON object / array (tolerating fenced blocks + surrounding prose) instead of the strict parser.
+3. **Never throws on a parse miss** — even an unparseable verdict yields a well-formed result (`result.error` populated, `result.data` undefined), so a flaky judge degrades instead of crashing the flow.
+
+`ai.agent.judge(config, judge?)` is sugar for `ai.agent({ ...config, judge })`; the bare `ai.agent({ judge: true })` option does the same. `judge: {}` ≡ `judge: true` (every field falls back to its resilient default); `judge: { repairAttempts: 0 }` keeps the lenient parser + never-throw guarantee but disables repair.
+
+**Trade-off — resilience over strictness.** The lenient parse can recover JSON the strict parser would (correctly) reject — leave `judge` **off** for normal structured output, where a hard parse failure is a useful signal. Off by default; omitting it parses strictly and never auto-enables repair, byte-for-byte as before. (This is the same Nova-safe judge the unified prompt `validate()` uses — see [`@warlock.js/ai/manage-prompts/SKILL.md`](@warlock.js/ai/manage-prompts/SKILL.md).)
+
 ## Pattern — image attachments
 
 ```ts

package/skills/run-ai-rag/SKILL.md

@@ -0,0 +1,139 @@
+---
+name: run-ai-rag
+description: 'Retrieval-augmented generation with ai.rag({...}) — a chunk → embed → vector-store → retrieve → rerank → cite pipeline that reuses ai.embedder + a @warlock.js/cache CacheDriver. Covers index() / retrieve() / clear() / asTool(), chunking strategies (recursive | markdown | sentence | fixed), Citation / RetrievedChunk provenance, and the opt-in rerankers ai.rag.keywordReranker / ai.rag.llmReranker. Triggers: `ai.rag`, `rag.index`, `rag.retrieve`, `rag.clear`, `rag.asTool`, `RagConfig`, `RagDocument`, `RetrieveOptions`, `RetrieveResult`, `RetrievedChunk`, `Citation`, `ChunkOptions`, `ChunkType`, `ai.rag.keywordReranker`, `ai.rag.llmReranker`, `cacheVectorStore`, `VectorStore`, `topK`, `threshold`, `candidates`; ''build a knowledge base'', ''retrieve relevant chunks for a query'', ''cite the source of an answer'', ''chunk markdown for embedding'', ''rerank retrieval results'', ''expose retrieval as a tool''; typical import `import { ai } from "@warlock.js/ai"`. Skip: raw single-string embedding — `@warlock.js/ai/embed-text/SKILL.md`; exact + vector LLM-response cache — `@warlock.js/ai/attach-ai-middleware/SKILL.md` (ai.middleware.semanticCache); tool wiring — `@warlock.js/ai/define-ai-tool/SKILL.md`; competing libs `langchain`, `llamaindex`.'
+---
+
+# `ai.rag()` — chunk → embed → retrieve → rerank → cite
+
+A self-contained retrieval pipeline. It reuses the embedder you already have (`provider.embedder(...)`), a `@warlock.js/cache` vector-capable `CacheDriver` as the store, and the composite-as-tool engine for `asTool()`. Zero new dependencies. `ai.rag` is a native core verb — present the moment `@warlock.js/ai` is imported (no module augmentation, no side-effect import).
+
+## Factory shape
+
+```ts
+import { ai } from "@warlock.js/ai";
+import { MemoryCacheDriver } from "@warlock.js/cache";
+import { OpenAISDK } from "@warlock.js/ai-openai";
+
+const openai = new OpenAISDK({ apiKey: process.env.OPENAI_API_KEY! });
+
+const kb = ai.rag({
+  name: "docs",                                            // default "rag"
+  embedder: openai.embedder({ name: "text-embedding-3-small" }), // REQUIRED
+  store: new MemoryCacheDriver(),                          // or ai.config({ defaultStore })
+  namespace: "ai.rag.docs",                                // default `ai.rag.<name>`
+  chunk: { type: "markdown", size: 800, overlap: 120 },    // index() defaults
+  reranker: ai.rag.keywordReranker(),                      // OFF by default (cosine-only)
+  retrieve: { topK: 4, threshold: 0.5 },                   // default retrieval knobs
+});
+```
+
+Resolution is **loud at construction** (mirrors `ai.memory`):
+
+- `embedder` is **required** — a provider with no embedder must be caught here, not at first `index()`.
+- `store` falls back to `ai.config({ defaultStore })`; if neither resolves, the factory throws.
+
+## Surface — `Rag`
+
+```ts
+interface Rag {
+  readonly name: string;
+  index(docs: RagDocument[], chunk?: ChunkOptions): Promise<{ chunks: number }>;
+  retrieve(query: string, options?: RetrieveOptions): Promise<RetrieveResult>;
+  clear(): Promise<void>;
+  asTool(options?: RagAsToolOptions): ToolContract<{ query: string }, RetrieveResult>;
+}
+```
+
+## `index()` — chunk, embed (batched), store
+
+```ts
+await kb.index([
+  { id: "guide", text: longMarkdown, metadata: { url: "/guide" }, tags: ["frontend"] },
+  { id: "faq", text: faqText },
+]);
+```
+
+A `RagDocument` is `{ id, text, metadata?, tags? }` — **you** load + parse documents to text (loaders are out of scope for v1). Each doc is split into chunks, embedded in sub-batches of 96 texts per `embedMany()` call (so one giant doc never blows the provider's per-request cap), and upserted. Returns the chunk count written. Empty / whitespace-only documents yield zero chunks — nothing is embedded. The per-call `chunk` arg overrides `config.chunk` for that index.
+
+## Chunking — `ChunkOptions`
+
+All sizing is in **characters** (tokenizer-free; the embedder owns token counting). `chunk(text, options)` is also exported standalone.
+
+```ts
+type ChunkType = "recursive" | "sentence" | "fixed" | "markdown";
+
+{
+  type?: ChunkType,          // default "recursive"
+  size?: number,             // target chars per chunk, default 1000
+  overlap?: number,          // chars carried between adjacent chunks, default 200
+  separators?: string[],     // recursive only; default ["\n\n", "\n", ". ", " ", ""]
+}
+```
+
+- **`recursive`** (default) — separator-aware greedy packing, largest unit first.
+- **`markdown`** — heading/section-aware, then recursive within each section.
+- **`sentence`** — packs whole sentences up to `size`.
+- **`fixed`** — back-to-back character windows.
+
+Every chunk records its exact `[start, end)` span in the original text, so a `Citation.span` is precise.
+
+## `retrieve()` — embed query, fetch, rerank, slice, cite
+
+```ts
+const { query, chunks } = await kb.retrieve("how do I configure caching?", {
+  topK: 4,          // returned AFTER reranking. default 5
+  threshold: 0.5,   // cosine floor at the store stage. default 0.5
+  candidates: 16,   // pool fetched before rerank. default topK * 4 (clamped >= topK)
+  tags: ["frontend"], // restrict to chunks whose source had one of these tags
+});
+
+for (const hit of chunks) {
+  console.log(hit.score, hit.text);
+  console.log(hit.citation.sourceId, hit.citation.chunkIndex, hit.citation.span);
+}
+```
+
+`retrieve()` is **return-only** — it never auto-injects into a prompt. The caller formats the cited chunks (or uses `asTool()` for the agent loop). A `RetrievedChunk` carries `{ text, score, citation }`; the `Citation` is `{ sourceId, chunkIndex, span, score, metadata? }`. The reranker is **OFF by default** (cosine ranking only) unless `config.reranker` is set.
+
+## Rerankers — opt-in, on `ai.rag.*`
+
+Both are exposed as namespaced helpers on the factory (`ai.rag.keywordReranker`, `ai.rag.llmReranker`).
+
+```ts
+// Zero-dependency lexical reranker (BM25-lite keyword overlap).
+ai.rag.keywordReranker({ weight: 0.5 }); // weight in [0,1]; 1 = pure keyword, 0 = keep cosine
+
+// Model-backed reranker — one or more model calls per retrieval.
+ai.rag.llmReranker({ model: openai.model({ name: "gpt-4o-mini" }), batchSize: 10 });
+```
+
+- **`keywordReranker`** — blends lexical query-term overlap with the original cosine score by `weight`; ties keep the incoming cosine order. Costs nothing beyond string splits. Reach for it when embedding-only ranking buries a keyword-rich chunk.
+- **`llmReranker`** — asks an LLM to grade each over-fetched candidate `0..1` and sorts by that. Candidates the model fails to score keep their cosine score, so a garbled reply degrades gracefully. Opt in only when precision beats latency/cost. Both implement the `RagReranker` contract, so you can write your own.
+
+## `asTool()` — drop retrieval into an agent's `tools: []`
+
+```ts
+const agent = ai.agent({
+  model: openai.model({ name: "gpt-4o" }),
+  tools: [kb.asTool({ name: "search_docs", retrieve: { topK: 6 } })],
+});
+```
+
+Input is `{ query: string }`; output is the `RetrieveResult`. Default tool name is `retrieve_<rag.name>`; `description` and a per-tool `retrieve` override are optional. Built via the same composite-as-tool engine every other primitive uses.
+
+## `clear()`
+
+```ts
+await kb.clear(); // drops every entry written under this rag's namespace
+```
+
+## Advanced
+
+- `cacheVectorStore(driver)` + the `VectorStore` contract are exported for swapping in a custom store.
+- A stored chunk's namespaced key is `${namespace}.${sourceId}.${chunkIndex}`.
+
+## See also
+
+- [`@warlock.js/ai/embed-text/SKILL.md`](@warlock.js/ai/embed-text/SKILL.md) — the `sdk.embedder` primitive this consumes
+- [`@warlock.js/ai/define-ai-tool/SKILL.md`](@warlock.js/ai/define-ai-tool/SKILL.md) — what `asTool()` produces
+- [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — wiring the retrieval tool into an agent

package/skills/run-ai-team/SKILL.md

@@ -0,0 +1,107 @@
+---
+name: run-ai-team
+description: 'Manager-led multi-agent teams with ai.team({...}) — transparent sugar over ai.supervisor that maps a manager → route/router, members → intents, and a gate → evaluate, returning a REAL SupervisorContract (no new loop, no new contract). Covers the built-in gate strings "quality" (review-then-fix) and "verify" (test-then-fix), a custom gate function, role mapping (roles / gateKey), and the verbatim supervisor pass-throughs (goal / output / state / maxIterations / snapshotStore / on / observe). Triggers: `ai.team`, `TeamConfig`, `TeamGate`, `TeamGateFn`, `TeamMemberValue`, `manager`, `members`, `gate`, `roles`, `gateKey`, `buildQualityGate`, `buildVerifyGate`, `SupervisorContract`, `ReportType`; ''build a team of agents'', ''manager that delegates to members'', ''review then fix loop'', ''test then fix loop'', ''quality gate for a multi-agent run'', ''report type team''; typical import `import { ai } from "@warlock.js/ai"`. Skip: routing one input to a fixed roster directly — `@warlock.js/ai/run-supervisor/SKILL.md` (team is sugar over it); durable cross-turn sessions — `@warlock.js/ai/run-orchestrator/SKILL.md`; LLM-generated plans — `@warlock.js/ai/run-planner/SKILL.md`; competing libs `crewai`, `autogen`.'
+---
+
+# `ai.team()` — manager + members + a quality gate
+
+`ai.team(config)` is **thin, transparent sugar over `ai.supervisor`**. It builds a `SupervisorConfig` from the team-shaped config, calls `supervisor(...)`, and returns the **unchanged** `SupervisorContract<TOutput>` — the exact object `ai.supervisor` returns. So `ctx.intents.<member>.execute()`, `.asTool()`, `.resume()`, snapshots, and events all stay intact. `team()` owns no loop of its own.
+
+The mapping:
+
+| team field | becomes supervisor field |
+| --- | --- |
+| `manager` | `route` (deterministic `{ route }`) XOR `router` (an agent / `RouterEntry`) |
+| `members` | `intents` |
+| `gate` | `evaluate` |
+
+Everything else passes through 1:1 — the sole exception is the report/result `type`, which is stamped `"team"` (see [Pass-throughs](#pass-throughs-verbatim-supervisor-semantics)).
+
+## Shape
+
+```ts
+import { ai } from "@warlock.js/ai";
+import { v } from "@warlock.js/seal";
+
+const codeTeam = ai.team({
+  name: "code-team",
+  goal: "Ship a tested module that passes review.",
+  manager: techLeadRouter,                 // an agent / RouterEntry → router; or { route } → deterministic
+  members: { builder, reviewer, fixer },   // role-name → agent | workflow
+  gate: "quality",                         // "quality" | "verify" | (ctx) => EvaluateResult
+  output: v.object({ code: v.string() }),
+  maxIterations: 6,                        // default 10 (supervisor's)
+});
+
+const { data, report } = await codeTeam.execute("Build a debounce<T> utility.");
+```
+
+A `member` is an `AgentContract` or a `WorkflowInstance` (the `TeamMemberValue` union — the autocomplete-friendly common case; callback / full-entry intent shapes still work when forwarded). The keys are both the role names the manager routes to AND the keys `ctx.intents.<role>` exposes (the supervisor escape hatch is preserved).
+
+## The manager — `route` XOR `router`
+
+```ts
+// LLM-driven manager: an agent (or RouterEntry) → becomes SupervisorConfig.router
+manager: techLeadRouter
+
+// Deterministic manager: { route } → becomes SupervisorConfig.route
+manager: { route: (ctx) => (ctx.iteration === 0 ? "builder" : "reviewer") }
+```
+
+Exactly one form is forwarded — mutually exclusive, mirroring the supervisor's own `router` XOR `route` rule. A malformed manager surfaces the existing `SupervisorFailedError` downstream.
+
+## Gates — `"quality"` | `"verify"` | a function
+
+A `gate` string selects a pre-built `evaluate` strategy; both desugar to a concrete `evaluate` callback that leans entirely on the already-shipped `EvaluateResult` semantics (`satisfied` terminates, `reassignTo` re-dispatches the fixer, `feedback` threads forward) — **no new termination or loop code**.
+
+### `gate: "quality"` — review-then-fix
+
+After each iteration's members settle and merge into supervisor `state`, the gate reads `state.approved` (the `gateKey`, default `"approved"`). If truthy → `{ satisfied: true }`; otherwise → `{ reassignTo: "fixer", feedback: String(state.notes ?? "") }`. The reviewer's feedback (`state.notes`) threads into the next iteration.
+
+### `gate: "verify"` — test-then-fix
+
+Identical shape but keyed on the tester's pass/fail slice `state.passed` (default `gateKey`) rather than a subjective score. On failure it re-dispatches the fixer; there is no feedback channel for a pass/fail signal, so none is threaded.
+
+> The named member whose `output` schema writes the gate slice must produce a boolean into `gateKey`.
+
+### A custom gate (full escape hatch)
+
+```ts
+gate: (ctx) => {
+  if (ctx.state.score >= 0.9) return { satisfied: true };
+  return { reassignTo: "fixer", feedback: ctx.state.review };
+}
+```
+
+Supplying a `TeamGateFn` instead of a string opts out of the sugar entirely while keeping the rest of `team()`'s wiring — it forwards straight to `SupervisorConfig.evaluate` with zero wrapping.
+
+## Role mapping — `roles` + `gateKey`
+
+The string gates default to canonical role names. Override when your `members` keys differ:
+
+```ts
+ai.team({
+  name: "qa-team",
+  manager,
+  members: { author, critic, patcher },
+  gate: "quality",
+  roles: { reviewer: "critic", fixer: "patcher" }, // map gate roles → your member keys
+  gateKey: "ok",                                    // state slice the gate reads
+});
+```
+
+**Construction-time validation:** when the gate is a string, the resolved `fixer` (and, for `"quality"`, the `reviewer`) role is checked against `members`. A missing role throws an authoring-style `SupervisorFailedError` (`context: { authoring: true }`) immediately — rather than silently starving until `maxIterations`.
+
+## Pass-throughs (verbatim supervisor semantics)
+
+`goal`, `output`, `state`, `maxIterations`, `snapshotStore`, `on`, `observe`, and `version` are forwarded unchanged. Because the returned object IS a supervisor, observability rides the same generic `Observer` seam every other flow uses (see `observe-ai-flows`), and snapshot resume works exactly as on a bare supervisor.
+
+The one behavioural difference from a bare supervisor: a team stamps **`type: "team"`** on both its report (a first-class `ReportType`, was `"supervisor"`) and its result, so Panoptic and any `Observer` can distinguish, group, filter, and label team runs as their own type rather than folding them into plain supervisor runs. Everything else passes through 1:1.
+
+A member callback that calls `agent.execute()` **directly** still nests `member → agent → tool` under the member span with usage rolled up — the same ambient-`RunFrame` auto-nesting as a bare supervisor. See [`@warlock.js/ai/run-supervisor/SKILL.md`](@warlock.js/ai/run-supervisor/SKILL.md).
+
+## See also
+
+- [`@warlock.js/ai/run-supervisor/SKILL.md`](@warlock.js/ai/run-supervisor/SKILL.md) — the primitive team desugars into (intents, route/router, evaluate, ctx.intents)
+- [`@warlock.js/ai/run-orchestrator/SKILL.md`](@warlock.js/ai/run-orchestrator/SKILL.md) — wrap a team in durable cross-turn session state
+- [`@warlock.js/ai/observe-ai-flows/SKILL.md`](@warlock.js/ai/observe-ai-flows/SKILL.md) — the `observe` seam a team inherits

package/skills/run-orchestrator/SKILL.md

@@ -86,6 +86,8 @@ result.compaction;                // CompactionResult when a turn compacted (and
 
 `report.children[]` carries ONLY the current turn's dispatched primitive reports. Full session history lives on `report.turns[]` — a `children[]` walker will NOT reach prior turns (intentional). Child `supervisor.*` / `agent.*` events bubble up unmodified under their own identity.
 
+A turn callback that calls `agent.execute()` **directly** (not via `ctx.run` / `ctx.intents`) still nests `callback → agent → tool` inside the turn's report tree, with usage rolled up and the session's `sessionId` stamped onto the captured subtree — an ambient `RunFrame` handles the self-attach. See [`@warlock.js/ai/run-supervisor/SKILL.md`](@warlock.js/ai/run-supervisor/SKILL.md).
+
 **`awaiting-input` is the only non-terminal status across the unified result tree.** Code branching on `status === "completed"` MUST explicitly handle `"awaiting-input"` as a session-continues path, not a failure.
 
 ## `iterate` — single dispatch vs. internal supervisor

package/skills/run-planner/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: run-planner
-description: 'Goal-driven planning with ai.planner({...}) — an LLM GENERATES an ordered execution plan over your registered capabilities (agents / workflows / supervisors / tools), then the planner EXECUTES that plan step-by-step, threading each step output into the next, and returns the unified {data, report, usage, error} envelope with report.type "planner". A plan step may delegate via ai.spawnSubAgent({...}) — a GENERAL one-shot-agent helper (a fresh agent + optional per-task budget), covered fully in `@warlock.js/ai/run-ai-agent/SKILL.md`; it is not planner-specific and the planner engine does not require it. Triggers: `ai.planner`, `planner.execute`, `spawnSubAgent`, `PlannerConfig`, `PlannerCapability`, `PlannerResult`, `PlannerReport`, `PlannerPlan`, `PlannerStep`, `maxSteps`, `report.plan`, `report.executedSteps`; ''let the model plan the steps'', ''dynamic plan from a goal'', ''decompose a goal into capability calls'', ''spawn a sub-agent for a subtask''; typical import `import { ai } from "@warlock.js/ai"`. Skip: a FIXED known pipeline — `@warlock.js/ai/run-ai-workflow/SKILL.md`; routing one input to a specialist each turn — `@warlock.js/ai/run-supervisor/SKILL.md`; a single model + tools call — `@warlock.js/ai/run-ai-agent/SKILL.md`; competing libs `langgraph`, `crewai`.'
+description: 'Goal-driven planning with ai.planner({...}) — an LLM GENERATES an ordered execution plan over your registered capabilities (agents / workflows / supervisors / tools), then the planner EXECUTES it, threading each step output into the next, and returns the unified {data, report, usage, error} envelope with report.type "planner". Supports DAG scheduling (dag:true + maxConcurrency off dependsOn), adaptive re-planning (replan:{maxReplans} + the onStep continue/abort/replan directive), and plan-only / approval (mode:"plan-only" → status "awaiting-approval" → approvedPlan). A plan step may delegate via ai.spawnSubAgent({...}) — a GENERAL one-shot-agent helper covered in `@warlock.js/ai/run-ai-agent/SKILL.md`; it is not planner-specific. Triggers: `ai.planner`, `planner.execute`, `spawnSubAgent`, `PlannerConfig`, `PlannerCapability`, `PlannerResult`, `PlannerReport`, `PlannerPlan`, `PlannerStep`, `PlannerStepDirective`, `PlannerPlanInvalidError`, `maxSteps`, `dag`, `maxConcurrency`, `dependsOn`, `replan`, `onStep`, `mode`, `approvedPlan`, `awaiting-approval`, `report.plan`, `report.executedSteps`; ''let the model plan the steps'', ''dynamic plan from a goal'', ''run independent steps in parallel'', ''re-plan when a step fails'', ''generate a plan for approval before running it''; typical import `import { ai } from "@warlock.js/ai"`. Skip: a FIXED known pipeline — `@warlock.js/ai/run-ai-workflow/SKILL.md`; routing one input to a specialist each turn — `@warlock.js/ai/run-supervisor/SKILL.md`; a single model + tools call — `@warlock.js/ai/run-ai-agent/SKILL.md`; competing libs `langgraph`, `crewai`.'
 ---
 
 # `ai.planner()` — LLM-generated, then executed, plans
@@ -12,16 +12,19 @@ A planner turns a free-form **goal** into an ordered **plan** the LLM writes its
 - **`agent`** — one model + tools, single task. No multi-step decomposition.
 - **`workflow`** — a FIXED pipeline you author by hand (`steps: [...]`). The steps are known at design time.
 - **`supervisor`** — routes one input to the right specialist each turn; loops on a quality verdict.
-- **`planner`** — the steps are NOT known in advance. The LLM generates the ordered plan from the goal, then the planner executes it. Bounded v1: strictly sequential, no DAG scheduling, no mid-plan re-planning.
+- **`planner`** — the steps are NOT known in advance. The LLM generates the ordered plan from the goal, then the planner executes it. Sequential by default; opt into **DAG** scheduling, **adaptive re-planning**, and **plan-only / approval** as needed (below).
 
 ## Shape
 
 ```ts
 import { ai } from "@warlock.js/ai";
+import { OpenAISDK } from "@warlock.js/ai-openai";
+
+const openai = new OpenAISDK({ apiKey: process.env.OPENAI_API_KEY! });
 
 const research = ai.planner({
   name: "research-assistant",
-  model: ai.openai.model({ name: "gpt-4o" }), // the plan-GENERATION brain
+  model: openai.model({ name: "gpt-4o" }), // the plan-GENERATION brain
   capabilities: [
     { name: "search", description: "Search the web for sources", executable: searchAgent },
     { name: "summarize", description: "Summarize text into bullet points", executable: summarizer },
@@ -42,19 +45,81 @@ for (const step of report.executedSteps) {  // forensic, in execution order
 - `planner` lets you bring your own fully-configured planning agent (custom prompt, middleware). The planner injects the plan schema as that agent's per-call `output`.
 - A `capability` is `{ name, description, executable }`. The `name` is what the LLM references per step; the `description` is what it reads to pick. `executable` is any `ExecutableContract` (agent / workflow / supervisor / tool).
 
-## Execution model (bounded v1)
+## Execution model
 
-1. **Generate** — the planning agent is asked for a `{ steps, summary? }` plan via a generated schema whose `capability` field is an `enum` of your capability names.
-2. **Execute** — steps run **strictly in array order**. Each completed step's output is threaded into the next step's input as "Context from earlier steps". `dependsOn` on a step is advisory metadata only — recorded, not scheduled on.
+1. **Generate** — the planning agent is asked for a `{ steps, summary? }` plan via a generated schema whose `capability` field is an `enum` of your capability names. Each `PlannerStep` is `{ capability, input, id?, reason?, dependsOn? }`.
+2. **Execute** — by default steps run **strictly in array order**; each completed step's output is threaded into the next step's input as "Context from earlier steps". (Set `dag: true` to schedule on `dependsOn` instead — below.)
 3. **Finalize** — when `output` is set (factory or per-call), the LAST completed step's structured output is validated into `result.data`. A capability that should feed typed output to the planner's `output` should declare its own `output` schema (the planner reads `data`, falling back to an agent's raw `text`).
 
-`report.type === "planner"`; `report.children[]` carries every dispatched capability report (plus the planning trip), with usage rolled up. Lazy capability loading is **deferred** — every capability is fully constructed up front.
+`report.type === "planner"`; `report.children[]` carries every dispatched capability report (plus the planning trip), with usage rolled up. `report.executedSteps` is the authoritative per-step record (`PlannerStepSnapshot[]`). Lazy capability loading is **deferred** — every capability is fully constructed up front.
+
+## DAG scheduling — `dag: true` + `maxConcurrency`
+
+Run independent steps in parallel instead of array-order:
+
+```ts
+const research = ai.planner({
+  name: "research",
+  model,
+  capabilities,
+  dag: true,            // build a DAG from each step's `id` / `dependsOn`
+  maxConcurrency: 4,    // max steps in flight at once. default 4
+});
+```
+
+With `dag: true` the planner builds a DAG from step `id` / `dependsOn`, runs each **ready level concurrently** (up to `maxConcurrency`), and feeds each step **only its dependencies' outputs** (not the whole prior transcript). A **cycle** or a `dependsOn` naming an unknown step raises a typed `PlannerPlanInvalidError` **before any step runs**. Default `false` ⇒ the strict array-order loop, byte-for-byte unchanged (where `dependsOn` is advisory-only metadata).
+
+## Adaptive re-planning — `replan: { maxReplans }` + `onStep`
+
+When set, a **failed step** (or a `replan` verdict from the `onStep` hook) **revises the REMAINING plan** instead of aborting — re-asking the planning agent for a fresh plan seeded with the executed-step digest plus the feedback. Bounded by `maxReplans`; on exhaustion the run ends with the last failure.
+
+```ts
+const planner = ai.planner({
+  name: "adaptive",
+  model,
+  capabilities,
+  replan: { maxReplans: 2 },
+});
+
+await planner.execute(goal, {
+  onStep: (snapshot, plan) => {
+    // fired after EACH step settles (both the sequential and the DAG path)
+    if (snapshot.status === "completed" && looksWrong(snapshot.output)) {
+      return { type: "replan", feedback: "The summary missed the pricing section." };
+    }
+    // return nothing / { type: "continue" } to proceed; { type: "abort" } to stop
+  },
+});
+```
+
+The `onStep` directive (`PlannerStepDirective`):
+
+- `{ type: "continue" }` (or returning nothing) — proceed.
+- `{ type: "abort" }` — stop; remaining steps recorded `skipped` (exactly as a failure aborts).
+- `{ type: "replan"; feedback }` — re-plan the remainder, seeded with the digest + `feedback`. **A `replan` directive with no `replan` config is treated as `continue`** (no-op). Default off ⇒ a failure aborts exactly as before.
+
+## Plan-only / approval — `mode: "plan-only"` + `approvedPlan`
+
+Generate (and validate) a plan, return it for human sign-off, then execute the approved plan in a follow-up call:
+
+```ts
+// 1. Generate WITHOUT executing.
+const draft = await planner.execute(goal, { mode: "plan-only" });
+// draft.report.status === "awaiting-approval"; draft.plan carries the generated PlannerPlan.
+
+// 2. (human reviews draft.plan) ... then execute it verbatim.
+const final = await planner.execute(goal, { approvedPlan: draft.plan! });
+```
+
+- `mode: "plan-only"` generates + validates the plan and returns **without executing** — `report.status === "awaiting-approval"` (a planner-specific NON-terminal status) and `result.plan` carries the generated plan.
+- `approvedPlan` executes that exact plan, **skipping plan generation entirely**. It is still validated against the **live** capabilities, so a stale plan naming a capability the planner no longer has surfaces a `PlannerPlanInvalidError`.
+- `mode: "plan-only"` **with** `approvedPlan` is contradictory — `approvedPlan` wins (the plan executes).
 
 ## Failure + cancellation
 
 `execute()` never throws — failures surface on `result.error`:
 
-- **`PlannerPlanInvalidError`** (`PLANNER_PLAN_INVALID`, category `schema`) — empty plan or a step naming an unknown capability; also a final-output validation failure.
+- **`PlannerPlanInvalidError`** (`PLANNER_PLAN_INVALID`, category `schema`) — empty plan, a step naming an unknown capability, a DAG cycle, a `dependsOn` naming an unknown step, a stale `approvedPlan`, or a final-output validation failure.
 - **`PlannerCancelledError`** (`PLANNER_CANCELLED`, category `cancelled`) — the `AbortSignal` fired. `report.status === "cancelled"`, `report.cancelledAt` set; remaining steps are `skipped`.
 - A child capability's own error (agent / tool / provider) flows through unchanged on the failing step's snapshot and as `result.error`. The planner stops at the first failed step and marks the rest `skipped`.
 - **`PlannerFailedError`** is the base for the `PLANNER_*` family.