@warlock.js/ai 4.4.0 → 4.6.0

package/skills/generate-speech/SKILL.md

@@ -0,0 +1,139 @@
+---
+name: generate-speech
+description: 'Text-to-speech via ai.speech({ model: sdk.speech({ name }), text }) — the audio-OUTPUT verb (Theme I), returning the uniform never-throws { data, error, usage, report } envelope with cost-truth + panoptic observation. Models come from an adapter''s speech() factory: OpenAI tts-1 / tts-1-hd (per-character) or gpt-4o-mini-tts (per-token). Synthesized audio is a discriminated GeneratedAudio = { type: "base64"; base64; mediaType }. Options: voice / format / speed / instructions / signal. Triggers: `ai.speech`, `sdk.speech`, `openai.speech`, `SpeechModelContract`, `GeneratedAudio`, `SpeechModelPricing`, `SpeechOptions`, `MockSpeechModel`; ''text to speech'', ''TTS'', ''synthesize voice'', ''read this aloud'', ''tts-1'', ''gpt-4o-mini-tts'', ''voice narration'', ''audio output'', ''speak this text''; typical import `import { ai } from "@warlock.js/ai"` + `import { OpenAISDK } from "@warlock.js/ai-openai"`. Skip: speech-to-text / transcribing a voice note — [[transcribe-audio]]; image OUTPUT — [[generate-images]]; competing libs raw `openai.audio.speech.create`, `elevenlabs` SDK.'
+---
+
+# Generate speech — the text-to-speech verb (`ai.speech`)
+
+`ai.speech()` is the audio-output counterpart to `ai.image()` on the output-modality track (Theme I). Text-in / audio-out, wrapped in the same uniform result contract every executable returns — so a synthesized voicemail slots into cost dashboards and panoptic traces exactly like an agent run.
+
+This is audio **output** (TTS). For audio **input** (speech-to-text on a WhatsApp voice note or a meeting recording), see [[transcribe-audio]].
+
+## Shape
+
+```ts
+// 1. Build a speech model from an adapter's speech() factory.
+const model = openai.speech({ name: "tts-1", voice: "alloy" }); // SpeechModelContract
+
+// 2. Run the verb — never throws; failures land on result.error.
+const { data, error, usage, report } = await ai.speech({ model, text: "Your order has shipped." });
+
+if (error) {
+  console.warn(error.code);          // typed AIError (auth / rate-limit / content-filter / …)
+} else {
+  const { base64, mediaType } = data.audio; // GeneratedAudio (always base64 today)
+  await fs.writeFile("ship.mp3", Buffer.from(base64, "base64"));
+}
+```
+
+`SpeechModelContract` mirrors `EmbedderContract` / `ImageModelContract` — a peer primitive produced by the adapter's optional `speech?()` factory. An adapter without a TTS API simply doesn't define `speech()`, so calling it is a **compile-time** error, not a silent runtime failure. A non-TTS model id (`openai.speech({ name: "gpt-4o" })`) throws `InvalidRequestError` **at construction** — fail fast, like the embedder/image guards.
+
+## The result envelope
+
+```ts
+type SpeechResult = {
+  type: "speech";
+  data?: { audio: GeneratedAudio }; // undefined on failure
+  error?: AIError;                  // undefined on success — NEVER thrown
+  usage: Usage;                     // tokens (gpt-4o-mini-tts) + cost when priced
+  report: SpeechReport;             // type:"speech", model, characters, lineage
+};
+
+type GeneratedAudio = {
+  type: "base64";
+  base64: string;    // base64-encoded audio bytes
+  mediaType: string; // IANA type, e.g. "audio/mpeg", "audio/wav"
+};
+```
+
+`GeneratedAudio` is a discriminated union with a single `base64` variant today — the union leaves room for a future hosted-`url` variant without a breaking change, so always branch on `audio.type` rather than assuming `base64`.
+
+## Generation options (provider-neutral)
+
+```ts
+await ai.speech({
+  model,
+  text: "Welcome aboard. Let's get you set up.",
+  voice: "verse",                 // voice id/name; overrides the model's default
+  format: "wav",                  // "mp3" | "opus" | "aac" | "flac" | "wav" | "pcm"
+  speed: 1.25,                    // playback multiplier (OpenAI 0.25–4.0)
+  instructions: "calm, warm",     // tone/delivery steering (gpt-4o-mini-tts only)
+  signal,                         // AbortSignal
+  observe: collector,             // route the report to an Observer (panoptic), like agents
+  sessionId: "onboarding-42",     // group into a session for flat cost/trace queries
+  options: { /* provider passthrough */ },
+});
+```
+
+Each adapter maps the options its API supports and forwards `options` verbatim. On OpenAI the container defaults to `mp3` (→ `audio/mpeg`); `speed` and `instructions` are only sent when set, and the default voice is `alloy` when neither the call nor the model config supplies one.
+
+## OpenAI — tts-1 (per-character) + gpt-4o-mini-tts (per-token)
+
+```ts
+import { ai } from "@warlock.js/ai";
+import { OpenAISDK } from "@warlock.js/ai-openai";
+
+const openai = new OpenAISDK({ apiKey: process.env.OPENAI_API_KEY! });
+
+// tts-1 / tts-1-hd — billed per INPUT CHARACTER.
+const classic = openai.speech({ name: "tts-1", voice: "alloy", pricing: { perMillionCharacters: 15 } });
+
+// gpt-4o-mini-tts — billed per TOKEN like a chat model; supports `instructions`.
+const steered = openai.speech({ name: "gpt-4o-mini-tts", pricing: { input: 0.6, output: 12 } });
+
+const { data } = await ai.speech({ model: steered, text: "Read this warmly.", instructions: "gentle" });
+```
+
+## Cost-truth — one rollup, two metering models
+
+`ai.speech` fills `usage.cost` (a USD breakdown) so TTS spend folds into the **same** `Usage.cost` rollup as text — no second accounting path:
+
+- **Per-character** (`tts-1` / `tts-1-hd`): `{ perMillionCharacters }` × `report.characters` → `cost.input`. The Speech API reports no token usage, so `usage` tokens stay `{ 0, 0, 0 }` and spend is priced entirely from the input character count.
+- **Token-metered** (`gpt-4o-mini-tts`): `{ input, output }` USD-per-1M-tokens → standard `computeCost` against the returned token usage.
+
+Per-character wins when both are set. An unpriced model leaves `usage.cost` **`undefined`** (honest "cost unknown", never a false zero); a pre-priced adapter response is honored, not overwritten.
+
+## Pattern — order-confirmation voice line in a workflow `run` step
+
+```ts
+ai.step({
+  name: "voiceLine",
+  run: async (ctx) => {
+    const { data, error } = await ai.speech({
+      model: openai.speech({ name: "tts-1", voice: "alloy" }),
+      text: `Order ${ctx.steps.order.output.id} confirmed. Thank you!`,
+      format: "mp3",
+    });
+    if (error) throw error; // step retry/backoff handles transient provider faults
+    ctx.state.audio = data.audio; // { type:"base64", base64, mediaType:"audio/mpeg" }
+  },
+});
+```
+
+## Observability
+
+The completed `SpeechReport` (with `report.characters` and cost/latency attributed to `report.model`) routes to any registered `Observer` (panoptic, OTel, …) through the shared `observe` seam — pass `observe: true` (global), an `Observer` object (flow-local), or rely on observe-all. See [[observe-ai-flows]]. Provider faults surface as typed `AIError`s on `result.error`; see [[handle-ai-errors]].
+
+## Testing
+
+`MockSpeechModel(name, responses, pricing?)` is a deterministic `SpeechModelContract` double — no HTTP. Script audio/usage/errors and inspect `model.calls`. `MockSDK({ speechResponses, speechPricing }).speech({ name })` wires the same double behind a full adapter.
+
+```ts
+import { MockSpeechModel } from "@warlock.js/ai";
+import { speech } from "@warlock.js/ai";
+
+const model = new MockSpeechModel("tts-1", [{}], { perMillionCharacters: 15 });
+const { data, usage } = await speech({ model, text: "abcdefghij" }); // 10 chars
+// data.audio → { type:"base64", base64:"AAAA", mediaType:"audio/mpeg" }
+// usage.cost.input → (10 * 15) / 1_000_000
+// model.calls[0] records { text, options } for assertions
+```
+
+Scripting `[{ error: new ProviderRateLimitError("slow down") }]` drives the never-throws path — `result.error` is the typed error and `result.data` is `undefined`.
+
+## See also
+
+- [[transcribe-audio]] — the inverse verb (`ai.transcribe`), audio → text
+- [[generate-images]] — the sibling image-output verb (`ai.image`)
+- [[observe-ai-flows]] — routing the `SpeechReport` to panoptic / OTel
+- [[handle-ai-errors]] — the typed `AIError` taxonomy on `result.error`

package/skills/guard-input-output/SKILL.md

@@ -0,0 +1,117 @@
+---
+name: guard-input-output
+description: 'Build the composed guardrail middleware with @warlock.js/ai-guard and wire it into an agent — `ai.guardrail({ input, output, tool, toolNames, escalation })`. Triggers: `ai.guardrail`, `guard`, `GuardOptions`, `GuardrailVerdict`, `GuardrailDetector`, `GuardrailPhase`, `GuardrailMatch`, `GuardrailViolationError`, `ai.guardrail.topic`, `ai.guardrail.injection`, `topicFilter`, `injectionDetector`, `toolNames`, `forTool`; ''add a guardrail to my agent'', ''block prompt injection'', ''filter banned topics'', ''guard agent input and output'', ''stop the model leaking data into a tool call'', ''scope a detector to one tool''; typical import `import "@warlock.js/ai-guard"` (registers `ai.guardrail`) or `import { guard } from "@warlock.js/ai-guard"`. Skip: PII detection/redaction specifically — `@warlock.js/ai-guard/detect-and-redact-pii/SKILL.md`; routing a block to a human — `@warlock.js/ai-guard/escalate-block-to-human/SKILL.md`; the core middleware pipeline / hook contract — `@warlock.js/ai/run-ai-agent/SKILL.md`.'
+---
+
+# Guard agent input, output, and tool args
+
+`ai.guardrail(...)` is a **middleware factory**. It produces one `AgentMiddleware` that runs your detectors at three hook points and maps each verdict onto the agent pipeline's existing throw / return / record mechanics. Importing the package registers the verb (and its attached detector factories) on the shared `ai` namespace:
+
+```ts
+import { ai } from "@warlock.js/ai";
+import "@warlock.js/ai-guard"; // registers ai.guardrail + ai.guardrail.pii/.topic/.injection/.moderation
+
+const policy = ai.guardrail({
+  name: "compliance",
+  input: [ai.guardrail.injection({ onMatch: "block" })],
+  output: [ai.guardrail.topic({ deny: [/medical advice/i, "diagnosis"], onMatch: "block" })],
+});
+
+const agent = ai.agent({ model, middleware: [policy] });
+```
+
+A named-export form is available for callers who prefer not to rely on the augmented namespace:
+
+```ts
+import { guard, topic, injection } from "@warlock.js/ai-guard";
+const policy = guard({ input: [injection({ onMatch: "block" })] });
+```
+
+## The three phases
+
+| Phase | Hook | Inspected text | Set with |
+|---|---|---|---|
+| **input** | `trip.before` | the outbound prompt (`extractUserText(ctx.messages)`) | `input: [...]` |
+| **output** | `trip.after` | `response.content` | `output: [...]` |
+| **tool** | `tool.before` | `JSON.stringify(toolArgs)` | `tool: [...]` |
+
+Each phase array runs its detectors in **registration order**; the first non-`allow` verdict decides the action for that phase (short-circuit). A phase you don't configure is inert — a guard with no detectors is a no-op middleware.
+
+## The verdict model
+
+A detector inspects text and returns a `GuardrailVerdict`, discriminated by `type` (never `kind`):
+
+| `type` | Effect |
+|---|---|
+| `allow` | Pass to the next detector. |
+| `redact` | Rewrite the inspected text and continue — **output phase only** (see limitation below). |
+| `block` | Short-circuit with the existing `GuardrailViolationError`. |
+| `flag` | Pass, but append a `FlagRecord` into `ctx.state` under `<name>.flags` for a downstream observer (panoptic, the caller). |
+
+`agent.execute()` **never throws** — a `block` surfaces on `result.error` as a `GuardrailViolationError`, exactly like every other `AIError`. Branch on it after the run:
+
+```ts
+const result = await agent.execute(userInput);
+
+if (result.error instanceof ai.errors.GuardrailViolationError) {
+  // result.error.phase is "input" | "output" | "tool"
+  // result.error.reason / result.error.guardrail carry the detail
+}
+```
+
+## Built-in detectors
+
+Three zero-dependency detectors ship (a fourth, `moderation`, is an optional `openai` peer — see [`detect-and-redact-pii/SKILL.md`](@warlock.js/ai-guard/detect-and-redact-pii/SKILL.md)):
+
+- **`ai.guardrail.injection(options?)`** — jailbreak / prompt-injection marker phrases. Extra `markers` (string | RegExp); `onMatch` defaults to `"flag"`, callers commonly use `"block"` on input.
+- **`ai.guardrail.topic(options)`** — `deny` (string substring | RegExp) and/or `allow` (allow-list miss triggers `onMatch`). `onMatch` is `"block" | "flag"`, default `"block"`.
+- **`ai.guardrail.pii(options?)`** — PII regex + dictionary (its own skill).
+
+```ts
+const policy = ai.guardrail({
+  input: [
+    ai.guardrail.injection({ onMatch: "block", markers: ["ignore previous instructions"] }),
+    ai.guardrail.topic({ deny: ["competitor-name"], onMatch: "block" }),
+  ],
+});
+```
+
+## Scope tool detectors to specific tools
+
+`tool` detectors fire on **every** tool call by default. Set `toolNames` to scope them — the whole middleware is wrapped with the core `forTool(toolNames, mw)` helper so the `tool` hooks fire only for those names; `input` / `output` (`trip`) hooks are unaffected:
+
+```ts
+const policy = ai.guardrail({
+  tool: [ai.guardrail.pii({ onMatch: "block" })], // stop PII reaching the tool
+  toolNames: ["send_email", "post_webhook"],      // ...only for these tools
+});
+
+const agent = ai.agent({ model, tools: [sendEmail, postWebhook, lookup], middleware: [policy] });
+// `lookup` runs unguarded; `send_email` / `post_webhook` block on PII in their args.
+```
+
+A `block` from `tool.before` aborts that tool dispatch and surfaces on `result.error` with `phase: "tool"` — the agent run itself still never crashes.
+
+## Install order
+
+A guard is a normal `AgentMiddleware`; registration order is execution order (`before` top-down, `after` bottom-up). The canonical order is `[cache, budget, guardrail, observability]`. A `semanticCache` that short-circuits `trip.before` runs *before* the guard — a cached response then skips the **output** detectors, so place the guard before the cache if you don't trust cached contents.
+
+## Input-redaction limitation (v1)
+
+The core `trip.before` hook can only **short-circuit** (return a `ModelResponse`); it cannot rewrite the outbound prompt and continue. So:
+
+- **Input detectors are `block` / `flag` only.** A `redact` verdict on an input detector is treated as a `block` rather than silently passing an un-redacted prompt.
+- **Output redaction works** — `trip.after` returns a replacement `ModelResponse` with the rewritten `content`.
+- **Tool-arg `redact` is also withheld** — it downgrades to a `block` (`reason: "tool-arg-redaction-unsupported"`), because silently rewriting tool arguments changes the call's side-effects unpredictably.
+
+Lifting the input limitation needs a small, non-breaking core affordance and is deferred.
+
+## Failure isolation
+
+A detector's `check()` **rejecting** is an infrastructure fault, not a content violation — it is recorded as a `flag` (`<detector>.error`) into `ctx.state` and the fold **continues** (fail-open). A moderation-API outage degrades to missing annotation, never a failed agent run.
+
+## See also
+
+- [`@warlock.js/ai-guard/detect-and-redact-pii/SKILL.md`](@warlock.js/ai-guard/detect-and-redact-pii/SKILL.md) — the `pii` detector (detect/redact/block), the `mask` template, and the optional `moderation` peer.
+- [`@warlock.js/ai-guard/escalate-block-to-human/SKILL.md`](@warlock.js/ai-guard/escalate-block-to-human/SKILL.md) — routing a `block` to a human-review surface via `escalation.onBlock`.
+- [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — running the agent, the middleware pipeline, and the `GuardrailViolationError` on `result.error`.

package/skills/manage-prompts/SKILL.md

@@ -0,0 +1,186 @@
+---
+name: manage-prompts
+description: 'Unified prompt registry — ai.prompts: one process-wide store of named, versioned systemPrompt(...) builders keyed by name@version. Register by giving a prompt a meta.name (auto-registers), resolve by get(name) / resolve(name, versionOrTag, placeholders) / the inline name@selector form, bulk-register with define(name, versions), pin tags with tag(name, tag, version), compare with diff(name, from, to), round-trip with export() / import(snapshot), and quality-check with a unified validate(target, options) (deterministic missing-placeholder check + optional Nova-safe LLM-as-judge with verdict caching). Compose registered prompts into new ones with systemPrompt().merge(name, { fromVersion }) — provenance recorded in meta.composedFrom. ai.prompt is now a thin FACADE over ai.prompts (BREAKING vs the old standalone registry). Triggers: `ai.prompts`, `ai.prompt`, `PromptsManagerContract`, `PromptsManagerEntry`, `SystemPromptContract`, `SystemPromptMeta`, `SystemPromptMergeOptions`, `PromptsValidateOptions`, `PromptValidationResult`, `PromptValidateTarget`, `PromptTemplateVersion`, `PromptDiff`, `ExportedRegistry`, `defaultPromptsManager`, `prompts()`, `promptKey`, `meta`, `name`, `version`, `composedFrom`, `fromVersion`, `register`, `create`, `get`, `has`, `list`, `versions`, `resolve`, `define`, `tag`, `validate`, `diff`, `export`, `import`, `merge`, `judge`, `judgeCache`; ''register a prompt by name'', ''resolve a prompt by name@version or tag'', ''pin a production tag to a prompt version'', ''diff two prompt versions'', ''export / import the prompt registry'', ''validate a prompt for missing placeholders'', ''merge a registered prompt into another''; typical import `import { ai } from "@warlock.js/ai"`. Skip: composing a single prompt from persona + instruction blocks (the builder itself) — `@warlock.js/ai/write-system-prompt/SKILL.md`; runtime loadable skill bodies — `@warlock.js/ai/use-runtime-skills/SKILL.md`; eval scoring of agent outputs — `@warlock.js/ai/eval-datasets-and-ci/SKILL.md`; competing libs `langfuse` (direct), `promptfoo`.'
+---
+
+# `ai.prompts` — the unified prompt registry
+
+`ai.prompts` is ONE process-wide registry of named, versioned `systemPrompt(...)` builders keyed by `name@version`. A `systemPrompt(input, { name })` (or any `.meta({ name })` rename) auto-registers here; `ai.prompts.get(name)` / `.resolve(name)` read them back; `systemPrompt().merge(name)` folds a registered prompt into a new one. There is exactly **one storage shape** behind the whole prompt surface — a `SystemPromptContract` keyed by `name@version` — and `ai.prompt(...)` is now a thin facade over it (see the migration note below).
+
+```ts
+import { ai } from "@warlock.js/ai";
+
+// Register: any named systemPrompt auto-registers in ai.prompts.
+ai.systemPrompt("You are support for {{product}}.", { name: "support" });
+
+// Resolve back — latest version, or a version / pinned tag.
+ai.prompts.get("support");                 // → the SystemPromptContract
+ai.prompts.resolve("support", undefined, { product: "Warlock" }); // → final string
+```
+
+`ai.prompts` is the process-wide default (`defaultPromptsManager()`). For an **isolated** registry (parallel test suites, multi-tenant apps) call the `prompts()` factory — same `PromptsManagerContract`, its own store, no global side effects.
+
+## Identity — `SystemPromptMeta` (`meta.name` / `version` / `description` / `required` / `composedFrom`)
+
+A prompt's identity rides on its `meta`. Read it with the no-argument accessor; update it immutably with the one-argument form:
+
+```ts
+const base = ai.systemPrompt("You are support.", {
+  name: "support",
+  version: "1",
+  description: "Tier-1 support persona.",
+  required: ["product"],
+});
+
+base.meta();                          // → { name: "support", version: "1", description, required }
+const v2 = base.meta({ version: "2" }); // new builder, shallow-merged meta; original untouched
+```
+
+- **`name`** — when present, the prompt auto-registers in `ai.prompts` under `name@version`. Anonymous prompts (no `name`) are never registered.
+- **`version`** — free-form label (`"1"`, `"2025-draft"`). Defaults to the **next integer** for that name when omitted.
+- **`description`** — human-readable purpose (carried through `export`).
+- **`required`** — placeholder keys callers must supply; `validate()` reads them.
+- **`composedFrom`** — deterministic source labels a prompt was merged from (e.g. `["base@2", "global@1"]`). No random suffixes — the same merge always yields the same labels.
+
+## Register / resolve — `register` / `get` / `resolve` / `has` / `list` / `versions`
+
+```ts
+const registry = ai.prompts;              // or prompts() for an isolated one
+
+registry.register(ai.systemPrompt("You are support.", { name: "support" }));
+registry.versions("support");             // ["1"] — version derived as next integer
+
+registry.get("support");                  // latest SystemPromptContract
+registry.get("support@1");                // inline name@selector
+registry.resolve("support", "1", { product: "Warlock" }); // pick version + render in one call
+
+registry.has("support");                  // boolean
+registry.list();                          // every registered name, first-seen order
+```
+
+- **Version selection** — `get(name)` / `resolve(name)` return the **latest** by insertion order; pass a version label, a pinned tag, or fold it into the first arg as `name@selector` (`get("support@1")`, `resolve("support@production")`).
+- **Duplicates** — re-registering the same `name@version` throws `InvalidRequestError` **unless** the content is byte-identical (idempotent re-registration is a no-op).
+- **Unknown name / version / tag** → `InvalidRequestError`.
+- `register()` throws if the prompt has no `meta.name`.
+
+## `create()` — build + register in one entry point
+
+`ai.prompts.create(input?, meta?)` is a documented alias of `ai.systemPrompt(...)` — identical input forms (no arg → empty builder; a string → one instruction; an array of blocks → verbatim). Pass `meta.name` to auto-register, so authoring and lookup read side-by-side:
+
+```ts
+ai.prompts.create("You are support for {{product}}.", { name: "support" });
+ai.prompts.resolve("support", undefined, { product: "Warlock" });
+```
+
+## `define()` — bulk-register many versions
+
+```ts
+ai.prompts.define("agent", [
+  { version: "1", template: "You are v1." },
+  { version: "2", template: [ai.persona("You are Alex."), ai.instruction("Be concise.")] },
+]);
+```
+
+A `PromptTemplateVersion`'s `template` is a raw string (wrapped into one instruction block) or an explicit ordered block list (verbatim). Versions register **oldest-first** in array order; the same duplicate / idempotency rule applies per `name@version`. Returns the manager for chaining.
+
+## `tag()` — pin a moving label to a version
+
+```ts
+ai.prompts.tag("agent", "production", "2");   // pin "production" → version 2
+
+ai.prompts.get("agent", "production");        // resolves through the tag
+ai.prompts.resolve("agent", "production");
+ai.prompts.get("agent@production");           // inline form
+```
+
+Re-pinning an existing tag moves it. An unknown name / version throws `InvalidRequestError`. Tags survive `export` / `import`.
+
+## `validate()` — unified deterministic + optional LLM-judge
+
+```ts
+const report = await ai.prompts.validate("support", {
+  placeholders: { product: "Warlock" }, // values you intend to supply
+  declare: ["language"],                  // extra keys to treat as known
+  judge: judgeModel,                      // optional — turns on the LLM-as-judge pass
+});
+
+report.ok;       // true iff no required placeholder is missing (DETERMINISTIC verdict alone)
+report.missing;  // placeholder keys referenced with no default, unsupplied, undeclared
+report.score;    // 0..1 — present ONLY when a judge ran and produced a usable verdict
+report.issues;   // advisory judge reasons / a degrade note — present only when a judge was supplied
+```
+
+- **Always** runs the deterministic check: every `{{key}}` with no inline default that is neither supplied (`placeholders`), declared (`declare`), nor in the prompt's `meta.required` lands in `missing`; `ok` is `true` iff `missing` is empty.
+- **`judge`** adds a **Nova-safe** LLM-as-judge quality pass — it **never throws** and degrades to an `issues` note (leaving `score` undefined) on failure, so a flaky judge can **never flip `ok`**.
+- **`target`** is a registered name (or `name@selector`), a `SystemPromptContract` instance, or a raw prompt string.
+- **`judgeCache`** (per-call or via the `prompts({ judgeCache })` factory option) memoizes judge verdicts by a content hash of the resolved body + the judge model id — a structural `{ get, set }` subset of `@warlock.js/cache`'s `CacheDriver`, so the cache package stays a strictly **optional** peer.
+
+`systemPrompt().validate(options?)` is the per-builder sugar — `ai.prompts.validate(this, options)` under the hood, same result shape.
+
+## `diff()` — block-level version diff
+
+```ts
+const diff = ai.prompts.diff("agent", "1", "2");
+
+diff.identical;  // true when both versions have identical blocks in identical order
+diff.added;      // blocks in `to` not at the same position in `from`
+diff.removed;    // blocks in `from` not at the same position in `to`
+diff.changed;    // [{ from, to }] — same position, type/text changed
+```
+
+Blocks are matched **positionally**. Unknown name / version → `InvalidRequestError`.
+
+## `export()` / `import()` — portable JSON round-trip
+
+```ts
+const snapshot = ai.prompts.export();   // ExportedRegistry — every name, version, pinned tag, description/required
+otherRegistry.import(snapshot);          // rehydrate (same duplicate / idempotency rule; tags restored)
+```
+
+Each version flattens to `{ type, text }` blocks so the registry round-trips without live builder instances — commit a snapshot, ship it, restore it elsewhere.
+
+## Compose registered prompts — `systemPrompt().merge(name, { fromVersion })`
+
+`merge` folds another prompt's blocks into a new builder (persona **replaces**, instructions **append**) and records `meta.composedFrom`:
+
+```ts
+ai.systemPrompt("Always answer in {{language|English}}.", { name: "global", version: "1" });
+
+const supportPrompt = ai.systemPrompt("You are support for {{product}}.")
+  .merge("global", { fromVersion: "1" });  // fold the registered prompt by name
+
+supportPrompt.meta()?.composedFrom;        // ["…", "global@1"] — deterministic provenance
+```
+
+`merge` accepts three source forms: a pre-built block, another `SystemPromptContract`, or a **registered name** resolved from `ai.prompts` (latest version unless `options.fromVersion` selects another — an unknown name / version throws `InvalidRequestError`).
+
+## `ai.prompt(...)` — now a thin facade (⚠ breaking vs the old registry)
+
+`ai.prompt` has **two** call forms, both backed by the unified manager — there is no longer a separate prompt store:
+
+```ts
+// (a) Resolve a globally-registered prompt from ai.prompts by name.
+ai.systemPrompt("You are support.", { name: "support" });
+const sp = ai.prompt("support");          // → SystemPromptContract (latest)
+const v1 = ai.prompt("support", "1");     // → a specific version / pinned tag
+
+// (b) Build an ISOLATED legacy-shaped registry (PromptRegistryContract).
+const reg = ai.prompt({
+  prompts: [{ name: "summarizer", versions: [{ version: "1", template: "Summarize: {{text}}" }] }],
+});
+const resolved = reg.resolve("summarizer", { placeholders: { text } });
+resolved.toSystemPrompt();                 // drop-in for ai.agent({ systemPrompt })
+```
+
+**⚠ Migration.** Before unification, `ai.prompt(...)` only built a standalone, self-contained registry with its **own private** storage. It now:
+
+1. Adds the **string overload** `ai.prompt(name, versionOrTag?)` → resolves from the shared `ai.prompts` manager. (New capability — `ai.prompt("x")` used to be a type error.)
+2. Backs the **options form** (`ai.prompt({ ... })` → `PromptRegistryContract`) by an internal `PromptsManagerContract`, so its storage shape and validation primitives are now the unified ones. The legacy method surface (`register` / `add` / `versions` / `resolve` / `validate` / `sync` + the `{ score, notes }` report shape) is **unchanged**, and each `ai.prompt({ ... })` call still returns its **own isolated** registry — no shared global state.
+
+If you only ever called `ai.prompt({ ... })` and used the returned registry, **no code change is needed**. The new behavior is additive: prefer `ai.prompts` (the unified manager) for new code; reach for `ai.prompt({ ... })` only when you want the legacy `ResolvedPrompt` / `toSystemPrompt()` ergonomics or the optional Langfuse sync. The legacy facade's reference — `register` / `add` / `resolve(name, { version, placeholders })` / `validate` (`{ score, notes }`) / `sync()` (lazy `langfuse` peer) — is documented inline in `src/prompt/prompt.ts`.
+
+## See also
+
+- [`@warlock.js/ai/write-system-prompt/SKILL.md`](@warlock.js/ai/write-system-prompt/SKILL.md) — the `systemPrompt()` / `persona()` / `instruction()` builder, `.meta()`, and `merge()` this registry stores and composes
+- [`@warlock.js/ai/eval-datasets-and-ci/SKILL.md`](@warlock.js/ai/eval-datasets-and-ci/SKILL.md) — the eval `judge` scorer `validate()`'s LLM pass reuses
+- [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — wiring a resolved prompt into an agent, plus the judge-safe agent preset (`ai.agent.judge`)

package/skills/observe-ai-flows/SKILL.md

@@ -0,0 +1,94 @@
+---
+name: observe-ai-flows
+description: 'The core Observer seam — a generic, tool-agnostic observability hook every flow routes its completed ExecutionReport through. Covers the per-flow `observe?: boolean | Observer` option on ai.agent / workflow / supervisor / team, the global registry (registerObserver / getObservers / setObserveAll / isObserveAll / clearObservers), resolveObservers / notifyObservers resolution, the opt-in AgentConfig.captureMessages → AgentReport.messages full-history capture, the onConfigApplied dependency-inversion seam, and that @warlock.js/ai-panoptic is the batteries-included Observer. Triggers: `Observer`, `observe`, `registerObserver`, `getObservers`, `setObserveAll`, `isObserveAll`, `clearObservers`, `resolveObservers`, `notifyObservers`, `FlowObserveOption`, `ExecutionReport`, `captureMessages`, `AgentReport.messages`, `CapturedMessage`, `onConfigApplied`, `observeAll`; ''observe an agent run'', ''send finished reports to a collector'', ''capture the full message history'', ''observe every flow by default'', ''wire panoptic / tracing''; typical import `import { ai, registerObserver } from "@warlock.js/ai"`. Skip: structured logging of events — `@warlock.js/ai/log-ai-calls/SKILL.md`; reading the report tree shape (trips / children) — `@warlock.js/ai/run-ai-agent/SKILL.md`; per-call cost / usage rollup — `@warlock.js/ai/handle-ai-errors/SKILL.md`. The batteries-included Observer is the `@warlock.js/ai-panoptic` package.'
+---
+
+# The `Observer` seam — generic, tool-agnostic observability
+
+Core defines a structural `Observer` and a tiny registry; it never imports any observability package (panoptic, OTel, Langfuse, …). A flow that resolves to "observed" hands its completed `ExecutionReport` to every registered observer. An observability tool **implements `Observer` and registers itself**, so `observe: true` / observe-all route reports without coupling core to the tool — the dependency inversion that keeps the two sides decoupled.
+
+```ts
+export interface Observer {
+  collect(report: ExecutionReport): void | Promise<void>;
+}
+```
+
+`collect` may be sync or async — the flow awaits it. A throw is **swallowed** by the flow (never breaks the run), mirroring the existing `onUsage` / `onComplete` hooks.
+
+## Per-flow `observe` option
+
+`observe?: boolean | Observer` (`FlowObserveOption`) is accepted on **`ai.agent`, `ai.workflow`, `ai.supervisor`, and `ai.team`** (a team forwards it verbatim to the supervisor it desugars into):
+
+```ts
+const collector: Observer = { collect(report) { exporter.send(report); } };
+
+ai.agent({ model, observe: true });       // → the globally registered observers, even if observe-all is off
+ai.agent({ model, observe: false });      // → opt out entirely, even when observe-all is on
+ai.agent({ model, observe: collector });  // → a flow-LOCAL collector; only this flow's report, only to it
+ai.agent({ model });                      // → undefined: follow the global observe-all flag
+```
+
+Resolution (`resolveObservers(observe)`):
+
+- `false` → `[]` (opted out).
+- `true` → the globally registered observers.
+- an `Observer` object → just that one (flow-local; the global observers are skipped).
+- `undefined` → the global observers when observe-all is on, otherwise `[]`.
+
+`notifyObservers(observe, report)` routes a completed report to each resolved observer, awaiting each `collect` (so async exporters finish before the flow returns) and swallowing any throw. The object form is typed as the structural `Observer` (NOT a panoptic-specific type), so a panoptic flow-local collector — which implements `Observer` — can be passed directly.
+
+## The global registry
+
+```ts
+import {
+  registerObserver, getObservers, setObserveAll, isObserveAll, clearObservers,
+} from "@warlock.js/ai";
+
+registerObserver(collector);  // an observability tool registers ONE collector when its config is applied
+getObservers();               // read-only snapshot of the registered observers (do not mutate)
+
+setObserveAll(true);          // "observe every flow by default" — flows without their own `observe` get observed
+isObserveAll();               // read the flag (default false — opt-in observability)
+
+clearObservers();             // test-only: reset observers + the observe-all flag for spec isolation
+```
+
+`observeAll` defaults to `false` (opt-in). A flow that never sets `observe` is observed **only** when observe-all is on; individual flows still opt out with `observe: false`.
+
+## Full-history capture — `captureMessages` → `AgentReport.messages`
+
+Off by default. When `ai.agent({ captureMessages: true })` is set, the agent normalizes the real assembled turn array onto `AgentReport.messages` as a `CapturedMessage[]`:
+
+```ts
+const { report } = await ai.agent({ model, tools, captureMessages: true }).execute("Go");
+report.messages; // CapturedMessage[] — every role (system/user/assistant/tool), every trip
+```
+
+A `CapturedMessage` is a JSON-safe projection: `{ role, content, toolCalls?, toolCallId? }` — `content` is always a string (tool results stringified), assistant turns that triggered tools carry `toolCalls`, tool-result turns carry the `toolCallId` they answer. Unlike `trips[].input` (which stubs non-first trips with `"[tool results]"`), this preserves the **real** turn array. Omitted ⇒ the field is **absent** and the report is byte-for-byte as before. Opt-in because messages can be large and sensitive (full prompts, tool inputs/outputs) — and **required for panoptic full-history capture**.
+
+## Callback sub-agents nest in the report tree
+
+The `ExecutionReport` an observer receives reflects **full** lineage: a supervisor / team / orchestrator callback that calls `agent.execute()` directly auto-nests `callback → agent → tool` (via an ambient `RunFrame`), so usage / cost roll up and panoptic renders the sub-agent under its callback instead of as a lone `$0` span. No observer-side change is needed — the tree arrives already nested. A team's root span carries `type: "team"` (a first-class `ReportType`, not `"supervisor"`), so observers can distinguish, group, and label team runs as their own type. See [`@warlock.js/ai/run-supervisor/SKILL.md`](@warlock.js/ai/run-supervisor/SKILL.md).
+
+## The config seam — `onConfigApplied`
+
+An observability tool reacts to its own augmented config slot without core importing it. Core lets tools attach an opaque slot (e.g. `panoptic?`) via declaration merging on `AIConfig`, then fires registered listeners after each `ai.config(...)` merge:
+
+```ts
+import { onConfigApplied, getAIConfig } from "@warlock.js/ai";
+
+onConfigApplied((config) => applyPanopticConfig(config.panoptic)); // react on every config merge
+applyPanopticConfig(getAIConfig().panoptic);                        // catch a pre-set config
+```
+
+A misbehaving listener's throw is swallowed (same swallow-on-throw discipline as the observer hooks). This mirrors the `Observer` registry's dependency inversion: a tool flips `setObserveAll(true)` and calls `registerObserver(...)` from inside its `onConfigApplied` listener.
+
+## The batteries-included Observer
+
+`@warlock.js/ai-panoptic` is the shipped, full-featured `Observer` — install it, configure it via `ai.config({ panoptic })`, and it registers its collector + (optionally) flips observe-all for you. Core stays dependency-free; this skill documents the seam panoptic plugs into.
+
+## See also
+
+- [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — the `AgentReport` / `ExecutionReport` tree (`trips`, `children`) an observer receives
+- [`@warlock.js/ai/log-ai-calls/SKILL.md`](@warlock.js/ai/log-ai-calls/SKILL.md) — event-level structured logging (vs. report-level observation)
+- [`@warlock.js/ai/run-ai-team/SKILL.md`](@warlock.js/ai/run-ai-team/SKILL.md) — a team inherits `observe` through the supervisor it forwards to