@warlock.js/ai 4.3.0 → 4.5.0

package/skills/durable-resume/SKILL.md

@@ -0,0 +1,128 @@
+---
+name: durable-resume
+description: 'Persist a gated tool call and resume it from another process hours later — ships in @warlock.js/ai core: `ai.human.resume(interruptId, decision, options)`, the `InterruptStore` (`ai.human.interrupt.{memory,pg,redis}()`), `PendingInterrupt`, and the `InterruptSuspendedError` suspend sentinel. Triggers: `ai.human.resume`, `resume(interruptId, decision)`, `InterruptStore`, `ai.human.interrupt.memory`, `ai.human.interrupt.pg`, `ai.human.interrupt.redis`, `interruptMemory`, `interruptPg`, `interruptRedis`, `PendingInterrupt`, `InterruptSuspendedError`, `ResumeOptions`, `ResumeResult`, `PgClientLike`, `RedisClientLike`; ''approve hours later from a webhook'', ''persist the approval request and resume in another process'', ''durable human-in-the-loop'', ''store the interrupt in Postgres/Redis'', ''re-run the agent turn once the human approves''. Typical import `import { ai, InterruptSuspendedError } from "@warlock.js/ai"`. Skip: the in-process await gate and the policy/decision shapes — `@warlock.js/ai/approve-tool-calls/SKILL.md`.'
+---
+
+# Durable resume — persist the interrupt, approve from another process
+
+Interactive approval `await`s the operator in-process. **Durable** approval is for when the reviewer rules out-of-band — a Slack button, a webhook, hours later, in a different process. The flow: the handler **persists** the request to an `InterruptStore` and **throws** `InterruptSuspendedError` to suspend the run; the caller surfaces the `interruptId`; later, `ai.human.resume(interruptId, decision, { store })` applies the ruling.
+
+> **v1 durable resume re-runs the turn** with the decision pre-seeded — it does **not** rehydrate an in-flight supervisor mid-call (that's the deferred v2 lift). Re-running is idempotent because the prompt and the seeded decision fully determine the gated call's outcome.
+
+## Process A — suspend and surface the id
+
+```ts
+import { ai, InterruptSuspendedError } from "@warlock.js/ai";
+
+const store = ai.human.interrupt.memory(); // swap for pg / redis in production
+
+const agent = ai.agent({
+  model,
+  tools: [deleteAccount],
+  middleware: [
+    ai.human.approval({
+      policy: { type: "predicate", requiresApproval: (c) => c.toolName === "deleteAccount" },
+      store,
+      handler: async (req) => {
+        // 1. persist the pending interrupt
+        await store.save({
+          interruptId: req.interruptId,
+          request: req,
+          status: "pending",
+          savedAt: new Date().toISOString(),
+        });
+        // 2. notify the reviewer out-of-band
+        await slack.postApproval(req);
+        // 3. suspend the run — the middleware recognizes its OWN sentinel
+        throw new InterruptSuspendedError("Awaiting human approval", {
+          interruptId: req.interruptId,
+        });
+      },
+    }),
+  ],
+});
+
+const result = await agent.execute("Delete account #88");
+
+// execute() never throws — the suspend rides result.error:
+if (result.error instanceof InterruptSuspendedError) {
+  return { status: "awaiting-approval", interruptId: result.error.interruptId };
+}
+```
+
+The middleware catches the **sentinel** (`instanceof InterruptSuspendedError`) and short-circuits a failed `ToolInvokeResult` carrying it, so `error.interruptId` is on `result.error`. Hand that id to the reviewer.
+
+## Process B — resume hours later
+
+```ts
+import { ai } from "@warlock.js/ai";
+
+// Re-run the turn with the decision pre-seeded:
+const outcome = await ai.human.resume(
+  interruptId,
+  { type: "edit", args: { confirm: true } },
+  { store, agent },
+);
+
+if (outcome.type === "applied" && outcome.result) {
+  console.log(outcome.result.text); // the re-run completed with the ruling applied
+}
+```
+
+`ai.human.resume(interruptId, decision, options)` loads the `PendingInterrupt`, validates the decision shape, deletes the record, and — when an `agent` is supplied — re-executes the original prompt with the decision **pre-seeded** so the gated tool call resolves to the ruling instead of pausing again. The prompt comes from `request.context.originalInput`; pass `options.input` to override (e.g. to append the reviewer's note), and `options.executeOptions` to forward history / output schema / signal to the re-run.
+
+### Two resume shapes
+
+| Shape | Pass | Behavior |
+|---|---|---|
+| **re-run** | `{ store, agent }` | Loads, deletes, re-executes the turn; `ResultResult.result` carries the `AgentResult`. |
+| **apply-only** | `{ store }` (no `agent`) | Loads, validates, deletes; returns `{ type: "applied", decision }` for a caller-owned re-drive (custom transport). No turn re-run. |
+
+### Idempotent by construction
+
+```ts
+type ResumeResult =
+  | { type: "applied"; interruptId: string; decision: ApprovalDecision; result?: AgentResult }
+  | { type: "already-resolved"; interruptId: string };
+```
+
+A second resume of an already-resolved (deleted) or never-raised interrupt returns `{ type: "already-resolved" }` — it never double-applies the decision or re-runs the turn. The record is deleted **before** the re-run, so even a re-run that itself raises a fresh interrupt can't collide with the one being resolved. A malformed decision (`reject` with no `reason`, `edit` with no `args`, an unknown `type`) throws a `TypeError` loudly rather than silently mis-driving the re-run.
+
+## The `InterruptStore`
+
+`ai.human.interrupt.{memory,pg,redis}()` build the store. The contract mirrors `@warlock.js/ai`'s `CheckpointStore` / `SnapshotStore` — `save` / `load` / `delete` / optional `list(prefix?)` / `schema()` — so a consumer already running an orchestrator can reuse the **same** pool for the interrupt table.
+
+| Factory | Backing | Deps |
+|---|---|---|
+| `ai.human.interrupt.memory()` | process-local `Map` | none — zero runtime deps |
+| `ai.human.interrupt.pg(options)` | one Postgres row per interrupt, keyed by `interrupt_id` | lazily imports the optional `pg` peer |
+| `ai.human.interrupt.redis(options)` | one namespaced JSON value + a self-maintained id index | lazily imports the optional `redis` peer |
+
+```ts
+// Memory — dev / tests / single-process:
+const store = ai.human.interrupt.memory();
+
+// Postgres — pass a live pool (core never imports pg in that case):
+import { Pool } from "pg";
+const pool = new Pool({ connectionString: process.env.DATABASE_URL });
+const store = ai.human.interrupt.pg({ client: pool });
+// Once, via your migration tool — the framework never auto-migrates:
+// await pool.query(store.schema());
+
+// …or let the store build its own pool (lazily import("pg")):
+const store = ai.human.interrupt.pg({ connectionString: process.env.DATABASE_URL });
+
+// Redis — pass a connected client, or a url:
+const store = ai.human.interrupt.redis({ url: process.env.REDIS_URL });
+```
+
+### Optional peers are lazy
+
+`pg` and `redis` are **optional** peer dependencies — neither is a hard dependency. The driver is imported only inside the store that needs it, and only when you pass a `connectionString` / `url` (passing a live `client` imports nothing). If the driver is absent, a **curated install string** surfaces on first use, never a raw module-resolution stack trace at import — so a memory-only consumer always loads cleanly. `PgClientLike` / `RedisClientLike` are structural interfaces, so any compatible pool/client satisfies them.
+
+`schema()` returns the reference DDL for the Postgres store (run it through your migration tool once) and an empty string for memory / redis, so callers treat `schema()` uniformly across drivers.
+
+## See also
+
+- [`@warlock.js/ai/approve-tool-calls/SKILL.md`](@warlock.js/ai/approve-tool-calls/SKILL.md) — the gate itself: the interrupt policy, the approve / reject / edit decision union, and the interactive (in-process await) handler.
+- `@warlock.js/ai` — the `CheckpointStore` / `SnapshotStore` the `InterruptStore` mirrors, and the `ai.agent(...)` re-run target.

package/skills/escalate-block-to-human/SKILL.md

@@ -0,0 +1,85 @@
+---
+name: escalate-block-to-human
+description: 'Route a hard guardrail block to a human-review surface with @warlock.js/ai-guard — the `escalation.onBlock` seam and an `escalate: true` verdict. Triggers: `escalation`, `onBlock`, `GuardrailEscalation`, `GuardrailBlockEvent`, `escalate: true`, `{ type: "block", escalate: true }`, ''escalate a block to a human'', ''human review queue for guardrail'', ''page an operator on a guardrail block'', ''human-in-the-loop guardrail'', ''compose a block with a review surface'', ''custom detector that escalates''; typical import `import "@warlock.js/ai-guard"` then `ai.guardrail({ escalation: { onBlock } })`. Skip: composing the guard / phases / verdict model — `@warlock.js/ai-guard/guard-input-output/SKILL.md`; PII/moderation detectors — `@warlock.js/ai-guard/detect-and-redact-pii/SKILL.md`; durable suspend/resume human-step machinery (deferred) — not in this package.'
+---
+
+# Escalate a block to a human
+
+A `block` verdict can carry `escalate: true`. When it does, the guard `await`s your `escalation.onBlock(...)` handler **before** throwing the `GuardrailViolationError` — the seam to a human-review queue, an operator page, or any out-of-band approval surface.
+
+```ts
+import { ai } from "@warlock.js/ai";
+import "@warlock.js/ai-guard";
+
+const policy = ai.guardrail({
+  output: [ai.guardrail.moderation({ blockOn: ["self-harm"] })],
+  escalation: {
+    async onBlock(event) {
+      await reviewQueue.enqueue({
+        phase: event.phase,   // "input" | "output" | "tool"
+        reason: event.reason, // the detector's human-readable reason
+      });
+    },
+  },
+});
+
+const agent = ai.agent({ model, middleware: [policy] });
+```
+
+## When `onBlock` fires
+
+`onBlock` fires **only** for a verdict of `{ type: "block", escalate: true }` — not for an ordinary `block`, and never for `allow` / `redact` / `flag`. It is **awaited before** the `GuardrailViolationError` is thrown, so your handler runs to completion (enqueue succeeds, the page is sent) before the error surfaces on `result.error`. The run still aborts: escalation is a *signal*, not a recovery — `execute()` returns with `result.error` populated as usual.
+
+## The `GuardrailBlockEvent` payload
+
+`onBlock(event)` receives:
+
+| Field | Type | Meaning |
+|---|---|---|
+| `phase` | `"input" \| "output" \| "tool"` | where the block fired |
+| `reason` | `string` | the detector's human-readable reason |
+| `matches` | `readonly GuardrailMatch[] \| undefined` | what tripped the rule (rule id, span, label), when reported |
+| `ctx` | `MiddlewareTripContext` | the live trip context — `state`, `messages`, `agent`, `model`, `signal` |
+
+`ctx` lets the handler enrich the review item with run context (session id from `ctx.state`, the offending messages, etc.).
+
+## Producing an escalating verdict
+
+The built-in detectors return ordinary `block` verdicts (no `escalate`). To escalate, author a tiny custom `GuardrailDetector` that sets `escalate: true` on its `block`:
+
+```ts
+import type { GuardrailDetector } from "@warlock.js/ai-guard";
+
+const wirePolicy: GuardrailDetector = {
+  name: "wire-transfer",
+  check(text) {
+    if (/wire \$?\d{5,}/i.test(text)) {
+      return {
+        type: "block",
+        reason: "large wire transfer requires human approval",
+        escalate: true, // <- routes through escalation.onBlock
+        matches: [{ rule: "wire-transfer.large", label: "wire" }],
+      };
+    }
+
+    return { type: "allow" };
+  },
+};
+
+const policy = ai.guardrail({
+  tool: [wirePolicy],
+  toolNames: ["initiate_transfer"],
+  escalation: { async onBlock(e) { await approvals.request(e); } },
+});
+```
+
+A `check()` may be sync or async (async = call an external service); the guard awaits either.
+
+## A plain callback by design
+
+`escalation.onBlock` is a **plain callback** — `ai-guard` takes **no** dependency on the deferred durable human-step machinery (suspend/resume). The callback is the decoupling seam: inside it you wire your own review queue, and (where your stack supports it) a `workflow.resume(...)` loop. This package only emits the *signal*; it does not own durable suspension. When the typed human-step handoff ships, `onBlock` upgrades to it without a breaking change here.
+
+## See also
+
+- [`@warlock.js/ai-guard/guard-input-output/SKILL.md`](@warlock.js/ai-guard/guard-input-output/SKILL.md) — composing the guard, the phases, the verdict model, and how a `block` surfaces on `result.error`.
+- [`@warlock.js/ai-guard/detect-and-redact-pii/SKILL.md`](@warlock.js/ai-guard/detect-and-redact-pii/SKILL.md) — the `pii` detector and the optional `moderation` peer that commonly drives an escalation.

package/skills/eval-datasets-and-ci/SKILL.md

@@ -0,0 +1,117 @@
+---
+name: eval-datasets-and-ci
+description: 'Datasets + regression-gated eval CI with ai.dataset({...}) feeding agent.eval({cases,baseline,tolerance}). Covers the immutable filterable/shardable dataset (cases / fromFile JSONL), DatasetEntry tags, EvalReport.regression (regressed/added/removed/passed) against a baseline, and the ai.eval reporters toJUnit / toJSON / fromJSON for CI artifacts + committed baselines. Triggers: `ai.dataset`, `DatasetContract`, `DatasetEntry`, `DatasetOptions`, `dataset.filter`, `dataset.shard`, `fromFile`, `agent.eval`, `EvalOptions`, `EvalReport`, `EvalCaseResult`, `EvalRegression`, `baseline`, `tolerance`, `ai.eval.toJUnit`, `ai.eval.toJSON`, `ai.eval.fromJSON`, `diff`, JSONL; ''eval dataset from a JSONL file'', ''shard an eval suite across CI jobs'', ''fail CI on an eval regression'', ''emit a JUnit report'', ''snapshot an eval baseline''; typical import `import { ai } from "@warlock.js/ai"`. Skip: the scorers + LLM-as-judge + Vitest matchers themselves — `@warlock.js/ai/ai-dx-helpers/SKILL.md` (registerAiMatchers / ai.eval.exact|contains|predicate|judge); record/replay of model calls for deterministic tests — `@warlock.js/ai/record-replay-llm/SKILL.md`; competing libs `promptfoo`, `braintrust`.'
+---
+
+# `ai.dataset()` + `agent.eval()` regression CI
+
+Turn a corpus of cases into a regression-gated CI signal. `ai.dataset(...)` wraps cases into an immutable, filterable, shardable collection; `agent.eval({ cases, baseline, tolerance })` runs them, scores them, and diffs against a prior report; the `ai.eval.*` reporters serialize the result for CI ingestion and tomorrow's baseline.
+
+> This skill is the **dataset + CI** layer. The scorers, LLM-as-judge config, and Vitest matchers live in [`@warlock.js/ai/ai-dx-helpers/SKILL.md`](@warlock.js/ai/ai-dx-helpers/SKILL.md); `agent.eval`'s core scoring loop is in [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md).
+
+## `ai.dataset()` — immutable, filterable, shardable
+
+```ts
+import { ai } from "@warlock.js/ai";
+
+const ds = ai.dataset({
+  name: "support",
+  cases: [{ name: "greeting", input: "hi", expected: "Hello" }], // inline entries
+  fromFile: "./eval/support.jsonl",  // JSONL read ONCE, synchronously, at construction
+});
+
+ds.name;   // "support"
+ds.cases;  // DatasetEntry[] (inline first, then file entries appended)
+```
+
+- A `DatasetEntry` is an `EvalCase` plus optional `tags?: string[]` for filtering / sharding.
+- **`fromFile`** reads a JSONL file (one JSON object per line; blank lines skipped) synchronously at construction — mirroring `SystemPrompt.fromFile`. A malformed line throws an **`InvalidRequestError` naming the 1-based line number**; a missing/unreadable path throws too. `cases` and `fromFile` combine (file entries append after inline).
+
+### `filter` / `shard` — derive new datasets
+
+```ts
+const smoke = ds.filter((entry) => entry.tags?.includes("smoke"));
+const shard = ds.shard(0, 4); // first of four parallel CI shards
+```
+
+Both return a **fresh dataset sharing nothing mutable**. `shard(index, total)` is deterministic round-robin by position: every entry lands in exactly one shard, so the union of all `total` shards reproduces the full list with no gaps or overlaps. `shard` validates its args (positive integer `total`, `index` in `[0, total)`) and throws `InvalidRequestError` otherwise.
+
+## `agent.eval({ cases })` — accepts a dataset directly
+
+```ts
+const report = await myAgent.eval({
+  cases: ds,                       // a DatasetContract OR a raw EvalCase[]
+  scorers: [ai.eval.contains()],
+});
+
+expect(report.passed).toBe(true); // true only when EVERY case passed
+```
+
+The runner reads `.cases` off a dataset. Full `EvalReport`: `{ agentName, total, passedCount, failedCount, passRate, meanScore, passed, cases, duration, regression? }`. Each `EvalCaseResult` carries the case, the full `AgentResult`, every scorer's `scores`, the mean `score`, `passed`, and `duration`.
+
+## Regression gating — `baseline` + `tolerance`
+
+```ts
+import { readFile, writeFile } from "node:fs/promises";
+
+const baseline = ai.eval.fromJSON(await readFile("./eval/baseline.json", "utf8"));
+
+const report = await myAgent.eval({
+  cases: ds,
+  scorers: [ai.eval.exact()],
+  baseline,         // a prior EvalReport to diff against
+  tolerance: 0.05,  // max allowed per-case score DROP before it regresses. default 0 (any drop)
+});
+
+if (report.regression && !report.regression.passed) {
+  console.error("Regressed:", report.regression.regressed); // [{ name, before, after }]
+  process.exit(1);
+}
+```
+
+When `baseline` is set the report carries a `regression` block (`EvalRegression`), joining cases by `name`:
+
+- **`regressed`** — `[{ name, before, after }]` for cases whose new score fell more than `tolerance` below baseline.
+- **`added`** / **`removed`** — case names present in only one report. Adding or dropping a case **never fails the gate by itself**.
+- **`passed`** — `true` when `regressed` is empty.
+
+The pure `diff(report, baseline, tolerance)` function (exported as `diff`) is the same logic, decoupled from the runner — depends only on the two reports and the tolerance, mutates neither.
+
+## CI reporters — `ai.eval.toJUnit` / `toJSON` / `fromJSON`
+
+Pure functions over a finished `EvalReport`:
+
+```ts
+// JUnit-XML artifact for CI ingestion — one <testsuite> (the agent), one <testcase> per case,
+// a <failure> on each non-passing case (joined scorer reasons), times in SECONDS.
+await writeFile("./report.junit.xml", ai.eval.toJUnit(report));
+
+// Round-trippable snapshot — today's report becomes tomorrow's baseline.
+await writeFile("./eval/baseline.json", ai.eval.toJSON(report));
+const restored = ai.eval.fromJSON(await readFile("./eval/baseline.json", "utf8"));
+```
+
+`toJSON`/`fromJSON` preserve `result` payloads, per-case `scores`, timings, and any attached `regression` block, so a parsed report drives regression diffing exactly as the in-memory one. `toJUnit` hand-emits XML (no `xml` dependency) and entity-escapes every dynamic value.
+
+## Typical CI shard job
+
+```ts
+const shard = ai.dataset({ name: "support", fromFile: "./eval/support.jsonl" })
+  .shard(Number(process.env.SHARD_INDEX), Number(process.env.SHARD_TOTAL));
+
+const report = await agent.eval({
+  cases: shard,
+  scorers: [ai.eval.contains()],
+  baseline: ai.eval.fromJSON(await readFile("./eval/baseline.json", "utf8")),
+  tolerance: 0.05,
+});
+
+await writeFile(`./out/report-${process.env.SHARD_INDEX}.junit.xml`, ai.eval.toJUnit(report));
+if (report.regression && !report.regression.passed) process.exit(1);
+```
+
+## See also
+
+- [`@warlock.js/ai/ai-dx-helpers/SKILL.md`](@warlock.js/ai/ai-dx-helpers/SKILL.md) — `ai.eval.{exact,contains,predicate,judge}` scorers + Vitest matchers
+- [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — `agent.eval` scoring loop, `EvalCase` / `EvalJudge`
+- [`@warlock.js/ai/record-replay-llm/SKILL.md`](@warlock.js/ai/record-replay-llm/SKILL.md) — `ai.vcr` for deterministic, offline eval runs

package/skills/guard-input-output/SKILL.md

@@ -0,0 +1,117 @@
+---
+name: guard-input-output
+description: 'Build the composed guardrail middleware with @warlock.js/ai-guard and wire it into an agent — `ai.guardrail({ input, output, tool, toolNames, escalation })`. Triggers: `ai.guardrail`, `guard`, `GuardOptions`, `GuardrailVerdict`, `GuardrailDetector`, `GuardrailPhase`, `GuardrailMatch`, `GuardrailViolationError`, `ai.guardrail.topic`, `ai.guardrail.injection`, `topicFilter`, `injectionDetector`, `toolNames`, `forTool`; ''add a guardrail to my agent'', ''block prompt injection'', ''filter banned topics'', ''guard agent input and output'', ''stop the model leaking data into a tool call'', ''scope a detector to one tool''; typical import `import "@warlock.js/ai-guard"` (registers `ai.guardrail`) or `import { guard } from "@warlock.js/ai-guard"`. Skip: PII detection/redaction specifically — `@warlock.js/ai-guard/detect-and-redact-pii/SKILL.md`; routing a block to a human — `@warlock.js/ai-guard/escalate-block-to-human/SKILL.md`; the core middleware pipeline / hook contract — `@warlock.js/ai/run-ai-agent/SKILL.md`.'
+---
+
+# Guard agent input, output, and tool args
+
+`ai.guardrail(...)` is a **middleware factory**. It produces one `AgentMiddleware` that runs your detectors at three hook points and maps each verdict onto the agent pipeline's existing throw / return / record mechanics. Importing the package registers the verb (and its attached detector factories) on the shared `ai` namespace:
+
+```ts
+import { ai } from "@warlock.js/ai";
+import "@warlock.js/ai-guard"; // registers ai.guardrail + ai.guardrail.pii/.topic/.injection/.moderation
+
+const policy = ai.guardrail({
+  name: "compliance",
+  input: [ai.guardrail.injection({ onMatch: "block" })],
+  output: [ai.guardrail.topic({ deny: [/medical advice/i, "diagnosis"], onMatch: "block" })],
+});
+
+const agent = ai.agent({ model, middleware: [policy] });
+```
+
+A named-export form is available for callers who prefer not to rely on the augmented namespace:
+
+```ts
+import { guard, topic, injection } from "@warlock.js/ai-guard";
+const policy = guard({ input: [injection({ onMatch: "block" })] });
+```
+
+## The three phases
+
+| Phase | Hook | Inspected text | Set with |
+|---|---|---|---|
+| **input** | `trip.before` | the outbound prompt (`extractUserText(ctx.messages)`) | `input: [...]` |
+| **output** | `trip.after` | `response.content` | `output: [...]` |
+| **tool** | `tool.before` | `JSON.stringify(toolArgs)` | `tool: [...]` |
+
+Each phase array runs its detectors in **registration order**; the first non-`allow` verdict decides the action for that phase (short-circuit). A phase you don't configure is inert — a guard with no detectors is a no-op middleware.
+
+## The verdict model
+
+A detector inspects text and returns a `GuardrailVerdict`, discriminated by `type` (never `kind`):
+
+| `type` | Effect |
+|---|---|
+| `allow` | Pass to the next detector. |
+| `redact` | Rewrite the inspected text and continue — **output phase only** (see limitation below). |
+| `block` | Short-circuit with the existing `GuardrailViolationError`. |
+| `flag` | Pass, but append a `FlagRecord` into `ctx.state` under `<name>.flags` for a downstream observer (panoptic, the caller). |
+
+`agent.execute()` **never throws** — a `block` surfaces on `result.error` as a `GuardrailViolationError`, exactly like every other `AIError`. Branch on it after the run:
+
+```ts
+const result = await agent.execute(userInput);
+
+if (result.error instanceof ai.errors.GuardrailViolationError) {
+  // result.error.phase is "input" | "output" | "tool"
+  // result.error.reason / result.error.guardrail carry the detail
+}
+```
+
+## Built-in detectors
+
+Three zero-dependency detectors ship (a fourth, `moderation`, is an optional `openai` peer — see [`detect-and-redact-pii/SKILL.md`](@warlock.js/ai-guard/detect-and-redact-pii/SKILL.md)):
+
+- **`ai.guardrail.injection(options?)`** — jailbreak / prompt-injection marker phrases. Extra `markers` (string | RegExp); `onMatch` defaults to `"flag"`, callers commonly use `"block"` on input.
+- **`ai.guardrail.topic(options)`** — `deny` (string substring | RegExp) and/or `allow` (allow-list miss triggers `onMatch`). `onMatch` is `"block" | "flag"`, default `"block"`.
+- **`ai.guardrail.pii(options?)`** — PII regex + dictionary (its own skill).
+
+```ts
+const policy = ai.guardrail({
+  input: [
+    ai.guardrail.injection({ onMatch: "block", markers: ["ignore previous instructions"] }),
+    ai.guardrail.topic({ deny: ["competitor-name"], onMatch: "block" }),
+  ],
+});
+```
+
+## Scope tool detectors to specific tools
+
+`tool` detectors fire on **every** tool call by default. Set `toolNames` to scope them — the whole middleware is wrapped with the core `forTool(toolNames, mw)` helper so the `tool` hooks fire only for those names; `input` / `output` (`trip`) hooks are unaffected:
+
+```ts
+const policy = ai.guardrail({
+  tool: [ai.guardrail.pii({ onMatch: "block" })], // stop PII reaching the tool
+  toolNames: ["send_email", "post_webhook"],      // ...only for these tools
+});
+
+const agent = ai.agent({ model, tools: [sendEmail, postWebhook, lookup], middleware: [policy] });
+// `lookup` runs unguarded; `send_email` / `post_webhook` block on PII in their args.
+```
+
+A `block` from `tool.before` aborts that tool dispatch and surfaces on `result.error` with `phase: "tool"` — the agent run itself still never crashes.
+
+## Install order
+
+A guard is a normal `AgentMiddleware`; registration order is execution order (`before` top-down, `after` bottom-up). The canonical order is `[cache, budget, guardrail, observability]`. A `semanticCache` that short-circuits `trip.before` runs *before* the guard — a cached response then skips the **output** detectors, so place the guard before the cache if you don't trust cached contents.
+
+## Input-redaction limitation (v1)
+
+The core `trip.before` hook can only **short-circuit** (return a `ModelResponse`); it cannot rewrite the outbound prompt and continue. So:
+
+- **Input detectors are `block` / `flag` only.** A `redact` verdict on an input detector is treated as a `block` rather than silently passing an un-redacted prompt.
+- **Output redaction works** — `trip.after` returns a replacement `ModelResponse` with the rewritten `content`.
+- **Tool-arg `redact` is also withheld** — it downgrades to a `block` (`reason: "tool-arg-redaction-unsupported"`), because silently rewriting tool arguments changes the call's side-effects unpredictably.
+
+Lifting the input limitation needs a small, non-breaking core affordance and is deferred.
+
+## Failure isolation
+
+A detector's `check()` **rejecting** is an infrastructure fault, not a content violation — it is recorded as a `flag` (`<detector>.error`) into `ctx.state` and the fold **continues** (fail-open). A moderation-API outage degrades to missing annotation, never a failed agent run.
+
+## See also
+
+- [`@warlock.js/ai-guard/detect-and-redact-pii/SKILL.md`](@warlock.js/ai-guard/detect-and-redact-pii/SKILL.md) — the `pii` detector (detect/redact/block), the `mask` template, and the optional `moderation` peer.
+- [`@warlock.js/ai-guard/escalate-block-to-human/SKILL.md`](@warlock.js/ai-guard/escalate-block-to-human/SKILL.md) — routing a `block` to a human-review surface via `escalation.onBlock`.
+- [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — running the agent, the middleware pipeline, and the `GuardrailViolationError` on `result.error`.

package/skills/manage-prompts/SKILL.md

@@ -0,0 +1,186 @@
+---
+name: manage-prompts
+description: 'Unified prompt registry — ai.prompts: one process-wide store of named, versioned systemPrompt(...) builders keyed by name@version. Register by giving a prompt a meta.name (auto-registers), resolve by get(name) / resolve(name, versionOrTag, placeholders) / the inline name@selector form, bulk-register with define(name, versions), pin tags with tag(name, tag, version), compare with diff(name, from, to), round-trip with export() / import(snapshot), and quality-check with a unified validate(target, options) (deterministic missing-placeholder check + optional Nova-safe LLM-as-judge with verdict caching). Compose registered prompts into new ones with systemPrompt().merge(name, { fromVersion }) — provenance recorded in meta.composedFrom. ai.prompt is now a thin FACADE over ai.prompts (BREAKING vs the old standalone registry). Triggers: `ai.prompts`, `ai.prompt`, `PromptsManagerContract`, `PromptsManagerEntry`, `SystemPromptContract`, `SystemPromptMeta`, `SystemPromptMergeOptions`, `PromptsValidateOptions`, `PromptValidationResult`, `PromptValidateTarget`, `PromptTemplateVersion`, `PromptDiff`, `ExportedRegistry`, `defaultPromptsManager`, `prompts()`, `promptKey`, `meta`, `name`, `version`, `composedFrom`, `fromVersion`, `register`, `create`, `get`, `has`, `list`, `versions`, `resolve`, `define`, `tag`, `validate`, `diff`, `export`, `import`, `merge`, `judge`, `judgeCache`; ''register a prompt by name'', ''resolve a prompt by name@version or tag'', ''pin a production tag to a prompt version'', ''diff two prompt versions'', ''export / import the prompt registry'', ''validate a prompt for missing placeholders'', ''merge a registered prompt into another''; typical import `import { ai } from "@warlock.js/ai"`. Skip: composing a single prompt from persona + instruction blocks (the builder itself) — `@warlock.js/ai/write-system-prompt/SKILL.md`; runtime loadable skill bodies — `@warlock.js/ai/use-runtime-skills/SKILL.md`; eval scoring of agent outputs — `@warlock.js/ai/eval-datasets-and-ci/SKILL.md`; competing libs `langfuse` (direct), `promptfoo`.'
+---
+
+# `ai.prompts` — the unified prompt registry
+
+`ai.prompts` is ONE process-wide registry of named, versioned `systemPrompt(...)` builders keyed by `name@version`. A `systemPrompt(input, { name })` (or any `.meta({ name })` rename) auto-registers here; `ai.prompts.get(name)` / `.resolve(name)` read them back; `systemPrompt().merge(name)` folds a registered prompt into a new one. There is exactly **one storage shape** behind the whole prompt surface — a `SystemPromptContract` keyed by `name@version` — and `ai.prompt(...)` is now a thin facade over it (see the migration note below).
+
+```ts
+import { ai } from "@warlock.js/ai";
+
+// Register: any named systemPrompt auto-registers in ai.prompts.
+ai.systemPrompt("You are support for {{product}}.", { name: "support" });
+
+// Resolve back — latest version, or a version / pinned tag.
+ai.prompts.get("support");                 // → the SystemPromptContract
+ai.prompts.resolve("support", undefined, { product: "Warlock" }); // → final string
+```
+
+`ai.prompts` is the process-wide default (`defaultPromptsManager()`). For an **isolated** registry (parallel test suites, multi-tenant apps) call the `prompts()` factory — same `PromptsManagerContract`, its own store, no global side effects.
+
+## Identity — `SystemPromptMeta` (`meta.name` / `version` / `description` / `required` / `composedFrom`)
+
+A prompt's identity rides on its `meta`. Read it with the no-argument accessor; update it immutably with the one-argument form:
+
+```ts
+const base = ai.systemPrompt("You are support.", {
+  name: "support",
+  version: "1",
+  description: "Tier-1 support persona.",
+  required: ["product"],
+});
+
+base.meta();                          // → { name: "support", version: "1", description, required }
+const v2 = base.meta({ version: "2" }); // new builder, shallow-merged meta; original untouched
+```
+
+- **`name`** — when present, the prompt auto-registers in `ai.prompts` under `name@version`. Anonymous prompts (no `name`) are never registered.
+- **`version`** — free-form label (`"1"`, `"2025-draft"`). Defaults to the **next integer** for that name when omitted.
+- **`description`** — human-readable purpose (carried through `export`).
+- **`required`** — placeholder keys callers must supply; `validate()` reads them.
+- **`composedFrom`** — deterministic source labels a prompt was merged from (e.g. `["base@2", "global@1"]`). No random suffixes — the same merge always yields the same labels.
+
+## Register / resolve — `register` / `get` / `resolve` / `has` / `list` / `versions`
+
+```ts
+const registry = ai.prompts;              // or prompts() for an isolated one
+
+registry.register(ai.systemPrompt("You are support.", { name: "support" }));
+registry.versions("support");             // ["1"] — version derived as next integer
+
+registry.get("support");                  // latest SystemPromptContract
+registry.get("support@1");                // inline name@selector
+registry.resolve("support", "1", { product: "Warlock" }); // pick version + render in one call
+
+registry.has("support");                  // boolean
+registry.list();                          // every registered name, first-seen order
+```
+
+- **Version selection** — `get(name)` / `resolve(name)` return the **latest** by insertion order; pass a version label, a pinned tag, or fold it into the first arg as `name@selector` (`get("support@1")`, `resolve("support@production")`).
+- **Duplicates** — re-registering the same `name@version` throws `InvalidRequestError` **unless** the content is byte-identical (idempotent re-registration is a no-op).
+- **Unknown name / version / tag** → `InvalidRequestError`.
+- `register()` throws if the prompt has no `meta.name`.
+
+## `create()` — build + register in one entry point
+
+`ai.prompts.create(input?, meta?)` is a documented alias of `ai.systemPrompt(...)` — identical input forms (no arg → empty builder; a string → one instruction; an array of blocks → verbatim). Pass `meta.name` to auto-register, so authoring and lookup read side-by-side:
+
+```ts
+ai.prompts.create("You are support for {{product}}.", { name: "support" });
+ai.prompts.resolve("support", undefined, { product: "Warlock" });
+```
+
+## `define()` — bulk-register many versions
+
+```ts
+ai.prompts.define("agent", [
+  { version: "1", template: "You are v1." },
+  { version: "2", template: [ai.persona("You are Alex."), ai.instruction("Be concise.")] },
+]);
+```
+
+A `PromptTemplateVersion`'s `template` is a raw string (wrapped into one instruction block) or an explicit ordered block list (verbatim). Versions register **oldest-first** in array order; the same duplicate / idempotency rule applies per `name@version`. Returns the manager for chaining.
+
+## `tag()` — pin a moving label to a version
+
+```ts
+ai.prompts.tag("agent", "production", "2");   // pin "production" → version 2
+
+ai.prompts.get("agent", "production");        // resolves through the tag
+ai.prompts.resolve("agent", "production");
+ai.prompts.get("agent@production");           // inline form
+```
+
+Re-pinning an existing tag moves it. An unknown name / version throws `InvalidRequestError`. Tags survive `export` / `import`.
+
+## `validate()` — unified deterministic + optional LLM-judge
+
+```ts
+const report = await ai.prompts.validate("support", {
+  placeholders: { product: "Warlock" }, // values you intend to supply
+  declare: ["language"],                  // extra keys to treat as known
+  judge: judgeModel,                      // optional — turns on the LLM-as-judge pass
+});
+
+report.ok;       // true iff no required placeholder is missing (DETERMINISTIC verdict alone)
+report.missing;  // placeholder keys referenced with no default, unsupplied, undeclared
+report.score;    // 0..1 — present ONLY when a judge ran and produced a usable verdict
+report.issues;   // advisory judge reasons / a degrade note — present only when a judge was supplied
+```
+
+- **Always** runs the deterministic check: every `{{key}}` with no inline default that is neither supplied (`placeholders`), declared (`declare`), nor in the prompt's `meta.required` lands in `missing`; `ok` is `true` iff `missing` is empty.
+- **`judge`** adds a **Nova-safe** LLM-as-judge quality pass — it **never throws** and degrades to an `issues` note (leaving `score` undefined) on failure, so a flaky judge can **never flip `ok`**.
+- **`target`** is a registered name (or `name@selector`), a `SystemPromptContract` instance, or a raw prompt string.
+- **`judgeCache`** (per-call or via the `prompts({ judgeCache })` factory option) memoizes judge verdicts by a content hash of the resolved body + the judge model id — a structural `{ get, set }` subset of `@warlock.js/cache`'s `CacheDriver`, so the cache package stays a strictly **optional** peer.
+
+`systemPrompt().validate(options?)` is the per-builder sugar — `ai.prompts.validate(this, options)` under the hood, same result shape.
+
+## `diff()` — block-level version diff
+
+```ts
+const diff = ai.prompts.diff("agent", "1", "2");
+
+diff.identical;  // true when both versions have identical blocks in identical order
+diff.added;      // blocks in `to` not at the same position in `from`
+diff.removed;    // blocks in `from` not at the same position in `to`
+diff.changed;    // [{ from, to }] — same position, type/text changed
+```
+
+Blocks are matched **positionally**. Unknown name / version → `InvalidRequestError`.
+
+## `export()` / `import()` — portable JSON round-trip
+
+```ts
+const snapshot = ai.prompts.export();   // ExportedRegistry — every name, version, pinned tag, description/required
+otherRegistry.import(snapshot);          // rehydrate (same duplicate / idempotency rule; tags restored)
+```
+
+Each version flattens to `{ type, text }` blocks so the registry round-trips without live builder instances — commit a snapshot, ship it, restore it elsewhere.
+
+## Compose registered prompts — `systemPrompt().merge(name, { fromVersion })`
+
+`merge` folds another prompt's blocks into a new builder (persona **replaces**, instructions **append**) and records `meta.composedFrom`:
+
+```ts
+ai.systemPrompt("Always answer in {{language|English}}.", { name: "global", version: "1" });
+
+const supportPrompt = ai.systemPrompt("You are support for {{product}}.")
+  .merge("global", { fromVersion: "1" });  // fold the registered prompt by name
+
+supportPrompt.meta()?.composedFrom;        // ["…", "global@1"] — deterministic provenance
+```
+
+`merge` accepts three source forms: a pre-built block, another `SystemPromptContract`, or a **registered name** resolved from `ai.prompts` (latest version unless `options.fromVersion` selects another — an unknown name / version throws `InvalidRequestError`).
+
+## `ai.prompt(...)` — now a thin facade (⚠ breaking vs the old registry)
+
+`ai.prompt` has **two** call forms, both backed by the unified manager — there is no longer a separate prompt store:
+
+```ts
+// (a) Resolve a globally-registered prompt from ai.prompts by name.
+ai.systemPrompt("You are support.", { name: "support" });
+const sp = ai.prompt("support");          // → SystemPromptContract (latest)
+const v1 = ai.prompt("support", "1");     // → a specific version / pinned tag
+
+// (b) Build an ISOLATED legacy-shaped registry (PromptRegistryContract).
+const reg = ai.prompt({
+  prompts: [{ name: "summarizer", versions: [{ version: "1", template: "Summarize: {{text}}" }] }],
+});
+const resolved = reg.resolve("summarizer", { placeholders: { text } });
+resolved.toSystemPrompt();                 // drop-in for ai.agent({ systemPrompt })
+```
+
+**⚠ Migration.** Before unification, `ai.prompt(...)` only built a standalone, self-contained registry with its **own private** storage. It now:
+
+1. Adds the **string overload** `ai.prompt(name, versionOrTag?)` → resolves from the shared `ai.prompts` manager. (New capability — `ai.prompt("x")` used to be a type error.)
+2. Backs the **options form** (`ai.prompt({ ... })` → `PromptRegistryContract`) by an internal `PromptsManagerContract`, so its storage shape and validation primitives are now the unified ones. The legacy method surface (`register` / `add` / `versions` / `resolve` / `validate` / `sync` + the `{ score, notes }` report shape) is **unchanged**, and each `ai.prompt({ ... })` call still returns its **own isolated** registry — no shared global state.
+
+If you only ever called `ai.prompt({ ... })` and used the returned registry, **no code change is needed**. The new behavior is additive: prefer `ai.prompts` (the unified manager) for new code; reach for `ai.prompt({ ... })` only when you want the legacy `ResolvedPrompt` / `toSystemPrompt()` ergonomics or the optional Langfuse sync. The legacy facade's reference — `register` / `add` / `resolve(name, { version, placeholders })` / `validate` (`{ score, notes }`) / `sync()` (lazy `langfuse` peer) — is documented inline in `src/prompt/prompt.ts`.
+
+## See also
+
+- [`@warlock.js/ai/write-system-prompt/SKILL.md`](@warlock.js/ai/write-system-prompt/SKILL.md) — the `systemPrompt()` / `persona()` / `instruction()` builder, `.meta()`, and `merge()` this registry stores and composes
+- [`@warlock.js/ai/eval-datasets-and-ci/SKILL.md`](@warlock.js/ai/eval-datasets-and-ci/SKILL.md) — the eval `judge` scorer `validate()`'s LLM pass reuses
+- [`@warlock.js/ai/run-ai-agent/SKILL.md`](@warlock.js/ai/run-ai-agent/SKILL.md) — wiring a resolved prompt into an agent, plus the judge-safe agent preset (`ai.agent.judge`)