@dsalazardev/ai-hub-cli 0.0.0-stage → 0.1.0

package/skills/shared/sqlite-turso/references/embedded-replicas.md

@@ -0,0 +1,96 @@
+# Embedded replicas — deep reference
+
+Offloaded from SKILL.md because it is branch-specific and longer than the body should carry. Read
+this when you are actually building or operating an embedded-replica deployment.
+
+## What a replica actually is
+
+An embedded replica is a real local SQLite file (`url: "file:local.db"`) plus a `syncUrl` pointing at
+a remote primary. The client maintains the local copy and pulls a stream of **frames** (the unit of
+replication — roughly a batch of committed changes) from the primary. Reads execute against the local
+file with in-process latency. Writes are forwarded to the primary; they do not mutate the local file
+directly, they come back via the next sync.
+
+This is why the local file is mandatory: there is no replica without somewhere local to materialize
+the data. A `syncUrl` next to a `libsql://` url is a contradiction — there is no local store to sync
+into, so the client behaves as a plain remote client and never replicates.
+
+## Sync mechanics and frame numbers
+
+`await client.sync()` returns `{ frame_no, frames_synced }`:
+
+- `frame_no` — the replication frame the local replica is now at (a monotonically advancing
+  position in the primary's change log). Useful for logging/observability and reasoning about how far
+  behind a replica was.
+- `frames_synced` — how many frames this particular `sync()` call pulled. `0` means the replica was
+  already current.
+
+Two ways to drive sync:
+
+- **`syncInterval` (seconds)** in `createClient` — background periodic pull. Good for read-mostly
+  apps that tolerate eventual consistency up to one interval. Pick the interval from how stale a read
+  may be, not from a default.
+- **Manual `client.sync()`** — pull on demand: at request start, after a known remote write, on app
+  resume from background. Combine both: a slow `syncInterval` safety net plus targeted manual syncs
+  where freshness matters.
+
+Calling `sync()` on a client without a `syncUrl` + local `file:` (a remote-only or pure HTTP/WS
+client) throws `LibsqlError` with code `SYNC_NOT_SUPPORTED`. Guard the call by construction — only
+sync clients you built as replicas.
+
+## `readYourWrites` semantics
+
+Default is `true`. With it on, a write you issue through the client is immediately visible to your
+own subsequent reads on that client, even before the next background sync — the client tracks your
+write position and waits for the local replica to catch up to it on read. This is what most apps
+want: you click "save", you re-read, you see your change.
+
+Set `readYourWrites: false` only when you deliberately want a sync's effect to be observable on its
+own timeline — e.g. tests that assert "this row is not visible until I explicitly `sync()`", or
+benchmarks measuring pure replica staleness. If you turn it off in a normal app you will get
+confusing "I just wrote that, why is it gone?" reports.
+
+## Offline and bidirectional writes
+
+Classic libSQL embedded replicas are read-local / write-through-primary: you need connectivity to
+the primary to commit a write. True **offline writes** with bidirectional sync (write locally while
+disconnected, reconcile later) is a capability of **Turso Database (the Rust rewrite)**, not the
+production libSQL path. If a requirement says "must accept writes fully offline and merge later",
+that is the Rust engine's roadmap territory — flag it as not-yet-default-production rather than
+promising it on libSQL today.
+
+## Database-per-user / per-tenant patterns
+
+Turso's unlimited-databases free tier makes a separate database per tenant economical, which gives
+hard data isolation without row-level-security gymnastics.
+
+- **One primary per tenant**, each with its own `libsql://` URL and token. Mint tokens per database
+  with the `turso` CLI.
+- **Connection management**: do not create one `createClient` per request — cache clients keyed by
+  tenant id (a `Map<tenantId, Client>`), and reuse. Each embedded replica holds an open local file;
+  thousands of unmanaged replicas will exhaust file handles and disk.
+- **Replica per tenant** only where the access pattern is local-heavy for that tenant (e.g. a desktop
+  or edge node dedicated to one tenant). For a multi-tenant server hitting many tenants, prefer
+  remote clients (shape #2) and let the edge do the work, or you will be syncing a fleet of files.
+- **Provisioning**: tenant signup creates a database (`turso db create`) and a scoped token; store
+  the URL/token in your secrets store keyed by tenant, never in source.
+
+## Staleness, failure, and consistency
+
+- A replica serves the **last successfully synced state**. If sync fails (network, auth), reads keep
+  working against the stale local copy — this is a feature for availability, a trap for correctness.
+  Surface sync errors; do not swallow them.
+- `frames_synced: 0` over many calls when you expect changes means the primary genuinely has nothing
+  new, or your client is pointed at the wrong primary — check `syncUrl` and token scope.
+- Writes are linearized at the primary, so cross-tenant/global invariants belong at the primary, not
+  asserted against a possibly-stale replica.
+- For "must always read the absolute latest", call `sync()` immediately before the read, or use a
+  remote-only client (shape #2) for that path and accept the round-trip.
+
+## Where this hands off
+
+- Type-safe schema and queries on top of any of these clients → `drizzle-orm`.
+- Versioned, ordered, reversible schema changes (up/down files, a runner) → `db-migrations`. This
+  reference only covers ad-hoc DDL through `execute`/`batch`.
+- Backup/restore and point-in-time strategy as the goal → `backups`.
+- Generic SQL authoring independent of engine (window functions, query tuning) → `sql`.

package/skills/shared/sqlite-turso/scripts/verify.sh

@@ -0,0 +1,95 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+# verify.sh — sqlite-turso skill gate. Read-only, no network, deterministic.
+#
+# Statically checks @libsql/client integration file(s) for the foot-guns this skill exists to
+# prevent. Pass a file or a directory; with no arg it scans the current directory.
+#
+#   bash verify.sh                 # scan ./ for *.ts / *.js / *.mjs / *.mts
+#   bash verify.sh src/db.ts       # check one file
+#   bash verify.sh src/            # scan a directory
+#
+# What it checks PER FILE that imports @libsql/client:
+#   1. uses createClient (else: not really wiring the client) ......... FAIL
+#   2. if `syncUrl` appears, a local `file:` url must also appear ...... FAIL
+#      (an embedded replica must be backed by a local file)
+#   3. no hardcoded auth token literal (require process.env/Deno.env) .. FAIL
+#   4. file-backed client (`file:`) should set PRAGMA ... WAL .......... advisory [warn]
+#
+# Exit code: non-zero ONLY when a FAIL is recorded. Files with no @libsql/client import are
+# skipped. An empty/clean target prints nothing actionable and exits 0 (no false failure).
+
+YELLOW=$'\033[33m'; GREEN=$'\033[32m'; RED=$'\033[31m'; NC=$'\033[0m'
+EXIT=0
+warn() { printf '%s[warn]%s %s\n' "$YELLOW" "$NC" "$*"; }
+ok()   { printf '%s[ok]%s %s\n'   "$GREEN"  "$NC" "$*"; }
+err()  { printf '%s[fail]%s %s\n' "$RED"    "$NC" "$*"; EXIT=1; }
+
+TARGET="${1:-.}"
+
+# Collect candidate files (bash 3.2: no mapfile; NUL-delimited into an initialised array).
+FILES=()
+if [ -f "$TARGET" ]; then
+  FILES+=("$TARGET")
+elif [ -d "$TARGET" ]; then
+  while IFS= read -r -d '' f; do
+    FILES+=("$f")
+  done < <(
+    find "$TARGET" \
+      \( -path '*/node_modules/*' -o -path '*/.git/*' -o -path '*/dist/*' -o -path '*/build/*' \) -prune -o \
+      -type f \( -name '*.ts' -o -name '*.tsx' -o -name '*.js' -o -name '*.mjs' -o -name '*.mts' \) -print0 2>/dev/null
+  )
+else
+  warn "target not found: $TARGET (nothing to check)"
+  exit 0
+fi
+
+CHECKED=0
+for f in "${FILES[@]+"${FILES[@]}"}"; do
+  # Only inspect files that actually pull in the libSQL client.
+  grep -Eq "@libsql/client" "$f" 2>/dev/null || continue
+  CHECKED=$((CHECKED + 1))
+
+  # 1. must call createClient
+  if grep -Eq "createClient[[:space:]]*\(" "$f"; then
+    ok "$f: imports @libsql/client and calls createClient"
+  else
+    err "$f: imports @libsql/client but never calls createClient"
+  fi
+
+  # 2. embedded replica must be file-backed
+  if grep -Eq "syncUrl" "$f"; then
+    if grep -Eq "[\"'\`]file:" "$f"; then
+      ok "$f: syncUrl is backed by a local file: url (embedded replica)"
+    else
+      err "$f: syncUrl present but no local 'file:' url — that is a confused remote client, not a replica"
+    fi
+  fi
+
+  # 3. no hardcoded auth token literal. authToken must come from an env source.
+  if grep -Eq "authToken" "$f"; then
+    if grep -Eq "authToken[[:space:]]*:[[:space:]]*[\"'\`]" "$f"; then
+      err "$f: authToken assigned a string literal — read it from process.env / Deno.env instead"
+    elif grep -Eq "authToken[[:space:]]*:[[:space:]]*(process\.env|Deno\.env|import\.meta\.env|Bun\.env)" "$f"; then
+      ok "$f: authToken read from an environment source"
+    else
+      warn "$f: authToken present — confirm it comes from the environment, not a literal"
+    fi
+  fi
+
+  # 4. advisory: a file-backed client should set WAL
+  if grep -Eq "[\"'\`]file:" "$f"; then
+    if grep -Eiq "journal_mode[[:space:]]*=[[:space:]]*WAL" "$f"; then
+      ok "$f: file-backed client sets journal_mode=WAL"
+    else
+      warn "$f: file-backed client — consider PRAGMA journal_mode=WAL (advisory)"
+    fi
+  fi
+done
+
+if [ "$CHECKED" -eq 0 ]; then
+  ok "no @libsql/client integration files found — nothing to verify"
+fi
+
+exit "$EXIT"

package/skills/shared/structured-extraction/SKILL.md

@@ -0,0 +1,230 @@
+---
+name: structured-extraction
+description: "Use when text must become a typed, schema-conformant object you can trust — pulling fields into a fixed JSON shape, extracting line items as typed records, classifying into enums, and building the Pydantic or Zod model plus the validate-and-retry loop. Covers extractors that throw parse errors, leak markdown fences, or fabricate a value where the field is absent instead of returning null. NOT getting the text out of a PDF, scan or DOCX first (that is `document-processing`), NOT general prompt craft untied to a schema (that is `prompt-engineering`)."
+tags: [structured-outputs, json-schema, pydantic, extraction, validation, llm]
+recommends: [document-processing, prompt-engineering, llm-pipeline, agent-eval, rag, data-cleaning]
+profiles: []
+origin: ai-hub
+---
+
+# Structured extraction — text in, a typed object you can trust out
+
+The deliverable is **a typed object that conforms to a schema you defined** — not prose, not "roughly JSON."
+The whole skill rests on one distinction the rest of the file keeps returning to:
+
+> Native structured outputs make the JSON **valid and typed**. They never make the values **correct**.
+
+Constrained decoding guarantees the model cannot emit a token that breaks your schema, so `JSON.parse`
+errors, missing keys, wrong types, and stray markdown fences disappear at the source. It does **nothing**
+to stop the model from putting a plausible-but-wrong email in a `string` field, snapping a fuzzy category to
+the wrong enum, or coercing `"$1,200"` into `1200.0` when the currency mattered. Owning both halves — the
+shape (decoding) and the values (validation) — is this skill. If you only do the first half you ship a
+database full of well-typed lies.
+
+**Boundary test (bytes vs. schema).** If the input is a PDF, scan, DOCX, or HTML and the deliverable is the
+*raw text/Markdown/cells* of that document, that is upstream: [`document-processing`](../document-processing/SKILL.md)
+produces the text, this skill turns that text into typed fields. If you're holding text and want it shaped,
+you're in the right place.
+
+Current as of 2026-06-02: OpenAI Structured Outputs (`strict: true` json_schema), Anthropic Structured
+Outputs (GA since the 2025-11-14 public beta; `output_config.format`), and Instructor (built on Pydantic,
+~3M downloads/month). Exact request/response shapes and the per-provider limit tables live in
+[`references/providers.md`](references/providers.md) so this file stays lean.
+
+## Default: native constrained decoding when the provider has it
+
+If the model and provider support native structured outputs, use them. This is not a tuning knob — it is the
+difference between ~100% schema conformance and hoping a regex catches the fence.
+
+**Bad — prompt-and-pray, then parse raw text:**
+
+```python
+resp = client.chat.completions.create(
+    model="gpt-5.1",
+    messages=[{"role": "user", "content": f"Return JSON with name and email:\n{text}"}],
+)
+data = json.loads(resp.choices[0].message.content)  # markdown fence / preamble / missing key -> crash
+```
+
+**Good — OpenAI strict json_schema (Chat Completions):**
+
+```python
+resp = client.chat.completions.create(
+    model="gpt-5.1",
+    messages=[{"role": "user", "content": text}],
+    response_format={
+        "type": "json_schema",
+        "json_schema": {
+            "name": "contact",
+            "strict": True,
+            "schema": {
+                "type": "object",
+                "additionalProperties": False,
+                "required": ["name", "email"],
+                "properties": {
+                    "name": {"type": "string", "description": "Full name as written."},
+                    "email": {"type": ["string", "null"],
+                              "description": "Email exactly as written, or null if none is stated."},
+                },
+            },
+        },
+    },
+)
+data = json.loads(resp.choices[0].message.content)  # now guaranteed valid + typed
+```
+
+On the OpenAI Responses API the same block moves under `text.format` instead of `response_format`. On
+Anthropic, the equivalent is `output_config={"format": {"type": "json_schema", "schema": {...}}}` on Claude
+Opus 4.5–4.8 / Sonnet 4.5+ / Haiku 4.5; Anthropic compiles your schema into a grammar and **caches it for
+24h**, and the SDKs ship helpers (`client.messages.parse(...)` in Python, `zodOutputFormat(schema)` in TS).
+The older deprecated `output_format` param and the deprecated `structured-outputs-2025-11-13` beta header
+still work in a transition window — do not write new code against them. Full shapes in
+[`references/providers.md`](references/providers.md).
+
+**The non-negotiable strict-schema rule (OpenAI and Anthropic both):**
+
+- every object sets `"additionalProperties": false`;
+- **every** property is listed in `required`;
+- "optional" is expressed as a **union with `null`** (`"type": ["string", "null"]`), never by leaving the
+  field out of `required`. Omitting it is the single most common strict-mode error.
+
+## Schema design rules (each prevents a specific failure)
+
+| You want | Express it as | Because |
+| --- | --- | --- |
+| A field that may be absent | nullable union `["string","null"]` + `description: "...or null if not stated"` | A non-null type *forces* a value, so the model fabricates one. Nullable + instruction yields `null` instead. |
+| A closed set of categories | `enum: ["open","pending","closed"]` | Free-text `string` drifts ("Open", "in progress", "closd"); an enum makes drift impossible to emit. |
+| Many items of one kind | one object schema + a top-level `{"items": {"type":"array","items": <object>}}` wrapper | One object per extraction unit keeps each record independently validatable; arrays of scalars lose field structure. |
+| The model to read your intent | a `description` on every property | The model reads field descriptions at decode time; "amount in cents, no currency symbol" beats a bare `integer`. |
+| A number in a range / a regex / a length cap | leave it **out** of the schema; enforce in a post-decode validator | Strict modes reject or silently ignore `minimum`/`maximum`/`minLength`/`maxLength`/complex regex — see the unsupported-features table in references. |
+| A deeply nested or recursive shape | flatten it, or split into two extractions | Native modes reject recursion and cap nesting/complexity; flat schemas decode reliably. |
+
+Keep schemas **flat and shallow**. If you find yourself nesting four levels deep or describing a tree, that
+is two extractions, not one heroic schema.
+
+## The reliability ladder — escalate only on failure, and cap it
+
+Climb from the cheapest mechanism upward. Each rung catches what the rung below cannot; you stop at the
+first rung that holds for your data.
+
+| Rung | Mechanism | Catches | Does NOT catch | When you stop here |
+| --- | --- | --- | --- | --- |
+| 1 | Native constrained decoding | parse errors, wrong types, missing keys, fences | wrong values, bad units, wrong enum | shape+types only, latest single provider |
+| 2 | Pydantic / Zod validation after decode | out-of-range, bad format, cross-field contradictions, null-vs-absent | nothing the model genuinely got wrong | value rules you can express as code |
+| 3 | Bounded reask (Instructor or hand-rolled) | semantic errors the model can fix when shown the validation message | systematic model blind spots | residual errors; **cap retries (e.g. 2) and log every reask** |
+| 4 | Human / log review | everything still wrong after 3 | — | high-stakes fields or low-confidence rows |
+
+Rung 1 is mandatory when available. Rung 2 is mandatory the moment any field has a *value* rule (a range, a
+format, a "must match the order date") — because rung 1 structurally cannot enforce values. Rungs 3 and 4
+are opt-in. **Never** make rung 3 unbounded: a retry loop with no cap turns one bad document into an
+unbounded bill.
+
+## Value validation the schema can't enforce
+
+This is the half native decoding leaves on the table. Validate values *after* you have a typed object.
+
+**Pydantic — value rules + normalization the schema can't carry:**
+
+```python
+from pydantic import BaseModel, field_validator
+
+class Order(BaseModel):
+    amount_cents: int
+    discount_pct: float | None  # nullable: may be absent
+    order_date: str             # we'll normalize to ISO
+
+    @field_validator("discount_pct")
+    @classmethod
+    def pct_in_range(cls, v):
+        if v is not None and not (0 <= v <= 100):
+            raise ValueError("discount_pct must be between 0 and 100")
+        return v
+
+    @field_validator("amount_cents", mode="before")
+    @classmethod
+    def strip_currency(cls, v):
+        if isinstance(v, str):  # "$1,200.00" -> 120000
+            return int(round(float(v.replace("$", "").replace(",", "")) * 100))
+        return v
+```
+
+The Zod equivalent uses `.refine()` for cross-field and range checks and `.transform()` for normalization.
+Three normalizations bite constantly: **currency** (`"$1,200"` vs `1200` vs `120000` cents — pick one and
+enforce it), **dates** (free text → ISO 8601, and decide what a missing year means), and **enum snapping**
+(the model rounds "kinda urgent" to `urgent`; validate that the snap was legitimate, or widen the enum).
+
+**Null vs. absent.** A nullable field with a clear instruction is the entire fix for "the model invents an
+email." `"email": {"type": ["string","null"], "description": "...or null if the text states no email"}` plus
+a one-line system instruction ("use null for any field not present in the source; never guess"). If you make
+the field non-nullable, you have *told the model to produce a value* — it will.
+
+**Bounded reask with Instructor** — failed validation is fed back to the model as an error message:
+
+```python
+import instructor
+
+client = instructor.from_provider("openai/gpt-5.1")
+order = client.chat.completions.create(
+    response_model=Order,            # your Pydantic model, validators and all
+    max_retries=2,                   # BOUND it; each retry is another paid call
+    messages=[{"role": "user", "content": text}],
+)
+```
+
+On a validation failure Instructor reasks with the `ValueError` text, so `@field_validator` rules the model
+never saw in the schema still get enforced through the loop. Log every reask (count + reason): a quietly
+climbing reask rate is your early signal that a field's instruction or schema is wrong.
+
+## Multi-provider: Instructor `from_provider`
+
+When you want one Pydantic model to run across OpenAI, Anthropic, and local backends without rewriting per
+SDK, use Instructor's unified entrypoint:
+
+```python
+client = instructor.from_provider("anthropic/claude-opus-4-8")  # or "openai/gpt-5.1", "ollama/llama3.3"
+```
+
+Reach for Instructor when you need **provider portability** or **value-level validation with reask**. Reach
+for the **native SDK helper** (`client.messages.parse`, `zodOutputFormat`) when you're on one provider and
+want the simplest path with the fewest dependencies. Both sit on the same native decoding underneath.
+
+## Scale and accuracy live next door
+
+This skill is the single extraction node and its per-call validation loop. Two concerns are explicitly *not*
+here:
+
+- Running 10k documents — batching, idempotency, retries-across-calls, cost/latency budgeting, multi-step
+  chains → [`llm-pipeline`](../llm-pipeline/SKILL.md).
+- Measuring extraction quality offline — a golden set, precision/recall, a regression gate that fails CI when
+  accuracy drops → [`agent-eval`](../agent-eval/SKILL.md). This skill *builds* the extractor; that one *scores* it.
+- Answering questions over a corpus by retrieving chunks → [`rag`](../rag/SKILL.md).
+- Post-extraction tabular cleanup (dedupe rows, coerce columns, normalize categories across a whole dataset)
+  → [`data-cleaning`](../data-cleaning/SKILL.md).
+- Improving a prompt that isn't bound to a schema → [`prompt-engineering`](../prompt-engineering/SKILL.md).
+
+(Some routed siblings may not be built in this collection yet; the routing decision still holds.)
+
+## Anti-patterns
+
+| Bad | Why it bites | Good |
+| --- | --- | --- |
+| `json.loads(resp.text)` on raw model output | markdown fence, chatty preamble, or a missing key crashes at runtime | native structured outputs; parse only a decoder-guaranteed string |
+| Stripping ` ```json ` fences with a regex | treats the symptom; the model can still drop a key or change a type | turn on native decoding — the fence never appears |
+| `"type": "string"` on a field that's often absent | forces a value, so the model fabricates a plausible wrong one | nullable union `["string","null"]` + "null if not stated" |
+| Omitting an optional field from `required` (strict mode) | OpenAI/Anthropic strict reject it — *every* property must be in `required` | keep it in `required`, make its type a union with `null` |
+| `minimum`/`maxLength`/lookahead-regex inside a strict schema | rejected or silently ignored — the constraint does nothing | leave value rules out of the schema; enforce in a Pydantic/Zod validator |
+| `max_retries` unbounded (or a `while` reask loop) | one bad doc becomes an unbounded bill and a hung job | cap at 2–3, log each reask, route the rest to review |
+| Deep/recursive schema in one call | native modes reject recursion and cap complexity → compile failure | flatten, or split into multiple extractions |
+| Trusting decoding to make values *correct* | valid+typed ≠ true; you ship well-formed wrong data | add the rung-2 validation step for every value rule |
+| Building on Anthropic `output_format` / `structured-outputs-2025-11-13` header | deprecated transition-window API | use `output_config={"format": {...}}` |
+| One giant array of scalars for "many things" | loses per-item field structure and per-item validation | one object schema per unit, wrapped in a top-level `items` array |
+
+## Checklist before you ship an extractor
+
+1. Native structured outputs ON (or a documented reason the provider has no native mode).
+2. Every object has `additionalProperties: false`; every property is in `required`.
+3. Every maybe-absent field is nullable with a "use null if not stated" instruction.
+4. Closed sets are enums; value rules (ranges/format/cross-field) live in validators, not the schema.
+5. A reask loop, if any, is **bounded and logged**.
+6. You can state, for each field, whether a wrong output would be caught by decoding (shape) or only by
+   validation (value) — and you have the validation for the value cases.

package/skills/shared/structured-extraction/evals/README.md

@@ -0,0 +1,13 @@
+# Evals — structured-extraction
+
+These cases are read by a human or an LLM grader; nothing here runs a model automatically. To check the
+skill, eyeball each `should_trigger` prompt against the SKILL.md `description` and confirm the routing
+language would pull this skill in — the set deliberately includes a symptom-led, non-obvious case (the
+"markdown fence / JSON.parse blows up" prompt, whose real fix is native structured outputs, not fence
+stripping) and a Spanish/Catalan case. Then check each `should_not_trigger` prompt routes to the named
+sibling instead (`document-processing` for raw text-out, `prompt-engineering` for schema-less prompt craft,
+`llm-pipeline` for multi-node/batch, `agent-eval` for offline scoring, `rag` for retrieval, `data-cleaning`
+for tabular cleanup) — those are the boundary lines the description must hold. The `capability` case is a
+rubric: read it as "could an agent following SKILL.md produce an answer hitting every `must_include` bullet?"
+If the body can't satisfy a bullet, fix the body, not the rubric. To statically lint the skill's own example
+code (no network, no SDK calls), run `scripts/verify.sh`.

package/skills/shared/structured-extraction/evals/cases.yaml

@@ -0,0 +1,70 @@
+skill: structured-extraction
+
+# Prompts that MUST load `structured-extraction`. The skill owns turning text
+# into a typed, schema-conformant object: defining the schema, choosing the
+# native decoding mechanism, and building the value-validation + bounded-reask
+# loop. The deliverable is a typed object that conforms to a schema you defined
+# — both its SHAPE (decoding) and its VALUES (validation).
+should_trigger:
+  - prompt: "Extract name, email, and total amount from these invoices into a fixed JSON schema."
+    why: "Canonical text-in/typed-object-out: a defined schema plus extraction is exactly this skill."
+
+  - prompt: "My LLM keeps returning a markdown fence and a chatty intro, JSON.parse blows up — make extraction reliable."
+    why: "Non-obvious: the symptom reads like a parsing bug, but the real fix is native structured outputs, not regex fence-stripping."
+
+  - prompt: "When the document has no phone number the model invents one — I want null instead."
+    why: "The value-correctness half: null-vs-absent is solved with a nullable schema field plus a 'use null if not stated' instruction, not by re-prompting harder."
+
+  - prompt: "Build the Pydantic model and a validate-and-retry loop to pull line items from 5k receipts."
+    why: "Schema design + bounded reask loop for value validation — the rung-2/rung-3 core of this skill."
+
+  - prompt: "Convertir estos contratos en datos estructurados con un esquema fijo y validar los campos."
+    why: "Spanish/Catalan: turn contracts into structured data with a fixed schema and validate the fields — squarely this skill."
+
+  - prompt: "Classify each support ticket into one of these 6 enums and pull the order id as structured output."
+    why: "Closed-set enum classification plus field extraction, emitted as a schema-conformant object."
+
+# NEAR-MISS prompts that must NOT load `structured-extraction`. The recurring
+# trap: the real deliverable is raw document bytes/text, a generic prompt, a
+# multi-node pipeline, an offline accuracy scorecard, or a retrieval index.
+should_not_trigger:
+  - prompt: "Get the text and tables out of this scanned PDF."
+    route_to: "document-processing"
+    why: "Produces the unstructured text this skill consumes; bytes-out, not schema-out."
+
+  - prompt: "Help me write a better system prompt with few-shot examples for my chatbot."
+    route_to: "prompt-engineering"
+    why: "General prompt craft not bound to a target schema; schema design and constrained decoding live here, 'make this prompt better' lives there."
+
+  - prompt: "Orchestrate a 4-step chain that summarizes then routes then drafts, and batch 50k docs cheaply."
+    route_to: "llm-pipeline"
+    why: "Multi-node orchestration, batching, and cost/latency — not a single extraction node."
+
+  - prompt: "Score my extractor's precision and recall against a labeled golden set and gate regressions in CI."
+    route_to: "agent-eval"
+    why: "Offline measurement harness; this skill builds the extractor, agent-eval scores it."
+
+  - prompt: "Answer questions over my document corpus by retrieving the relevant chunks."
+    route_to: "rag"
+    why: "Retrieval over a corpus, not parsing one document into a fixed schema."
+
+  - prompt: "Dedupe and normalize the categories across this 200k-row CSV of customers."
+    route_to: "data-cleaning"
+    why: "Tabular cleanup across a whole dataset, not extracting typed fields out of free text."
+
+# Capability: graded against the must_include rubric.
+capability:
+  - scenario: >
+      Given a paragraph of free text describing a customer order — sometimes
+      missing the email or the discount percent, sometimes writing the amount
+      as "$1,200.00" — produce a reliable typed extraction (vendor, amount,
+      email, discount_pct, order_date). Outline the schema and the code you
+      would write.
+    must_include:
+      - "Uses a native structured-outputs mode (OpenAI strict:true json_schema OR Anthropic output_config.format), NOT prompt-and-json.loads."
+      - "Schema sets additionalProperties:false and lists every property in required."
+      - "Maybe-absent fields (email, discount_pct) are modeled as a nullable union with an instruction to use null when absent, so the model does not fabricate them."
+      - "Adds a post-decode value-validation step (Pydantic @field_validator / Zod .refine) for ranges, currency normalization ('$1,200' -> int), and date->ISO that the schema cannot enforce."
+      - "Any reask/retry on validation failure is BOUNDED (e.g. max_retries=2) and logged, never an unbounded loop."
+      - "Explicitly distinguishes 'valid JSON' (decoding) from 'correct values' (validation) and assigns each failure mode to the rung that catches it."
+      - "Uses current APIs, not the deprecated Anthropic output_format / structured-outputs-2025-11-13 header."

package/skills/shared/structured-extraction/references/providers.md

@@ -0,0 +1,152 @@
+# Provider reference — exact shapes, limits, and feature support
+
+Current as of 2026-06-02. Pins and shapes verified against the OpenAI Structured Outputs guide, the
+Anthropic Structured Outputs docs (GA after the 2025-11-14 public beta), and Instructor
+(python.useinstructor.com / github.com/567-labs/instructor). The SKILL body carries the rules; this file
+carries the literal request blocks and the limit tables so the body stays lean.
+
+## OpenAI
+
+### Chat Completions — `response_format`
+
+```python
+resp = client.chat.completions.create(
+    model="gpt-5.1",
+    messages=[{"role": "user", "content": text}],
+    response_format={
+        "type": "json_schema",
+        "json_schema": {
+            "name": "invoice",
+            "strict": True,
+            "schema": {
+                "type": "object",
+                "additionalProperties": False,
+                "required": ["vendor", "total_cents", "due_date"],
+                "properties": {
+                    "vendor":     {"type": "string"},
+                    "total_cents":{"type": "integer", "description": "Total in cents, no symbol."},
+                    "due_date":   {"type": ["string", "null"], "description": "ISO date or null."},
+                },
+            },
+        },
+    },
+)
+data = json.loads(resp.choices[0].message.content)
+```
+
+### Responses API — `text.format`
+
+Same json_schema block, relocated:
+
+```python
+resp = client.responses.create(
+    model="gpt-5.1",
+    input=text,
+    text={"format": {"type": "json_schema", "name": "invoice", "strict": True, "schema": {...}}},
+)
+```
+
+### OpenAI strict rules
+
+- `strict: true` gives constrained decoding: the model **cannot** emit tokens that violate the schema
+  (~100% schema conformance in OpenAI's evals).
+- Every object **must** set `"additionalProperties": false`.
+- **Every** property must appear in `required`. Optional = a union with `null`
+  (`"type": ["string","null"]`), never omission from `required`.
+- `response_format` (shape the final answer) and `tools`/function calling (the model calls your code) are
+  different jobs — pick function calling when you want a tool invoked, structured outputs when you want a
+  shaped answer. Don't use both to shape the same final answer.
+
+## Anthropic (Claude Developer Platform)
+
+### `output_config.format` (current, GA)
+
+```python
+resp = client.messages.create(
+    model="claude-opus-4-8",
+    max_tokens=2048,
+    messages=[{"role": "user", "content": text}],
+    output_config={"format": {"type": "json_schema", "schema": {
+        "type": "object",
+        "additionalProperties": False,
+        "required": ["vendor", "total_cents", "due_date"],
+        "properties": {
+            "vendor":      {"type": "string"},
+            "total_cents": {"type": "integer"},
+            "due_date":    {"type": ["string", "null"]},
+        },
+    }}},
+)
+```
+
+- Anthropic compiles the schema into a grammar and **caches it for 24h**, then constrains decoding
+  token-by-token.
+- Supported models: Claude Opus 4.5 / 4.6 / 4.7 / 4.8, Sonnet 4.5 / 4.6, Haiku 4.5. Also on Bedrock,
+  Vertex, and Foundry (beta).
+- SDK helpers: Python `client.messages.parse(...)` returns a parsed object; TS `zodOutputFormat(schema)` /
+  `jsonSchemaOutputFormat(schema)`.
+- **Deprecated** (transition window only — do not write new code against these): the `output_format` param
+  and the `structured-outputs-2025-11-13` beta header.
+
+### Strict tool use (separate feature)
+
+To constrain a *tool call's* input, set `"strict": true` on the tool's `input_schema`. This is independent
+of `output_config.format`, which shapes the assistant's final answer.
+
+```python
+tools=[{"name": "save_order", "input_schema": {"type": "object", "strict": True, ...}}]
+```
+
+### Anthropic schema constraints (the runtime gotchas)
+
+- `additionalProperties` must be `false` for all objects.
+- **Not supported**: recursive schemas; numeric bounds (`minimum`, `maximum`, `multipleOf`); string length
+  (`minLength`, `maxLength`); array constraints beyond `minItems` of 0 or 1; complex regex (lookahead,
+  lookbehind, backreferences).
+- Complexity limits per request (combined across all strict schemas): **≤20 strict tools**, **≤24 optional
+  parameters** total, **≤16 union-typed (`anyOf`) parameters**, **180s** schema-compile timeout.
+- Implication: the schema enforces **shape and types only, never values or ranges**. Push every value rule
+  into a post-decode validator.
+
+## Instructor (cross-provider, Pydantic-based)
+
+```python
+import instructor
+from pydantic import BaseModel
+
+class Order(BaseModel):
+    vendor: str
+    total_cents: int
+    due_date: str | None
+
+client = instructor.from_provider("anthropic/claude-opus-4-8")  # or "openai/gpt-5.1", "ollama/llama3.3"
+order = client.chat.completions.create(
+    response_model=Order,
+    max_retries=2,            # bound the reask loop
+    messages=[{"role": "user", "content": text}],
+)
+```
+
+- Built on **Pydantic**; unified `from_provider("<provider>/<model>", ...)` across 15+ backends (OpenAI,
+  Anthropic, Gemini, Ollama, DeepSeek, …). TS / Go / Ruby ports exist.
+- Core value beyond native decoding: the **validate→retry** loop. A failed Pydantic validation — including
+  custom `@field_validator` rules the model never saw in the schema — is fed back to the model as the error
+  message; `max_retries` caps the reasks.
+- Use it for **provider portability** or **value-level validation with reask**. For a single provider and
+  the simplest dependency footprint, the native SDK helper (`messages.parse` / `zodOutputFormat`) is enough.
+- Add a hook to log reasks (count + reason) so a rising reask rate surfaces a bad field instruction early.
+
+## Consolidated: unsupported JSON Schema features (push these to validators)
+
+| Feature | OpenAI strict | Anthropic | Where it must go instead |
+| --- | --- | --- | --- |
+| `additionalProperties: false` | required | required | n/a (must set it) |
+| Optional via omission from `required` | rejected | rejected | union with `null`, keep in `required` |
+| `minimum` / `maximum` / `multipleOf` | not enforced | not supported | Pydantic `@field_validator` / Zod `.refine` |
+| `minLength` / `maxLength` | not enforced | not supported | validator after decode |
+| Array `minItems`/`maxItems` (beyond 0/1) | limited | beyond `minItems` 0/1 unsupported | validator after decode |
+| Complex regex (lookahead/backref) | unsupported | unsupported | validator with a real regex engine |
+| Recursive schemas | limited | not supported | flatten / split into multiple extractions |
+
+The single safe assumption: **strict/native modes enforce shape and type only.** Every range, length,
+format, cross-field, or enum-legitimacy rule is your validator's job.