@dsalazardev/ai-hub-cli 0.0.0-stage → 0.1.0

package/skills/shared/level-design/references/guidance-and-readability.md

@@ -0,0 +1,152 @@
+# Guidance & readability — direct the player without hand-holding
+
+How to make a player always know where to go, understand what they can do, and
+feel clever for exploring — using the environment instead of arrows. Also
+covers the playtest protocol, traversal telemetry, and reward/secret placement,
+because those are how you *diagnose and fix* readability.
+
+## The layers of guidance (strongest to weakest)
+
+The eye is drawn, roughly in this priority, to: **motion → light/brightness →
+high contrast → saturated color → the different/odd → framed shapes.** Stack
+these on the path you want taken; strip them from paths you don't.
+
+### Light leads
+
+Players move toward light — the moth effect — and toward higher value contrast.
+
+- Light the **critical path, the objective, and the exit**; keep dead-ends,
+  non-paths, and cut branches dim.
+- A bright opening at the end of a dark corridor is an irresistible pull.
+- Use light *in greybox* as a design tool, not a final-art afterthought — the
+  guidance is authored in placeholder lighting and art merely honors it.
+- Contrast beats absolute brightness: a shaft of light in gloom guides harder
+  than a uniformly bright room.
+
+### Color as language
+
+Reserve **one saturated accent color** to mean "you can interact / go here" and
+use it *nowhere else*.
+
+- Precedents: Mirror's Edge's "runner red" for the traversal path; Dishonored's
+  faint glint on usable objects; the near-ubiquitous "yellow = climbable/grab"
+  convention.
+- The value of the convention *is its consistency*. The moment the interact
+  color appears on decoration, you've lied — the player learns to distrust it
+  and starts checking everything, which is the opposite of guidance.
+- Keep the rest of the palette desaturated relative to the accent so it pops.
+
+### Composition
+
+Compose the space like a photographer composes a shot.
+
+- **Leading lines** — rails, beams, roads, pipes, tiling that point toward the
+  objective.
+- **Framing** — arches, doorways, foreground silhouettes that box the eye onto
+  the target.
+- **Negative space** and **contrast** to isolate what matters.
+- From every decision point, the critical path should be the most-framed,
+  highest-contrast, best-lit option.
+
+### Affordances & signifiers
+
+(Norman/Gibson.) An affordance is what the environment *lets* you do; a signifier
+is the visual cue that advertises it.
+
+- A ledge that looks grabbable **must** be grabbable; a wall that looks solid
+  **must** be solid. A door-shaped thing that never opens teaches the player to
+  ignore doors.
+- Establish the visual language **early** and hold it: if a specific trim, decal,
+  or material means "climb here," that cue appears **only** where you can climb.
+- Inconsistent affordances are the #1 source of "I didn't know I could do that"
+  and "I thought I could do that and died." Consistency > cleverness.
+- Prefer **diegetic signifiers** (a rope, scuff marks, a worn path, a ladder)
+  over UI prompts; save prompts for genuinely non-obvious systemic actions.
+
+### Landmarks & self-location
+
+- One **dominant landmark** per area — a large, distinct, distant silhouette
+  (Disney's "weenie": the visual magnet that pulls you forward). It answers
+  "where am I / where's the goal" without a minimap.
+- Keep landmarks visible from key decision points; if the player can't see the
+  landmark when they need to choose, it isn't doing its job.
+- **Deny wrong turns visually** — subtle dead-ends, unlit gaps, a distinct
+  "not-a-path" material — rather than relying on invisible collision, which
+  feels arbitrary and cheap.
+
+### The critical-path readability self-test
+
+Stand at each decision point in the greybox and ask: *with no HUD and no prior
+knowledge, is the intended next step the most obvious thing here?* If you have
+to argue for it, a fresh player will get lost. Fix with light/framing/landmark
+before reaching for a waypoint.
+
+## Playtesting — observe, don't coach
+
+You cannot see your own level fresh; your knowledge of it is the one thing a
+player won't have.
+
+1. **Fresh, uncoached tester.** Say "play." Then stay silent.
+2. **Log four signals:**
+   - **Hesitation / stopping** → readability gap (they can't tell where to go).
+   - **Wrong turn / backtrack** → navigation gap (guidance points wrong or is
+     absent).
+   - **Repeated death** → telegraph or difficulty gap (threat unfair or
+     unreadable).
+   - **Skipped content** → reward-placement or guidance gap.
+3. **Never answer "where do I go?"** — that question *is* the finding. Answering
+   it destroys the data and hides the bug.
+4. **Ask after, not during**: "Where did you think you were supposed to go? What
+   did you expect that button to do? Where did you feel stuck?" Open questions,
+   no leading.
+5. Test on the **greybox**, early and often — fixes are cheap while it's grey.
+
+## Traversal telemetry & heatmaps
+
+Turn anecdote into pattern by logging player data across many sessions:
+
+| Metric | What it shows | The fix when it's bad |
+|---|---|---|
+| **Position heatmap** | Where players spend time; **cold zones** they never enter | Cold zone = dead space: re-light it, add a landmark/reward, or cut it |
+| **Death heatmap** | Where players die; **clusters** = unfair/unreadable threats | Fix the telegraph, the arena, or the difficulty — not the player |
+| **Path traces** | Are they on the golden path or lost? | Divergence from intended path = guidance failure at that fork |
+| **Dwell time** at forks | Long dwell = unclear guidance | Add light/framing/landmark to make the choice obvious |
+| **Look / aim heatmap** | Did they even *see* the landmark or reward? | If not looked-at, it's mis-placed or out-competed — reposition |
+| **Time-to-objective** | Pacing vs intent (too fast/slow) | Adjust distance, breather length, or beat spacing |
+
+Read the map as instructions: a branch nobody takes needs a visible reward at
+its mouth; a region everyone avoids isn't pulling them — fix the pull or remove
+the region.
+
+## Reward & secret placement
+
+- **Reward the wandering eye.** Put secrets where a *curious* player naturally
+  looks — behind the waterfall, up the ledge you already lit and framed, down
+  the branch you breadcrumbed. Reward **observation**, not pixel-hunting.
+- **Breadcrumb every secret.** A glint, an out-of-place object, a suspicious
+  seam, an odd bit of geometry visible from the path. The player should feel
+  clever — which means the game showed them the thread first. A secret with no
+  breadcrumb is just frustration.
+- **Never hide mandatory progression like a secret.** A required key or door
+  disguised as optional is a stuck-player generator. Secrets are optional by
+  definition; the critical path stays legible.
+
+## Risk vs reward
+
+- Gate the **best** optional reward behind the **biggest** optional risk or
+  detour — a hard side-arena, a long climb, a resource gamble, a no-checkpoint
+  stretch.
+- Keep the **critical path rewarding on its own** so risk-averse players aren't
+  punished for playing it safe; put the *extra* behind the *extra* effort.
+- Make the risk and the reward both *visible before committing* so the choice is
+  informed: the player should see the treasure across the deadly gap and decide.
+
+### Reward taxonomy (vary these so exploration stays fresh)
+
+| Type | Example | Notes |
+|---|---|---|
+| **Power / upgrade** | new ability, weapon, permanent buff | Strongest pull; use sparingly |
+| **Resources** | ammo, health, currency | Steady exploration incentive |
+| **Lore / collectible** | note, audio log, vista, cosmetic | For the curious; light them in breathers |
+| **Shortcut unlock** | a door back to the hub, a lowered ladder | Best long-term reward — reshapes the level, cuts backtracking |
+| **Spectacle** | a view, a scripted moment, an Easter egg | Pure delight; costs no systemic balance |

package/skills/shared/level-design/references/pacing-and-encounters.md

@@ -0,0 +1,142 @@
+# Pacing & encounters — rhythm, arenas, and teaching through space
+
+A level is a piece of music, not a list of rooms. This reference covers the
+intensity graph, the breather, the difficulty curve, the encounter template, the
+introduce→develop→twist→combine teaching pattern, and arena-shape design.
+
+## The intensity graph
+
+Plot intended intensity (danger + cognitive load + stimulus) against time for
+the whole level. The target shape is a **sawtooth trending upward**: a peak
+(combat, chase, platforming gauntlet, tense stealth), a sharp drop to a valley
+(a breather), then a higher peak. Overall the line climbs, ending on the level's
+hardest, most spectacular moment.
+
+```text
+intensity
+  ^                                        ___/\   <- climax (hardest, last)
+  |            ___/\           ___/\      /
+  |     __/\  /     \    __/\  /     \___/
+  |  __/    \/       \__/    \/            <- breathers (valleys)
+  +---------------------------------------------> time
+     intro  peak  rest  peak  rest  build  climax
+```
+
+Two failure shapes to avoid:
+
+- **Flat-high** — sustained maximum intensity. The player fatigues and the peaks
+  stop reading as peaks; it flattens into undifferentiated noise. This is the
+  most common "my level is exhausting / boring despite lots of action" cause.
+- **Flat-low** — long stretches with no tension or stimulus. Reads as padding
+  and boredom even if each individual moment is competent.
+
+Draw this graph *before* building encounters; it tells you where fights go and
+how long the gaps between them should be.
+
+## The breather (safe room)
+
+After each peak, a valley: a **safe room** with no threat, low stimulus, and
+usually a save point and/or restock. The breather does three jobs:
+
+1. **Lets adrenaline drop** so the next peak spikes instead of blending in.
+   (Halo's cadence is often described as "thirty seconds of fun" on repeat — the
+   calm between is what makes the fun read.)
+2. **Rewards observation** — a quiet room is where lore, a secret, or a vista
+   lands, because the player has attention to spare.
+3. **Sets up the next escalation** — telegraph what's coming (a locked blast
+   door, distant sound, a corpse) so the next peak feels earned, not random.
+
+Vary breather length with the peak that preceded it: a brutal fight earns a
+longer rest; a light skirmish gets a short beat.
+
+## Difficulty curve within a level
+
+The macro trend is up, but it **resets at checkpoints** — the sawtooth again.
+Let the player re-enter a checkpoint competent and confident, then ramp. A curve
+that only climbs with no resets feels punishing; one that never climbs feels
+trivial. Signposting difficulty (a visibly tougher enemy, a bigger arena) lets
+the player brace, which is fairer than a surprise spike.
+
+## Encounter design
+
+An **encounter = enemy composition × space × objective × available resources.**
+Change any single factor to create a fresh encounter without new assets:
+
+- **Composition** — which enemy types, how many, in what waves.
+- **Space** — the arena shape (below); the same enemies feel new in a new arena.
+- **Objective** — kill all / survive N seconds / reach the exit / protect / hold
+  ground / retreat. The objective reshapes how the space is used.
+- **Resources** — ammo, health, cover, high ground the player can seize.
+
+### The encounter template
+
+```text
+ENCOUNTER — <name>
+  Teaches / tests ..... which mechanic or threat (and which of I/D/T/C beat)
+  Space ............... arena shape, sightline length, cover, flanks, verticality
+  Composition ......... enemy types + counts + wave timing
+  Objective ........... win condition (kill / survive / reach / hold / escort)
+  Resources ........... ammo/health/cover/high-ground and where they sit
+  Entry / exit ........ how the player arrives (telegraph) and leaves (release valve)
+  Intensity ........... where this sits on the level's graph (peak height)
+```
+
+## Introduce -> develop -> twist -> combine
+
+The four-beat progression that teaches a new element through space, without
+text. (The pattern behind Portal's test chambers, Half-Life 2's set-pieces, and
+much Nintendo level craft.)
+
+1. **Introduce** — meet it where it cannot hurt you. A turret across a safe gap;
+   one tile of the new hazard with a wall to watch from. Zero-risk learning.
+2. **Develop** — use it for real, low stakes. The turret now guards a corridor
+   you must cross; the hazard is a gap you must jump.
+3. **Twist** — recontextualize it. Two turrets flanking; the turret behind glass
+   you must lure enemies past; the hazard now moves. The rule you learned bends.
+4. **Combine** — the mastery test: the new element + everything learned earlier,
+   at full intensity, as the section's climax.
+
+### Worked example — a "gravity plate" that flings the player upward
+
+- **Introduce**: one plate in a safe room; step on it, get flung to a ledge with
+  a reward. No threat. Player learns "plate = launch."
+- **Develop**: cross a pit by chaining two plates — miss and you just fall back,
+  retry. Low stakes, real use.
+- **Twist**: a plate launches you *into* a turret's line of fire — now you must
+  time the launch, or use the plate to reach the turret's flank.
+- **Combine**: an arena where plates are the only way to reach high cover, under
+  fire, while enemies also use them. Everything at once.
+
+## Arena shape — the geometry is the difficulty knob
+
+The shape a fight happens in matters as much as the enemies:
+
+- **Sightline length ↔ threat range.** Long lanes favor ranged play and
+  sniping; tight rooms favor shotguns/melee and force close engagements. Set the
+  lane length to the range you want to be dominant.
+- **Cover rhythm.** Space cover so the player *advances in beats* — peek, move,
+  peek — rather than turtling in one spot. Low cover (peek over) vs full cover
+  (peek around) change the fight's tempo.
+- **Flanks.** Give at least one flank route so a fight isn't a static peek-war;
+  a flank forces the player to move and makes the space three-dimensional in
+  play, not just in geometry.
+- **Verticality.** High ground is pressure and reward. The Doom (2016) arena
+  is the reference: multi-tier, with resources and enemies placed to pull the
+  player *through* and *up* the space rather than hunker in a corner.
+- **Chokepoints + release valves.** A choke concentrates danger (good for a
+  last stand); always pair it with an exit or a way to break the standoff, or
+  the fight stalls into a stalemate.
+- **Entry and exit.** Telegraph the fight before the player commits (see the
+  arena from a safe threshold), and give a clear release when it's won.
+
+### Arena patterns
+
+| Pattern | Shape | Good for |
+|---|---|---|
+| **Corridor / lane** | Long, narrow, few flanks | Tension, forced pace, ranged duels |
+| **Open arena** | Wide, multi-cover, multi-flank | Skill expression, mobile fights |
+| **Vertical arena** | Stacked tiers, ramps/lifts | High-ground contests, push-forward combat |
+| **Hub** | Central space, spokes off it | Wave defense, "hold ground" objectives |
+| **Pinch** | Wide → choke → wide | Escalation, last-stand beats |
+
+Match the pattern to the encounter's job on the intensity graph, not to habit.

package/skills/shared/llm-pipeline/SKILL.md

@@ -0,0 +1,150 @@
+---
+name: llm-pipeline
+description: "Use when wiring several LLM calls into a production flow: typed contracts between steps, a router/gateway so 429s, timeouts and outages fail over instead of taking you down, and cost control via caching, model tiers and abort caps. NOT single-prompt wording (that is `prompt-engineering`), NOT a model-driven tool loop (that is `building-agents`)."
+tags: [llm-orchestration, llm-gateway, fallbacks, prompt-caching, cost-control, litellm, reliability]
+recommends: [prompt-engineering, structured-extraction, building-agents, cost-tracking, agent-eval, rag, observability, parallel]
+origin: ai-hub
+---
+
+# llm-pipeline
+
+Wire multiple LLM calls into a reliable, controllable production pipeline. You chain steps where one call's validated output feeds the next, put a router in front of providers so an outage fails over instead of taking you down, and engineer the cross-cutting concerns: timeouts, bounded retries, fallbacks, caching, and cost caps.
+
+Treat the LLM as an unreliable network dependency, not a local function call. Every rule below follows from that: providers have outages, rate limits, and latency tails, so no single provider is a single point of failure and no call is allowed to run unbounded.
+
+## Do you even need a pipeline?
+
+This skill is the orchestration *around* calls. If you only have one call, you are in the wrong place.
+
+| Situation | Go to |
+| --- | --- |
+| Make one prompt better, few-shot, system-prompt design | [../prompt-engineering/SKILL.md](../prompt-engineering/SKILL.md) |
+| One call must return a typed object validated against a schema | [../structured-extraction/SKILL.md](../structured-extraction/SKILL.md) |
+| The model decides its own next step / tool to call | [../building-agents/SKILL.md](../building-agents/SKILL.md) |
+| Chunk/embed/retrieve context to stuff into a prompt | [../rag/SKILL.md](../rag/SKILL.md) |
+| Pure spend ledger / attribution / dashboard | [../cost-tracking/SKILL.md](../cost-tracking/SKILL.md) |
+| **Fixed multi-step flow + reliability layer** | **here** |
+
+A pipeline is a *DAG you designed*. The moment the model picks its own next step, it is an agent — go build that instead.
+
+## Design the chain as a typed DAG
+
+Each step is a pure-ish function: `(typed input) -> (typed output via structured output)`. Chaining small single-purpose steps beats one mega-prompt — reported ~20% output-quality gain — because each step is debuggable, cacheable, and retryable in isolation.
+
+Rules:
+
+- **The structured output of step N is the input contract of step N+1.** Validate it (Pydantic / JSON Schema) at the seam. A schema-valid object that fails validation here never poisons the next call.
+- **Keep steps small and single-purpose.** "Extract entities" and "classify sentiment" are two steps, not one prompt doing both. Smaller steps route to cheaper models and cache better.
+- **Mark independent steps for parallel fan-out.** If step B and step C both only need step A's output, run them concurrently — see [../parallel/SKILL.md](../parallel/SKILL.md). Sequential only where there is a real data dependency.
+- **Tag each step idempotent or side-effecting.** Retries and replays must be safe; a step that writes to a DB or sends an email is not safe to blindly retry.
+
+```python
+# Bad: free text flows between steps; step 2 silently mis-parses step 1
+entities = client.responses.create(model="gpt-4o", input=f"Extract entities: {doc}").output_text
+summary  = client.responses.create(model="gpt-4o", input=f"Summarize for {entities}").output_text
+
+# Good: each step emits a validated object; the seam is a contract you assert on
+from pydantic import BaseModel
+
+class Entities(BaseModel):
+    people: list[str]
+    orgs: list[str]
+
+step1 = client.responses.parse(model="gpt-4o", input=f"Extract: {doc}",
+                               text_format=Entities, timeout=30)
+ents: Entities = step1.output_parsed          # parse fails HERE, not downstream
+summary = summarize(ents)                      # typed input, not a string blob
+```
+
+## Put a gateway/router in front
+
+Use a router so a failure on one deployment fails over to another, and so you can swap or load-balance models from config without touching call sites.
+
+**LiteLLM** is the de-facto open-source LLM gateway (current stable line **v1.83.3-stable**). It exposes a unified OpenAI-format `completion()` across 100+ providers, with built-in retry/fallback, cost tracking, and budget management — usable as a Python SDK or as a Proxy Server.
+
+Fallback semantics worth knowing: a request to an `order=1` deployment that fails (connection error, 404, 429, ...) auto-tries `order=2`, then `order=3`; each order level gets its own `num_retries` before escalating; exhausting orders falls through to configured fallbacks. There are specialized buckets — `content_policy_fallbacks` (ContentPolicyViolationError), `context_window_fallbacks` (ContextWindowExceededError), and `default_fallbacks`.
+
+```python
+from litellm import Router
+
+router = Router(
+    model_list=[
+        {"model_name": "smart",
+         "litellm_params": {"model": "anthropic/claude-sonnet-4-6", "timeout": 30}},
+        {"model_name": "smart-backup",
+         "litellm_params": {"model": "openai/gpt-4o", "timeout": 30}},
+    ],
+    fallbacks=[{"smart": ["smart-backup"]}],
+    context_window_fallbacks=[{"smart": ["smart-backup"]}],
+    num_retries=2,            # per order level, bounded
+    timeout=30,               # hard cap, never unbounded
+)
+resp = router.completion(model="smart",
+                         messages=[{"role": "user", "content": prompt}])
+```
+
+**Build vs buy:** raw SDK + a thin retry/timeout wrapper (fewest deps, fine for one provider) → LiteLLM SDK/proxy (widest provider coverage, fallbacks for free) → self-hosted open-source gateway (Bifrost: Go, Apache-2.0, per-key budgets and rate limits) → hosted gateway (Portkey, when you want it operated for you). Full config — all fallback buckets, redis cache params, budget/rate-limit settings, cost callbacks — is in [references/litellm-router.md](references/litellm-router.md).
+
+## Reliability mechanics
+
+- **Always set a timeout.** A 30–60s hard cap per call. Without it, one slow upstream hangs the request indefinitely.
+- **Bounded retries with exponential backoff + jitter.** LiteLLM defaults: backoff from `INITIAL_RETRY_DELAY` 0.2s up to `MAX_RETRY_DELAY` 10s, with jitter to avoid thundering herds. Never `while True`.
+- **Only retry idempotent steps.** Retrying a step that sent an email or charged a card double-fires. Gate retries behind the idempotency tag from the DAG design.
+- **Circuit-break and degrade gracefully.** When a provider is down and fallbacks are exhausted, return a partial result, a cached result, or a cheaper-model result — never hang and never 500 the user if a degraded answer exists.
+
+```python
+# Bad: naked call in a for-loop; no timeout, unbounded effect, no fallback
+for _ in range(10000):
+    try:
+        return client.chat.completions.create(model="gpt-4o", messages=msgs)
+    except Exception:
+        continue          # hammers a down provider, blows the budget, may never exit
+
+# Good: router does bounded retries + backoff + fallback; you degrade on exhaustion
+try:
+    return router.completion(model="smart", messages=msgs)   # timeout + num_retries set
+except Exception:
+    return cached_or_cheaper_answer(msgs)                     # graceful degradation
+```
+
+## Caching: two distinct layers
+
+The cheapest, fastest, most reliable call is the one you never made — so cache and tier before you tune prompts. Wording is the last lever, not the first.
+
+| Layer | What it matches | Safety | Enable when |
+| --- | --- | --- | --- |
+| Prefix / prompt cache (provider-native) | Exact prefix of the prompt | Always safe (same input → same cached compute) | Always; put the stable prefix first |
+| Semantic cache (your gateway) | Embedding-similar prior query | Risky: weak embedder → false hits | Paraphrased FAQ-style queries, approximate answers OK |
+
+**Prefix cache** is transparent and free to enable. OpenAI caches automatically at ~50% off cached input tokens, no write penalty, no storage fee — first request full price, prefix hits half price. Anthropic is explicit (you mark cache breakpoints) and deeper: cache *read* = 0.1× base input (~90% off), 5-minute *write* = 1.25× base, 1-hour *write* = 2× base, delivering ~90% cost and ~85% latency reduction on long stable prefixes. For both, **put the stable content first** (system prompt, instructions, fixed context) and the variable content last so the prefix matches.
+
+**Semantic cache** matches an embedding-similar prior query and returns that prior response. The quality is dominated by the embedding model — a weak embedder produces **false cache hits: a confidently wrong answer for a similar-but-different question** (GPTCache is documented returning incorrect saved responses for similar prompts). So: strong embedder, tuned similarity threshold, and **never on correctness-critical paths**. Threshold tuning, embedding choice, TTL, and the multi-tier `semantic → prefix → inference` order are in [references/caching-layers.md](references/caching-layers.md).
+
+## Cost & latency control
+
+- **Model-tier routing, cheap-first.** Run the cheap tier, escalate only on low confidence or detected complexity. Anchor prices (Anthropic, 2026): Haiku 4.5 $1/$5 per M in/out, Sonnet 4.6 $3/$15, Opus 4.7 $5/$25 — flagship-for-everything is 5× the cost for zero quality gain on easy calls.
+- **Budget caps that ABORT, not just log.** Per-request and per-tenant caps that kill a runaway loop. A logged-but-uncapped budget still lets a bug spend $10k overnight.
+- **Token and latency budgets per step**, and stream output for perceived latency on user-facing calls.
+
+The *ledger* — attribution, per-team dashboards, monthly reporting — is not this skill; that is [../cost-tracking/SKILL.md](../cost-tracking/SKILL.md). This skill owns the *controls* (tiers, caching, caps that abort).
+
+## Observability hooks
+
+Log per step, every call: `model`, `tokens_in/out`, `cost`, `latency_ms`, `cache_hit`, `fallback_used`, `retry_count`. These are exactly the fields you debug a production incident from ("why did p99 spike?" → fallback_used + retry_count). Wire them into the tracing backbone in [../observability/SKILL.md](../observability/SKILL.md), and measure output *quality* with [../agent-eval/SKILL.md](../agent-eval/SKILL.md) — logging is not evaluation.
+
+## Anti-patterns
+
+| Anti-pattern | Why it bites | Do instead |
+| --- | --- | --- |
+| Naked call, no timeout | One hung provider stalls the whole request | Hard timeout on every call (30–60s) |
+| Unbounded `while True` retry | Thundering herd, blown budget, infinite hang | Bounded retries + exp backoff (0.2s→10s) + jitter |
+| Retrying a side-effecting step | Double-writes, double-charges | Idempotency tag; only retry pure steps |
+| Free text between steps | Step N+1 silently mis-parses | Validated structured output as the contract |
+| Semantic cache with a weak embedder | Confident WRONG answers from false hits | Strong embedder + tuned threshold, or prefix cache only |
+| Flagship model for everything | 5–25× the cost, no quality gain on easy calls | Tier routing, cheap-first, escalate |
+| No fallback configured | Provider outage = your outage | Router model group + fallbacks |
+| Treating schema-valid as correct | Perfectly-shaped wrong answers ship | Validate semantics + eval (agent-eval) |
+
+## Verify
+
+Run [scripts/verify.sh](scripts/verify.sh) `<file-or-dir>` against your pipeline/gateway code. It is offline and read-only, and checks statically: every completion/chat call site has an explicit timeout, retries are bounded (no `while True` retry loops), at least one fallback is configured when a router/model_list is present, no hardcoded `sk-`/provider key literals (must be env-sourced), and any YAML/JSON config parses and lists ≥2 model entries so a fallback target exists. It prints PASS/FAIL per check and exits non-zero on any FAIL; an empty or clean target exits 0.

package/skills/shared/llm-pipeline/evals/README.md

@@ -0,0 +1,3 @@
+# Evals for llm-pipeline
+
+`cases.yaml` is read by the repo's skill-eval harness. The `should_trigger` and `should_not_trigger` sets check routing precision — that orchestration/reliability/caching/cost-control requests land here, while prompt-wording goes to `prompt-engineering`, single schema-bound calls to `structured-extraction`, retrieval to `rag`, autonomous decision loops to `building-agents`, and spend dashboards to `cost-tracking`. The `capability` case checks that the agent produces a reliability-complete pipeline design (typed contracts between steps, timeouts, bounded retries, router fallback, the right caching choice, tier routing with an aborting budget cap, and per-step observability). Run it through the repo's eval runner the same way as the other skills; all cases are static-routing and design-rubric checks, so no live API calls or keys are required.

package/skills/shared/llm-pipeline/evals/cases.yaml

@@ -0,0 +1,44 @@
+skill: llm-pipeline
+
+should_trigger:
+  - prompt: "Chain three LLM calls: extract entities, then summarize, then classify — and make it not fall over."
+    why: "Canonical multi-step orchestration with data flowing between steps plus a reliability requirement — the core of this skill."
+  - prompt: "We keep getting random 429s and timeouts from OpenAI and it takes our feature down. Add fallbacks."
+    why: "Reliability layer: router fallback, retries, circuit-breaking around provider failures."
+  - prompt: "Put a gateway in front of OpenAI and Claude so we can swap models and load-balance."
+    why: "LLM gateway/router (LiteLLM) is the central tool of the skill."
+  - prompt: "Our LLM bill tripled — we call the model fresh on every request even for repeats."
+    why: "Non-obvious: framed as a cost problem, but the real fix is caching + tier routing, which are this skill's controls (not pure accounting)."
+  - prompt: "Necessito encadenar crides a l'LLM amb fallback i cachejar les respostes."
+    why: "Catalan phrasing for chaining + fallback + caching — exactly this skill, in another language."
+  - prompt: "Route easy requests to Haiku and only escalate to Opus when confidence is low."
+    why: "Model-tier routing, cheap-first with escalation, is the standard cost lever here."
+
+should_not_trigger:
+  - prompt: "Help me write a better system prompt and few-shot examples for this classification task."
+    route_to: prompt-engineering
+    why: "This is prompt wording, not the orchestration around prompts."
+  - prompt: "Make this single call return JSON matching my Pydantic schema."
+    route_to: structured-extraction
+    why: "One schema-bound call, not a chain — this skill uses structured output as a contract but does not own single extraction."
+  - prompt: "Chunk my docs, embed them, and retrieve relevant context for the prompt."
+    route_to: rag
+    why: "Building a retriever is retrieval, not call-chaining; a pipeline may call a retriever as a step but does not build it."
+  - prompt: "Build an agent that decides which tools to call to book travel."
+    route_to: building-agents
+    why: "An autonomous decision loop where the model picks its next step — not a fixed DAG you designed."
+  - prompt: "Build me a dashboard attributing our monthly AI spend per team."
+    route_to: cost-tracking
+    why: "Spend ledger and reporting, not the pipeline controls (tiers, caching, abort caps) this skill owns."
+
+capability:
+  - scenario: "Design a production pipeline that classifies an inbound support ticket, then drafts a reply, behind a router with fallback, with caching and a per-request cost cap."
+    must_include:
+      - "Typed/structured-output contract between the classify step and the draft step (validated at the seam)"
+      - "Explicit timeout on every LLM call (30-60s)"
+      - "Bounded retries with exponential backoff + jitter, never an unbounded loop"
+      - "Router with a fallback model, and mention of content_policy / context_window fallback buckets"
+      - "Prefix cache enabled (stable prefix first); semantic cache only if safe with a strong embedder + tuned threshold"
+      - "Model-tier routing cheap-first (e.g. Haiku then escalate), and a per-request budget cap that ABORTS"
+      - "Per-step observability fields: model, tokens, cost, latency_ms, cache_hit, fallback_used, retry_count"
+      - "Note that schema-valid does not mean correct; defer quality measurement to agent-eval"

package/skills/shared/llm-pipeline/references/caching-layers.md

@@ -0,0 +1,60 @@
+# Caching layers: prefix vs semantic
+
+Two caches, different mechanisms, different risk profiles. Do not conflate them.
+
+## Layer 1 — prefix / prompt cache (provider-native)
+
+Matches an **exact prefix** of your prompt at the provider and reuses the cached compute. Transparent and safe: identical input → identical cached state. Free upside, no correctness risk.
+
+**OpenAI** — automatic, on by default, ~50% off cached input tokens. No write penalty, no storage fee. First request full price; subsequent prefix hits half price.
+
+**Anthropic** — explicit: you mark cache breakpoints (`cache_control`). Deeper discount:
+
+| Operation | Multiplier vs base input |
+| --- | --- |
+| Cache read | 0.1× (~90% off) |
+| 5-minute cache write | 1.25× |
+| 1-hour cache write | 2× |
+
+On long, stable prefixes this is ~90% cost and ~85% latency reduction.
+
+### Order prompts for maximum prefix hits
+
+Put content **most stable first, most variable last**:
+
+1. System prompt / role instructions (never changes)
+2. Fixed reference context, tool definitions, few-shot exemplars (changes rarely)
+3. Retrieved/dynamic context (changes per request)
+4. The user's actual turn (changes every time)
+
+If you put the user's variable input first, the prefix never matches and you cache nothing. For Anthropic, set the breakpoint at the end of the stable block (1–2).
+
+## Layer 2 — semantic cache (your gateway)
+
+Embeds the incoming query, looks up an **embedding-similar prior query**, and returns *that prior response*. Lives in your gateway: GPTCache, a Redis vector store, Bifrost, etc.
+
+The danger: it returns a **different** prior answer for a **similar** query. Quality is dominated by the embedding model — a weak embedder yields **false cache hits**, i.e. a confidently wrong answer to a question that only looked similar (GPTCache is documented returning incorrect saved responses for similar prompts; cf. Microsoft GenCache, NeurIPS 2025).
+
+### Enable it only when all hold
+
+- Queries are paraphrase-heavy (FAQ, support, repeated intents).
+- An approximate / near-match answer is acceptable for the use case.
+- It is **not** a correctness-critical path (no legal, medical, financial, or contract logic).
+
+### Tuning
+
+- **Embedding model:** use a strong, current embedder. The cheap default is where false hits come from.
+- **Similarity threshold:** start strict (e.g. cosine ≥ 0.95) and loosen only with eval data on real query pairs. Too loose = false hits; too strict = no hits.
+- **TTL:** short for time-sensitive answers (prices, status), longer for stable knowledge. Stale-but-confident is its own failure mode.
+- **Scope keys:** namespace the cache by tenant/locale/model so you never serve tenant A's answer to tenant B.
+
+## Multi-tier order
+
+Check caches cheapest-and-safest first, fall through to inference:
+
+```text
+semantic cache  ->  prefix cache  ->  live inference
+   (gateway)        (provider)         (full price)
+```
+
+Only reach inference on a miss. But if correctness matters more than cost, drop the semantic tier entirely and rely on prefix cache + inference — prefix cache never changes the answer.