@damiangilgonzalez/harness 1.3.7

package/skills/linkedin-strategy/evals/cases.yaml

@@ -0,0 +1,49 @@
+skill: linkedin-strategy
+
+should_trigger:
+  - prompt: "What should I be known for on LinkedIn and how often should I post?"
+    why: Positioning/POV plus cadence — two account-level decisions, the skill's core.
+  - prompt: "Should our content go on the company page or the founder's profile?"
+    why: The personal-vs-company-page distribution decision; humans-as-main-event with employee advocacy.
+  - prompt: "I post regularly but get no inbound — what's wrong with my LinkedIn rhythm?"
+    why: Non-obvious, no 'strategy' word; it is the engage-vs-create split and SSI rhythm, owned here.
+  - prompt: "Quarterly LinkedIn review — which content pillar is working and what should I double down on?"
+    why: The read-wiki + double-down loop, the skill's spine.
+  - prompt: "Qué debería publicar en LinkedIn y cada cuánto para que me tomen como referente?"
+    why: Spanish for pillars + cadence + thought-leadership positioning, all account-level.
+  - prompt: "My SSI is stuck at 48 — how do I build a habit system to raise it?"
+    why: SSI as a four-dimension weekly rhythm and steering metric, owned by this skill (fact 6).
+  - prompt: "My company page is basically dead — where should the content actually live?"
+    why: Non-obvious phrasing of the distribution decision; pages get ~5% feed, profiles carry reach.
+
+should_not_trigger:
+  - prompt: "Write the post copy and hook for tomorrow's update."
+    route_to: linkedin-content
+    why: Per-post craft fills a pillar; strategy sets the pillar, it does not write the post.
+  - prompt: "Build me a 10-slide carousel deck on this topic."
+    route_to: linkedin-carousels
+    why: Single-asset format craft, not the presence system.
+  - prompt: "Draft connection requests and a DM sequence for these 20 prospects."
+    route_to: linkedin-outreach
+    why: 1:1 thread execution; strategy sets the rhythm, not the individual messages.
+  - prompt: "Pull my last 90 days of post analytics from the LinkedIn API."
+    route_to: linkedin-api
+    why: Data plumbing / OAuth; strategy consumes the numbers, it does not fetch them.
+  - prompt: "Schedule and cross-post this to Twitter and Instagram too."
+    route_to: social-publisher
+    why: Cross-platform scheduling, not LinkedIn presence architecture.
+
+capability:
+  - scenario: >
+      A B2B founder has a near-dead company page and a personal profile with sporadic posting.
+      docs/wiki/linkedin/ already holds: SSI ~48, the top-2 dwell-time posts are both carousels
+      on one topic, and a stated cadence of "3/week" that slipped to ~1. Produce a strategy
+      decision for the presence.
+    must_include:
+      - Reads docs/wiki/linkedin/ first (prior positioning, current pillars, whether cadence held, top dwell-time performers, SSI trend, killed experiments) — or bootstraps the dir if empty.
+      - Moves distribution to the founder's personal profile with the company page as amplification plus employee advocacy, grounded in the personal-vs-page reach gap (~8x engagement, ~5% vs ~65% feed).
+      - Defines or refines 3-5 content pillars aligned to the founder's professional graph, doubling down on the carousel topic that already earns the best dwell time.
+      - Gives a cadence recommendation justified by the account's own data (e.g. a holdable 3/week weighted toward carousels/docs for dwell time) with the 8-week sustainability gate, not a generic number.
+      - Sets a social-selling rhythm — the ~80/20 engage-vs-create split and SSI as a four-dimension weekly habit with a target (e.g. 48 -> 65).
+      - Writes a dated decision record back to docs/wiki/linkedin/ naming the single metric it bets on and a review date.
+      - Routes single-asset work out: post copy -> linkedin-content, carousels -> linkedin-carousels, DMs -> linkedin-outreach, raw metrics -> linkedin-api.

package/skills/linkedin-strategy/references/ssi-and-pillars.md

@@ -0,0 +1,59 @@
+# SSI dimensions, pillar design, and distribution mechanics
+
+Offloaded depth for `SKILL.md` §3, §4, §5. Use it when a session needs the worked detail.
+
+## The four SSI dimensions as weekly habits
+
+SSI is 0-100, four 25-point dimensions, refreshed daily on a trailing 90-day window, free at `linkedin.com/sales/ssi`. It is benchmarked relative to your industry and your network, not absolute — so the target is *direction*, not a leaderboard rank. It responds within ~90 days to habit changes, which is what makes it steerable.
+
+Targets: **65+** active B2B seller, **75+** thought-leader territory, all-user average ~40-50.
+
+| Dimension | What it measures | Concrete weekly habit | Common failure |
+|---|---|---|---|
+| Establish a professional brand | Complete profile, content published inside a clear domain, post engagement | 100% profile; 3-4 posts/week inside your pillars; a carousel/week | Sporadic, off-topic posting that suppresses topical authority |
+| Find the right people | Using search/filters to reach the right prospects | A weekly targeted-connect block inside your niche | Mass-connecting outside the graph; inflates count, not score |
+| Engage with insights | Sharing and commenting with substance on others' content | The ~80% engage block — substantive comments daily | Like-only drive-bys; no dwell signal generated |
+| Build relationships | Connecting with and converting senior/relevant people | Convert engaged commenters into 1:1 threads (hand to outreach) | Never moving beyond the public feed |
+
+The dimension that moves most for stuck accounts is usually **Engage with insights** — it is the directly controllable lever and it feeds the others.
+
+## Pillar-design worked examples
+
+Pillars must align to the professional graph (title, skills, background) so the 2026 topic cross-reference reads them as on-domain authority. 3-5 pillars; each one a recurring theme, not a single post.
+
+**B2B SaaS founder (data platform):**
+1. Pipeline reliability war stories (graph-aligned: the actual job)
+2. Hiring and scaling a data team
+3. Build-vs-buy decision frameworks
+4. Behind-the-scenes founder lessons
+
+**Solo consultant (GTM advisor):**
+1. Diagnosed GTM failures (anonymized case teardowns)
+2. The one framework they are known for
+3. Contrarian takes on a common GTM myth
+4. Client-result proof, told as a story
+
+**Company page (secondary amplification only):**
+1. Customer outcomes / case studies
+2. Product/release notes for retargeting base
+3. Hiring and culture
+   (The page never carries reach — humans do. See distribution below.)
+
+A pillar that consistently underperforms on dwell time across a quarter is a sunset candidate: kill it, log it, reallocate the slot.
+
+## Personal-vs-company-page + employee-advocacy mechanics (2026 numbers)
+
+- Personal-profile content engages **~8x** more than company-page content.
+- Company pages lost **~60-66%** organic reach 2024-2026; they now get **~5%** of feed allocation vs **~65%** for personal profiles.
+- Employee advocacy yields **~561%** greater reach and **~7x** more lead conversion than the page alone.
+- CEO/founder content earns **~4x** more engagement than the average company-page post.
+
+Operating model that follows from the numbers:
+
+```text
+Founder profile  -> carries the POV and the reach (the main event)
+Company page     -> reshares as amplification + serves as the ad-retargeting base
+Employee advocacy-> N employees reshare/comment the same week -> reach compounds across graphs
+```
+
+Decision shortcut: if the goal is reach, trust, or inbound, the human profile carries it. The page is a base of record and an amplifier, never the primary channel.

package/skills/linkedin-strategy/references/wiki-records.md

@@ -0,0 +1,88 @@
+# Wiki records — `docs/wiki/linkedin/`
+
+The read/write loop's storage. Everything is append-only markdown so future strategy sessions compound instead of restarting from generic advice.
+
+Every `.md` file written here is part of the `docs/wiki/` OKF v0.1 bundle, so each carries YAML frontmatter with a non-empty `type:` (the only required OKF field). Cross-references between files use **standard markdown links** (`[Text](./pillars.md)`) — never wikilinks (`[[...]]`). The domain lines `verify.sh` greps for (the dated heading, `Decision:`, the metric/bet field) stay exactly as below.
+
+## File layout
+
+```text
+docs/wiki/linkedin/
+  positioning.md        # current POV, niche of authority, who-it's-for; updated in place when it shifts
+  pillars.md            # the live 3-5 pillars + any KILLED pillars with the date and reason
+  decisions/            # one append-only file per decision (preferred), dated filenames
+    2026-06-02-distribution-to-founder.md
+    2026-09-02-q3-review.md
+  what-worked.md        # running log of what specific posts/pillars/formats earned dwell time + engagement
+```
+
+A running `decisions.md` log at the root is also accepted instead of `decisions/*.md` — pick one and stay consistent. `verify.sh` checks both shapes.
+
+## Bootstrap (empty/absent dir)
+
+When `docs/wiki/linkedin/` does not exist, create it with `positioning.md`, `pillars.md`, `decisions/`, and `what-worked.md` before you decide. Seed `positioning.md` with whatever the account already implies (title, current posts) so the first decision has a baseline to react to. Each of `positioning.md`, `pillars.md`, and `what-worked.md` gets OKF frontmatter — use `type: linkedin-positioning`, `type: linkedin-pillars`, and `type: linkedin-record` respectively (templates below).
+
+## Decision-record template
+
+Every record carries: a dated heading, a context-read line (what you grounded in), a decision line, the single metric it bets on, and a review date. Missing the metric or the review date means it is not a checkable bet.
+
+```markdown
+---
+type: linkedin-decision
+title: <short decision title>
+description: <one-sentence summary of the choice>
+tags: [linkedin, <pillars|cadence|distribution|ssi>]
+timestamp: <YYYY-MM-DDTHH:MM:SSZ>
+topic: linkedin
+status: stable
+---
+
+# 2026-06-02 — <short decision title>
+
+Context read: <what you pulled from the wiki — prior positioning, current pillars,
+whether cadence held, top dwell-time performers, SSI trend, killed experiments>.
+
+Decision: <the concrete choice — positioning / pillars / cadence / distribution /
+social-selling rhythm, stated so the next session can act on it>.
+
+Bets on metric: <the single steering metric + from -> to + window, e.g.
+SSI 48 -> 65 within 90 days, or median dwell time +X%, or inbound DMs/week>.
+
+Review date: <YYYY-MM-DD — when the next session checks whether the bet paid>.
+```
+
+`type` (required) + `timestamp` (ISO 8601, last meaningful edit) are the OKF surface; everything below the frontmatter is the domain content `verify.sh` checks. A running `decisions.md` log at the root carries one frontmatter block at the top (`type: linkedin-decision`); its dated `## ` entries are prepended newest-first and never edited.
+
+## what-worked template
+
+`what-worked.md` is a single running file; new review blocks are prepended newest-first under one frontmatter block at the top.
+
+```markdown
+---
+type: linkedin-record
+title: What worked — LinkedIn
+description: Running log of which pillars, formats, and posts earned dwell time and SSI movement.
+tags: [linkedin, what-worked, dwell-time, ssi]
+timestamp: 2026-09-02T00:00:00Z
+topic: linkedin
+status: stable
+---
+
+## 2026-09-02 review
+
+- Pillar "pipeline reliability" — RISING. 2 carousels in top-3 dwell time. Double down.
+- Pillar "founder lessons" — FLAT. Text-only; try carousel format before sunsetting.
+- Pillar "industry hot takes" — KILLED. Off-graph topic, suppressed reach. Slot reallocated to reliability.
+- Format: carousels ~2.4x dwell vs text on this account. Weight calendar toward them.
+- SSI: 48 -> 61 in 90 days. "Engage with insights" dimension moved most.
+```
+
+## What `scripts/verify.sh` checks
+
+Read-only, network-free, pure bash + grep. Given `docs/wiki/linkedin/`:
+
+1. The target directory exists.
+2. At least one decision record exists (`decisions/*.md` or a root `decisions.md`).
+3. Each record carries a date (`YYYY-MM-DD`), a `Decision:` line, and a named metric/bet field — a record missing the metric is a hard failure (a decision with no metric is an opinion).
+
+A missing or empty target (a fresh presence with no wiki yet) is reported and exits 0 — no false failure. It enforces the compounding loop's integrity, not strategy quality (that is the capability eval).

package/skills/linkedin-strategy/scripts/verify.sh

@@ -0,0 +1,120 @@
+#!/usr/bin/env bash
+#
+# verify.sh — structural lint for a linkedin-strategy wiki under docs/wiki/linkedin/.
+#
+# WHAT IT DOES (read-only; never edits a file)
+#   Static, network-free checks on the decision records this skill writes back.
+#     1. The target directory exists.
+#     2. At least one decision record is present: a file under decisions/*.md
+#        OR a running decisions.md log at the target root.
+#     3. Each decision record carries the required signals:
+#          - a date              (a YYYY-MM-DD anywhere in the file)
+#          - a Decision: line     (the concrete choice)
+#          - a named metric/bet   (a "Bets on metric:" / "metric" / "bet" field)
+#        A record missing the metric/bet is a HARD FAILURE — a decision with no
+#        metric is an opinion, not a checkable bet. Missing date or Decision line
+#        is also a hard failure. A missing review date is a soft warning.
+#
+#   A missing/empty target (a fresh presence with no wiki yet) is reported and
+#   exits 0 — no false failure. Pure bash + grep, no dependencies. It is a lint,
+#   not a strategy oracle: it never judges whether the strategy is good.
+#
+# HOW TO RUN
+#   ./verify.sh docs/wiki/linkedin/   # lint a presence's strategy wiki
+#   ./verify.sh                          # no target -> nothing to check, exit 0
+#
+# EXIT CODES
+#   0  clean, or nothing to check
+#   1  a hard structural failure
+#   2  bad usage
+#
+# Runs on stock macOS bash 3.2.
+
+set -euo pipefail
+
+if [ -t 1 ]; then
+  RED=$'\033[31m'; GREEN=$'\033[32m'; YELLOW=$'\033[33m'; NC=$'\033[0m'
+else
+  RED=''; GREEN=''; YELLOW=''; NC=''
+fi
+
+ok_count=0; warn_count=0; fail_count=0
+ok()   { printf '%s[ ok ]%s %s\n' "$GREEN"  "$NC" "$*"; ok_count=$((ok_count + 1)); }
+warn() { printf '%s[warn]%s %s\n' "$YELLOW" "$NC" "$*"; warn_count=$((warn_count + 1)); }
+fail() { printf '%s[fail]%s %s\n' "$RED"    "$NC" "$*"; fail_count=$((fail_count + 1)); }
+
+usage() { sed -n '2,33p' "$0" | sed 's/^# \{0,1\}//'; }
+
+summary_exit() {
+  printf '\nok=%d warn=%d fail=%d\n' "$ok_count" "$warn_count" "$fail_count"
+  [ "$fail_count" -gt 0 ] && exit 1
+  exit 0
+}
+
+# --- arg parse --------------------------------------------------------------
+TARGET=""
+while [ $# -gt 0 ]; do
+  case "$1" in
+    -h|--help) usage; exit 0 ;;
+    -*) printf 'unknown flag: %s\n' "$1" >&2; usage >&2; exit 2 ;;
+    *) TARGET="$1"; shift ;;
+  esac
+done
+
+# No target, or missing dir -> nothing to check, clean exit (no false fail).
+if [ -z "$TARGET" ]; then
+  ok "no wiki path given — nothing to lint"
+  summary_exit
+fi
+if [ ! -d "$TARGET" ]; then
+  warn "directory not found: $TARGET — nothing to lint (fresh presence?)"
+  summary_exit
+fi
+ok "wiki directory exists: $TARGET"
+
+# --- collect decision records ----------------------------------------------
+# Per-decision files under decisions/, plus an optional running decisions.md log.
+RECORDS=""
+if [ -d "$TARGET/decisions" ]; then
+  for f in "$TARGET"/decisions/*.md; do
+    [ -f "$f" ] && RECORDS="$RECORDS$f
+"
+  done
+fi
+[ -f "$TARGET/decisions.md" ] && RECORDS="$RECORDS$TARGET/decisions.md
+"
+
+# Trim blank lines / handle empty.
+RECORDS="$(printf '%s' "$RECORDS" | sed '/^$/d')"
+
+if [ -z "$RECORDS" ]; then
+  warn "no decision records yet (decisions/*.md or decisions.md) — nothing to lint"
+  summary_exit
+fi
+
+# --- per-record structural checks ------------------------------------------
+DATE_RE='[0-9]{4}-[0-9]{2}-[0-9]{2}'
+
+while IFS= read -r rec; do
+  [ -z "$rec" ] && continue
+  if [ ! -s "$rec" ]; then
+    fail "$rec: empty record"
+    continue
+  fi
+
+  miss=""
+  grep -Eq  "$DATE_RE"                       "$rec" || miss="$miss date"
+  grep -Eiq '^[[:space:]-]*Decision:'        "$rec" || miss="$miss decision-line"
+  grep -Eiq '(bets on metric:|metric|bet:)'  "$rec" || miss="$miss metric/bet"
+
+  if [ -n "$miss" ]; then
+    fail "$rec: missing required field(s):$miss"
+  else
+    ok "$rec: date + decision + metric present"
+    grep -Eiq 'review date:' "$rec" || warn "$rec: no review date — bet has no follow-up"
+  fi
+done <<EOF
+$RECORDS
+EOF
+
+summary_exit

package/skills/llm-pipeline/SKILL.md

@@ -0,0 +1,150 @@
+---
+name: llm-pipeline
+description: "Use when wiring several LLM calls into a production flow: typed contracts between steps, a router/gateway so 429s, timeouts and outages fail over instead of taking you down, and cost control via caching, model tiers and abort caps. NOT single-prompt wording (that is `prompt-engineering`), NOT a model-driven tool loop (that is `building-agents`)."
+tags: [llm-orchestration, llm-gateway, fallbacks, prompt-caching, cost-control, litellm, reliability]
+recommends: [prompt-engineering, structured-extraction, building-agents, cost-tracking, agent-eval, rag, observability]
+origin: risco
+---
+
+# llm-pipeline
+
+Wire multiple LLM calls into a reliable, controllable production pipeline. You chain steps where one call's validated output feeds the next, put a router in front of providers so an outage fails over instead of taking you down, and engineer the cross-cutting concerns: timeouts, bounded retries, fallbacks, caching, and cost caps.
+
+Treat the LLM as an unreliable network dependency, not a local function call. Every rule below follows from that: providers have outages, rate limits, and latency tails, so no single provider is a single point of failure and no call is allowed to run unbounded.
+
+## Do you even need a pipeline?
+
+This skill is the orchestration *around* calls. If you only have one call, you are in the wrong place.
+
+| Situation | Go to |
+| --- | --- |
+| Make one prompt better, few-shot, system-prompt design | [../prompt-engineering/SKILL.md](../prompt-engineering/SKILL.md) |
+| One call must return a typed object validated against a schema | [../structured-extraction/SKILL.md](../structured-extraction/SKILL.md) |
+| The model decides its own next step / tool to call | [../building-agents/SKILL.md](../building-agents/SKILL.md) |
+| Chunk/embed/retrieve context to stuff into a prompt | [../rag/SKILL.md](../rag/SKILL.md) |
+| Pure spend ledger / attribution / dashboard | [../cost-tracking/SKILL.md](../cost-tracking/SKILL.md) |
+| **Fixed multi-step flow + reliability layer** | **here** |
+
+A pipeline is a *DAG you designed*. The moment the model picks its own next step, it is an agent — go build that instead.
+
+## Design the chain as a typed DAG
+
+Each step is a pure-ish function: `(typed input) -> (typed output via structured output)`. Chaining small single-purpose steps beats one mega-prompt — reported ~20% output-quality gain — because each step is debuggable, cacheable, and retryable in isolation.
+
+Rules:
+
+- **The structured output of step N is the input contract of step N+1.** Validate it (Pydantic / JSON Schema) at the seam. A schema-valid object that fails validation here never poisons the next call.
+- **Keep steps small and single-purpose.** "Extract entities" and "classify sentiment" are two steps, not one prompt doing both. Smaller steps route to cheaper models and cache better.
+- **Mark independent steps for parallel fan-out.** If step B and step C both only need step A's output, run them concurrently — see `superpowers:dispatching-parallel-agents`. Sequential only where there is a real data dependency.
+- **Tag each step idempotent or side-effecting.** Retries and replays must be safe; a step that writes to a DB or sends an email is not safe to blindly retry.
+
+```python
+# Bad: free text flows between steps; step 2 silently mis-parses step 1
+entities = client.responses.create(model="gpt-4o", input=f"Extract entities: {doc}").output_text
+summary  = client.responses.create(model="gpt-4o", input=f"Summarize for {entities}").output_text
+
+# Good: each step emits a validated object; the seam is a contract you assert on
+from pydantic import BaseModel
+
+class Entities(BaseModel):
+    people: list[str]
+    orgs: list[str]
+
+step1 = client.responses.parse(model="gpt-4o", input=f"Extract: {doc}",
+                               text_format=Entities, timeout=30)
+ents: Entities = step1.output_parsed          # parse fails HERE, not downstream
+summary = summarize(ents)                      # typed input, not a string blob
+```
+
+## Put a gateway/router in front
+
+Use a router so a failure on one deployment fails over to another, and so you can swap or load-balance models from config without touching call sites.
+
+**LiteLLM** is the de-facto open-source LLM gateway (current stable line **v1.83.3-stable**). It exposes a unified OpenAI-format `completion()` across 100+ providers, with built-in retry/fallback, cost tracking, and budget management — usable as a Python SDK or as a Proxy Server.
+
+Fallback semantics worth knowing: a request to an `order=1` deployment that fails (connection error, 404, 429, ...) auto-tries `order=2`, then `order=3`; each order level gets its own `num_retries` before escalating; exhausting orders falls through to configured fallbacks. There are specialized buckets — `content_policy_fallbacks` (ContentPolicyViolationError), `context_window_fallbacks` (ContextWindowExceededError), and `default_fallbacks`.
+
+```python
+from litellm import Router
+
+router = Router(
+    model_list=[
+        {"model_name": "smart",
+         "litellm_params": {"model": "anthropic/claude-sonnet-4-6", "timeout": 30}},
+        {"model_name": "smart-backup",
+         "litellm_params": {"model": "openai/gpt-4o", "timeout": 30}},
+    ],
+    fallbacks=[{"smart": ["smart-backup"]}],
+    context_window_fallbacks=[{"smart": ["smart-backup"]}],
+    num_retries=2,            # per order level, bounded
+    timeout=30,               # hard cap, never unbounded
+)
+resp = router.completion(model="smart",
+                         messages=[{"role": "user", "content": prompt}])
+```
+
+**Build vs buy:** raw SDK + a thin retry/timeout wrapper (fewest deps, fine for one provider) → LiteLLM SDK/proxy (widest provider coverage, fallbacks for free) → hosted gateway (Bifrost/Portkey, when you want it operated for you). Full config — all fallback buckets, redis cache params, budget/rate-limit settings, cost callbacks — is in [references/litellm-router.md](references/litellm-router.md).
+
+## Reliability mechanics
+
+- **Always set a timeout.** A 30–60s hard cap per call. Without it, one slow upstream hangs the request indefinitely.
+- **Bounded retries with exponential backoff + jitter.** LiteLLM defaults: backoff from `INITIAL_RETRY_DELAY` 0.2s up to `MAX_RETRY_DELAY` 10s, with jitter to avoid thundering herds. Never `while True`.
+- **Only retry idempotent steps.** Retrying a step that sent an email or charged a card double-fires. Gate retries behind the idempotency tag from the DAG design.
+- **Circuit-break and degrade gracefully.** When a provider is down and fallbacks are exhausted, return a partial result, a cached result, or a cheaper-model result — never hang and never 500 the user if a degraded answer exists.
+
+```python
+# Bad: naked call in a for-loop; no timeout, unbounded effect, no fallback
+for _ in range(10000):
+    try:
+        return client.chat.completions.create(model="gpt-4o", messages=msgs)
+    except Exception:
+        continue          # hammers a down provider, blows the budget, may never exit
+
+# Good: router does bounded retries + backoff + fallback; you degrade on exhaustion
+try:
+    return router.completion(model="smart", messages=msgs)   # timeout + num_retries set
+except Exception:
+    return cached_or_cheaper_answer(msgs)                     # graceful degradation
+```
+
+## Caching: two distinct layers
+
+The cheapest, fastest, most reliable call is the one you never made — so cache and tier before you tune prompts. Wording is the last lever, not the first.
+
+| Layer | What it matches | Safety | Enable when |
+| --- | --- | --- | --- |
+| Prefix / prompt cache (provider-native) | Exact prefix of the prompt | Always safe (same input → same cached compute) | Always; put the stable prefix first |
+| Semantic cache (your gateway) | Embedding-similar prior query | Risky: weak embedder → false hits | Paraphrased FAQ-style queries, approximate answers OK |
+
+**Prefix cache** is transparent and free to enable. OpenAI caches automatically at ~50% off cached input tokens, no write penalty, no storage fee — first request full price, prefix hits half price. Anthropic is explicit (you mark cache breakpoints) and deeper: cache *read* = 0.1× base input (~90% off), 5-minute *write* = 1.25× base, 1-hour *write* = 2× base, delivering ~90% cost and ~85% latency reduction on long stable prefixes. For both, **put the stable content first** (system prompt, instructions, fixed context) and the variable content last so the prefix matches.
+
+**Semantic cache** matches an embedding-similar prior query and returns that prior response. The quality is dominated by the embedding model — a weak embedder produces **false cache hits: a confidently wrong answer for a similar-but-different question** (GPTCache is documented returning incorrect saved responses for similar prompts). So: strong embedder, tuned similarity threshold, and **never on correctness-critical paths**. Threshold tuning, embedding choice, TTL, and the multi-tier `semantic → prefix → inference` order are in [references/caching-layers.md](references/caching-layers.md).
+
+## Cost & latency control
+
+- **Model-tier routing, cheap-first.** Run the cheap tier, escalate only on low confidence or detected complexity. Anchor prices (Anthropic, 2026): Haiku 4.5 $1/$5 per M in/out, Sonnet 4.6 $3/$15, Opus 4.7 $5/$25 — flagship-for-everything is 5× the cost for zero quality gain on easy calls.
+- **Budget caps that ABORT, not just log.** Per-request and per-tenant caps that kill a runaway loop. A logged-but-uncapped budget still lets a bug spend $10k overnight.
+- **Token and latency budgets per step**, and stream output for perceived latency on user-facing calls.
+
+The *ledger* — attribution, per-team dashboards, monthly reporting — is not this skill; that is [../cost-tracking/SKILL.md](../cost-tracking/SKILL.md). This skill owns the *controls* (tiers, caching, caps that abort).
+
+## Observability hooks
+
+Log per step, every call: `model`, `tokens_in/out`, `cost`, `latency_ms`, `cache_hit`, `fallback_used`, `retry_count`. These are exactly the fields you debug a production incident from ("why did p99 spike?" → fallback_used + retry_count). Wire them into the tracing backbone in [../observability/SKILL.md](../observability/SKILL.md), and measure output *quality* with [../agent-eval/SKILL.md](../agent-eval/SKILL.md) — logging is not evaluation.
+
+## Anti-patterns
+
+| Anti-pattern | Why it bites | Do instead |
+| --- | --- | --- |
+| Naked call, no timeout | One hung provider stalls the whole request | Hard timeout on every call (30–60s) |
+| Unbounded `while True` retry | Thundering herd, blown budget, infinite hang | Bounded retries + exp backoff (0.2s→10s) + jitter |
+| Retrying a side-effecting step | Double-writes, double-charges | Idempotency tag; only retry pure steps |
+| Free text between steps | Step N+1 silently mis-parses | Validated structured output as the contract |
+| Semantic cache with a weak embedder | Confident WRONG answers from false hits | Strong embedder + tuned threshold, or prefix cache only |
+| Flagship model for everything | 5–25× the cost, no quality gain on easy calls | Tier routing, cheap-first, escalate |
+| No fallback configured | Provider outage = your outage | Router model group + fallbacks |
+| Treating schema-valid as correct | Perfectly-shaped wrong answers ship | Validate semantics + eval (agent-eval) |
+
+## Verify
+
+Run [scripts/verify.sh](scripts/verify.sh) `<file-or-dir>` against your pipeline/gateway code. It is offline and read-only, and checks statically: every completion/chat call site has an explicit timeout, retries are bounded (no `while True` retry loops), at least one fallback is configured when a router/model_list is present, no hardcoded `sk-`/provider key literals (must be env-sourced), and any YAML/JSON config parses and lists ≥2 model entries so a fallback target exists. It prints PASS/FAIL per check and exits non-zero on any FAIL; an empty or clean target exits 0.

package/skills/llm-pipeline/evals/README.md

@@ -0,0 +1,3 @@
+# Evals for llm-pipeline
+
+`cases.yaml` is read by the repo's skill-eval harness. The `should_trigger` and `should_not_trigger` sets check routing precision — that orchestration/reliability/caching/cost-control requests land here, while prompt-wording goes to `prompt-engineering`, single schema-bound calls to `structured-extraction`, retrieval to `rag`, autonomous decision loops to `building-agents`, and spend dashboards to `cost-tracking`. The `capability` case checks that the agent produces a reliability-complete pipeline design (typed contracts between steps, timeouts, bounded retries, router fallback, the right caching choice, tier routing with an aborting budget cap, and per-step observability). Run it through the repo's eval runner the same way as the other skills; all cases are static-routing and design-rubric checks, so no live API calls or keys are required.

package/skills/llm-pipeline/evals/cases.yaml

@@ -0,0 +1,44 @@
+skill: llm-pipeline
+
+should_trigger:
+  - prompt: "Chain three LLM calls: extract entities, then summarize, then classify — and make it not fall over."
+    why: "Canonical multi-step orchestration with data flowing between steps plus a reliability requirement — the core of this skill."
+  - prompt: "We keep getting random 429s and timeouts from OpenAI and it takes our feature down. Add fallbacks."
+    why: "Reliability layer: router fallback, retries, circuit-breaking around provider failures."
+  - prompt: "Put a gateway in front of OpenAI and Claude so we can swap models and load-balance."
+    why: "LLM gateway/router (LiteLLM) is the central tool of the skill."
+  - prompt: "Our LLM bill tripled — we call the model fresh on every request even for repeats."
+    why: "Non-obvious: framed as a cost problem, but the real fix is caching + tier routing, which are this skill's controls (not pure accounting)."
+  - prompt: "Necessito encadenar crides a l'LLM amb fallback i cachejar les respostes."
+    why: "Catalan phrasing for chaining + fallback + caching — exactly this skill, in another language."
+  - prompt: "Route easy requests to Haiku and only escalate to Opus when confidence is low."
+    why: "Model-tier routing, cheap-first with escalation, is the standard cost lever here."
+
+should_not_trigger:
+  - prompt: "Help me write a better system prompt and few-shot examples for this classification task."
+    route_to: prompt-engineering
+    why: "This is prompt wording, not the orchestration around prompts."
+  - prompt: "Make this single call return JSON matching my Pydantic schema."
+    route_to: structured-extraction
+    why: "One schema-bound call, not a chain — this skill uses structured output as a contract but does not own single extraction."
+  - prompt: "Chunk my docs, embed them, and retrieve relevant context for the prompt."
+    route_to: rag
+    why: "Building a retriever is retrieval, not call-chaining; a pipeline may call a retriever as a step but does not build it."
+  - prompt: "Build an agent that decides which tools to call to book travel."
+    route_to: building-agents
+    why: "An autonomous decision loop where the model picks its next step — not a fixed DAG you designed."
+  - prompt: "Build me a dashboard attributing our monthly AI spend per team."
+    route_to: cost-tracking
+    why: "Spend ledger and reporting, not the pipeline controls (tiers, caching, abort caps) this skill owns."
+
+capability:
+  - scenario: "Design a production pipeline that classifies an inbound support ticket, then drafts a reply, behind a router with fallback, with caching and a per-request cost cap."
+    must_include:
+      - "Typed/structured-output contract between the classify step and the draft step (validated at the seam)"
+      - "Explicit timeout on every LLM call (30-60s)"
+      - "Bounded retries with exponential backoff + jitter, never an unbounded loop"
+      - "Router with a fallback model, and mention of content_policy / context_window fallback buckets"
+      - "Prefix cache enabled (stable prefix first); semantic cache only if safe with a strong embedder + tuned threshold"
+      - "Model-tier routing cheap-first (e.g. Haiku then escalate), and a per-request budget cap that ABORTS"
+      - "Per-step observability fields: model, tokens, cost, latency_ms, cache_hit, fallback_used, retry_count"
+      - "Note that schema-valid does not mean correct; defer quality measurement to agent-eval"

package/skills/llm-pipeline/references/caching-layers.md

@@ -0,0 +1,60 @@
+# Caching layers: prefix vs semantic
+
+Two caches, different mechanisms, different risk profiles. Do not conflate them.
+
+## Layer 1 — prefix / prompt cache (provider-native)
+
+Matches an **exact prefix** of your prompt at the provider and reuses the cached compute. Transparent and safe: identical input → identical cached state. Free upside, no correctness risk.
+
+**OpenAI** — automatic, on by default, ~50% off cached input tokens. No write penalty, no storage fee. First request full price; subsequent prefix hits half price.
+
+**Anthropic** — explicit: you mark cache breakpoints (`cache_control`). Deeper discount:
+
+| Operation | Multiplier vs base input |
+| --- | --- |
+| Cache read | 0.1× (~90% off) |
+| 5-minute cache write | 1.25× |
+| 1-hour cache write | 2× |
+
+On long, stable prefixes this is ~90% cost and ~85% latency reduction.
+
+### Order prompts for maximum prefix hits
+
+Put content **most stable first, most variable last**:
+
+1. System prompt / role instructions (never changes)
+2. Fixed reference context, tool definitions, few-shot exemplars (changes rarely)
+3. Retrieved/dynamic context (changes per request)
+4. The user's actual turn (changes every time)
+
+If you put the user's variable input first, the prefix never matches and you cache nothing. For Anthropic, set the breakpoint at the end of the stable block (1–2).
+
+## Layer 2 — semantic cache (your gateway)
+
+Embeds the incoming query, looks up an **embedding-similar prior query**, and returns *that prior response*. Lives in your gateway: GPTCache, a Redis vector store, Bifrost, etc.
+
+The danger: it returns a **different** prior answer for a **similar** query. Quality is dominated by the embedding model — a weak embedder yields **false cache hits**, i.e. a confidently wrong answer to a question that only looked similar (GPTCache is documented returning incorrect saved responses for similar prompts; cf. Microsoft GenCache, NeurIPS 2025).
+
+### Enable it only when all hold
+
+- Queries are paraphrase-heavy (FAQ, support, repeated intents).
+- An approximate / near-match answer is acceptable for the use case.
+- It is **not** a correctness-critical path (no legal, medical, financial, or contract logic).
+
+### Tuning
+
+- **Embedding model:** use a strong, current embedder. The cheap default is where false hits come from.
+- **Similarity threshold:** start strict (e.g. cosine ≥ 0.95) and loosen only with eval data on real query pairs. Too loose = false hits; too strict = no hits.
+- **TTL:** short for time-sensitive answers (prices, status), longer for stable knowledge. Stale-but-confident is its own failure mode.
+- **Scope keys:** namespace the cache by tenant/locale/model so you never serve tenant A's answer to tenant B.
+
+## Multi-tier order
+
+Check caches cheapest-and-safest first, fall through to inference:
+
+```text
+semantic cache  ->  prefix cache  ->  live inference
+   (gateway)        (provider)         (full price)
+```
+
+Only reach inference on a miss. But if correctness matters more than cost, drop the semantic tier entirely and rely on prefix cache + inference — prefix cache never changes the answer.