rsc-universal 0.1.1

package/skills/fly-io/evals/cases.yaml

@@ -0,0 +1,42 @@
+skill: fly-io
+
+should_trigger:
+  - prompt: "Deploy my Dockerized API to Fly.io in Frankfurt and Virginia"
+    why: Core use — a multi-region Fly deploy with explicit region placement (fra + iad).
+  - prompt: "My fly.toml scales to zero but cold starts are slow, how do I keep one machine warm?"
+    why: Autostop tuning (min_machines_running tradeoff) — Fly-specific, not generic scaling theory.
+  - prompt: "Users in Sydney get high latency and my app only runs in iad"
+    why: Non-obvious — no 'Fly' keyword, but 'iad' region code + latency-by-distance is a Fly region-placement problem (add a Machine in syd).
+  - prompt: "Need to attach a persistent disk to my Fly machine for SQLite"
+    why: Fly Volumes — region/machine-pinned disk attached via [[mounts]].
+  - prompt: "Vull desplegar a Fly.io amb secrets i rèpliques per regió"
+    why: Catalan — deploy to Fly.io with secrets and per-region replicas; secrets + multi-region.
+  - prompt: "How do I forward writes to the primary region while reads stay local on Fly?"
+    why: Non-obvious — the fly-replay write-forwarding pattern for stateful multi-region.
+
+should_not_trigger:
+  - prompt: "Deploy my app to Railway with git push"
+    route_to: railway
+    why: Railway is a git-push PaaS with no Machines/regions model — different platform.
+  - prompt: "Host my Next.js site on the edge with a CDN"
+    route_to: vercel
+    why: Frontend/serverless edge hosting, not Fly Machines.
+  - prompt: "Write a multi-stage Dockerfile for my Go service"
+    route_to: docker
+    why: Image build mechanics — Fly consumes the Dockerfile but does not author it.
+  - prompt: "Design a load-shedding and queue strategy for traffic spikes"
+    route_to: scaling
+    why: Generic horizontal-scaling theory, platform-agnostic.
+
+capability:
+  - scenario: >
+      A single-region (iad) Express app talks to Postgres via DATABASE_URL. Make it
+      serve EU + US users and scale to zero outside business hours, keeping cold-start
+      pain in mind.
+    must_include:
+      - "fly.toml with primary_region = \"iad\" and an [http_service] with internal_port"
+      - "auto_stop_machines and auto_start_machines set together (not one alone)"
+      - "min_machines_running = 0 for scale-to-zero, with the cold-start tradeoff called out (set to 1 to keep one warm; honored only in primary region)"
+      - "fly scale count with --region to add an EU region (e.g. ams) for nearby users"
+      - "secrets via `fly secrets set DATABASE_URL=...`, never in [env]"
+      - "note that Fly Volumes/Postgres are region-pinned with no replication; cross-region writes need a replica + fly-replay or a managed/external DB"

package/skills/fly-io/references/fly-toml.md

@@ -0,0 +1,155 @@
+# fly.toml field reference
+
+The full configuration surface. The SKILL.md body covers the common case; reach here for a key it did not show. Source: https://fly.io/docs/reference/configuration/ (accessed 2026-06-02).
+
+## Top level
+
+```toml
+app              = "my-api"      # required, unique app name
+primary_region   = "iad"        # 3-letter region code; the write/anchor region
+kill_signal      = "SIGINT"     # signal sent on shutdown
+kill_timeout     = "5s"         # grace period before SIGKILL
+swap_size_mb     = 512          # optional swap file (helps memory-spiky workloads)
+
+[experimental]
+  # rarely needed; flags for unreleased behavior
+```
+
+## [build]
+
+```toml
+[build]
+  dockerfile      = "Dockerfile"   # default if present
+  # builder        = "paketobuildpacks/builder:base"   # buildpack instead of Dockerfile
+  # image          = "flyio/myimage:latest"            # deploy a prebuilt image, skip build
+  [build.args]
+    NODE_VERSION = "22"
+```
+
+The image mechanics belong to ../docker/SKILL.md — Fly only consumes the build.
+
+## Services: [http_service] vs [[services]]
+
+`[http_service]` is the sugar for the common single HTTP service. Use `[[services]]` when you need raw TCP, multiple ports, or several services.
+
+```toml
+# Lower-level, explicit form
+[[services]]
+  internal_port = 8080
+  protocol      = "tcp"
+  processes     = ["web"]
+
+  [[services.ports]]
+    port     = 443
+    handlers = ["tls", "http"]
+  [[services.ports]]
+    port        = 80
+    handlers    = ["http"]
+    force_https = true
+
+  [services.concurrency]
+    type       = "requests"   # "requests" | "connections"
+    soft_limit = 200
+    hard_limit = 250
+
+  [[services.tcp_checks]]
+    interval = "15s"
+    timeout  = "2s"
+  [[services.http_checks]]
+    interval = "15s"
+    timeout  = "2s"
+    method   = "get"
+    path     = "/health"
+```
+
+`internal_port` is the port your process listens on inside the container. `handlers` tell the Proxy how to terminate (`tls`, `http`). Exactly one service should define the public HTTP entrypoint.
+
+## [[vm]]
+
+Formerly `[[compute]]`. Bind to process groups via `processes`.
+
+```toml
+[[vm]]
+  size       = "shared-cpu-1x"   # preset; sets cpus + memory together
+  memory     = "512mb"           # override the preset memory
+  cpu_kind   = "shared"          # "shared" | "performance"
+  cpus       = 1
+  processes  = ["web"]
+```
+
+Common sizes: `shared-cpu-1x/2x/4x/8x` (burstable), `performance-1x/2x/4x/8x` (dedicated). Memory presets scale with the size; override with `memory` for memory-heavy, low-CPU work.
+
+## [processes]
+
+```toml
+[processes]
+  web    = "node server.js"
+  worker = "node worker.js"
+```
+
+Each key is a process group running the same image with a different command. `[[vm]]`, `[http_service]`, `[[mounts]]`, and `[[services]]` all target groups via `processes`. A group with no service takes no inbound traffic.
+
+## [[mounts]]
+
+```toml
+[[mounts]]
+  source             = "data"     # volume name
+  destination        = "/data"    # mount path in the guest
+  processes          = ["web"]
+  initial_size       = "1gb"
+  snapshot_retention = 7          # days of automatic volume snapshots
+```
+
+One volume per Machine, region-pinned, no replication. See the SKILL.md Volumes section and `multi-region.md`.
+
+## [deploy]
+
+```toml
+[deploy]
+  release_command = "npm run migrate"   # one-shot Machine run before the rollout goes live
+  strategy        = "rolling"            # rolling | bluegreen | canary | immediate
+  max_unavailable = 0.33                 # fraction (or int) of Machines down at once during rolling
+```
+
+`release_command` runs in a temporary Machine with the new image + secrets; if it exits nonzero, the deploy aborts. Ideal for migrations.
+
+## [[statics]]
+
+Serve static assets directly from the Proxy / object storage instead of your process:
+
+```toml
+[[statics]]
+  guest_path = "/app/public"
+  url_prefix = "/static/"
+```
+
+## [[files]]
+
+Write files into the guest at boot — including secrets as files:
+
+```toml
+[[files]]
+  guest_path  = "/etc/app/config.yaml"
+  local_path  = "config/prod.yaml"     # ship a local file
+[[files]]
+  guest_path  = "/etc/app/sa.json"
+  secret_name = "GCP_SA_JSON"          # materialize a secret as a file
+[[files]]
+  guest_path  = "/etc/app/banner.txt"
+  raw_value   = "aGVsbG8="             # base64 inline content
+```
+
+Use `secret_name` for credentials a library expects on disk (e.g. a service-account JSON) rather than in an env var.
+
+## [restart] and [metrics]
+
+```toml
+[restart]
+  policy = "on-failure"   # "no" | "on-failure" | "always"
+
+[metrics]
+  port = 9091
+  path = "/metrics"       # Prometheus scrape endpoint Fly ingests
+```
+
+The `[metrics]` endpoint is what `fly-autoscaler` reads to scale by custom metrics (see `multi-region.md` / the SKILL.md scaling table).

package/skills/fly-io/references/multi-region.md

@@ -0,0 +1,66 @@
+# Multi-region playbook
+
+Most deploys do not need this — run Machines in more regions and let anycast do the rest. Reach here when you have **state** that must stay correct across regions. Sources: https://fly.io/docs/networking/dynamic-request-routing/ , https://fly.io/docs/blueprints/multi-region-fly-replay/ , https://fly.io/docs/launch/scale-count/ (accessed 2026-06-02).
+
+## Decide the shape
+
+| Your state lives in... | Pattern |
+| --- | --- |
+| Nowhere local (DB is external/managed, app is stateless) | **Stateless replicate** — just add regions |
+| A Fly Volume / embedded DB (SQLite, LiteFS) | **Primary + read replicas** with `fly-replay` write forwarding |
+| Fly Postgres | One primary region; route writes there, reads to local replicas |
+
+## Stateless replicate
+
+The easy path. No volumes, no write coordination.
+
+```bash
+fly scale count 1 --region iad,ams,syd   # one Machine in each of three regions
+fly scale show
+```
+
+Anycast sends each caller to the nearest Machine. All Machines are identical and talk to the same external datastore. Done.
+
+## Primary + read replicas (fly-replay write forwarding)
+
+When reads can be local but **writes must hit one region** (the `primary_region`), use the `fly-replay` header. Replicas serve reads instantly; on a write, the app tells the Proxy to replay the entire request in the primary region.
+
+Header forms the app can return:
+
+```text
+fly-replay: region=iad        # replay this request in region iad
+fly-replay: instance=<id>      # replay on a specific Machine
+fly-replay: app=other-app      # replay against a different app
+fly-replay: elsewhere=true     # "not me" — let the Proxy pick another Machine
+```
+
+Sketch (Node/Express), forwarding writes to the primary region:
+
+```javascript
+const PRIMARY = process.env.PRIMARY_REGION;   // Fly injects PRIMARY_REGION
+const HERE    = process.env.FLY_REGION;       // and FLY_REGION for the current Machine
+
+app.use((req, res, next) => {
+  const isWrite = req.method !== "GET" && req.method !== "HEAD";
+  if (isWrite && HERE !== PRIMARY) {
+    res.set("fly-replay", `region=${PRIMARY}`);
+    return res.status(409).end();   // body is discarded; Proxy re-runs the request in PRIMARY
+  }
+  next();
+});
+```
+
+Why a status like 409: the response is never sent to the client — the Proxy intercepts the `fly-replay` header and re-runs the original request (method, path, body) in the target. Your replica code just needs to *refuse locally and tag it*.
+
+`PRIMARY_REGION` and `FLY_REGION` are environment variables Fly injects into every Machine — read them to know "am I the writer."
+
+## Volume strategy across regions
+
+- A volume is pinned to one Machine in one region; there is no cross-region replication.
+- For per-region read replicas, give **each region its own volume** and replicate at the application layer.
+- **LiteFS** is Fly's distributed SQLite layer: it ships the primary's writes to replica nodes and integrates with `fly-replay` so writes route to the primary automatically. Use it when you want SQLite semantics with multi-region reads.
+- For Postgres, run a primary in `primary_region` and read replicas elsewhere; send writes to the primary (via connection routing or `fly-replay`). Operating the engine itself belongs to ../postgresdb/SKILL.md.
+
+## Capacity caveat
+
+`fly scale count N --region a,b,c` places N Machines in **each** listed region. If **any** region is out of capacity the whole operation fails atomically — nothing is placed. Retry with fewer regions, a different code (`fly platform regions` for alternatives), or place granularly with `fly machine clone <id> --region <code>`.

package/skills/fly-io/scripts/verify.sh

@@ -0,0 +1,90 @@
+#!/usr/bin/env bash
+# verify.sh — lint a fly.toml. Read-only. Exits 0 on a clean/empty target.
+#
+# Usage: scripts/verify.sh [path/to/fly.toml]   (defaults to ./fly.toml)
+#
+# Prefers `fly config validate` when flyctl is on PATH; otherwise runs
+# structural grep checks. Prints PASS:/FAIL: lines; nonzero exit on any FAIL.
+
+set -u
+
+TARGET="${1:-fly.toml}"
+fails=0
+
+pass() { printf 'PASS: %s\n' "$1"; }
+fail() { printf 'FAIL: %s\n' "$1"; fails=$((fails + 1)); }
+info() { printf 'INFO: %s\n' "$1"; }
+
+# Empty/clean target: nothing to lint, do not invent a failure.
+if [ ! -e "$TARGET" ]; then
+  info "no fly.toml at '$TARGET' — nothing to verify"
+  exit 0
+fi
+if [ ! -s "$TARGET" ]; then
+  info "'$TARGET' is empty — nothing to verify"
+  exit 0
+fi
+
+pass "found $TARGET"
+
+# Prefer the real validator when available.
+FLY_BIN=""
+if command -v fly >/dev/null 2>&1; then
+  FLY_BIN="fly"
+elif command -v flyctl >/dev/null 2>&1; then
+  FLY_BIN="flyctl"
+fi
+
+if [ -n "$FLY_BIN" ]; then
+  info "$FLY_BIN on PATH — running '$FLY_BIN config validate'"
+  if "$FLY_BIN" config validate --config "$TARGET" >/dev/null 2>&1; then
+    pass "$FLY_BIN config validate"
+  else
+    fail "$FLY_BIN config validate reported errors (run '$FLY_BIN config validate --config $TARGET')"
+  fi
+  [ "$fails" -eq 0 ] && exit 0 || exit 1
+fi
+
+info "flyctl not found — falling back to structural checks"
+
+# app = "..."
+if grep -Eq '^[[:space:]]*app[[:space:]]*=' "$TARGET"; then
+  pass "app is set"
+else
+  fail "missing top-level 'app ='"
+fi
+
+# primary_region = "..."
+if grep -Eq '^[[:space:]]*primary_region[[:space:]]*=' "$TARGET"; then
+  pass "primary_region is set"
+else
+  fail "missing 'primary_region ='"
+fi
+
+# an internal_port under [http_service] or [[services]]
+if grep -Eq '^[[:space:]]*internal_port[[:space:]]*=' "$TARGET"; then
+  pass "internal_port is defined"
+else
+  fail "no 'internal_port' found (expected under [http_service] or [[services]])"
+fi
+
+# autostop pair lint: if either key present, both must be present
+has_stop=0; has_start=0
+grep -Eq '^[[:space:]]*auto_stop_machines[[:space:]]*='  "$TARGET" && has_stop=1
+grep -Eq '^[[:space:]]*auto_start_machines[[:space:]]*=' "$TARGET" && has_start=1
+if [ "$has_stop" -eq 1 ] || [ "$has_start" -eq 1 ]; then
+  if [ "$has_stop" -eq 1 ] && [ "$has_start" -eq 1 ]; then
+    pass "auto_stop_machines and auto_start_machines set together"
+  else
+    fail "set auto_stop_machines and auto_start_machines together (only one is present)"
+  fi
+else
+  info "no autostop keys — skipping pair lint"
+fi
+
+if [ "$fails" -eq 0 ]; then
+  printf '\nOK: %s passed structural checks\n' "$TARGET"
+  exit 0
+fi
+printf '\n%d check(s) failed\n' "$fails"
+exit 1

package/skills/forecasting/SKILL.md

@@ -0,0 +1,139 @@
+---
+name: forecasting
+description: "Use when you need to project sales, demand, units, revenue, signups or traffic forward from historical data and report a defensible number with an error band. Triggers: 'forecast next quarter's sales', 'how many units of SKU X next month', 'project revenue 12 months with a confidence band', 'is our forecast actually better than just repeating last quarter', lumpy/intermittent demand with lots of zero-sales weeks, 'prevé las ventas del próximo trimestre', 'prediu la demanda del proper mes per producte'. NOT building an assumption-driven P&L (that is financial-model), NOT sizing reorder points or safety stock (that is inventory)."
+tags: [forecasting, demand-planning, time-series, sales-forecast, statsforecast]
+recommends: [inventory, financial-model, unit-economics, data-cleaning, analyze, duckdb]
+origin: risco
+---
+
+# Forecasting
+
+A forecast that cannot beat "repeat last period" is noise. Baseline first, fancy second. The naive forecast is free, instant, and the bar every model must clear — if your AutoARIMA loses to last-quarter-repeated, ship the repeat and say so.
+
+You are not done when a model produces a number. You are done when you can defend the number: which method, why that method for this data, how it scored against the naive baseline in a backtest, and the interval around the point. A point estimate with no error band is a guess wearing a lab coat.
+
+## The deliverable contract
+
+Every forecast you ship is a reproducible artifact, not a number pasted in chat:
+
+1. A **script** that reads the history and regenerates the forecast (no manual steps).
+2. A **CSV/Parquet** with columns `ds, forecast, lo, hi` — timestamp, point, interval bounds.
+3. A **one-paragraph accuracy readout**: WAPE + bias from a rolling-origin backtest, and MASE vs the naive baseline (MASE < 1.0 = you beat naive; ≥ 1.0 = ship the naive forecast instead).
+
+If you cannot produce all three, you have not forecast — you have guessed. `scripts/verify.sh` checks the artifact has these columns, the right row count, and an accuracy line.
+
+## The loop
+
+Run these in order. Skipping step 3 is the most common failure.
+
+1. **Frame it.** Pin down the horizon `h` (how many periods forward), the granularity (daily / weekly / monthly), and exactly what is being predicted (units? revenue? per-SKU or aggregate?). Forecast at the level you will *act* on — if you reorder per SKU, forecast per SKU, then sanity-check against the aggregate.
+2. **Establish the series.** Regular timestamps, one row per period, gaps filled *explicitly* (a missing month is not zero unless it truly is). Flag promotions, stockouts, and outliers — they distort the signal. If the input is dirty (dupes, missing rows, mixed units), stop and hand off to `data-cleaning` before modeling. Garbage history, garbage forecast.
+3. **Build the naive + seasonal-naive baseline.** This is the bar. Naive = repeat last value. Seasonal-naive = repeat the value from one season ago (e.g. last December for this December). Compute its backtest error now — every fancier method must beat it or lose.
+4. **Pick the method by data shape** (table below). Do not reach for ARIMA on instinct.
+5. **Backtest with rolling-origin cross-validation.** Never a single holdout. Compute WAPE + bias + MASE vs the naive baseline across multiple cutoffs.
+6. **Report.** Point + interval, the one-line method rationale, the accuracy readout. Then hand off downstream (`inventory`, `financial-model`).
+
+## Method selection
+
+Match the method to the *shape* of the history, not to what sounds sophisticated. statsforecast (Nixtla, v2.0.3) provides all of these with built-in intervals.
+
+| Data shape | Method | statsforecast call | Why |
+|---|---|---|---|
+| Flat, no trend or season | Moving average or SES | `AutoCES()` / 3-period MA | Nothing to model; a mean is honest. |
+| Trend, with or without season | ETS | `AutoETS(season_length=m)` | ETS captures level+trend+season cleanly, no manual order. |
+| Strong known seasonality / autocorrelation | ARIMA | `AutoARIMA(season_length=m)` | Handles autocorrelated errors; ~20x faster than pmdarima. |
+| Many zeros (intermittent / lumpy demand) | Croston / SBA | `CrostonOptimized()` | SES is *provably* wrong on sporadic demand (Croston 1972); SBA debiases it. |
+| < 2 full seasonal cycles of history | SeasonalNaive **only** | `SeasonalNaive(season_length=m)` | Too little data to fit a model. Do not fit one. Full stop. |
+
+When in doubt between two, fit both plus the baseline in one `StatsForecast` run and let the backtest decide. Theta (`AutoTheta`) is a strong, cheap default that often wins on monthly business series.
+
+## Accuracy and honesty
+
+The metrics are not decoration — they decide what you ship.
+
+- **WAPE, not MAPE.** MAPE divides by the actual, so it explodes and misleads whenever actuals approach zero (constant in SKU and intermittent data). WAPE = total absolute error / total actual volume — it weights error by volume and stays stable. Use WAPE as the default magnitude metric.
+- **Pair WAPE with bias.** WAPE tells you *how big* the error is; bias tells you the *direction* — whether you systematically over- or under-forecast. A 10% WAPE with +9% bias means you are almost always forecasting high; that is an actionable, different problem than random error.
+- **MASE < 1.0 is the pass/fail line.** MASE is scale-free and compares your error to the naive forecast's error. < 1.0 = you beat naive (good). ≥ 1.0 = the naive forecast was better — ship the naive one and stop pretending. This is the single most important number in the readout.
+- **Rolling-origin, never a single holdout.** Time-series CV repeats the train/test split across multiple cutoffs (expanding window), giving a far more reliable estimate than one lucky/unlucky split. Use `cross_validation(h=…, n_windows=…)`.
+- **Always emit an interval.** A point forecast cannot express uncertainty, and point metrics cannot evaluate a distribution. Report `level=[80]` or `[95]`. The interval is not optional polish — it is half the deliverable.
+
+Formulas (WAPE, MASE, bias, pinball, coverage), rolling-origin mechanics, and how to read a backtest table are in `references/accuracy-and-backtesting.md`. Per-method when-to-use and the exact statsforecast one-liner for each are in `references/methods-cheatsheet.md`.
+
+## Minimal pipeline
+
+The full pipeline: long-format dataframe, fit competing methods + baseline, backtest, forecast with an interval, write the artifact.
+
+```python
+# pip install statsforecast  (Nixtla, v2.0.3)
+import pandas as pd
+from statsforecast import StatsForecast
+from statsforecast.models import SeasonalNaive, AutoETS, AutoARIMA
+
+# long format: unique_id, ds, y  (one row per series per period)
+df = pd.read_csv("history.csv", parse_dates=["ds"])
+m, h = 12, 12  # monthly seasonality; forecast 12 periods ahead
+
+sf = StatsForecast(
+    models=[SeasonalNaive(season_length=m), AutoETS(season_length=m), AutoARIMA(season_length=m)],
+    freq="MS",
+)
+
+# rolling-origin backtest BEFORE trusting any forecast
+cv = sf.cross_validation(df=df, h=h, n_windows=3, step_size=h)
+def wape(a, f): return (a - f).abs().sum() / a.abs().sum()
+for col in ["SeasonalNaive", "AutoETS", "AutoARIMA"]:
+    print(col, "WAPE", round(wape(cv["y"], cv[col]), 4))  # pick the lowest that beats SeasonalNaive
+
+# refit on full history, forecast with an 80% interval
+fc = sf.forecast(df=df, h=h, level=[80])
+# choose the winning model column from the backtest; here AutoETS as example
+out = fc.rename(columns={"AutoETS": "forecast", "AutoETS-lo-80": "lo", "AutoETS-hi-80": "hi"})
+out[["ds", "forecast", "lo", "hi"]].to_csv("forecast.csv", index=False)
+```
+
+Zero-dependency fallback when you cannot install statsforecast — a seasonal-naive baseline in pure pandas. This is also the thing every model must beat, so it is always worth computing:
+
+```python
+import pandas as pd
+
+def seasonal_naive(y: pd.Series, m: int, h: int) -> pd.Series:
+    """Repeat the last full season forward h periods."""
+    last_season = y.iloc[-m:].to_numpy()
+    return pd.Series([last_season[i % m] for i in range(h)])
+
+s = pd.read_csv("history.csv", parse_dates=["ds"]).set_index("ds")["y"]
+fc = seasonal_naive(s, m=12, h=12)
+# crude interval from historical residual spread; honest is better than absent
+resid_std = (s - s.shift(12)).dropna().std()
+out = pd.DataFrame({"forecast": fc, "lo": fc - 1.28 * resid_std, "hi": fc + 1.28 * resid_std})
+out.to_csv("forecast.csv", index=False)
+```
+
+## Edge cases
+
+- **New product, no history.** Do not fit a model on three points. Use analogues — a comparable product's curve scaled to expected volume — and say it is an assumption, not a forecast.
+- **Short history (< 2 seasonal cycles).** SeasonalNaive only. A fitted model will overfit noise and report a falsely tight interval.
+- **Structural breaks.** A pricing change, a relaunch, a regime shift. Do not train across the break — train on the post-break segment, even if it is short, or the model averages two different worlds.
+- **Promotions and outliers.** A promo spike is not baseline demand. Mark promo periods and either model them as a regressor or exclude them from the level estimate; otherwise the forecast inherits a spike that will not recur.
+- **Granularity.** Forecast at the level you act on. If you must report higher, aggregate the forecasts — and check the aggregate is plausible (this often catches per-SKU nonsense).
+- **Hierarchy reconciliation.** When SKU forecasts must sum to a category total, reconcile (bottom-up or MinT). Brief note here; the mechanics belong in `references/methods-cheatsheet.md`.
+
+## Anti-patterns
+
+| Bad | Good | Why |
+|---|---|---|
+| Report a single point number | Report point + 80/95% interval | A point hides uncertainty the reader needs to plan around. |
+| Tune ARIMA orders before any baseline | Compute naive/seasonal-naive first | If you cannot beat the free baseline, the tuning was wasted. |
+| Score with MAPE on intermittent demand | Use WAPE + bias | MAPE explodes near zero actuals and lies about accuracy. |
+| Single train/test holdout | Rolling-origin CV (`n_windows≥3`) | One split is one sample; CV estimates real out-of-sample error. |
+| Fit a model on 8 months of monthly data | SeasonalNaive only under 2 cycles | Too few points; the model overfits and underreports its own error. |
+| "The model picked it, so it's right" | State method + MASE vs naive | A forecast you cannot defend is worse than no forecast. |
+| Trust SKU forecasts without checking the sum | Sanity-check vs the aggregate | Per-SKU errors compound; the total exposes nonsense fast. |
+
+## Handoffs
+
+- **`../inventory/SKILL.md`** — feed it the demand number; it sizes reorder points and safety stock. Forecasting produces the demand; it does not size the stock.
+- **`../financial-model/SKILL.md`** — when the projection is driven by *assumptions and drivers* (pricing, hiring), not history, that is a model, not a forecast.
+- **`../unit-economics/SKILL.md`** — contribution margin, CAC/LTV, payback. No time series, route there.
+- **`data-cleaning`** — dirty input (dupes, missing rows, mixed units) goes here *before* you model.
+- **`../analyze/SKILL.md`** — when the question is "why" or a backward-looking metric/aggregation rather than forward extrapolation.

package/skills/forecasting/evals/README.md

@@ -0,0 +1,13 @@
+# Evals — forecasting
+
+These cases are read by the catalog eval harness, not by a standalone runner here.
+`should_trigger` and `should_not_trigger` check routing: each near-miss in
+`should_not_trigger` names the real sibling skill it actually belongs to
+(`financial-model`, `unit-economics`, `inventory`, `data-cleaning`, `analyze`), so a
+correct router sends it there instead of to forecasting. The `capability` case is a
+rubric scored by a judge against its `must_include` list — it grades whether a
+forecast actually establishes a baseline, backtests with rolling origin, reports
+WAPE/bias/MASE, and ships an interval. Separately, run `scripts/verify.sh` to confirm
+the pipeline emits a valid artifact (`ds,forecast` plus an interval column, correct
+row count, and an accuracy line); that script is read-only against your project and
+exits 0 on a clean/empty target.

package/skills/forecasting/evals/cases.yaml

@@ -0,0 +1,47 @@
+skill: forecasting
+
+should_trigger:
+  - prompt: "We have 3 years of monthly sales — forecast the next 4 quarters."
+    why: Core horizon-plus-history time-series projection, the central task.
+  - prompt: "How many units of SKU-1182 will we sell next month?"
+    why: Single-series demand forecast at SKU granularity.
+  - prompt: "Project signups forward 12 months and give me a confidence band."
+    why: Forward extrapolation with an explicit interval request.
+  - prompt: "Half our weeks have zero sales for this part — can you still forecast demand?"
+    why: Non-obvious — intermittent/lumpy demand routes to Croston/SBA, not plain ETS, but it is squarely forecasting.
+  - prompt: "Is our sales forecast actually better than just repeating last quarter?"
+    why: Non-obvious — this is a baseline/backtest (MASE vs naive) question, the core honesty check of forecasting.
+  - prompt: "Prevé las ventas del próximo trimestre a partir del histórico mensual."
+    why: Spanish phrasing of the core next-period sales forecast.
+  - prompt: "Prediu la demanda del proper mes per producte."
+    why: Catalan phrasing of per-SKU demand forecast.
+
+should_not_trigger:
+  - prompt: "Build a 3-year P&L from our pricing and hiring assumptions."
+    route_to: financial-model
+    why: Assumption- and driver-based projection, not extrapolation of a historical time series.
+  - prompt: "Compute our blended CAC and LTV and the payback period."
+    route_to: unit-economics
+    why: Margin and unit metrics with no forward time-series component.
+  - prompt: "Given expected demand, set reorder points and safety stock."
+    route_to: inventory
+    why: Stock sizing consumes a forecast; it does not produce one.
+  - prompt: "There are duplicate and missing rows in the sales export — fix it before anything else."
+    route_to: data-cleaning
+    why: Data hygiene precedes modeling and is not forecasting itself.
+  - prompt: "Why did revenue drop in March versus February?"
+    route_to: analyze
+    why: Backward-looking explanatory analysis, not forward extrapolation.
+
+capability:
+  - scenario: >
+      Given 24 months of monthly unit sales for one product, produce a 12-month
+      forecast with an 80% prediction interval, choosing a method appropriate to
+      the data and proving it beats a naive baseline.
+    must_include:
+      - establishes a seasonal-naive (or naive) baseline before fitting any model
+      - selects a method by data shape (e.g. AutoETS for trend+season) with a one-line rationale
+      - uses rolling-origin / expanding-window cross-validation, not a single holdout
+      - reports WAPE and bias (not MAPE) and MASE vs naive, noting MASE < 1 means it beat naive
+      - outputs an artifact with ds and forecast plus lo/hi interval columns
+      - notes uncertainty and when to fall back to the naive forecast (MASE >= 1)

package/skills/forecasting/references/accuracy-and-backtesting.md

@@ -0,0 +1,104 @@
+# Accuracy and backtesting
+
+The metrics decide what you ship. This is the reference for the formulas, the
+rolling-origin mechanics, and how to read a backtest table.
+
+## Metrics
+
+### WAPE (a.k.a. WMAPE) — the default magnitude metric
+
+```
+WAPE = sum(|actual - forecast|) / sum(|actual|)
+```
+
+Weights error by volume. Stable when actuals approach zero, which is exactly where
+MAPE fails. Report as a percentage. This is your headline accuracy number.
+
+### MAPE — avoid for demand
+
+```
+MAPE = mean(|actual - forecast| / |actual|)
+```
+
+Divides by each actual, so a single near-zero period sends it to infinity. Fine for
+smooth high-volume aggregate series; wrong for SKU-level and intermittent demand.
+Default to WAPE and only use MAPE when actuals are reliably far from zero.
+
+### Bias — direction of error
+
+```
+bias = sum(forecast - actual) / sum(|actual|)
+```
+
+Positive = systematic over-forecast, negative = under-forecast. Always report
+alongside WAPE: WAPE is magnitude, bias is direction. A persistent bias is a
+fixable, structural problem (wrong seasonality, ignored trend); random error around
+zero bias is not.
+
+### MASE — the pass/fail line vs naive
+
+```
+MASE = MAE(forecast) / MAE(naive forecast on the training set)
+```
+
+Scale-free. **MASE < 1.0 means you beat the naive forecast.** MASE ≥ 1.0 means the
+naive forecast was more accurate — ship the naive one. For seasonal series the
+denominator is the seasonal-naive MAE. This is the single number that justifies the
+existence of your model.
+
+### Interval quality — coverage and pinball
+
+A point metric cannot grade an interval. Use:
+
+- **Coverage:** the fraction of actuals that fall inside the interval. An 80%
+  interval should contain ~80% of actuals in the backtest. Far below = overconfident
+  (intervals too tight); far above = useless (too wide).
+- **Pinball loss (quantile loss):** the proper scoring rule for a quantile forecast.
+  Lower is better; it penalizes both miscalibration and width.
+
+## Rolling-origin cross-validation
+
+Never trust a single holdout — it is one sample of out-of-sample error and can be
+lucky or unlucky. Rolling-origin (expanding-window) CV repeats the split across
+several cutoffs and averages the error.
+
+Mechanics (statsforecast `cross_validation`):
+
+```
+window 1:  train [.................]  test [###]
+window 2:  train [....................]  test [###]
+window 3:  train [.......................]  test [###]
+```
+
+- `h` — forecast horizon per window (match your real horizon).
+- `n_windows` — how many cutoffs (≥ 3; more is better if history allows).
+- `step_size` — gap between cutoffs (commonly `= h` for non-overlapping tests).
+
+```python
+cv = sf.cross_validation(df=df, h=12, n_windows=3, step_size=12)
+# cv has: ds, cutoff, y, and one column per model
+```
+
+## Reading a backtest table
+
+`cross_validation` returns the actual `y` and each model's prediction per cutoff.
+Collapse to one row per model:
+
+| model         | WAPE | bias  | MASE | verdict                         |
+|---------------|------|-------|------|---------------------------------|
+| SeasonalNaive | 0.18 | +0.02 | 1.00 | the bar (MASE is 1 by definition)|
+| AutoETS       | 0.12 | -0.01 | 0.67 | beats naive — ship this         |
+| AutoARIMA     | 0.13 | +0.06 | 0.72 | beats naive but biased high      |
+
+Pick the lowest WAPE among models with **MASE < 1**. If no model has MASE < 1, ship
+SeasonalNaive and report it honestly. Prefer a slightly higher-WAPE model with near-
+zero bias over a lower-WAPE model with strong bias — bias compounds when you sum or
+roll forward.
+
+## Interval calibration
+
+After choosing a model, check the chosen interval's coverage in the backtest. If an
+80% interval covers only ~55% of actuals, the model is overconfident — widen the
+level, switch to a method whose intervals are better calibrated for the series, or
+state the limitation explicitly in the readout. An interval nobody calibrated is no
+better than the point alone.