@damiangilgonzalez/harness 1.3.7

package/skills/ollama/references/api.md

@@ -0,0 +1,148 @@
+# Ollama local API reference
+
+Daemon base URL: `http://localhost:11434`. Bind elsewhere with `OLLAMA_HOST=0.0.0.0:11434`.
+Two surfaces share the daemon: the **native** API (`/api/*`) and the **OpenAI-compatible** layer
+(`/v1/*`).
+
+## Native endpoint catalog
+
+| Method | Path | Purpose |
+| --- | --- | --- |
+| POST | `/api/generate` | single-turn completion (`prompt`) |
+| POST | `/api/chat` | multi-turn chat (`messages`); supports `tools`, `images`, `think` |
+| POST | `/api/embed` | embeddings (`input` string or array); legacy alias `/api/embeddings` |
+| POST | `/api/create` | create a model from a Modelfile / quantize |
+| POST | `/api/pull` | download a model |
+| POST | `/api/push` | upload a model |
+| POST | `/api/show` | model template, params, context length, quant |
+| POST | `/api/copy` | duplicate a model under a new name |
+| DELETE | `/api/delete` | remove a model |
+| GET | `/api/tags` | models on disk (≈ `ollama list`) |
+| GET | `/api/ps` | models loaded in memory (≈ `ollama ps`) |
+| GET | `/api/version` | daemon version |
+
+## Request fields (generate / chat)
+
+| Field | Type | Default | Notes |
+| --- | --- | --- | --- |
+| `model` | string | — | **required**; a pulled tag |
+| `prompt` | string | — | `/api/generate` only |
+| `messages` | array | — | `/api/chat`; `{role, content, images?, tool_calls?}` |
+| `stream` | bool | `true` | NDJSON stream; set `false` for a single object |
+| `format` | string \| object | — | `"json"` or a JSON Schema object to constrain output |
+| `options` | object | — | runtime params: `temperature`, `num_ctx`, `top_p`, `seed`, `num_predict`, `stop`, … |
+| `keep_alive` | string \| int | `"5m"` | how long to keep loaded; `0` unloads now, `-1` keeps forever |
+| `tools` | array | — | tool/function definitions for tool calling |
+| `images` | array | — | base64 images for multimodal models |
+| `think` | bool | — | enable/disable reasoning on models that support it |
+
+`options.num_ctx` is per-request and does not persist. Bake a permanent context window into a
+Modelfile with `PARAMETER num_ctx`.
+
+## Response fields
+
+Final object (streaming or not) carries: `done` (bool), `done_reason`, `total_duration`,
+`load_duration`, `prompt_eval_count`, `prompt_eval_duration`, `eval_count`, `eval_duration`.
+Tokens/sec ≈ `eval_count / (eval_duration / 1e9)`.
+
+Streaming = NDJSON: one JSON object per line; accumulate `message.content` (chat) or `response`
+(generate) until `done: true`.
+
+```python
+import requests, json
+
+with requests.post("http://localhost:11434/api/chat",
+                   json={"model": "qwen3:8b",
+                         "messages": [{"role": "user", "content": "Count to 3."}]},
+                   stream=True) as r:
+    for line in r.iter_lines():
+        if line:
+            obj = json.loads(line)
+            print(obj.get("message", {}).get("content", ""), end="", flush=True)
+            if obj.get("done"):
+                break
+```
+
+## Structured output via `format`
+
+Pass a JSON Schema as `format`; the model is constrained to emit conforming JSON. Pair with a low
+temperature and an explicit instruction to return JSON.
+
+```bash
+curl http://localhost:11434/api/chat -d '{
+  "model": "qwen3:8b",
+  "messages": [{"role": "user", "content": "List two EU capitals as JSON."}],
+  "stream": false,
+  "options": {"temperature": 0},
+  "format": {
+    "type": "object",
+    "properties": {
+      "capitals": {"type": "array", "items": {"type": "string"}}
+    },
+    "required": ["capitals"]
+  }
+}'
+```
+
+## Tool calling
+
+Define `tools`; the response may contain `message.tool_calls`. Execute the tool, append a
+`{"role": "tool", "content": "<result>"}` message, and call again.
+
+```bash
+curl http://localhost:11434/api/chat -d '{
+  "model": "qwen3:8b",
+  "stream": false,
+  "messages": [{"role": "user", "content": "Weather in Andorra?"}],
+  "tools": [{
+    "type": "function",
+    "function": {
+      "name": "get_weather",
+      "description": "Current weather for a city",
+      "parameters": {
+        "type": "object",
+        "properties": {"city": {"type": "string"}},
+        "required": ["city"]
+      }
+    }
+  }]
+}'
+```
+
+## Embeddings
+
+```bash
+curl http://localhost:11434/api/embed -d '{
+  "model": "nomic-embed-text",
+  "input": ["first chunk", "second chunk"]
+}'
+```
+
+Returns `embeddings` (array of float arrays). For the retrieval pipeline around these vectors, that's
+`rag` / `embeddings-search`, not this skill.
+
+## OpenAI-compatible layer (`/v1`)
+
+| OpenAI path | Ollama support |
+| --- | --- |
+| `POST /v1/chat/completions` | yes — drop-in for chat |
+| `POST /v1/completions` | yes — legacy text completion |
+| `POST /v1/embeddings` | yes |
+| `GET /v1/models` | lists pulled models |
+
+Point any OpenAI SDK at the base URL with any non-empty key (it is ignored):
+
+```python
+from openai import OpenAI
+client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
+
+# streaming
+for chunk in client.chat.completions.create(
+        model="qwen3:8b",
+        messages=[{"role": "user", "content": "hi"}],
+        stream=True):
+    print(chunk.choices[0].delta.content or "", end="")
+```
+
+Native-only knobs (`keep_alive`, `format` as a JSON Schema, `think`) aren't expressible through the
+OpenAI layer — use `/api/chat` when you need them.

package/skills/ollama/references/hardware-sizing.md

@@ -0,0 +1,87 @@
+# Hardware sizing for local Ollama
+
+Total memory = **weights + KV cache + overhead**. People size for weights and forget the KV cache,
+then OOM at long context. Budget all three.
+
+## The quant ladder
+
+| Quant | Bytes/param | Size vs fp16 | Quality | Use it for |
+| --- | --- | --- | --- | --- |
+| fp16 | 2.0 | 100% | reference | when VRAM is abundant or you need max fidelity |
+| Q8_0 | ~1.0 | ~50% | near-lossless | quality-sensitive work that still fits |
+| Q6_K | ~0.8 | ~40% | very close to Q8 | a middle step when Q8 is just too big |
+| **Q4_K_M** | ~0.5 | ~25% | ~3–5% loss; the default | everyday local inference |
+| Q3_K_M | ~0.4 | ~20% | noticeable degradation | last resort to fit |
+| Q2_K | ~0.3 | ~15% | often unusable | rarely worth it — change model size instead |
+
+Rule: drop **model size** before dropping below Q4. A 14B at Q4_K_M beats a 70B at Q2_K on both quality
+and fit.
+
+## Weights formula
+
+```text
+weights_GB ≈ params(B) × bytes_per_param × 1.2
+# the ×1.2 absorbs runtime/format overhead; KV cache is separate (below).
+```
+
+| Model | Q4_K_M (~0.5 B/p) | Q8_0 (~1.0) | fp16 (2.0) |
+| --- | --- | --- | --- |
+| 7–8B | ~5–6 GB | ~9 GB | ~16 GB |
+| 13–14B | ~9–10 GB | ~16 GB | ~30 GB |
+| 32B | ~20 GB | ~36 GB | ~70 GB |
+| 70B | ~40–48 GB | ~75 GB | ~140 GB |
+
+## KV-cache math
+
+KV cache holds keys+values for every token in context, per layer. It grows ~linearly with context
+length and is allocated on top of the weights:
+
+```text
+kv_GB ≈ 2 (K+V) × layers × ctx_tokens × kv_dim × bytes_per_elem / 1e9
+```
+
+You don't need the exact number — you need the shape: **doubling `num_ctx` roughly doubles the cache**,
+and on big models at long context the cache can rival or exceed the weights. Illustrative, large model:
+
+| Context (`num_ctx`) | KV cache (order of magnitude) |
+| --- | --- |
+| 4K | small (≪ weights) |
+| 32K | several GB |
+| 128K | tens of GB on a 70B — often more than the headroom you assumed |
+
+Shrink it: set `OLLAMA_KV_CACHE_TYPE=q8_0` (≈ half) or `q4_0` (≈ quarter) — a small accuracy cost for a
+big memory win. Or just cap `num_ctx` to what the task needs.
+
+## Apple Silicon (unified memory)
+
+On a Mac, weights + KV cache draw from the **single unified memory pool** shared with the OS and apps.
+Budget against total RAM minus what the system needs (leave several GB). A 64 GB Mac can comfortably run
+a 32B Q4_K_M; a 70B Q4_K_M is feasible but leaves little headroom for long context.
+
+**MLX backend gate.** Ollama can run an MLX backend on Apple Silicon (shipped in Ollama 0.19, per
+[ollama.com/blog/mlx](https://ollama.com/blog/mlx), 2026-03-30), but it requires a Mac with **more than
+32 GB of unified memory** — below that gate it stays on the default llama.cpp/GGUF engine. So MLX is not
+a universal Apple-Silicon win: a 16 GB or 32 GB Mac will not use it, and your sizing math is the same
+GGUF math as everywhere else. The 64 GB row above is where MLX actually kicks in.
+
+## Env knobs to fit tight boxes
+
+| Variable | Effect |
+| --- | --- |
+| `OLLAMA_KV_CACHE_TYPE` | `q8_0` / `q4_0` quantize the KV cache → less VRAM at long context |
+| `OLLAMA_FLASH_ATTENTION` | enable flash attention → lower memory + faster on supported GPUs |
+| `OLLAMA_NUM_PARALLEL` | concurrent requests per model; more = more KV cache allocated |
+| `OLLAMA_MAX_LOADED_MODELS` | how many models stay resident at once |
+| `OLLAMA_KEEP_ALIVE` | default unload timer (overrides the 5m default) |
+| `OLLAMA_HOST` | bind address/port |
+| `OLLAMA_MODELS` | where models are stored on disk |
+
+## Decision: fit, or leave the box
+
+1. Compute `weights_GB` for the model+quant you want.
+2. Add a KV-cache budget for your target `num_ctx` (and `OLLAMA_NUM_PARALLEL`).
+3. Add ~1–2 GB OS/overhead.
+4. If the total exceeds your VRAM/unified memory: shrink `num_ctx`, quantize the KV cache, or step the
+   model down one size — **not below Q4**.
+5. If it still doesn't fit at the quality you need: it's a remote-GPU job → `runpod` (rent a GPU),
+   `modal` (serverless + autoscale), or a hosted endpoint (`replicate`, `together-fireworks`, `fal`).

package/skills/ollama/scripts/verify.sh

@@ -0,0 +1,116 @@
+#!/usr/bin/env bash
+#
+# verify.sh — static checks for a local Ollama setup.
+#
+# Usage:
+#   ./scripts/verify.sh [TARGET]      # TARGET defaults to the current directory
+#
+# What it does (all read-only, no daemon needed):
+#   1. Modelfile lint  — if a Modelfile exists under TARGET: FAIL if it has no FROM line;
+#                        WARN on unknown instructions; WARN if PARAMETER num_ctx is absurdly
+#                        high (> 32768) — a likely OOM on consumer GPUs.
+#   2. App-code scan   — note whether code points at the local localhost:11434 / /v1 endpoint;
+#                        WARN if it only references remote LLM hosts when this is meant to be local.
+#   3. Ollama presence — only if `ollama` is on PATH: best-effort `ollama list` to confirm a model
+#                        is pulled (WARN, never FAIL, if none / daemon down).
+#
+# Emits PASS / WARN / FAIL lines. Exits non-zero ONLY on a real FAIL. An empty or clean
+# target passes cleanly (exit 0). Works on stock macOS bash 3.2 — no mapfile, no assoc arrays.
+
+set -euo pipefail
+
+TARGET="${1:-.}"
+
+if [ -t 1 ]; then
+  RED=$'\033[0;31m'; GREEN=$'\033[0;32m'; YELLOW=$'\033[0;33m'; RESET=$'\033[0m'
+else
+  RED=''; GREEN=''; YELLOW=''; RESET=''
+fi
+pass() { printf '%sPASS%s %s\n' "$GREEN" "$RESET" "$*"; }
+warn() { printf '%sWARN%s %s\n' "$YELLOW" "$RESET" "$*"; }
+fail() { printf '%sFAIL%s %s\n' "$RED" "$RESET" "$*"; FAILED=$((FAILED + 1)); }
+
+FAILED=0
+
+if [ ! -e "$TARGET" ]; then
+  warn "target '$TARGET' does not exist — nothing to verify"
+  pass "verify.sh: ok (nothing to verify)"
+  exit 0
+fi
+
+# --- 1. Modelfile lint -------------------------------------------------------
+# Match Modelfile and Modelfile.* (case-insensitive), excluding VCS/deps dirs.
+MODELFILES=$(find "$TARGET" \
+  \( -name node_modules -o -name .git -o -name .venv -o -name venv \) -prune -o \
+  -type f \( -iname 'Modelfile' -o -iname 'Modelfile.*' \) -print 2>/dev/null || true)
+
+if [ -z "$MODELFILES" ]; then
+  warn "no Modelfile found under '$TARGET' — skipping Modelfile lint"
+else
+  # Known Modelfile instructions (uppercased first token of a line).
+  KNOWN=" FROM PARAMETER SYSTEM TEMPLATE LICENSE ADAPTER MESSAGE STOP "
+  # Iterate without a pipe so FAILs increment FAILED in THIS shell (bash 3.2: a piped
+  # while runs in a subshell and would lose the counter).
+  OLD_IFS=$IFS; IFS=$'\n'
+  for mf in $MODELFILES; do
+    IFS=$OLD_IFS
+    [ -z "$mf" ] && continue
+    if grep -qiE '^[[:space:]]*FROM[[:space:]]+' "$mf"; then
+      pass "Modelfile '$mf' has a FROM line"
+    else
+      fail "Modelfile '$mf' has no FROM line (required)"
+    fi
+
+    # Unknown instructions: first uppercased token on non-blank, non-comment lines.
+    while IFS= read -r line; do
+      case "$line" in ''|'#'*) continue ;; esac
+      first=$(printf '%s' "$line" | awk '{print toupper($1)}')
+      [ -z "$first" ] && continue
+      case "$KNOWN" in
+        *" $first "*) : ;;
+        *) warn "Modelfile '$mf': unknown instruction '$first'" ;;
+      esac
+    done < "$mf"
+
+    # Absurd num_ctx.
+    ctx=$(grep -iE '^[[:space:]]*PARAMETER[[:space:]]+num_ctx[[:space:]]+[0-9]+' "$mf" \
+      | awk '{print $3}' | tail -n1 || true)
+    if [ -n "${ctx:-}" ] && [ "$ctx" -gt 32768 ] 2>/dev/null; then
+      warn "Modelfile '$mf': PARAMETER num_ctx $ctx is very high — likely OOM on consumer GPUs"
+    fi
+    IFS=$'\n'
+  done
+  IFS=$OLD_IFS
+fi
+
+# --- 2. App-code scan --------------------------------------------------------
+SCAN_GLOB='--include=*.py --include=*.js --include=*.ts --include=*.go --include=*.rb --include=*.sh --include=*.env --include=*.yaml --include=*.yml --include=*.toml'
+LOCAL_HITS=$(grep -rIl $SCAN_GLOB -e 'localhost:11434' -e '127.0.0.1:11434' -e ':11434/v1' "$TARGET" 2>/dev/null || true)
+REMOTE_HITS=$(grep -rIl $SCAN_GLOB -e 'api.openai.com' -e 'api.anthropic.com' -e 'api.together.xyz' -e 'api.replicate.com' "$TARGET" 2>/dev/null || true)
+
+if [ -n "$LOCAL_HITS" ]; then
+  pass "found local Ollama endpoint (localhost:11434) in app code"
+elif [ -n "$REMOTE_HITS" ]; then
+  warn "app code references only remote LLM hosts; for a local Ollama target use http://localhost:11434/v1"
+else
+  warn "no LLM endpoint references found in app code under '$TARGET' — nothing to check"
+fi
+
+# --- 3. Ollama presence (best-effort) ---------------------------------------
+if command -v ollama >/dev/null 2>&1; then
+  if models=$(ollama list 2>/dev/null) && [ "$(printf '%s\n' "$models" | sed -n '2,$p' | grep -c .)" -gt 0 ]; then
+    pass "ollama is on PATH and at least one model is pulled"
+  else
+    warn "ollama is on PATH but no model is pulled (or daemon is down) — try: ollama pull qwen3:8b"
+  fi
+else
+  warn "ollama not on PATH — skipping runtime check (static checks still ran)"
+fi
+
+# --- summary -----------------------------------------------------------------
+if [ "$FAILED" -gt 0 ]; then
+  fail "verify.sh: $FAILED failure(s)"
+  exit 1
+fi
+pass "verify.sh: ok"
+exit 0

package/skills/open-weights/SKILL.md

@@ -0,0 +1,239 @@
+---
+name: open-weights
+description: "Use when choosing an open-weight LLM and clearing it for use — which family and size fit the task, the hardware and the budget, and above all whether the license permits shipping. Owns the license-class map (OSI-open versus custom-community versus non-commercial), the always-verify-the-model-card rule, size-to-VRAM budgeting and the quant formats. This layer decides WHICH model and whether the license allows it, then hands off. NOT downloading or hosting on the Hub (that is `huggingface`), NOT running it locally (that is `ollama`), NOT serving at throughput (that is `vllm`), NOT fine-tuning it (that is `finetuning`)."
+tags: [open-weights, open-source-llm, model-selection, model-licenses, llama-qwen-mistral-gemma]
+recommends: [huggingface, ollama, finetuning, vllm]
+origin: risco
+---
+
+# Open weights — pick the model, then clear the license
+
+This is the CHOICE layer. Given a task, a box, and a shipping constraint, it tells you *which*
+open model to reach for and *whether the license lets you ship it*. It does not download, run,
+serve, or fine-tune anything — it feeds the skills that do (see [Related skills](#related-skills)).
+
+## The one rule (read before you name a model)
+
+**Model names and their licenses change monthly. Never state a license from memory — open the
+model card and confirm it, every time.** This file was authored against cards live in mid-2026;
+by the time you read it, versions have shipped and terms have moved. So:
+
+- Treat every specific version/license line below as **[verify the card]**, not as settled fact.
+- State the license **class** (OSI-open vs custom-community vs restricted), then send the reader to
+  the actual `LICENSE` / terms page for the exact model + size they intend to ship.
+- The *decision framework* and the *class taxonomy* are the durable parts. The version list is the
+  perishable part. Weight your trust accordingly.
+
+Real, verified-mid-2026 examples of *why* this rule exists — all confirmed against the source, and
+all things that were different at my Jan-2026 cutoff:
+
+- **Gemma 4** moved to **Apache-2.0**, while Gemma 1–3 stay on the custom Gemma Terms of Use
+  (confirmed on [ai.google.dev/gemma/terms](https://ai.google.dev/gemma/terms), accessed
+  2026-07-18). Same brand, opposite license class depending on version.
+- **Codestral 2** was relicensed **Apache-2.0** (per Mistral, Apr 2026), while the original
+  Codestral stays **non-production (MNPL)**. Same name, opposite shippability depending on release.
+- **DeepSeek-R1** is **MIT**, but the original **DeepSeek-V3** weights ship under a **custom
+  DeepSeek License Agreement with OpenRAIL-style use restrictions** (confirmed on the
+  [V3 LICENSE-MODEL](https://huggingface.co/deepseek-ai/DeepSeek-V3/blob/main/LICENSE-MODEL)),
+  not MIT. Same family, different license per model.
+
+If three of the load-bearing facts moved in one release cycle, yours have too. Check the card.
+
+## 1. How to choose an open model
+
+Five gates, in this order. The license gate can veto everything above it, so never fall in love
+with a model before you clear it.
+
+1. **Task fit** — match the model's design to the job:
+   - General chat / instruct → Llama, Qwen, Mistral, Gemma instruct variants.
+   - Coding → Qwen-Coder, Codestral (license!), DeepSeek-Coder, gpt-oss.
+   - Reasoning / math → DeepSeek-R1-style reasoning models, gpt-oss, Qwen reasoning variants.
+   - Multilingual → Qwen and Gemma tend to lead; check the card's language list.
+   - Multimodal (vision) → Llama vision, Qwen-VL, Gemma vision, PaliGemma, Phi multimodal.
+   - Embeddings / retrieval → a dedicated embedding model, **not** a chat LLM → `embeddings-search`.
+   - Small / edge → 0.5B–4B (Qwen small, Gemma small, Phi-mini, gpt-oss-20b class).
+2. **Size vs your hardware class** — can it even fit? See [§4](#4-size--vram). A 70B at fp16 is
+   ~140 GB; if you have one 24 GB GPU, that model is off the table unless you quantize *and* accept
+   the quality/throughput cost. Pick a size your box can hold before you compare quality.
+3. **License / shippability** — **the gate that matters most** (see [§3](#3-licenses)). Decide up
+   front: is this a hobby/internal use, or a commercial product you distribute? "Open weights" does
+   **not** mean "open source" and does **not** guarantee commercial rights. Verify the card.
+4. **Hardware target** — consumer GPU (8–24 GB), data-center GPU (A100/H100 80 GB), Apple unified
+   memory, or CPU/edge. This narrows both size and quant format (GGUF for CPU/Mac/llama.cpp; AWQ/GPTQ
+   for GPU serving).
+5. **Community support** — a proxy for "will this actually work Monday morning": are there
+   ready-made GGUF/AWQ quants on the Hub, is it supported by your runtime (llama.cpp/Ollama/vLLM),
+   are there fine-tunes and recent downloads? A model with no quants and no runtime support is a
+   research artifact, not a shippable choice.
+
+Rule of thumb: **smallest model that passes your eval wins.** Don't reach for 70B when a well-chosen
+8B clears the bar — it's cheaper to run, faster, and fits more hardware. Prove it with `agent-eval`.
+
+## 2. Model families
+
+Families and *typical* size ladders below. Specific version claims are marked — **[verify]** at
+author time; the family and its rough sizing are the durable part. Full per-family notes,
+strengths, and license pointers in [references/model-families.md](references/model-families.md).
+
+| Family | Maker | Typical open sizes | Notes / current line [verify] |
+|---|---|---|---|
+| **Llama** | Meta | ~1B–~400B (dense + MoE) | Llama 4 (Scout/Maverick, MoE) current line; custom **Community License**, gated |
+| **Qwen** | Alibaba | 0.5B–235B+ (dense + MoE), strong Coder/VL | mostly Apache-2.0 but **per-size variance** — check each card |
+| **Mistral / Mixtral** | Mistral AI | 7B dense, 8x7B/8x22B MoE, "Small/Large" | open ones Apache-2.0; **Codestral original = MNPL non-production** |
+| **Gemma** | Google | ~1B–~27B, plus PaliGemma/CodeGemma/ShieldGemma | Gemma 1–3 = **custom Gemma Terms**; Gemma 4 = Apache-2.0 [verify] |
+| **DeepSeek** | DeepSeek | V3-class MoE (~600B+), R1 reasoning, distills | **per-model license split** — R1 MIT, original V3 custom; verify |
+| **Phi** | Microsoft | ~3B–15B ("mini", reasoning, multimodal) | **MIT** across the Phi-4 family [verify] |
+| **gpt-oss** | OpenAI | 20B and 120B (MoE, open *weights*) | **Apache-2.0** [verify]; OpenAI's first open-weight LLMs since GPT-2 |
+| Others | various | — | SmolLM, OLMo (fully-open incl. data), Falcon, Yi, Command — check card |
+
+## 3. Licenses
+
+**This is the load-bearing section. Get it wrong and you ship something you have no right to
+ship.** Every model falls into one of three classes. Identify the class, then open the card.
+
+### The three classes
+
+| Class | Examples (class, not a promise) | What it means for shipping |
+|---|---|---|
+| **OSI-open** (Apache-2.0, MIT) | Qwen (most), Mistral open, Phi, gpt-oss, DeepSeek-R1, Gemma **4** | Commercial use, modify, redistribute — permissive. Still read the card for attribution/notice. |
+| **Custom / community** | **Llama** (Meta Community), **Gemma 1–3** (Gemma Terms) | Broad free use **but with conditions**: acceptable-use policy, attribution, sometimes a scale cap. NOT OSI-open. |
+| **Non-commercial / restricted** | original **Codestral** (MNPL), any "research-only", some RAIL, some "≥$X revenue → buy a license" | You **cannot** ship it in a commercial product without a separate license. Fatal if missed. |
+
+"Open weights" describes availability of the weights file — it says nothing about your legal rights.
+A model can be a free download and still be non-commercial. **The download button is not a license.**
+
+### Per-family license notes (state the class, then verify the exact card)
+
+- **Llama = Meta Llama Community License — NOT OSI-open.** Carries an Acceptable Use Policy, a
+  **"Built with Llama"** attribution requirement on derivatives/products, naming rules, and a
+  **>700M-monthly-active-users clause** (above that you must request a separate license from Meta,
+  granted at Meta's discretion). Gated on the Hub (accept terms first). Verify the *current* Llama
+  version's terms — Meta has revised them across releases.
+- **Gemma = custom Gemma Terms of Use + Prohibited Use Policy** for Gemma 1–3 (not Apache), with a
+  duty to pass the restrictions **downstream** to every user and to ship the terms/notice file.
+  Gemma **4** reportedly moved to **Apache-2.0** — a textbook reason to check the version's card.
+- **Qwen = mostly Apache-2.0, but with per-size variance.** Historically some sizes (often the very
+  largest or a special tier) carried a separate Qwen license instead of Apache. **Check each size's
+  card** — do not assume "Qwen = Apache" for the whole family.
+- **Mistral / Mixtral open models = Apache-2.0**, but the **original Codestral is MNPL
+  (non-production)** — free for research/eval, **not** for commercial deployment. Codestral **2**
+  was relicensed Apache-2.0; verify which Codestral release you actually have. Note some newer
+  Mistral models use a "Modified MIT" with a revenue-threshold commercial clause — verify.
+- **DeepSeek = per-model split.** Code repos are MIT; **DeepSeek-R1 weights are MIT**; but the
+  original **DeepSeek-V3 weights** ship under a **custom DeepSeek License Agreement with OpenRAIL-
+  style use-based restrictions**. "DeepSeek = MIT" is too simple — verify the specific model.
+- **Phi = MIT** across the Phi-4 family [verify] — genuinely permissive, commercial-friendly.
+- **gpt-oss (OpenAI open-weight models, 20B/120B) = Apache-2.0** [verify] — permissive.
+
+### Commercial-use checklist (run before you ship)
+
+- [ ] Opened the **exact model + size** card and read its `LICENSE` / terms — not a blog, not this file.
+- [ ] Confirmed the **class**: OSI-open, custom-community, or restricted/non-commercial.
+- [ ] If custom/community: identified the **conditions** (attribution string, acceptable-use policy,
+      MAU/revenue caps, downstream pass-through duty) and can meet them.
+- [ ] If restricted: confirmed you are **not** deploying commercially, or obtained a separate license.
+- [ ] Checked whether the model is **gated** (accept terms on the Hub before download).
+- [ ] Checked the **base model** license — a fine-tune inherits the base's obligations (a Llama
+      fine-tune still owes "Built with Llama"; a distill can inherit the teacher's terms).
+
+Deeper class breakdown, gated-model mechanics, and the fine-tune-inheritance trap:
+[references/licenses.md](references/licenses.md).
+
+## 4. Size ↔ VRAM
+
+A **selection-time** budget: will this size class even fit your hardware? (Operational per-request
+KV-cache math lives in `ollama`/`vllm` — this is the "before I pick it" estimate.)
+
+```text
+weights_GB ≈ params(B) × bytes_per_param      # then add KV cache + runtime overhead on top
+  fp16/bf16 → 2.0   (≈ 2 GB per 1B params)
+  8-bit     → ~1.0  (≈ 1 GB per 1B params)
+  4-bit     → ~0.6  (≈ 0.6 GB per 1B + cache + overhead; conservative ~0.5–0.6/param)
+```
+
+What that means per hardware class (weights only — leave headroom for context/KV cache):
+
+| Hardware | fp16 ceiling | Practical pick (4-bit) |
+|---|---|---|
+| 8 GB consumer GPU | ~3B | 7–8B at 4-bit (~5 GB) |
+| 16 GB | ~7B | 13–14B at 4-bit |
+| 24 GB (e.g. 3090/4090) | ~10B | ~32B at 4-bit (~20 GB) |
+| 80 GB (A100/H100) | ~34B | 70B at 4-bit (~40 GB), or a big-MoE like gpt-oss-120b |
+| Apple unified (e.g. 64 GB) | shares with the OS | budget against **total** unified memory |
+| multi-GPU / 2×80 GB+ | 70B fp16 (~140 GB) via tensor-parallel | full-precision large models |
+
+**A 70B at fp16 is ~140 GB — it does not fit one GPU.** You either quantize (4-bit ≈ ~40 GB, fits one
+80 GB card) or split across GPUs with **tensor parallelism** (that's a `vllm` job). Longer context
+grows the KV cache *on top of* weights, so keep a margin. Full derivation + KV math:
+[references/sizing-and-quant.md](references/sizing-and-quant.md).
+
+## 5. Quant / formats
+
+Quantization shrinks weights (fewer bits/param) to fit smaller hardware, trading a little quality.
+Which **format** you pick is driven by your runtime:
+
+| Format | Where it runs | Use it for |
+|---|---|---|
+| **GGUF** | llama.cpp, **Ollama**, LM Studio | CPU, Apple Silicon, single-box local; the everyday local format |
+| **AWQ** | vLLM, TGI, transformers | GPU serving — activation-aware 4-bit, strong quality/latency |
+| **GPTQ** | vLLM, TGI, transformers | GPU serving — older, widely available 4-bit |
+| bitsandbytes (NF4) | transformers | quick load-time 4/8-bit for experiments/fine-tuning |
+| EXL2 | ExLlamaV2 | flexible bit-rates on consumer GPUs |
+
+**Decoding a GGUF quant tag like `Q4_K_M`:** `Q4` = ~4-bit weights; `_K` = a k-quant (mixed
+precision — keeps the more sensitive tensors at higher bit-depth); `_M` = the medium size/quality
+tier (`_S` smaller/lower, `_L` larger/higher). **`Q4_K_M` is the standard default** — roughly half
+the memory of fp16 for a few percent quality loss. `Q8_0` is near-lossless (~1 byte/param); below
+`Q4`, quality drops off fast. Don't go sub-Q4 to force a too-big model onto a too-small box — pick a
+smaller model instead. More in [references/sizing-and-quant.md](references/sizing-and-quant.md).
+
+## 6. Where to get it + how to run it (routing)
+
+You've chosen a model and cleared its license. Now hand off — this skill stops here.
+
+- **Get the weights / host on HF** → `huggingface` (Hub download/upload, Inference Providers,
+  endpoints). Also where you convert to GGUF.
+- **Run it locally on one box** (laptop/desktop, GGUF via Ollama) → `ollama`.
+- **Serve it at throughput / with tensor-parallel** (production, batching, big models) → `vllm`.
+- **Fine-tune it** (LoRA/QLoRA, SFT, preference tuning) → `finetuning`.
+- **Prove the small model is enough** before committing → `agent-eval`.
+
+The Hub filters (task + license + size + recent downloads) and the Ollama library are where you
+actually find candidates — that mechanics lives in `huggingface` / `ollama`, not here.
+
+## Guardrails / gotchas
+
+- **Never assert a license from memory.** State the class, open the card. Licenses change per
+  version, per size, and per release month.
+- **"Open weights" ≠ "open source" ≠ "commercial-use OK".** Three different things.
+- **A fine-tune inherits its base license.** Fine-tuning Llama does not launder away "Built with
+  Llama" or the acceptable-use policy; a distill can inherit the teacher model's terms.
+- **Gated ≠ restricted.** Gemma/Llama are gated (accept terms to download) yet usable commercially
+  under their conditions; a "research-only" card is restricted regardless of gating.
+- **Don't sub-Q4 a too-big model onto a too-small GPU.** Quality collapses and it still may OOM at
+  real context. Choose a size that fits.
+- **Newest ≠ best for you.** The smallest model that passes your eval wins on cost, speed, and fit.
+- **MoE total vs active params.** A "120B" MoE may activate only ~5B/token (fast) but you still need
+  VRAM for *all* experts — size against total params, not active.
+
+## Related skills
+
+- **`huggingface`** — get/host: Hub download/upload, Inference Providers, endpoints, GGUF
+  conversion. This skill tells you *which* repo to pull; `huggingface` pulls it.
+- **`ollama`** — run one model locally (GGUF, single box, VRAM sizing at request time). This skill
+  says which model + quant *class* fits; `ollama` runs it and does the KV-cache math.
+- **`vllm`** — serve at throughput, batching, tensor-parallel for models too big for one GPU.
+- **`finetuning`** — adapt a chosen base with LoRA/QLoRA/SFT. Choose+clear the base here first.
+- Boundary: this skill is **selection + licensing knowledge only**. It never re-teaches calling,
+  running, serving, or training — it feeds those four.
+
+## Checklist
+
+- [ ] Named the **task** and matched a family designed for it (chat/code/reasoning/vision/embeddings/edge).
+- [ ] Picked the **smallest size** likely to pass the eval, and confirmed it **fits the hardware class**.
+- [ ] Identified the **license class** (OSI-open / custom-community / restricted).
+- [ ] Opened the **exact model + size card** and confirmed the license — not from memory, not from a blog.
+- [ ] For commercial use: met the conditions (attribution, acceptable-use, MAU/revenue caps) or confirmed permissive.
+- [ ] Checked the **base model** license if it's a fine-tune/distill.
+- [ ] Picked a **quant format** matching the runtime (GGUF local / AWQ-GPTQ serving) and a sane tier (`Q4_K_M` default).
+- [ ] Handed off: `huggingface` (get), `ollama` (run local), `vllm` (serve), `finetuning` (adapt).

package/skills/open-weights/evals/README.md

@@ -0,0 +1,14 @@
+# Evals for the `open-weights` skill
+
+Routing + capability checks read from `cases.yaml` by the skill eval harness — no network, no
+model downloads, no license fetches. The `should_trigger` prompts must select this skill: they
+cover general model selection, the Llama commercial-license question, Qwen per-size license
+variance, VRAM-fit sizing, the non-commercial (Codestral/MNPL) gate, and a Catalan phrasing.
+The `should_not_trigger` prompts must route to the named real sibling instead — `huggingface`
+(download/host), `ollama` (run local + KV cache), `vllm` (serve/tensor-parallel), `finetuning`
+(adapt), and `rag` (retrieval) — proving the tight "choose only, then hand off" boundary. The
+single `capability` case is a rubric: a good answer to the 24GB commercial-assistant scenario must
+verify the license on the exact card (never from memory), classify the license class, flag Llama as
+custom/non-OSI with its conditions, size the model to fit 24GB, note fine-tune license inheritance,
+and hand off to the four sibling skills. Run with whatever harness loads `evals/cases.yaml`;
+nothing to install or connect.

package/skills/open-weights/evals/cases.yaml

@@ -0,0 +1,42 @@
+skill: open-weights
+
+should_trigger:
+  - prompt: "Which open-weight model should I pick for a customer-support chatbot I plan to ship commercially?"
+    why: "Core selection + license-clearance question across families — the CHOICE layer this skill owns."
+  - prompt: "Can I use Llama commercially in my SaaS product, or does its license restrict that?"
+    why: "License-class question on Llama (Meta Community License, not OSI-open) — the load-bearing section."
+  - prompt: "Is Qwen Apache-2.0 across every size, or do I have to check each model card separately?"
+    why: "Per-size license variance — squarely the verify-the-card rule this skill teaches."
+  - prompt: "What is the largest open LLM I can realistically run on a single 24GB GPU?"
+    why: "Size-to-VRAM budgeting to narrow the model choice before picking — this skill's §4."
+  - prompt: "Is the original Codestral safe to use in a commercial product, or is it non-commercial only?"
+    why: "Restricted/non-commercial (MNPL) gate — the fatal-if-missed license class."
+  - prompt: "Quin model obert puc fer servir legalment en un producte comercial i que càpiga en una GPU de 16GB?"
+    why: "Catalan (non-English) phrasing; open-model selection + license + VRAM fit — this skill's whole remit."
+
+should_not_trigger:
+  - prompt: "Download the Llama weights from the Hub with the hf CLI and push a model card."
+    route_to: huggingface
+    why: "Getting/hosting weights on HF — huggingface's job; this skill only says WHICH repo, not how to pull it."
+  - prompt: "My 70B OOMs in Ollama at 128K context — how do I shrink the KV cache to make it fit?"
+    route_to: ollama
+    why: "Operational, single-box run + per-request KV-cache math — ollama, not the selection layer."
+  - prompt: "Serve a 70B across two 80GB GPUs with tensor parallelism and high request throughput."
+    route_to: vllm
+    why: "Production serving engine + tensor-parallel flags — vllm owns serving, not model choice."
+  - prompt: "Fine-tune Qwen on my instruction dataset with LoRA and export the adapter."
+    route_to: finetuning
+    why: "Adapting a chosen base with LoRA/SFT — finetuning; this skill only chooses+clears the base first."
+  - prompt: "Build a retrieval pipeline over our docs with chunking, an index, and reranking."
+    route_to: rag
+    why: "Designing a retrieval system, not selecting or licensing an open model."
+
+capability:
+  - scenario: "Choose an open-weight model for a commercial customer-support assistant that must run on a single 24GB GPU, and confirm its license lets you ship."
+    must_include:
+      - "verify the license on the EXACT model + size card, never assert it from memory — licenses change per version, size, and month"
+      - "classify the license: OSI-open (Apache-2.0/MIT) vs custom-community (Llama Meta Community, Gemma 1-3) vs non-commercial/restricted"
+      - "Llama is NOT OSI-open: Meta Community License with an acceptable-use policy, a 'Built with Llama' attribution requirement, and a >700M-monthly-active-users clause"
+      - "pick a size that fits 24GB: ~32B at 4-bit (~20GB) or an 8-14B, leaving headroom for KV cache — a 70B at fp16 (~140GB) does not fit one GPU"
+      - "a fine-tune or distill inherits its base model's license obligations"
+      - "hand off after choosing: huggingface to download/host, ollama to run locally, vllm to serve, finetuning to adapt"