@ericrisco/rsc 0.1.38 → 0.1.40

package/skills/open-weights/references/model-families.md

@@ -0,0 +1,83 @@
+# Open-weight model families — depth notes
+
+**Every version and license line here is [verify the card] — authored against cards live mid-2026
+(accessed 2026-07-18). Families and rough size ladders are durable; specific versions are not.**
+Do not quote a license from this file into a product decision — open the model's `LICENSE`.
+
+## Llama (Meta)
+
+- **Sizes:** historically ~1B–70B dense; current line (Llama 4: Scout, Maverick) uses a
+  **mixture-of-experts** design with large total-param counts but a smaller active count per token.
+- **Strengths:** huge ecosystem — the most fine-tunes, quants, tooling, and community docs of any
+  family. If "will it just work in my runtime" matters, Llama is the safe bet.
+- **License:** **Meta Llama Community License — custom, NOT OSI-open.** Acceptable-Use Policy,
+  "Built with Llama" attribution, naming rules, and a **>700M-MAU** clause. Gated on the Hub. See
+  [licenses.md](licenses.md).
+
+## Qwen (Alibaba)
+
+- **Sizes:** a very wide ladder — sub-1B up to 200B+ (dense **and** MoE). Strong specialized lines:
+  **Qwen-Coder** (coding), **Qwen-VL** (vision), reasoning variants. Recent lines (Qwen3 / Qwen3.5 /
+  Qwen3.6) [verify].
+- **Strengths:** excellent multilingual coverage, strong coding, a size for nearly every hardware
+  budget, generally permissive licensing.
+- **License:** **mostly Apache-2.0 but per-size variance** — some sizes historically shipped under a
+  separate Qwen license. Check **each size's** card; never generalize "Qwen = Apache" to the family.
+
+## Mistral / Mixtral (Mistral AI)
+
+- **Sizes:** 7B dense (Mistral 7B), MoE (Mixtral 8x7B / 8x22B), and named tiers (Small/Large/Nemo).
+- **Strengths:** efficient, strong for their size; Mixtral popularized accessible MoE.
+- **License:** open models **Apache-2.0**; but **Codestral (original, code model) = MNPL
+  non-production** — free to eval, not to ship commercially. **Codestral 2** was relicensed
+  Apache-2.0 (Apr 2026) [verify]. Some newer models use a "Modified MIT" with a revenue-threshold
+  commercial clause. Per-model check required — see [licenses.md](licenses.md).
+
+## Gemma (Google)
+
+- **Sizes:** ~1B–~27B, plus specialized siblings: **PaliGemma** (vision-language), **CodeGemma**,
+  **ShieldGemma** (safety), **RecurrentGemma**, EmbeddingGemma, DataGemma.
+- **Strengths:** strong small-model quality, good multilingual, tight integration with Google tooling.
+- **License:** **Gemma 1–3 = custom Gemma Terms of Use + Prohibited Use Policy** (NOT Apache), with
+  a downstream pass-through duty. **Gemma 4** reportedly moved to **Apache-2.0** (confirmed on
+  ai.google.dev/gemma/terms, 2026-07-18) — verify the version you use. See [licenses.md](licenses.md).
+
+## DeepSeek
+
+- **Sizes:** very large MoE base/chat (V3-class, hundreds of B total), **R1** reasoning models, plus
+  **distills** into smaller Llama/Qwen backbones (handy for consumer hardware).
+- **Strengths:** frontier-class reasoning and coding at open-weight cost; distills bring much of the
+  quality to small sizes.
+- **License:** **per-model split.** Code repos MIT; **DeepSeek-R1 weights MIT**; original
+  **DeepSeek-V3 weights = custom DeepSeek License Agreement with OpenRAIL-style use restrictions**.
+  Newer point releases have moved toward MIT [verify]. Never assume "DeepSeek = MIT" — check the model.
+
+## Phi (Microsoft)
+
+- **Sizes:** small — ~3B to ~15B ("mini", reasoning, multimodal variants in the Phi-4 family).
+- **Strengths:** punches above its weight on reasoning for the size (trained on curated/synthetic
+  data); great for edge and cost-sensitive deployments.
+- **License:** **MIT** across the Phi-4 family [verify] — genuinely permissive, commercial-friendly.
+
+## gpt-oss (OpenAI)
+
+- **Sizes:** **gpt-oss-20b** and **gpt-oss-120b** — MoE, open *weights*. 120B runs on a single 80 GB
+  GPU; 20B targets ~16 GB.
+- **Strengths:** OpenAI's first open-weight LLMs since GPT-2; strong reasoning/agentic/tool-use.
+- **License:** **Apache-2.0** [verify] — permissive. Confirmed on OpenAI's own model-card page and
+  the Hugging Face repo (2026-07-18).
+
+## Others worth knowing (check the card)
+
+- **OLMo (AI2)** — *fully* open: weights **and** training data/code released. The pick when
+  reproducibility/provenance matters, not just weight availability.
+- **SmolLM (HF)** — tiny, permissive, edge-friendly.
+- **Falcon (TII)**, **Yi (01.AI)**, **Command / Command-R (Cohere)** — Command family often carries
+  more restrictive terms (research vs commercial split); verify carefully before shipping.
+
+## The "will it work Monday" filter
+
+Beyond quality, a family/size is only a real option if the **ecosystem** supports it: ready-made
+GGUF/AWQ/GPTQ quants on the Hub, support in your runtime (llama.cpp/Ollama/vLLM/TGI), documented
+chat template, and recent downloads. A brand-new model with no quants and no runtime support is a
+research artifact — route the actual pull to `huggingface` and the run to `ollama`/`vllm`.

package/skills/open-weights/references/sizing-and-quant.md

@@ -0,0 +1,89 @@
+# Size ↔ VRAM and quant formats — selection-time depth
+
+This is the **"will it fit before I pick it"** estimate. The **operational** per-request KV-cache
+sizing and runtime flags live in `ollama` (single box) and `vllm` (serving) — do not duplicate them
+here. These numbers are rules of thumb for *choosing a size class*, not a per-model spec sheet;
+confirm the real footprint with `ollama show` or the model card.
+
+## The weights formula
+
+```text
+weights_GB ≈ params(B) × bytes_per_param
+  fp16 / bf16 → 2.0     (2 GB per 1B params)
+  int8 / 8-bit → ~1.0   (1 GB per 1B params)
+  4-bit        → ~0.5–0.6 (≈ 0.6 GB per 1B params, conservative)
+
+total_VRAM ≈ weights_GB + KV_cache + runtime_overhead
+```
+
+The 4-bit rate is a round number: measured k-quant rates land near ~4.9 bits/weight for Q4_K_M
+(~0.6 byte/param). Round **up** when budgeting so you don't OOM. **KV cache and overhead are extra**
+and grow with context length — always leave a margin above the weights figure.
+
+### Worked examples (weights only, fp16 unless noted)
+
+| Model | Params | fp16 | 8-bit | 4-bit |
+|---|---|---|---|---|
+| Small | 3B | ~6 GB | ~3 GB | ~2 GB |
+| Mid | 8B | ~16 GB | ~8 GB | ~5 GB |
+| Large-consumer | 32B | ~64 GB | ~32 GB | ~20 GB |
+| Big | 70B | **~140 GB** | ~70 GB | ~40 GB |
+
+**The 70B lesson:** fp16 ~140 GB does not fit any single GPU on the market for most teams. Options:
+(a) **quantize** to 4-bit (~40 GB → fits one 80 GB card), accepting some quality/throughput cost; or
+(b) **tensor-parallel** across multiple GPUs (e.g. 2×80 GB) to keep full precision — a `vllm` job.
+
+## What fits which hardware class
+
+| Hardware | fp16 ceiling (weights) | Practical pick |
+|---|---|---|
+| 8 GB consumer GPU | ~3B | 7–8B @ 4-bit (~5 GB) + small context |
+| 12 GB | ~5B | up to ~14B @ 4-bit |
+| 16 GB | ~7B | 13–14B @ 4-bit; 7–8B @ 8-bit |
+| 24 GB (3090/4090) | ~10B | ~32B @ 4-bit (~20 GB) |
+| 48 GB (or 2×24) | ~22B | 70B @ 4-bit is tight; better on 80 GB |
+| 80 GB (A100/H100) | ~34B | 70B @ 4-bit (~40 GB), or a big MoE (gpt-oss-120b) |
+| 2×80 GB+ | 70B fp16 (~140 GB) via TP | full-precision large models |
+| Apple unified (e.g. 64 GB) | shares with the whole OS | budget vs **total** unified memory, not a GPU number |
+| CPU / edge | — | small GGUF (≤3B) at 4-bit; slow but runs |
+
+## MoE: total vs active params
+
+Mixture-of-experts models (Mixtral, gpt-oss, Llama 4, big Qwen/DeepSeek) route each token through a
+**subset** of experts. So a "120B" MoE might activate only ~5B params per token (fast inference) —
+**but you still need VRAM to hold all experts.** Budget against **total** params for memory and
+against **active** params for the speed intuition. Don't size a 120B MoE as if it were a 5B dense.
+
+## Quant formats — pick by runtime
+
+| Format | Runtime | Notes |
+|---|---|---|
+| **GGUF** | llama.cpp, Ollama, LM Studio | The local default. CPU + Apple Silicon + single GPU. Self-contained files with quant in the name. |
+| **AWQ** | vLLM, TGI, transformers | Activation-aware weight quant; strong 4-bit quality for GPU serving. |
+| **GPTQ** | vLLM, TGI, transformers | Older post-training quant; very widely available 4-bit. |
+| **bitsandbytes (NF4/int8)** | transformers | Load-time quantization — quick for experiments and QLoRA fine-tuning. |
+| **EXL2** | ExLlamaV2 | Variable bit-rate; good on consumer GPUs, flexible size/quality. |
+
+Rule: **local/Mac/CPU → GGUF**; **GPU serving → AWQ or GPTQ**; **fine-tuning experiments → bnb NF4**.
+Getting or converting to a format is a `huggingface` (convert) / `ollama` / `vllm` job.
+
+## Decoding a GGUF quant tag
+
+`Q4_K_M`:
+- **`Q4`** — nominal ~4-bit weights.
+- **`_K`** — a **k-quant**: mixed precision that keeps the more error-sensitive tensors (attention,
+  some feed-forward) at higher bit-depth than the bulk. Better quality than the old flat `Q4_0`.
+- **`_M`** — size/quality **tier**: `_S` (small, lower), `_M` (medium — the balanced default),
+  `_L` (large, higher). There are also non-K legacy tags (`Q4_0`, `Q4_1`, `Q5_0`…).
+
+Quality ladder (rough): `fp16` ≥ `Q8_0` (near-lossless, ~1 byte/param) > `Q6_K` > `Q5_K_M` >
+**`Q4_K_M`** (standard default, ~half of fp16 for a few % loss) > `Q3_K` > `Q2_K` (noticeable
+degradation). **Do not drop below Q4 to cram a too-big model onto a too-small GPU** — the quality
+falls off fast and you'll often still OOM at real context. Pick a smaller model instead.
+
+## Selection heuristic
+
+1. Estimate weights_GB at 4-bit for the sizes you're considering.
+2. Keep it **comfortably under** your VRAM (leave ~20–30% for KV cache + overhead + context).
+3. If nothing that fits passes your eval, the honest options are: a better-tuned smaller model, a
+   bigger box (route to `runpod`/`modal`), or `vllm` tensor-parallel — **not** sub-Q4 desperation.

package/skills/training-data/SKILL.md

@@ -0,0 +1,226 @@
+---
+name: training-data
+description: "Use when assembling or curating the corpus a fine-tune trains on: turning raw examples into JSONL in the format a trainer expects (Alpaca instruction, conversational messages, DPO preference pairs, KTO binary-label), matching your data to the model's chat template, generating synthetic or distilled examples, deduplicating and decontaminating against eval sets, or quality-filtering. Triggers: 'what format should my fine-tuning dataset be in', 'build a DPO preference dataset', 'convert my Q&A into the messages/chat format', 'my chat template does not match my training data', 'generate synthetic instruction data with distilabel', 'dedup and decontaminate my training set', 'quants exemples necessito i com filtro la qualitat del dataset abans d'entrenar'. NOT cleaning tabular rows/nulls/dtypes (that is data-cleaning), NOT building a retrieval corpus of chunks+embeddings (that is embeddings-search), NOT running the trainer or picking LoRA hyperparameters (that is finetuning/unsloth)."
+tags: [training-data, fine-tuning-dataset, jsonl, chat-template, preference-data, dpo, kto, synthetic-data, decontamination, dataset-curation]
+recommends: [finetuning, unsloth, huggingface, data-cleaning]
+origin: risco
+---
+
+# training-data — the corpus a fine-tune actually eats
+
+You own the **training corpus**: the JSONL of chat turns, instruction triples, or preference
+pairs that a trainer reads. The deliverable is a validated, deduplicated, decontaminated,
+license-clean file in the **exact shape the trainer expects**, rendered through the **target
+model's chat template**. You stop the moment that file loads cleanly and round-trips through
+`apply_chat_template`. You do not choose LoRA rank or launch the run — that is
+[`finetuning`](../finetuning/SKILL.md) / [`unsloth`](../unsloth/SKILL.md).
+
+**Loud boundary.** This is *LLM training corpora* — messages, instruction triples, preference
+pairs. It is **not**:
+
+- Tabular row cleaning — nulls, dtypes, dedupe of CSV rows, category normalization → [`data-cleaning`](../data-cleaning/SKILL.md).
+- A *retrieval* corpus — chunking documents and embedding them for search → [`embeddings-search`](../embeddings-search/SKILL.md).
+- Actually training or serving — hyperparameters, the run, export → [`finetuning`](../finetuning/SKILL.md), [`unsloth`](../unsloth/SKILL.md), [`huggingface`](../huggingface/SKILL.md).
+
+**Version reality (verified July 2026 — re-verify, these move monthly).** TRL is on the **v1.x**
+line (its dataset-formats doc was tagged `v1.8.0` at author time); `transformers` is in the
+**4.57+** era (mixed text+vision data needs ≥4.57); `datasets` is **4.x** (the `Json()` feature
+type needs ≥4.7). Pin whatever you install — do not trust these numbers as current.
+
+## 1. The format the trainer expects (pick by trainer, not by taste)
+
+The trainer dictates the columns. Get this wrong and TRL either errors or, worse, trains on a
+mangled string. Two axes: **format** (`standard` = plain strings vs `conversational` =
+`messages` lists) and **type** (the task). One JSON object per line = JSONL.
+
+| Trainer | Dataset type | Required keys |
+| --- | --- | --- |
+| `SFTTrainer` | language-modeling **or** prompt-completion | `messages` / `text`, or `prompt`+`completion` |
+| `DPOTrainer`, `ORPOTrainer`, `CPOTrainer` | preference (explicit prompt recommended) | `prompt`, `chosen`, `rejected` |
+| `KTOTrainer`, `BCOTrainer` | unpaired preference (binary label) | `prompt`, `completion`, `label` |
+| `RewardTrainer` | preference (implicit prompt) | `chosen`, `rejected` |
+| `GRPOTrainer`, `RLOOTrainer`, `PPOTrainer` | prompt-only | `prompt` |
+
+Tiny JSONL of each (conversational values are **lists of `{role, content}`**; `label` is a JSON
+boolean):
+
+```jsonl
+# Alpaca instruction (standard) — classic; NOT a native TRL type, see below
+{"instruction": "Classify the sentiment.", "input": "The battery dies in an hour.", "output": "negative"}
+
+# Conversational messages (SFT) — the default for chat fine-tunes
+{"messages": [{"role": "system", "content": "You are a terse support agent."}, {"role": "user", "content": "My order never arrived."}, {"role": "assistant", "content": "Sorry about that — what is your order number?"}]}
+
+# Preference pair (DPO) — chosen beats rejected for the same prompt
+{"prompt": [{"role": "user", "content": "Define a hash map in one sentence."}], "chosen": [{"role": "assistant", "content": "A hash map stores key-value pairs and finds a value by hashing its key to a bucket, giving average O(1) lookup."}], "rejected": [{"role": "assistant", "content": "It's a fast dictionary thing."}]}
+
+# KTO / unpaired preference — one completion + a good/bad boolean label
+{"prompt": [{"role": "user", "content": "Define a hash map in one sentence."}], "completion": [{"role": "assistant", "content": "It's a fast dictionary thing."}], "label": false}
+```
+
+**Alpaca is not a native TRL type.** `{instruction, input, output}` is the Stanford-Alpaca
+convention, still common in Unsloth notebooks, but TRL trains on `text`/`messages`/`prompt`+
+`completion`. You must either (a) map it into `messages` (instruction+input → user, output →
+assistant), or (b) render it into a single `text` string via a prompt template — **and append
+the EOS token yourself**, or the model never learns to stop (the #1 Unsloth-Alpaca bug). Prefer
+(a) `messages` for chat models. Full field matrix, tool-calling (`tools` column) and vision
+(`images`) extras, and every type→type conversion live in `references/formats.md`.
+
+## 2. Chat templates — the silent run-wrecker
+
+A chat template is a **Jinja** string stored in the tokenizer (in `tokenizer_config.json` under
+`chat_template`, or a standalone `chat_template.jinja` in newer tokenizers). It maps a `messages`
+list to the exact token string the model was trained on — special tokens (`<|im_start|>`,
+`[INST]`, `<|start_header_id|>`, …) and all. You render it, you never hand-type it:
+
+```python
+from transformers import AutoTokenizer
+tok = AutoTokenizer.from_pretrained("<target-model>")   # the model you will fine-tune
+
+# TRAINING: no trailing generation prompt — the assistant turn is already in the data
+text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=False)
+
+# INFERENCE: add_generation_prompt=True appends the assistant turn-start so the model continues
+prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
+```
+
+Three ways this silently destroys a run — **no error, just a worse model**:
+
+- **Hand-formatting the tokens.** Writing `<|im_start|>user\n…` strings yourself and getting one
+  token, one newline, or the BOS wrong. Train-time string ≠ inference-time string → the model
+  learns a distribution it is never served. Always render via `apply_chat_template`.
+- **Using the wrong model's template.** The template must be the one of the model you are
+  fine-tuning. Copy Llama's template onto a Qwen fine-tune and every example is subtly malformed.
+- **A base model with no template at all.** Base (non-instruct) checkpoints often ship
+  `chat_template = None`. `apply_chat_template` then raises — you must **choose and attach** a
+  template (e.g. ChatML) and use that same one at inference forever after.
+
+Also decide **loss masking**: for chat SFT you usually train only on the assistant tokens
+(`completion_only_loss` / assistant-only masking in SFTTrainer, or a completion-only collator),
+so the model is not penalized for "predicting" the user's words. TRL applies the template for
+you when the dataset is conversational — let it, rather than pre-flattening to `text`.
+
+## 3. Synthetic data & distillation
+
+Not enough real examples? Generate them. Two workhorses: **Self-Instruct** (seed a few
+hand-written examples, prompt a strong model to produce more, filter) and **Evol-Instruct**
+(iteratively mutate prompts to be harder/deeper). Wrap them in a pipeline framework rather than
+ad-hoc loops (see §7).
+
+**Licensing trap — read before you distill.** Generating your training data from *another
+model's outputs* ("distillation") is a **terms-of-service question, not just a quality one**.
+Some providers' terms restrict using their outputs to train competing models; some open-weight
+licenses carry naming/derivative obligations (e.g. Llama-derived data/models may inherit naming
+requirements). Never assert a model's license from memory — check the specific model card and
+provider ToS at author time (licenses change). If in doubt, distill from an
+openly-licensed-for-this-use model, and record the provenance per example.
+
+## 4. Dedup + decontamination (skip these and your numbers lie)
+
+- **Dedup.** Exact dedup is trivial (hash the text). Real corpora need **near-dup** removal:
+  **MinHash + LSH** (Jaccard similarity over shingles) catches templated/boilerplate repeats
+  that inflate a few patterns. Dupes waste compute and bias the model toward whatever is
+  over-represented.
+- **Decontamination — the one people forget.** Remove any training example that overlaps your
+  **eval / benchmark test sets** (n-gram overlap, e.g. long-n-gram match against MMLU, GSM8K,
+  your own held-out set). If test items leak into training, your eval score is **inflated** and
+  meaningless — you measured memorization, not capability. Decontaminate *against every metric
+  you will report*, including your private eval. Code for both in `references/synthesis-dedup-quality.md`.
+
+## 5. Quality filtering — a few thousand clean beats a noisy dump
+
+**LIMA** (*Less Is More for Alignment*, arXiv **2305.11206**) is the anchor: ~1,000 carefully
+curated examples produced a strong instruction-follower — for **alignment/style** SFT, quality
+and diversity dominate raw volume. (This is about teaching *behavior/format*, not injecting a
+lot of new *knowledge* — a broad knowledge shift still wants scale.) Cheap, high-leverage
+filters, applied before you spend GPU hours:
+
+- **Length/format**: drop empty or truncated turns, runaway-length outliers, malformed JSON,
+  wrong-role sequences (two `assistant` turns in a row, missing final assistant turn for SFT).
+- **Dedup + decontam** from §4.
+- **Diversity**: cluster/embed and prune near-identical intents so the set is not 80% one task.
+- **Model/heuristic scoring**: rate helpfulness/correctness (a reward model or an LLM judge) and
+  keep the top slice — but audit the judge, LLM-as-judge has its own biases.
+
+## 6. Licensing — two separate questions
+
+1. **The dataset's own license** — what *you* release the JSONL under, and whether you *can*
+   release it (aggregating others' data does not launder their licenses).
+2. **Source-usage restrictions** — the terms on where each example *came from*: scraped-site
+   ToS, the license of any base dataset you built on, and the model-output ToS from §3. These
+   bind even if you never publish. Keep a provenance column so an audit can trace every row.
+
+State the license *class* and point at the source; never freeze a license as bare fact.
+
+## 7. Tooling
+
+- **distilabel** (Argilla, now under Hugging Face) — the go-to synthetic-data / AI-feedback
+  pipeline framework: composable `Step`/`Task` graphs (`TextGeneration`, `UltraFeedback`,
+  `EvolInstruct`), serializable to YAML/JSON, outputs a `Distiset` you push to the Hub. v1.x.
+- **Argilla** — human-in-the-loop annotation/review UI to label and vet examples.
+- **HF `datasets`** — load/`map`/`filter`/`push_to_hub`; the substrate everything else speaks.
+- **Lilac** — dataset exploration/clustering for quality triage. **[verify — the open-source
+  repo was archived (read-only) around July 2025 after the Databricks acquisition]**; treat as
+  unmaintained OSS and confirm before depending on it.
+
+## Worked lifecycle (build → validate → dedup → decontaminate → format → push)
+
+```python
+from datasets import load_dataset
+from transformers import AutoTokenizer
+
+ds  = load_dataset("json", data_files="raw.jsonl", split="train")
+tok = AutoTokenizer.from_pretrained("<target-model>")
+
+# 1. VALIDATE shape + render every row through the template (catches template errors NOW,
+#    not after 3 GPU-hours). A base model with chat_template=None raises here — attach one.
+def render(ex):
+    return {"text": tok.apply_chat_template(ex["messages"], tokenize=False,
+                                            add_generation_prompt=False)}
+ds = ds.filter(lambda ex: isinstance(ex.get("messages"), list) and ex["messages"]
+               and ex["messages"][-1]["role"] == "assistant")   # SFT: must end on assistant
+ds = ds.map(render)
+
+# 2. DEDUP (near-dup) and 3. DECONTAMINATE against your eval set — see references for MinHash
+#    + n-gram code; both are one filter pass each.
+
+# 4. PUSH with a data card recording license + provenance.
+ds.push_to_hub("me/support-sft", private=True)
+```
+
+Deep code — MinHash/LSH dedup, n-gram decontamination, a distilabel Self-Instruct pipeline, and
+the full conversion matrix — is in `references/`.
+
+## Guardrails / gotchas
+
+- **Wrong format for the trainer** = hard error or silent garbage. Match the table in §1 to your
+  trainer before generating a single row.
+- **Hand-typed chat tokens** train a distribution you never serve. Render via `apply_chat_template`.
+- **No EOS in Alpaca-`text` formatting** → the model never stops. Append it.
+- **Skipped decontamination** → inflated eval; you measured leakage. Non-negotiable.
+- **Distilling model outputs** can violate ToS. Provenance + license check first.
+- **Quantity worship** — a noisy 500k dump loses to a curated few-thousand for alignment (LIMA).
+- `label` in KTO/unpaired data is a JSON boolean (`true`/`false`), not the strings `"true"`/`"1"`.
+
+## Related skills
+
+- [`finetuning`](../finetuning/SKILL.md) — consumes this corpus: chooses SFT vs DPO vs KTO,
+  LoRA/QLoRA, hyperparameters, runs trl/peft. You hand it the file; it trains.
+- [`unsloth`](../unsloth/SKILL.md) — one fast single-GPU training backend + GGUF export;
+  its notebooks expect exactly the Alpaca/`messages` shapes you produce here.
+- [`huggingface`](../huggingface/SKILL.md) — the Hub you `push_to_hub` the dataset to, model
+  cards, and hosted/routed inference of the *result*.
+- [`data-cleaning`](../data-cleaning/SKILL.md) — upstream when your raw source is dirty *tabular*
+  rows; it hands you clean rows, you turn rows into training examples.
+
+## Checklist
+
+- [ ] Format matches the target **trainer** (§1 table); one JSON object per line.
+- [ ] Every row renders through the **target model's** `apply_chat_template` without error.
+- [ ] SFT rows end on an `assistant` turn; preference rows have distinct `chosen`/`rejected`; KTO `label` is a boolean.
+- [ ] Loss masking / EOS handling decided (assistant-only loss; EOS appended if flattening to `text`).
+- [ ] Near-duplicates removed (MinHash/LSH); exact dupes gone.
+- [ ] **Decontaminated** against every eval/benchmark you will report.
+- [ ] Quality-filtered (length/format/diversity/score) — curated over bulk.
+- [ ] Dataset license set **and** source-usage/model-output ToS checked; provenance recorded.
+- [ ] Data card written; pushed (private first).

package/skills/training-data/evals/README.md

@@ -0,0 +1,13 @@
+# Evals for the `training-data` skill
+
+Routing + capability checks read from `cases.yaml` by the skill eval harness — no network, no
+model calls, nothing to install. The `should_trigger` prompts must select this skill: they cover
+format-by-trainer (DPO), converting raw pairs to `messages`, the non-obvious "model never stops /
+I hand-formatted the chat tokens" template-mismatch case, synthetic-data + dedup, decontamination
+against benchmarks, and a Catalan quality-filtering prompt. The `should_not_trigger` prompts must
+route to the named real sibling instead — `data-cleaning` (tabular rows), `embeddings-search`
+(retrieval corpus), `finetuning` / `unsloth` (running the trainer + export), `prompt-engineering`
+(inference prompt craft). The single `capability` case is a rubric: a good answer to the
+support-transcript scenario must hit every `must_include` bullet — the messages/JSONL shape, chat-
+template rendering via `apply_chat_template`, dedup + decontamination, LIMA quality-over-quantity,
+and the license/provenance + distillation-ToS guardrail.

package/skills/training-data/evals/cases.yaml

@@ -0,0 +1,41 @@
+skill: training-data
+
+should_trigger:
+  - prompt: "What format should my fine-tuning dataset be in if I want to run DPO?"
+    why: "Core format-by-trainer decision — preference type {prompt, chosen, rejected}. Squarely this skill."
+  - prompt: "I have thousands of Q&A pairs. Convert them into the conversational messages format for SFT."
+    why: "Turning raw examples into the messages/role-content JSONL a trainer eats — the central deliverable."
+  - prompt: "My fine-tuned chat model never stops generating and repeats itself. I formatted the training data myself with the chat tokens."
+    why: "Non-obvious: hand-typed chat tokens / missing EOS = train/inference template mismatch. User never names the fix; this skill diagnoses it."
+  - prompt: "Generate synthetic instruction-tuning data with distilabel and then deduplicate it before training."
+    why: "Synthetic data (§3) + dedup (§4) — distilabel pipeline and MinHash, both this skill."
+  - prompt: "Do I need to decontaminate my training set against MMLU before I trust my eval numbers?"
+    why: "Decontamination against benchmark test sets — the load-bearing guardrail; leakage inflates eval."
+  - prompt: "Quants exemples necessito i com filtro la qualitat del dataset abans d'entrenar un model?"
+    why: "Catalan; quality-over-quantity filtering (LIMA) before training. This skill's §5."
+
+should_not_trigger:
+  - prompt: "Clean this messy CSV: fix the nulls and sentinels, dedupe the rows, and normalize the category column."
+    route_to: data-cleaning
+    why: "Tabular row cleaning (nulls/dtypes/dupe rows) — the explicit boundary; not an LLM training corpus."
+  - prompt: "Chunk our documentation, embed it, and build an index so we can retrieve relevant passages."
+    route_to: embeddings-search
+    why: "A retrieval corpus of chunks+embeddings, not a training corpus of messages/preference pairs."
+  - prompt: "Set the LoRA rank, learning rate, and batch size and launch the QLoRA fine-tune."
+    route_to: finetuning
+    why: "Running the trainer and choosing hyperparameters — training-data stops at the validated file."
+  - prompt: "Run the fine-tune on my single 4090 with Unsloth and export the result to GGUF for Ollama."
+    route_to: unsloth
+    why: "The actual training backend + export, not corpus preparation."
+  - prompt: "Help me write a better system prompt so the model returns clean JSON at inference time."
+    route_to: prompt-engineering
+    why: "Prompt craft for inference, no training corpus involved."
+
+capability:
+  - scenario: "I have 8,000 raw customer-support transcripts and want to fine-tune a chat model on them. Prepare the dataset."
+    must_include:
+      - "convert to conversational `messages` (role/content) — or Alpaca instruction/input/output — as JSONL, one JSON object per line, ending each SFT example on an assistant turn"
+      - "render through the TARGET model's chat template via apply_chat_template (add_generation_prompt=False for training) instead of hand-writing special tokens, because a template mismatch silently wrecks the run with no error"
+      - "deduplicate near-duplicates (MinHash/LSH) and decontaminate against the eval/benchmark test sets, or the reported eval score is inflated"
+      - "quality-filter over raw volume (LIMA, arXiv 2305.11206) — a few thousand curated examples beat a large noisy dump for alignment"
+      - "record the dataset license AND check source-usage/model-output terms — distilling another model's outputs may violate its ToS; keep per-example provenance"

package/skills/training-data/references/formats.md

@@ -0,0 +1,118 @@
+# Dataset formats — full field matrix & conversions
+
+Grounded in the TRL "Dataset formats and types" doc (tagged `v1.8.0` at author time). TRL and
+`datasets` move monthly — verify field names against the current TRL docs before you rely on
+them. Field names below are verbatim from that doc.
+
+## The two axes
+
+- **Format** — `standard` (plain-text string values) vs `conversational` (values are lists of
+  `{"role", "content"}` messages). Conversational is the default for chat models.
+- **Type** — the task, which sets the columns.
+
+| Type | Standard | Conversational |
+| --- | --- | --- |
+| Language modeling | `{"text": "..."}` | `{"messages": [{"role","content"}, ...]}` |
+| Prompt-only | `{"prompt": "..."}` | `{"prompt": [ ... ]}` |
+| Prompt-completion | `{"prompt", "completion"}` | `{"prompt": [...], "completion": [...]}` |
+| Preference (explicit) | `{"prompt", "chosen", "rejected"}` | same keys, each a message list |
+| Preference (implicit) | `{"chosen", "rejected"}` (no `prompt`) | same, each a full conversation |
+| Unpaired preference (KTO) | `{"prompt", "completion", "label"}` | same, `label` a bool |
+| Stepwise supervision | `{"prompt", "completions": [...], "labels": [bool,...]}` | — |
+
+`label` / `labels` are JSON booleans (`true`/`false`).
+
+## Which trainer wants which type
+
+| Trainer | Type |
+| --- | --- |
+| `SFTTrainer` | language modeling **or** prompt-completion |
+| `DPOTrainer` | preference (explicit prompt recommended) |
+| `KTOTrainer` | unpaired preference, or preference |
+| `RewardTrainer` | preference (implicit prompt recommended) |
+| `GRPOTrainer`, `RLOOTrainer`, `OnlineDPO/Nash/XPO` | prompt-only |
+| `ORPOTrainer`, `CPOTrainer`, `BCOTrainer` | preference (explicit) / unpaired |
+| `PRMTrainer` | stepwise supervision |
+| `PPOTrainer` | tokenized language modeling |
+
+Experimental trainers live under `trl.experimental.*` — confirm the import path in the current
+release.
+
+## Alpaca instruction format (classic, not native TRL)
+
+```json
+{"instruction": "Summarize the paragraph.", "input": "<long text>", "output": "<summary>"}
+```
+
+`input` is optional (empty string when the instruction is self-contained). TRL does not train on
+these keys directly. Convert:
+
+**To `messages` (preferred for chat models):**
+
+```python
+def alpaca_to_messages(ex):
+    user = ex["instruction"] + (("\n\n" + ex["input"]) if ex.get("input") else "")
+    return {"messages": [{"role": "user", "content": user},
+                         {"role": "assistant", "content": ex["output"]}]}
+```
+
+**To a single `text` string via a prompt template (Unsloth-style) — append EOS:**
+
+```python
+TEMPLATE = ("### Instruction:\n{instruction}\n\n### Input:\n{input}\n\n### Response:\n{output}")
+def alpaca_to_text(ex, eos):
+    return {"text": TEMPLATE.format(**ex) + eos}   # eos = tokenizer.eos_token — REQUIRED
+```
+
+Without the EOS token the model never learns to stop generating. When formatting to `text` you
+are also bypassing the chat template, so this path is for base/completion-style training; for
+chat models prefer `messages` + `apply_chat_template`.
+
+## Tool-calling data
+
+Add a `tools` column (list of JSON-schema function specs); assistant turns carry `tool_calls`
+instead of `content`, and tool results come back as a `{"role": "tool", ...}` message.
+
+```python
+from transformers.utils import get_json_schema   # build the schema from a Python signature
+schema = get_json_schema(my_function)
+row = {"messages": [...], "tools": [schema]}
+```
+
+On `datasets` ≥4.7 use the `Json()` feature type for arbitrary tool-arg objects
+(`Dataset.from_list(data, on_mixed_types="use_json")`); on older `datasets` store `tools` as a
+`json.dumps(...)` string.
+
+## Vision (VLM) data
+
+Add an `images` key (list of PIL images per row) or `image` (single), and make each message
+`content` a list of typed parts:
+
+```python
+"content": [{"type": "image"}, {"type": "text", "text": "What is in the image?"}]
+```
+
+Mixing text-only and image rows in one dataset needs `transformers` ≥ 4.57.
+
+## Type→type conversions (with `datasets`)
+
+- **prompt-completion → language-modeling**: concatenate `prompt` + `completion` into `text`.
+- **preference → prompt-completion**: drop `rejected`, rename `chosen` → `completion`.
+- **preference → prompt-only**: drop `chosen` and `rejected`.
+- **implicit → explicit preference**: `trl.extract_prompt` pulls the shared prefix into `prompt`.
+- **preference → unpaired (KTO)**: `trl.unpair_preference_dataset` — **only valid if every
+  `chosen` is genuinely good and every `rejected` genuinely bad** (KTO's label is absolute
+  good/bad, not relative). If your pairs are "better vs worse" but both acceptable, unpairing
+  mislabels them.
+- **stepwise → unpaired**: join `completions`, merge `labels` (e.g. logical AND).
+
+Do conversions **before** `apply_chat_template`, on the structured `messages`, so they behave
+consistently across formats.
+
+## Sanity checks worth automating
+
+- Every SFT conversational row ends on an `assistant` turn.
+- Roles alternate sensibly; no empty `content`.
+- Preference `chosen != rejected`.
+- KTO `label` present and boolean; class balance is not 100/0.
+- The whole file round-trips through `apply_chat_template` for the *target* model with no raise.