@damiangilgonzalez/harness 1.3.7

package/skills/training-data/SKILL.md

@@ -0,0 +1,226 @@
+---
+name: training-data
+description: "Use when assembling or curating the corpus a fine-tune trains on — turning raw examples into the JSONL shape a trainer expects (instruction, conversational, preference-pair or binary-label), matching the data to the model's chat template, generating synthetic or distilled examples, deduplicating and decontaminating against eval sets, and quality-filtering. NOT cleaning tabular rows, nulls and dtypes (that is `data-cleaning`), NOT building a retrieval corpus of chunks and embeddings (that is `embeddings-search`), NOT running the trainer or picking hyperparameters (that is `finetuning`)."
+tags: [training-data, fine-tuning-dataset, jsonl, chat-template, preference-data, dpo, kto, synthetic-data, decontamination, dataset-curation]
+recommends: [finetuning, unsloth, huggingface, data-cleaning]
+origin: risco
+---
+
+# training-data — the corpus a fine-tune actually eats
+
+You own the **training corpus**: the JSONL of chat turns, instruction triples, or preference
+pairs that a trainer reads. The deliverable is a validated, deduplicated, decontaminated,
+license-clean file in the **exact shape the trainer expects**, rendered through the **target
+model's chat template**. You stop the moment that file loads cleanly and round-trips through
+`apply_chat_template`. You do not choose LoRA rank or launch the run — that is
+[`finetuning`](../finetuning/SKILL.md) / [`unsloth`](../unsloth/SKILL.md).
+
+**Loud boundary.** This is *LLM training corpora* — messages, instruction triples, preference
+pairs. It is **not**:
+
+- Tabular row cleaning — nulls, dtypes, dedupe of CSV rows, category normalization → [`data-cleaning`](../data-cleaning/SKILL.md).
+- A *retrieval* corpus — chunking documents and embedding them for search → [`embeddings-search`](../embeddings-search/SKILL.md).
+- Actually training or serving — hyperparameters, the run, export → [`finetuning`](../finetuning/SKILL.md), [`unsloth`](../unsloth/SKILL.md), [`huggingface`](../huggingface/SKILL.md).
+
+**Version reality (verified July 2026 — re-verify, these move monthly).** TRL is on the **v1.x**
+line (its dataset-formats doc was tagged `v1.8.0` at author time); `transformers` is in the
+**4.57+** era (mixed text+vision data needs ≥4.57); `datasets` is **4.x** (the `Json()` feature
+type needs ≥4.7). Pin whatever you install — do not trust these numbers as current.
+
+## 1. The format the trainer expects (pick by trainer, not by taste)
+
+The trainer dictates the columns. Get this wrong and TRL either errors or, worse, trains on a
+mangled string. Two axes: **format** (`standard` = plain strings vs `conversational` =
+`messages` lists) and **type** (the task). One JSON object per line = JSONL.
+
+| Trainer | Dataset type | Required keys |
+| --- | --- | --- |
+| `SFTTrainer` | language-modeling **or** prompt-completion | `messages` / `text`, or `prompt`+`completion` |
+| `DPOTrainer`, `ORPOTrainer`, `CPOTrainer` | preference (explicit prompt recommended) | `prompt`, `chosen`, `rejected` |
+| `KTOTrainer`, `BCOTrainer` | unpaired preference (binary label) | `prompt`, `completion`, `label` |
+| `RewardTrainer` | preference (implicit prompt) | `chosen`, `rejected` |
+| `GRPOTrainer`, `RLOOTrainer`, `PPOTrainer` | prompt-only | `prompt` |
+
+Tiny JSONL of each (conversational values are **lists of `{role, content}`**; `label` is a JSON
+boolean):
+
+```jsonl
+# Alpaca instruction (standard) — classic; NOT a native TRL type, see below
+{"instruction": "Classify the sentiment.", "input": "The battery dies in an hour.", "output": "negative"}
+
+# Conversational messages (SFT) — the default for chat fine-tunes
+{"messages": [{"role": "system", "content": "You are a terse support agent."}, {"role": "user", "content": "My order never arrived."}, {"role": "assistant", "content": "Sorry about that — what is your order number?"}]}
+
+# Preference pair (DPO) — chosen beats rejected for the same prompt
+{"prompt": [{"role": "user", "content": "Define a hash map in one sentence."}], "chosen": [{"role": "assistant", "content": "A hash map stores key-value pairs and finds a value by hashing its key to a bucket, giving average O(1) lookup."}], "rejected": [{"role": "assistant", "content": "It's a fast dictionary thing."}]}
+
+# KTO / unpaired preference — one completion + a good/bad boolean label
+{"prompt": [{"role": "user", "content": "Define a hash map in one sentence."}], "completion": [{"role": "assistant", "content": "It's a fast dictionary thing."}], "label": false}
+```
+
+**Alpaca is not a native TRL type.** `{instruction, input, output}` is the Stanford-Alpaca
+convention, still common in Unsloth notebooks, but TRL trains on `text`/`messages`/`prompt`+
+`completion`. You must either (a) map it into `messages` (instruction+input → user, output →
+assistant), or (b) render it into a single `text` string via a prompt template — **and append
+the EOS token yourself**, or the model never learns to stop (the #1 Unsloth-Alpaca bug). Prefer
+(a) `messages` for chat models. Full field matrix, tool-calling (`tools` column) and vision
+(`images`) extras, and every type→type conversion live in `references/formats.md`.
+
+## 2. Chat templates — the silent run-wrecker
+
+A chat template is a **Jinja** string stored in the tokenizer (in `tokenizer_config.json` under
+`chat_template`, or a standalone `chat_template.jinja` in newer tokenizers). It maps a `messages`
+list to the exact token string the model was trained on — special tokens (`<|im_start|>`,
+`[INST]`, `<|start_header_id|>`, …) and all. You render it, you never hand-type it:
+
+```python
+from transformers import AutoTokenizer
+tok = AutoTokenizer.from_pretrained("<target-model>")   # the model you will fine-tune
+
+# TRAINING: no trailing generation prompt — the assistant turn is already in the data
+text = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=False)
+
+# INFERENCE: add_generation_prompt=True appends the assistant turn-start so the model continues
+prompt = tok.apply_chat_template(msgs, tokenize=False, add_generation_prompt=True)
+```
+
+Three ways this silently destroys a run — **no error, just a worse model**:
+
+- **Hand-formatting the tokens.** Writing `<|im_start|>user\n…` strings yourself and getting one
+  token, one newline, or the BOS wrong. Train-time string ≠ inference-time string → the model
+  learns a distribution it is never served. Always render via `apply_chat_template`.
+- **Using the wrong model's template.** The template must be the one of the model you are
+  fine-tuning. Copy Llama's template onto a Qwen fine-tune and every example is subtly malformed.
+- **A base model with no template at all.** Base (non-instruct) checkpoints often ship
+  `chat_template = None`. `apply_chat_template` then raises — you must **choose and attach** a
+  template (e.g. ChatML) and use that same one at inference forever after.
+
+Also decide **loss masking**: for chat SFT you usually train only on the assistant tokens
+(`completion_only_loss` / assistant-only masking in SFTTrainer, or a completion-only collator),
+so the model is not penalized for "predicting" the user's words. TRL applies the template for
+you when the dataset is conversational — let it, rather than pre-flattening to `text`.
+
+## 3. Synthetic data & distillation
+
+Not enough real examples? Generate them. Two workhorses: **Self-Instruct** (seed a few
+hand-written examples, prompt a strong model to produce more, filter) and **Evol-Instruct**
+(iteratively mutate prompts to be harder/deeper). Wrap them in a pipeline framework rather than
+ad-hoc loops (see §7).
+
+**Licensing trap — read before you distill.** Generating your training data from *another
+model's outputs* ("distillation") is a **terms-of-service question, not just a quality one**.
+Some providers' terms restrict using their outputs to train competing models; some open-weight
+licenses carry naming/derivative obligations (e.g. Llama-derived data/models may inherit naming
+requirements). Never assert a model's license from memory — check the specific model card and
+provider ToS at author time (licenses change). If in doubt, distill from an
+openly-licensed-for-this-use model, and record the provenance per example.
+
+## 4. Dedup + decontamination (skip these and your numbers lie)
+
+- **Dedup.** Exact dedup is trivial (hash the text). Real corpora need **near-dup** removal:
+  **MinHash + LSH** (Jaccard similarity over shingles) catches templated/boilerplate repeats
+  that inflate a few patterns. Dupes waste compute and bias the model toward whatever is
+  over-represented.
+- **Decontamination — the one people forget.** Remove any training example that overlaps your
+  **eval / benchmark test sets** (n-gram overlap, e.g. long-n-gram match against MMLU, GSM8K,
+  your own held-out set). If test items leak into training, your eval score is **inflated** and
+  meaningless — you measured memorization, not capability. Decontaminate *against every metric
+  you will report*, including your private eval. Code for both in `references/synthesis-dedup-quality.md`.
+
+## 5. Quality filtering — a few thousand clean beats a noisy dump
+
+**LIMA** (*Less Is More for Alignment*, arXiv **2305.11206**) is the anchor: ~1,000 carefully
+curated examples produced a strong instruction-follower — for **alignment/style** SFT, quality
+and diversity dominate raw volume. (This is about teaching *behavior/format*, not injecting a
+lot of new *knowledge* — a broad knowledge shift still wants scale.) Cheap, high-leverage
+filters, applied before you spend GPU hours:
+
+- **Length/format**: drop empty or truncated turns, runaway-length outliers, malformed JSON,
+  wrong-role sequences (two `assistant` turns in a row, missing final assistant turn for SFT).
+- **Dedup + decontam** from §4.
+- **Diversity**: cluster/embed and prune near-identical intents so the set is not 80% one task.
+- **Model/heuristic scoring**: rate helpfulness/correctness (a reward model or an LLM judge) and
+  keep the top slice — but audit the judge, LLM-as-judge has its own biases.
+
+## 6. Licensing — two separate questions
+
+1. **The dataset's own license** — what *you* release the JSONL under, and whether you *can*
+   release it (aggregating others' data does not launder their licenses).
+2. **Source-usage restrictions** — the terms on where each example *came from*: scraped-site
+   ToS, the license of any base dataset you built on, and the model-output ToS from §3. These
+   bind even if you never publish. Keep a provenance column so an audit can trace every row.
+
+State the license *class* and point at the source; never freeze a license as bare fact.
+
+## 7. Tooling
+
+- **distilabel** (Argilla, now under Hugging Face) — the go-to synthetic-data / AI-feedback
+  pipeline framework: composable `Step`/`Task` graphs (`TextGeneration`, `UltraFeedback`,
+  `EvolInstruct`), serializable to YAML/JSON, outputs a `Distiset` you push to the Hub. v1.x.
+- **Argilla** — human-in-the-loop annotation/review UI to label and vet examples.
+- **HF `datasets`** — load/`map`/`filter`/`push_to_hub`; the substrate everything else speaks.
+- **Lilac** — dataset exploration/clustering for quality triage. **[verify — the open-source
+  repo was archived (read-only) around July 2025 after the Databricks acquisition]**; treat as
+  unmaintained OSS and confirm before depending on it.
+
+## Worked lifecycle (build → validate → dedup → decontaminate → format → push)
+
+```python
+from datasets import load_dataset
+from transformers import AutoTokenizer
+
+ds  = load_dataset("json", data_files="raw.jsonl", split="train")
+tok = AutoTokenizer.from_pretrained("<target-model>")
+
+# 1. VALIDATE shape + render every row through the template (catches template errors NOW,
+#    not after 3 GPU-hours). A base model with chat_template=None raises here — attach one.
+def render(ex):
+    return {"text": tok.apply_chat_template(ex["messages"], tokenize=False,
+                                            add_generation_prompt=False)}
+ds = ds.filter(lambda ex: isinstance(ex.get("messages"), list) and ex["messages"]
+               and ex["messages"][-1]["role"] == "assistant")   # SFT: must end on assistant
+ds = ds.map(render)
+
+# 2. DEDUP (near-dup) and 3. DECONTAMINATE against your eval set — see references for MinHash
+#    + n-gram code; both are one filter pass each.
+
+# 4. PUSH with a data card recording license + provenance.
+ds.push_to_hub("me/support-sft", private=True)
+```
+
+Deep code — MinHash/LSH dedup, n-gram decontamination, a distilabel Self-Instruct pipeline, and
+the full conversion matrix — is in `references/`.
+
+## Guardrails / gotchas
+
+- **Wrong format for the trainer** = hard error or silent garbage. Match the table in §1 to your
+  trainer before generating a single row.
+- **Hand-typed chat tokens** train a distribution you never serve. Render via `apply_chat_template`.
+- **No EOS in Alpaca-`text` formatting** → the model never stops. Append it.
+- **Skipped decontamination** → inflated eval; you measured leakage. Non-negotiable.
+- **Distilling model outputs** can violate ToS. Provenance + license check first.
+- **Quantity worship** — a noisy 500k dump loses to a curated few-thousand for alignment (LIMA).
+- `label` in KTO/unpaired data is a JSON boolean (`true`/`false`), not the strings `"true"`/`"1"`.
+
+## Related skills
+
+- [`finetuning`](../finetuning/SKILL.md) — consumes this corpus: chooses SFT vs DPO vs KTO,
+  LoRA/QLoRA, hyperparameters, runs trl/peft. You hand it the file; it trains.
+- [`unsloth`](../unsloth/SKILL.md) — one fast single-GPU training backend + GGUF export;
+  its notebooks expect exactly the Alpaca/`messages` shapes you produce here.
+- [`huggingface`](../huggingface/SKILL.md) — the Hub you `push_to_hub` the dataset to, model
+  cards, and hosted/routed inference of the *result*.
+- [`data-cleaning`](../data-cleaning/SKILL.md) — upstream when your raw source is dirty *tabular*
+  rows; it hands you clean rows, you turn rows into training examples.
+
+## Checklist
+
+- [ ] Format matches the target **trainer** (§1 table); one JSON object per line.
+- [ ] Every row renders through the **target model's** `apply_chat_template` without error.
+- [ ] SFT rows end on an `assistant` turn; preference rows have distinct `chosen`/`rejected`; KTO `label` is a boolean.
+- [ ] Loss masking / EOS handling decided (assistant-only loss; EOS appended if flattening to `text`).
+- [ ] Near-duplicates removed (MinHash/LSH); exact dupes gone.
+- [ ] **Decontaminated** against every eval/benchmark you will report.
+- [ ] Quality-filtered (length/format/diversity/score) — curated over bulk.
+- [ ] Dataset license set **and** source-usage/model-output ToS checked; provenance recorded.
+- [ ] Data card written; pushed (private first).

package/skills/training-data/evals/README.md

@@ -0,0 +1,13 @@
+# Evals for the `training-data` skill
+
+Routing + capability checks read from `cases.yaml` by the skill eval harness — no network, no
+model calls, nothing to install. The `should_trigger` prompts must select this skill: they cover
+format-by-trainer (DPO), converting raw pairs to `messages`, the non-obvious "model never stops /
+I hand-formatted the chat tokens" template-mismatch case, synthetic-data + dedup, decontamination
+against benchmarks, and a Catalan quality-filtering prompt. The `should_not_trigger` prompts must
+route to the named real sibling instead — `data-cleaning` (tabular rows), `embeddings-search`
+(retrieval corpus), `finetuning` / `unsloth` (running the trainer + export), `prompt-engineering`
+(inference prompt craft). The single `capability` case is a rubric: a good answer to the
+support-transcript scenario must hit every `must_include` bullet — the messages/JSONL shape, chat-
+template rendering via `apply_chat_template`, dedup + decontamination, LIMA quality-over-quantity,
+and the license/provenance + distillation-ToS guardrail.

package/skills/training-data/evals/cases.yaml

@@ -0,0 +1,41 @@
+skill: training-data
+
+should_trigger:
+  - prompt: "What format should my fine-tuning dataset be in if I want to run DPO?"
+    why: "Core format-by-trainer decision — preference type {prompt, chosen, rejected}. Squarely this skill."
+  - prompt: "I have thousands of Q&A pairs. Convert them into the conversational messages format for SFT."
+    why: "Turning raw examples into the messages/role-content JSONL a trainer eats — the central deliverable."
+  - prompt: "My fine-tuned chat model never stops generating and repeats itself. I formatted the training data myself with the chat tokens."
+    why: "Non-obvious: hand-typed chat tokens / missing EOS = train/inference template mismatch. User never names the fix; this skill diagnoses it."
+  - prompt: "Generate synthetic instruction-tuning data with distilabel and then deduplicate it before training."
+    why: "Synthetic data (§3) + dedup (§4) — distilabel pipeline and MinHash, both this skill."
+  - prompt: "Do I need to decontaminate my training set against MMLU before I trust my eval numbers?"
+    why: "Decontamination against benchmark test sets — the load-bearing guardrail; leakage inflates eval."
+  - prompt: "Quants exemples necessito i com filtro la qualitat del dataset abans d'entrenar un model?"
+    why: "Catalan; quality-over-quantity filtering (LIMA) before training. This skill's §5."
+
+should_not_trigger:
+  - prompt: "Clean this messy CSV: fix the nulls and sentinels, dedupe the rows, and normalize the category column."
+    route_to: data-cleaning
+    why: "Tabular row cleaning (nulls/dtypes/dupe rows) — the explicit boundary; not an LLM training corpus."
+  - prompt: "Chunk our documentation, embed it, and build an index so we can retrieve relevant passages."
+    route_to: embeddings-search
+    why: "A retrieval corpus of chunks+embeddings, not a training corpus of messages/preference pairs."
+  - prompt: "Set the LoRA rank, learning rate, and batch size and launch the QLoRA fine-tune."
+    route_to: finetuning
+    why: "Running the trainer and choosing hyperparameters — training-data stops at the validated file."
+  - prompt: "Run the fine-tune on my single 4090 with Unsloth and export the result to GGUF for Ollama."
+    route_to: unsloth
+    why: "The actual training backend + export, not corpus preparation."
+  - prompt: "Help me write a better system prompt so the model returns clean JSON at inference time."
+    route_to: prompt-engineering
+    why: "Prompt craft for inference, no training corpus involved."
+
+capability:
+  - scenario: "I have 8,000 raw customer-support transcripts and want to fine-tune a chat model on them. Prepare the dataset."
+    must_include:
+      - "convert to conversational `messages` (role/content) — or Alpaca instruction/input/output — as JSONL, one JSON object per line, ending each SFT example on an assistant turn"
+      - "render through the TARGET model's chat template via apply_chat_template (add_generation_prompt=False for training) instead of hand-writing special tokens, because a template mismatch silently wrecks the run with no error"
+      - "deduplicate near-duplicates (MinHash/LSH) and decontaminate against the eval/benchmark test sets, or the reported eval score is inflated"
+      - "quality-filter over raw volume (LIMA, arXiv 2305.11206) — a few thousand curated examples beat a large noisy dump for alignment"
+      - "record the dataset license AND check source-usage/model-output terms — distilling another model's outputs may violate its ToS; keep per-example provenance"

package/skills/training-data/references/formats.md

@@ -0,0 +1,118 @@
+# Dataset formats — full field matrix & conversions
+
+Grounded in the TRL "Dataset formats and types" doc (tagged `v1.8.0` at author time). TRL and
+`datasets` move monthly — verify field names against the current TRL docs before you rely on
+them. Field names below are verbatim from that doc.
+
+## The two axes
+
+- **Format** — `standard` (plain-text string values) vs `conversational` (values are lists of
+  `{"role", "content"}` messages). Conversational is the default for chat models.
+- **Type** — the task, which sets the columns.
+
+| Type | Standard | Conversational |
+| --- | --- | --- |
+| Language modeling | `{"text": "..."}` | `{"messages": [{"role","content"}, ...]}` |
+| Prompt-only | `{"prompt": "..."}` | `{"prompt": [ ... ]}` |
+| Prompt-completion | `{"prompt", "completion"}` | `{"prompt": [...], "completion": [...]}` |
+| Preference (explicit) | `{"prompt", "chosen", "rejected"}` | same keys, each a message list |
+| Preference (implicit) | `{"chosen", "rejected"}` (no `prompt`) | same, each a full conversation |
+| Unpaired preference (KTO) | `{"prompt", "completion", "label"}` | same, `label` a bool |
+| Stepwise supervision | `{"prompt", "completions": [...], "labels": [bool,...]}` | — |
+
+`label` / `labels` are JSON booleans (`true`/`false`).
+
+## Which trainer wants which type
+
+| Trainer | Type |
+| --- | --- |
+| `SFTTrainer` | language modeling **or** prompt-completion |
+| `DPOTrainer` | preference (explicit prompt recommended) |
+| `KTOTrainer` | unpaired preference, or preference |
+| `RewardTrainer` | preference (implicit prompt recommended) |
+| `GRPOTrainer`, `RLOOTrainer`, `OnlineDPO/Nash/XPO` | prompt-only |
+| `ORPOTrainer`, `CPOTrainer`, `BCOTrainer` | preference (explicit) / unpaired |
+| `PRMTrainer` | stepwise supervision |
+| `PPOTrainer` | tokenized language modeling |
+
+Experimental trainers live under `trl.experimental.*` — confirm the import path in the current
+release.
+
+## Alpaca instruction format (classic, not native TRL)
+
+```json
+{"instruction": "Summarize the paragraph.", "input": "<long text>", "output": "<summary>"}
+```
+
+`input` is optional (empty string when the instruction is self-contained). TRL does not train on
+these keys directly. Convert:
+
+**To `messages` (preferred for chat models):**
+
+```python
+def alpaca_to_messages(ex):
+    user = ex["instruction"] + (("\n\n" + ex["input"]) if ex.get("input") else "")
+    return {"messages": [{"role": "user", "content": user},
+                         {"role": "assistant", "content": ex["output"]}]}
+```
+
+**To a single `text` string via a prompt template (Unsloth-style) — append EOS:**
+
+```python
+TEMPLATE = ("### Instruction:\n{instruction}\n\n### Input:\n{input}\n\n### Response:\n{output}")
+def alpaca_to_text(ex, eos):
+    return {"text": TEMPLATE.format(**ex) + eos}   # eos = tokenizer.eos_token — REQUIRED
+```
+
+Without the EOS token the model never learns to stop generating. When formatting to `text` you
+are also bypassing the chat template, so this path is for base/completion-style training; for
+chat models prefer `messages` + `apply_chat_template`.
+
+## Tool-calling data
+
+Add a `tools` column (list of JSON-schema function specs); assistant turns carry `tool_calls`
+instead of `content`, and tool results come back as a `{"role": "tool", ...}` message.
+
+```python
+from transformers.utils import get_json_schema   # build the schema from a Python signature
+schema = get_json_schema(my_function)
+row = {"messages": [...], "tools": [schema]}
+```
+
+On `datasets` ≥4.7 use the `Json()` feature type for arbitrary tool-arg objects
+(`Dataset.from_list(data, on_mixed_types="use_json")`); on older `datasets` store `tools` as a
+`json.dumps(...)` string.
+
+## Vision (VLM) data
+
+Add an `images` key (list of PIL images per row) or `image` (single), and make each message
+`content` a list of typed parts:
+
+```python
+"content": [{"type": "image"}, {"type": "text", "text": "What is in the image?"}]
+```
+
+Mixing text-only and image rows in one dataset needs `transformers` ≥ 4.57.
+
+## Type→type conversions (with `datasets`)
+
+- **prompt-completion → language-modeling**: concatenate `prompt` + `completion` into `text`.
+- **preference → prompt-completion**: drop `rejected`, rename `chosen` → `completion`.
+- **preference → prompt-only**: drop `chosen` and `rejected`.
+- **implicit → explicit preference**: `trl.extract_prompt` pulls the shared prefix into `prompt`.
+- **preference → unpaired (KTO)**: `trl.unpair_preference_dataset` — **only valid if every
+  `chosen` is genuinely good and every `rejected` genuinely bad** (KTO's label is absolute
+  good/bad, not relative). If your pairs are "better vs worse" but both acceptable, unpairing
+  mislabels them.
+- **stepwise → unpaired**: join `completions`, merge `labels` (e.g. logical AND).
+
+Do conversions **before** `apply_chat_template`, on the structured `messages`, so they behave
+consistently across formats.
+
+## Sanity checks worth automating
+
+- Every SFT conversational row ends on an `assistant` turn.
+- Roles alternate sensibly; no empty `content`.
+- Preference `chosen != rejected`.
+- KTO `label` present and boolean; class balance is not 100/0.
+- The whole file round-trips through `apply_chat_template` for the *target* model with no raise.

package/skills/training-data/references/synthesis-dedup-quality.md

@@ -0,0 +1,134 @@
+# Synthesis, dedup, decontamination, quality, licensing
+
+Depth overflow for SKILL.md §3–§7. Library versions and, above all, model licenses change
+monthly — verify at author time; nothing here is a standing legal fact.
+
+## Synthetic data
+
+**Self-Instruct**: hand-write a small seed set of diverse instructions+responses, prompt a
+strong model to generate many more conditioned on the seeds, then filter for validity, dedup,
+and difficulty. **Evol-Instruct**: iteratively rewrite each instruction to be harder
+(add constraints, deepen, broaden) so the corpus spans a difficulty gradient. **AI feedback /
+UltraFeedback-style**: have a model rate candidate responses on axes (helpfulness, honesty,
+correctness) to build preference pairs.
+
+### distilabel skeleton (Argilla / Hugging Face, v1.x — verify)
+
+```python
+from distilabel.pipeline import Pipeline
+from distilabel.steps import LoadDataFromDicts
+from distilabel.steps.tasks import TextGeneration
+# LLM client import paths shift between releases — check current distilabel docs.
+
+with Pipeline(name="self-instruct") as pipe:
+    seeds = LoadDataFromDicts(data=[{"instruction": "Explain X."}, ...])
+    gen   = TextGeneration(llm=...)      # plug an LLM client
+    seeds >> gen
+
+distiset = pipe.run(parameters={...})
+distiset.push_to_hub("me/synthetic-sft")   # pushes data + the pipeline definition + a card
+```
+
+Distilabel serializes the whole pipeline to YAML/JSON and (on push) writes it into the dataset
+card, so a run is reproducible. Output is a `Distiset` (a `datasets.DatasetDict` variant).
+
+### Distillation licensing trap (do this before generating)
+
+Training on *another model's outputs* is governed by that model/provider's terms:
+
+- **Proprietary API outputs** may be contractually barred from training competing models — read
+  the current provider ToS, not a blog.
+- **Open-weight licenses** vary and are **not** all OSI-open: Llama ships under the Meta Llama
+  Community License (with acceptable-use + naming/derivative terms), Gemma has custom Google
+  terms, some models (e.g. Codestral-class) are non-production/research-only, and Qwen license
+  terms vary by size. **Never state a license as bare fact** — open the specific model card,
+  licenses change.
+- Record per-example provenance (which model/source produced it) so an audit can trace it.
+
+## Deduplication
+
+Exact dedup: hash normalized text, drop collisions. Near-dup (the one that matters) —
+**MinHash + LSH** over token/char shingles:
+
+```python
+from datasketch import MinHash, MinHashLSH
+
+def minhash(text, num_perm=128, k=5):
+    m = MinHash(num_perm=num_perm)
+    toks = text.split()
+    for i in range(len(toks) - k + 1):
+        m.update(" ".join(toks[i:i+k]).encode())
+    return m
+
+lsh, keep = MinHashLSH(threshold=0.8, num_perm=128), []
+for i, ex in enumerate(dataset):
+    mh = minhash(ex["text"])
+    if not lsh.query(mh):            # no near-duplicate already kept
+        lsh.insert(str(i), mh); keep.append(i)
+dedup = dataset.select(keep)
+```
+
+For very large corpora use a batched/distributed dedup (e.g. HF `datatrove`, `text-dedup`)
+rather than an in-memory loop. Tune `threshold` on a sample — too high keeps paraphrase dupes,
+too low deletes legitimately similar-but-distinct examples.
+
+## Decontamination (against eval/benchmark test sets)
+
+Leakage of test items into training inflates your eval — you measure memorization. Remove any
+training row that overlaps a test item, by **n-gram overlap** (the common convention is a long
+n-gram, e.g. 13-gram, exact match; also embed-and-threshold for paraphrased leakage):
+
+```python
+def ngrams(text, n=13):
+    t = text.lower().split()
+    return {" ".join(t[i:i+n]) for i in range(len(t) - n + 1)}
+
+contam = set().union(*(ngrams(x["text"]) for x in eval_set))   # build from EVERY eval you report
+clean  = dataset.filter(lambda ex: ngrams(ex["text"]).isdisjoint(contam))
+```
+
+Decontaminate against **every** benchmark you will quote *and* your own private held-out set —
+including any public dataset you synthesized from, since its test split may be in your training
+mix. `lm-eval-harness` and dataset-curation toolkits ship decontamination utilities; prefer a
+maintained one over hand-rolled for scale.
+
+## Quality filtering
+
+**LIMA** (*Less Is More for Alignment*, arXiv 2305.11206): ~1,000 curated examples produced a
+strong instruction-follower — for teaching **behavior/style/format**, curation and diversity
+beat raw volume. Caveat: LIMA is about *alignment*, not injecting large new *knowledge* — a
+broad domain-knowledge shift still wants scale. Practical filter ladder:
+
+1. **Format/structural** — drop empty/truncated turns, malformed JSON, wrong role order, missing
+   final assistant turn (SFT), identical `chosen`/`rejected` (preference).
+2. **Length** — cut extreme-short and runaway-long outliers (often degenerate/repetitive).
+3. **Dedup + decontam** (above).
+4. **Diversity** — embed and cluster; downsample over-represented intents so one task is not 80%
+   of the set. (Lilac was built for exactly this triage — but its OSS repo was archived
+   ~July 2025 after the Databricks acquisition; **[verify]** before depending on it.)
+5. **Scoring** — reward model or LLM-as-judge rates helpfulness/correctness; keep the top slice.
+   Audit the judge: it has position/length/self-preference biases; spot-check by hand.
+
+Always keep a discarded-rows sample so filtering decisions are reviewable, not a black hole.
+
+## Licensing — two separate questions
+
+1. **Your dataset's license** — what you release the JSONL under, and whether you *may* (you
+   cannot relicense others' data by aggregating it).
+2. **Source-usage restrictions** — the terms attached to where each row came from: scraped-site
+   ToS, the license of any base dataset, and the model-output ToS above. These bind even if you
+   never publish. A `source`/`license` provenance column per row makes an audit tractable.
+
+Write a **data card** (HF dataset README front-matter): license, sources, generation method,
+dedup/decontam done, known biases, intended use.
+
+## Tooling summary
+
+| Tool | Use | Note |
+| --- | --- | --- |
+| distilabel | synthetic data / AI-feedback pipelines | Argilla/HF, v1.x, → `Distiset` |
+| Argilla | human annotation/review UI | vet and label examples |
+| HF `datasets` | load/map/filter/push | the substrate |
+| datatrove / text-dedup | large-scale dedup | when datasketch loop is too small |
+| lm-eval-harness | eval + decontamination utils | report metrics on the clean split |
+| Lilac | dataset exploration/clustering | **OSS archived ~2025-07 [verify]** |