@ericrisco/rsc 0.1.38 → 0.1.40

package/skills/training-data/references/synthesis-dedup-quality.md

@@ -0,0 +1,134 @@
+# Synthesis, dedup, decontamination, quality, licensing
+
+Depth overflow for SKILL.md §3–§7. Library versions and, above all, model licenses change
+monthly — verify at author time; nothing here is a standing legal fact.
+
+## Synthetic data
+
+**Self-Instruct**: hand-write a small seed set of diverse instructions+responses, prompt a
+strong model to generate many more conditioned on the seeds, then filter for validity, dedup,
+and difficulty. **Evol-Instruct**: iteratively rewrite each instruction to be harder
+(add constraints, deepen, broaden) so the corpus spans a difficulty gradient. **AI feedback /
+UltraFeedback-style**: have a model rate candidate responses on axes (helpfulness, honesty,
+correctness) to build preference pairs.
+
+### distilabel skeleton (Argilla / Hugging Face, v1.x — verify)
+
+```python
+from distilabel.pipeline import Pipeline
+from distilabel.steps import LoadDataFromDicts
+from distilabel.steps.tasks import TextGeneration
+# LLM client import paths shift between releases — check current distilabel docs.
+
+with Pipeline(name="self-instruct") as pipe:
+    seeds = LoadDataFromDicts(data=[{"instruction": "Explain X."}, ...])
+    gen   = TextGeneration(llm=...)      # plug an LLM client
+    seeds >> gen
+
+distiset = pipe.run(parameters={...})
+distiset.push_to_hub("me/synthetic-sft")   # pushes data + the pipeline definition + a card
+```
+
+Distilabel serializes the whole pipeline to YAML/JSON and (on push) writes it into the dataset
+card, so a run is reproducible. Output is a `Distiset` (a `datasets.DatasetDict` variant).
+
+### Distillation licensing trap (do this before generating)
+
+Training on *another model's outputs* is governed by that model/provider's terms:
+
+- **Proprietary API outputs** may be contractually barred from training competing models — read
+  the current provider ToS, not a blog.
+- **Open-weight licenses** vary and are **not** all OSI-open: Llama ships under the Meta Llama
+  Community License (with acceptable-use + naming/derivative terms), Gemma has custom Google
+  terms, some models (e.g. Codestral-class) are non-production/research-only, and Qwen license
+  terms vary by size. **Never state a license as bare fact** — open the specific model card,
+  licenses change.
+- Record per-example provenance (which model/source produced it) so an audit can trace it.
+
+## Deduplication
+
+Exact dedup: hash normalized text, drop collisions. Near-dup (the one that matters) —
+**MinHash + LSH** over token/char shingles:
+
+```python
+from datasketch import MinHash, MinHashLSH
+
+def minhash(text, num_perm=128, k=5):
+    m = MinHash(num_perm=num_perm)
+    toks = text.split()
+    for i in range(len(toks) - k + 1):
+        m.update(" ".join(toks[i:i+k]).encode())
+    return m
+
+lsh, keep = MinHashLSH(threshold=0.8, num_perm=128), []
+for i, ex in enumerate(dataset):
+    mh = minhash(ex["text"])
+    if not lsh.query(mh):            # no near-duplicate already kept
+        lsh.insert(str(i), mh); keep.append(i)
+dedup = dataset.select(keep)
+```
+
+For very large corpora use a batched/distributed dedup (e.g. HF `datatrove`, `text-dedup`)
+rather than an in-memory loop. Tune `threshold` on a sample — too high keeps paraphrase dupes,
+too low deletes legitimately similar-but-distinct examples.
+
+## Decontamination (against eval/benchmark test sets)
+
+Leakage of test items into training inflates your eval — you measure memorization. Remove any
+training row that overlaps a test item, by **n-gram overlap** (the common convention is a long
+n-gram, e.g. 13-gram, exact match; also embed-and-threshold for paraphrased leakage):
+
+```python
+def ngrams(text, n=13):
+    t = text.lower().split()
+    return {" ".join(t[i:i+n]) for i in range(len(t) - n + 1)}
+
+contam = set().union(*(ngrams(x["text"]) for x in eval_set))   # build from EVERY eval you report
+clean  = dataset.filter(lambda ex: ngrams(ex["text"]).isdisjoint(contam))
+```
+
+Decontaminate against **every** benchmark you will quote *and* your own private held-out set —
+including any public dataset you synthesized from, since its test split may be in your training
+mix. `lm-eval-harness` and dataset-curation toolkits ship decontamination utilities; prefer a
+maintained one over hand-rolled for scale.
+
+## Quality filtering
+
+**LIMA** (*Less Is More for Alignment*, arXiv 2305.11206): ~1,000 curated examples produced a
+strong instruction-follower — for teaching **behavior/style/format**, curation and diversity
+beat raw volume. Caveat: LIMA is about *alignment*, not injecting large new *knowledge* — a
+broad domain-knowledge shift still wants scale. Practical filter ladder:
+
+1. **Format/structural** — drop empty/truncated turns, malformed JSON, wrong role order, missing
+   final assistant turn (SFT), identical `chosen`/`rejected` (preference).
+2. **Length** — cut extreme-short and runaway-long outliers (often degenerate/repetitive).
+3. **Dedup + decontam** (above).
+4. **Diversity** — embed and cluster; downsample over-represented intents so one task is not 80%
+   of the set. (Lilac was built for exactly this triage — but its OSS repo was archived
+   ~July 2025 after the Databricks acquisition; **[verify]** before depending on it.)
+5. **Scoring** — reward model or LLM-as-judge rates helpfulness/correctness; keep the top slice.
+   Audit the judge: it has position/length/self-preference biases; spot-check by hand.
+
+Always keep a discarded-rows sample so filtering decisions are reviewable, not a black hole.
+
+## Licensing — two separate questions
+
+1. **Your dataset's license** — what you release the JSONL under, and whether you *may* (you
+   cannot relicense others' data by aggregating it).
+2. **Source-usage restrictions** — the terms attached to where each row came from: scraped-site
+   ToS, the license of any base dataset, and the model-output ToS above. These bind even if you
+   never publish. A `source`/`license` provenance column per row makes an audit tractable.
+
+Write a **data card** (HF dataset README front-matter): license, sources, generation method,
+dedup/decontam done, known biases, intended use.
+
+## Tooling summary
+
+| Tool | Use | Note |
+| --- | --- | --- |
+| distilabel | synthetic data / AI-feedback pipelines | Argilla/HF, v1.x, → `Distiset` |
+| Argilla | human annotation/review UI | vet and label examples |
+| HF `datasets` | load/map/filter/push | the substrate |
+| datatrove / text-dedup | large-scale dedup | when datasketch loop is too small |
+| lm-eval-harness | eval + decontamination utils | report metrics on the clean split |
+| Lilac | dataset exploration/clustering | **OSS archived ~2025-07 [verify]** |

package/skills/unsloth/SKILL.md

@@ -0,0 +1,254 @@
+---
+name: unsloth
+description: "Use when fine-tuning an open-weight LLM fast on ONE GPU with low VRAM — Unsloth's FastLanguageModel/FastModel + 4-bit QLoRA + trl SFTTrainer, response-only loss masking via get_chat_template/train_on_responses_only, GRPO reasoning FT, and export to merged-16bit / GGUF / push_to_hub for Ollama or vLLM. Triggers: 'fine-tune Llama/Qwen/Gemma/gpt-oss on a single 16GB/24GB GPU', 'QLoRA OOMs on my GPU, make it fit', 'train ~2x faster with less VRAM', 'why is my loss covering the prompt / how do I mask the user turn', 'save my LoRA as GGUF Q4_K_M for Ollama', 'ajustar un modelo en una sola GPU sense quedar-me sense VRAM'. NOT whether/why/which method to fine-tune or trl/peft theory (that is finetuning), NOT running the exported GGUF locally (that is ollama), NOT serving-engine flags/throughput (that is vllm), NOT building the JSONL dataset (that is training-data)."
+tags: [unsloth, qlora, lora, single-gpu, gguf, fine-tuning]
+recommends: [finetuning, training-data, open-weights, ollama, huggingface]
+origin: risco
+---
+
+# Unsloth — fast, low-VRAM fine-tuning on one GPU
+
+Unsloth is a fine-tuning *backend*: hand-written Triton kernels + a patched LoRA/QLoRA path that make
+`transformers` + `trl` training run faster and fit a much bigger model on a single consumer GPU. You
+reach for it when the *decision to fine-tune is already made* and the problem is now "make this run on
+the one GPU I have." This skill owns the backend + the export mechanics. It does **not** decide
+whether fine-tuning is even the right move (`finetuning`), and it does **not** run the model you
+export (`ollama` / `vllm`).
+
+## Read this first (the two things that bite everyone)
+
+1. **Single-GPU is the free-tier assumption.** The open (Apache-2.0) core is built for one GPU.
+   Multi-GPU / multi-node "works but a better version is coming" per the docs, and the polished
+   multi-GPU + full-finetuning path is gated behind the paid Pro/Enterprise tiers — **verify the
+   current split at docs.unsloth.ai before you promise anyone `torchrun --nproc 8`.** If the plan is
+   truly multi-node from day one, that is an `axolotl`/native-`trl`/`accelerate` job, not this.
+2. **You must mask the prompt or your loss is wrong.** Fine-tuning a chat model means computing loss
+   on the *assistant* turn only. Use Unsloth's `get_chat_template` for the format and
+   `train_on_responses_only` for the mask. Skip it and the model trains on predicting the user's
+   words too — loss looks fine, behaviour is subtly broken (and on some templates you hit a
+   zero-loss trap). This is the single most common mistake; it is section 4 for a reason.
+
+### Version / performance reality (fast-moving — verify at author time)
+
+The headline, straight off docs.unsloth.ai (accessed 2026-07): **~2x faster training with ~70% less
+VRAM, no accuracy loss**, on a single GPU with the free core. Treat that as a *class* of improvement,
+not a contract:
+
+- The baseline is a **standard Hugging Face + FlashAttention-2 QLoRA** pipeline, not "raw PyTorch."
+- Numbers are **model-, GPU-, and config-specific.** Some pages/reviews cite ~60% VRAM; GRPO/RL
+  claims ~80% less VRAM; MoE (e.g. gpt-oss, Qwen3-family MoE) shows much larger multipliers on
+  specific hardware (up to ~7–12x on a B200 in Unsloth's own MoE post). **Do not quote a single
+  number as gospel — cite the docs page you read and hedge.**
+- Licensing/pricing also moves: core is Apache-2.0 and free; a paid Pro tier and an Enterprise
+  (contact-sales) tier add multi-GPU/multi-node, full-parameter training, and faster kernels. **Check
+  unsloth.ai/pricing for the live tiers and figures.**
+
+### Setup
+
+Notebook-centric by design — the fastest path is one of the maintained Colab/Kaggle notebooks
+(`unslothai/notebooks`). Locally:
+
+```bash
+pip install unsloth            # pulls unsloth + unsloth_zoo; expects a recent PyTorch + CUDA
+python -c "import unsloth; print(unsloth.__version__)"
+```
+
+NVIDIA is the first-class target (min ~CUDA-capable GPU, works down to ~a free-Colab T4 for small
+models). AMD (ROCm) and Intel GPU support have landed as install targets — **verify your hardware on
+the docs' requirements page before assuming it works.** Don't pin a brittle version in your head;
+`unsloth` ships frequently — install fresh and read its startup banner (it prints the versions it
+patched).
+
+## Supported models (verify the live list)
+
+Unsloth advertises **500+ models** across text, vision, and TTS/embeddings. Families you can expect
+(confirm the specific checkpoint at docs.unsloth.ai/models — new releases land within days):
+
+- **Text:** Llama, Qwen, Gemma, Mistral/Mixtral, Phi, DeepSeek, GLM, and **gpt-oss** (OpenAI's
+  open-weight MoE).
+- **Vision (VLM):** e.g. Qwen-VL, Gemma vision, Llama-vision — via `FastVisionModel`.
+- Plus TTS and embedding fine-tunes in the notebook zoo.
+
+Prefer Unsloth's **pre-quantized 4-bit repos** (`unsloth/<model>-unsloth-bnb-4bit`) — faster download,
+fewer OOMs. Which base model + **which license** is right for you is an `open-weights` question, not
+this one: **never assert a model's license from memory** (Llama = Meta Community license, Gemma =
+custom terms, gpt-oss/Qwen vary by size) — read the model card.
+
+## Canonical flow
+
+Three steps: **load 4-bit → attach LoRA → SFTTrainer**. `FastModel` is the newer unified loader (text
++ vision); `FastLanguageModel` is the text path and still owns `.get_peft_model`.
+
+```python
+from unsloth import FastLanguageModel
+import torch
+from trl import SFTTrainer, SFTConfig
+from datasets import load_dataset
+
+max_seq_length = 2048  # Unsloth does RoPE scaling internally — pick what you need
+
+# 1) Load a (pre-quantized) base in 4-bit. This is the QLoRA memory win.
+model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name    = "unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit",  # verify current id
+    max_seq_length= max_seq_length,
+    load_in_4bit  = True,   # QLoRA. False + load_in_16bit=True => 16-bit LoRA
+    # load_in_8bit / load_in_16bit / full_finetuning are the other switches
+    # token = "hf_...",     # only for gated repos
+)
+
+# 2) Attach LoRA adapters — you train ~1% of weights.
+model = FastLanguageModel.get_peft_model(
+    model,
+    r = 16,                 # rank: 8/16/32; higher = more capacity, more VRAM
+    lora_alpha = 16,        # a common default is alpha == r (some recipes use 2*r)
+    target_modules = ["q_proj","k_proj","v_proj","o_proj",
+                      "gate_proj","up_proj","down_proj"],
+    lora_dropout = 0,       # 0 is the optimized path
+    bias = "none",          # "none" is the optimized path
+    use_gradient_checkpointing = "unsloth",  # "unsloth" = ~30% less VRAM, fits longer context
+    random_state = 3407,
+)
+
+# 3) Train with trl's SFTTrainer (see section 4 before you call .train()).
+dataset = load_dataset("json", data_files="train.jsonl", split="train")  # your data
+trainer = SFTTrainer(
+    model = model, tokenizer = tokenizer, train_dataset = dataset,
+    args = SFTConfig(
+        max_seq_length = max_seq_length,
+        per_device_train_batch_size = 2,
+        gradient_accumulation_steps = 4,     # effective batch = 2*4
+        warmup_steps = 10,
+        max_steps = 60,                       # or num_train_epochs = 1
+        learning_rate = 2e-4,
+        logging_steps = 1,
+        optim = "adamw_8bit",                 # 8-bit optimizer = more VRAM saved
+        output_dir = "outputs",
+        seed = 3407,
+    ),
+)
+trainer.train()
+```
+
+MoE caveat: 4-bit QLoRA is **not** supported for MoE models yet — load MoE in 16-bit and LoRA the
+`gate_up_proj` / `down_proj` layers. (`load_in_4bit=False` for gpt-oss/Qwen3-MoE.) Verify on
+docs.unsloth.ai/basics/faster-moe.
+
+## Chat templates + response-only loss (do not skip)
+
+Format with Unsloth's template helper — **not** a hand-written string — so the special tokens match
+what the base model was trained on:
+
+```python
+from unsloth.chat_templates import get_chat_template
+
+tokenizer = get_chat_template(tokenizer, chat_template = "llama-3.1")  # match your base model
+# then map your messages -> a "text" column via tokenizer.apply_chat_template(...)
+```
+
+Then wrap the trainer so **loss is computed on the assistant turn only**:
+
+```python
+from unsloth.chat_templates import train_on_responses_only
+
+trainer = train_on_responses_only(
+    trainer,
+    instruction_part = "<|start_header_id|>user<|end_header_id|>\n\n",       # Llama-3
+    response_part    = "<|start_header_id|>assistant<|end_header_id|>\n\n",
+)
+# Gemma-3 would use: instruction_part="<start_of_turn>user\n", response_part="<start_of_turn>model\n"
+```
+
+The `instruction_part` / `response_part` strings are the **template's own turn markers** — they must
+match the chat template you applied, per model. Verify the mask worked before spending GPU-hours:
+
+```python
+# labels are -100 where masked. Decoding the non-masked tokens should show ONLY the answer.
+print(tokenizer.decode(trainer.train_dataset[0]["input_ids"]))
+print(tokenizer.decode([tokenizer.pad_token_id if x == -100 else x
+                        for x in trainer.train_dataset[0]["labels"]]))
+```
+
+More templates, thinking-mode (`enable_thinking`), and the vision path are in
+[references/masking-and-templates.md](references/masking-and-templates.md).
+
+## GRPO / reasoning fine-tuning (brief)
+
+Unsloth supports RL (GRPO and variants) with the same low-VRAM story — it plugs into `trl`'s
+`GRPOTrainer` / `GRPOConfig` and can use a built-in vLLM engine (`fast_inference=True`) for the
+rollout generation. Instead of imitating a target string, GRPO optimizes **reward functions** you
+write (e.g. "answer matches ground truth", "output obeys the `<reasoning>/<answer>` format"). The docs
+cite ~80% less VRAM for GRPO vs a standard setup — verify. This is how you turn a base model into a
+reasoning model on one GPU. The *choice* of SFT vs DPO vs GRPO is a `finetuning` decision; the
+mechanics + a runnable GSM8K reward example live in [references/grpo.md](references/grpo.md).
+
+## Export (the other half of this skill)
+
+After `trainer.train()` you have LoRA adapters. Pick an export by where it's going:
+
+```python
+# A) Merge LoRA into the base at 16-bit — the portable, high-quality artifact (vLLM, re-hosting).
+model.save_pretrained_merged("model_16bit", tokenizer, save_method = "merged_16bit")
+model.push_to_hub_merged("user/model", tokenizer, save_method = "merged_16bit", token = "hf_...")
+
+# B) Keep just the adapters (small, hot-swappable).
+model.save_pretrained_merged("model_lora", tokenizer, save_method = "lora")
+
+# C) GGUF for llama.cpp / Ollama — choose the quant that trades size vs quality.
+model.save_pretrained_gguf("model_gguf", tokenizer, quantization_method = "q4_k_m")
+model.push_to_hub_gguf("user/model-gguf", tokenizer,
+                       quantization_method = ["q4_k_m", "q8_0", "f16"], token = "hf_...")
+```
+
+Then **running** the GGUF is an `ollama` job (`ollama create` from the file, `ollama run`), and
+serving the merged-16bit at scale is a `vllm` job. Quant guidance: **Q4_K_M** is the everyday
+size/quality sweet spot, **Q8_0** near-lossless, **f16** the unquantized ceiling — lower quant = smaller
++ faster but real quality loss. Do **not** merge to 4-bit as your keeper artifact (quality drops;
+it's a niche path). Full export matrix + the Ollama/llama.cpp handoff:
+[references/export.md](references/export.md).
+
+## Guardrails / gotchas
+
+- **Single-GPU assumption (OSS).** Don't design a multi-node run on the free core; verify the paid
+  multi-GPU/multi-node status at docs.unsloth.ai first. `CUDA_VISIBLE_DEVICES` to one GPU if unsure.
+- **No mask = wrong training.** Without `train_on_responses_only`, loss covers the prompt; some
+  templates then show ~0 loss. Always decode-check the labels once.
+- **Wrong chat template = garbage.** The `chat_template` and the mask's `instruction_part`/
+  `response_part` must match the *base model's* markers. Use `get_chat_template`; never hand-roll.
+- **MoE ≠ 4-bit yet.** Load MoE models in 16-bit; QLoRA-4bit is unsupported for them (verify).
+- **GGUF quant is lossy.** Q4_K_M for size, Q8_0/f16 when quality matters. Merged-4bit is discouraged.
+- **`import unsloth` first.** Import it before `transformers`/`trl` so its patches apply; heed the
+  startup banner that prints patched versions.
+- **Numbers drift.** The 2x/70% headline is a *class*, not a guarantee — cite the docs page and hedge.
+
+## Related skills
+
+- **`finetuning`** — the method layer: FT-vs-RAG-vs-prompt, SFT/DPO/GRPO choice, hyperparameters, the
+  backend-agnostic `trl`/`peft` theory. Unsloth is *one fast backend* under it; go there for "should
+  I / how much / which method." This skill is "make it run on my GPU."
+- **`training-data`** — build the JSONL messages / preference pairs you feed the trainer. Data shape
+  and quality live there; this skill assumes you already have a dataset.
+- **`open-weights`** — choose the base model + read its license/size tradeoffs before you fine-tune.
+- **`ollama`** — run the GGUF you export, on one box. Export here, run there.
+- **`huggingface`** — get the base weights and host/push the result; `vllm` serves the merged-16bit at
+  throughput. This skill produces the artifact; those consume it.
+
+## Checklist
+
+- [ ] Fine-tuning is actually the right move and method is chosen (confirmed via `finetuning`).
+- [ ] Base model + license verified on its card (`open-weights`); using a current `unsloth/*-4bit` id.
+- [ ] `from_pretrained(load_in_4bit=True, max_seq_length=…)` → `get_peft_model(r, target_modules, …)`.
+- [ ] `get_chat_template` applied with the model's correct template.
+- [ ] `train_on_responses_only` applied AND the label mask decode-checked (answer-only).
+- [ ] Fits the one GPU (batch × grad-accum, `use_gradient_checkpointing="unsloth"`, `adamw_8bit`).
+- [ ] Exported for the target: merged_16bit (vLLM/re-host) or GGUF Q4_K_M/Q8_0 (Ollama/llama.cpp).
+- [ ] Any speed/VRAM/tier/model claim I stated is hedged + cited to docs.unsloth.ai (not memory).
+
+## References
+
+- [references/masking-and-templates.md](references/masking-and-templates.md) — `get_chat_template`
+  options, per-model `instruction_part`/`response_part` pairs, thinking-mode, the vision path, and the
+  label-mask sanity check.
+- [references/grpo.md](references/grpo.md) — GRPO end-to-end: `GRPOConfig`/`GRPOTrainer`, `vLLM`
+  `fast_inference`, a GSM8K reward-function set, and loss-type/DAPO knobs.
+- [references/export.md](references/export.md) — full export matrix (merged_16bit / lora / merged_4bit
+  / GGUF), quant-method table, `push_to_hub_*`, manual `convert_hf_to_gguf.py`, and the Ollama handoff.

package/skills/unsloth/evals/README.md

@@ -0,0 +1,10 @@
+# Evals for the `unsloth` skill
+
+`cases.yaml` is a routing + capability fixture, not an automated harness. For each `should_trigger`
+prompt, confirm the router picks `unsloth` (fast low-VRAM single-GPU fine-tuning + GGUF export). For
+each `should_not_trigger`, confirm it routes to the named sibling — method choice to `finetuning`,
+dataset to `training-data`, running the GGUF to `ollama`, throughput serving to `vllm`, base-model
+choice to `open-weights` — and not here. For the `capability` case, check the answer covers every
+`must_include` bullet: the from_pretrained→get_peft_model→SFTTrainer flow, response-only masking, GGUF
+export + the ollama handoff, the hedged speed/VRAM claim, and the single-GPU guardrail. No GPUs or
+network needed.

package/skills/unsloth/evals/cases.yaml

@@ -0,0 +1,42 @@
+skill: unsloth
+
+should_trigger:
+  - prompt: "I want to QLoRA fine-tune Llama-3.1-8B but it OOMs on my 16GB GPU — how do I make it fit?"
+    why: Core niche — fast, low-VRAM single-GPU fine-tuning; the fit problem is exactly what load_in_4bit + get_peft_model + adamw_8bit + unsloth gradient checkpointing solve.
+  - prompt: "How do I fine-tune gpt-oss on a single 24GB GPU roughly 2x faster with less VRAM?"
+    why: Names the backend's headline (2x/less-VRAM), a single GPU, and a supported MoE model — squarely this skill.
+  - prompt: "My chat fine-tune's loss looks like it's covering the user turn — how do I mask the prompt so it only trains on the response?"
+    why: The train_on_responses_only / get_chat_template masking problem, the load-bearing gotcha owned here.
+  - prompt: "After training my LoRA with Unsloth, how do I export it to GGUF Q4_K_M for Ollama?"
+    why: Export mechanics (save_pretrained_gguf, quant choice, handoff to ollama) are half of this skill.
+  - prompt: "Turn a base Qwen into a reasoning model with GRPO on one GPU using reward functions."
+    why: GRPO/reasoning FT via GRPOTrainer + fast_inference vLLM, the low-VRAM RL path this skill covers.
+  - prompt: "Vull ajustar un model a una sola GPU sense quedar-me sense VRAM amb Unsloth."
+    why: Catalan; single-GPU low-VRAM fine-tuning with Unsloth by name — the exact trigger.
+
+should_not_trigger:
+  - prompt: "Should I fine-tune, use RAG, or just prompt-engineer for my support bot? And SFT vs DPO?"
+    route_to: finetuning
+    why: The whether/why/which-method decision is the method layer, not the backend that makes it run.
+  - prompt: "How do I format my conversations into a JSONL messages dataset for fine-tuning?"
+    route_to: training-data
+    why: Dataset construction/quality; this skill assumes the dataset already exists.
+  - prompt: "I exported a GGUF — how do I run it locally, write a Modelfile, and size VRAM in Ollama?"
+    route_to: ollama
+    why: Running the exported artifact locally is Ollama's job; Unsloth stops at producing the file.
+  - prompt: "Serve my merged 16-bit model at high throughput with tensor parallelism and continuous batching."
+    route_to: vllm
+    why: Serving-engine flags/throughput are the serving skill, not the training backend.
+  - prompt: "Which open model and license should I pick as my base — Llama vs Qwen vs Gemma?"
+    route_to: open-weights
+    why: Base-model selection + license tradeoffs are the choosing layer, not the fine-tuning backend.
+
+capability:
+  - scenario: "I have one RTX 4090 (24GB). Fine-tune Llama-3.1-8B-Instruct on my chat JSONL with Unsloth so loss is only on the assistant turns, then export a Q4_K_M GGUF to run in Ollama. It keeps OOMing."
+    must_include:
+      - "FastLanguageModel.from_pretrained with load_in_4bit=True and max_seq_length, using a pre-quantized unsloth/*-bnb-4bit base id (verify the current id)."
+      - "get_peft_model with r/lora_alpha/target_modules and use_gradient_checkpointing='unsloth'; plus VRAM levers for OOM (batch size + gradient_accumulation_steps, optim='adamw_8bit')."
+      - "get_chat_template for the correct Llama-3.1 template AND train_on_responses_only with matching instruction_part/response_part so loss is response-only — with a decode/label sanity check."
+      - "Export via save_pretrained_gguf(quantization_method='q4_k_m') and the note that running it is an ollama job (create from the GGUF, run); merge to 16bit for vLLM instead."
+      - "The honesty hedge: the ~2x faster / ~70% less VRAM headline is model/GPU/config-specific vs a HF+FA2 QLoRA baseline — verify current numbers at docs.unsloth.ai and don't quote one number as gospel."
+      - "The single-GPU guardrail (OSS core is single-GPU; multi-GPU/multi-node is the paid/verify-live tier) and the boundary that method choice is finetuning, not this."

package/skills/unsloth/references/export.md

@@ -0,0 +1,75 @@
+# Export: from trained LoRA to a runnable artifact
+
+After `trainer.train()` you hold LoRA adapters on top of a (possibly 4-bit) base. Export depends on
+where it runs next. Tracks docs.unsloth.ai inference-and-deployment pages (accessed 2026-07).
+
+## The matrix
+
+| Target                     | Method                                                        | Notes |
+| -------------------------- | ------------------------------------------------------------- | ----- |
+| vLLM / re-host / portable  | `save_pretrained_merged(dir, tok, save_method="merged_16bit")`| Base + LoRA merged at 16-bit. The high-quality keeper. |
+| Small, hot-swappable       | `save_pretrained_merged(dir, tok, save_method="lora")`        | Adapters only (~MBs). Load onto the base at run time. |
+| Ollama / llama.cpp         | `save_pretrained_gguf(dir, tok, quantization_method="q4_k_m")`| Converts + quantizes to GGUF. |
+| DPO / niche 4-bit inference| `save_method="merged_4bit"`                                   | **Discouraged** — quality loss; only if a tool needs it. |
+
+## Merge to 16-bit (recommended keeper)
+
+```python
+model.save_pretrained_merged("model_16bit", tokenizer, save_method = "merged_16bit")
+# push instead of / in addition to saving:
+model.push_to_hub_merged("user/model", tokenizer,
+                         save_method = "merged_16bit", token = "hf_...")
+```
+
+`merged_16bit` is the format vLLM and most serving stacks expect. Prefer it over `merged_4bit` — merge
+at 16-bit, quantize downstream if you need to.
+
+## LoRA adapters only
+
+```python
+model.save_pretrained("finetuned_lora"); tokenizer.save_pretrained("finetuned_lora")
+# or save_pretrained_merged(..., save_method="lora")
+```
+
+Tiny and hot-swappable (e.g. vLLM LoRA hot-swap). Requires the base at inference time.
+
+## GGUF for Ollama / llama.cpp
+
+```python
+# one quant:
+model.save_pretrained_gguf("model_gguf", tokenizer, quantization_method = "q4_k_m")
+# several at once, pushed to the Hub:
+model.push_to_hub_gguf("user/model-gguf", tokenizer,
+                       quantization_method = ["q4_k_m", "q8_0", "f16"], token = "hf_...")
+```
+
+Quant guide (size vs quality — this is a real tradeoff, not free):
+
+| Method   | ~bits/weight | Use it for |
+| -------- | ------------ | ---------- |
+| `q4_k_m` | ~4.9         | Everyday default: ~half the size for a few % quality loss. |
+| `q5_k_m` | ~5.5         | A notch more quality than Q4 for a bit more size. |
+| `q8_0`   | ~8.5         | Near-lossless; larger. |
+| `f16`    | 16           | Unquantized ceiling; biggest. |
+
+Lower than Q4 (Q2/Q3) drops quality sharply — avoid unless desperate for size.
+
+### Manual conversion (when the helper can't, or you want control)
+
+Merge to 16-bit first, then run llama.cpp's converter:
+
+```bash
+python llama.cpp/convert_hf_to_gguf.py model_16bit \
+    --outfile model-F16.gguf --outtype f16 --split-max-size 50G
+# then quantize with llama.cpp's quantize tool to q4_k_m / q8_0, etc.
+```
+
+## Handoff — this skill stops at the file
+
+- **Run the GGUF locally →** `ollama`. Roughly: `ollama create <name> -f Modelfile` with
+  `FROM ./model-Q4_K_M.gguf`, then `ollama run <name>`. The Modelfile/quant/VRAM details are its job.
+  Unsloth's docs also show `OLLAMA_MODELS=unsloth ollama run merged_file.gguf` for a quick check.
+- **Serve merged-16bit at throughput →** `vllm` (engine flags, batching, LoRA hot-swap).
+- **Host/share the weights →** `huggingface` (the `push_to_hub_*` calls above land there).
+
+Export is the last thing this skill owns; running and serving belong to those siblings.

package/skills/unsloth/references/grpo.md

@@ -0,0 +1,91 @@
+# GRPO / reasoning fine-tuning with Unsloth
+
+Mechanics only. **Whether** to do RL vs SFT vs DPO is a `finetuning` decision. Tracks
+docs.unsloth.ai's RL guide (accessed 2026-07); reward APIs and loss-type flags move — verify.
+
+## The idea
+
+GRPO (Group Relative Policy Optimization) doesn't imitate a target string. It samples several
+completions per prompt, scores each with **reward functions you write**, and pushes weights toward the
+higher-reward samples. That's how you train a model to *reason* (produce correct answers in a required
+format) rather than memorize outputs. Unsloth's win here is memory — the docs cite ~80% less VRAM for
+GRPO vs a standard setup (verify), and it can drive generation through a built-in vLLM engine.
+
+## Load with fast inference
+
+```python
+from unsloth import FastLanguageModel
+
+model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name = "unsloth/Llama-3.1-8B-Instruct",  # verify current id
+    max_seq_length = 1024,          # room for the reasoning trace
+    load_in_4bit = True,
+    fast_inference = True,          # use the built-in vLLM engine for rollouts
+    gpu_memory_utilization = 0.8,   # lower if you OOM
+)
+model = FastLanguageModel.get_peft_model(model, r = 32, lora_alpha = 64,
+    target_modules = ["q_proj","k_proj","v_proj","o_proj",
+                      "gate_proj","up_proj","down_proj"],
+    use_gradient_checkpointing = "unsloth", random_state = 3407)
+# For memory-efficient GRPO with vLLM you may also set:
+#   os.environ["UNSLOTH_VLLM_STANDBY"] = "1"
+```
+
+## Reward functions (GSM8K-style)
+
+Each reward fn takes `completions` (+ `prompts`, `answer`, `**kwargs`) and returns a list of floats,
+one per completion. Combine several — correctness plus format shaping:
+
+```python
+import re
+
+def correctness_reward(prompts, completions, answer, **kw):
+    responses = [c[0]["content"] for c in completions]
+    got = [extract_xml_answer(r) for r in responses]
+    return [2.0 if g == a else 0.0 for g, a in zip(got, answer)]
+
+def strict_format_reward(completions, **kw):
+    pattern = r"^<reasoning>\n.*?\n</reasoning>\n<answer>\n.*?\n</answer>\n$"
+    responses = [c[0]["content"] for c in completions]
+    return [0.5 if re.match(pattern, r, re.S) else 0.0 for r in responses]
+
+def extract_xml_answer(text):
+    return text.split("<answer>")[-1].split("</answer>")[0].strip()
+```
+
+Keep rewards simple and hard to game — a sloppy reward degrades the model (reward hacking).
+
+## Train
+
+```python
+from trl import GRPOConfig, GRPOTrainer
+
+trainer = GRPOTrainer(
+    model = model, processing_class = tokenizer,
+    reward_funcs = [correctness_reward, strict_format_reward],
+    train_dataset = dataset,   # prompts + ground-truth "answer"
+    args = GRPOConfig(
+        learning_rate = 5e-6,
+        per_device_train_batch_size = 1,
+        gradient_accumulation_steps = 4,
+        num_generations = 8,       # completions sampled per prompt (the "group")
+        max_prompt_length = 256,
+        max_completion_length = 512,
+        max_steps = 250,
+        output_dir = "outputs_grpo",
+        # loss_type = "grpo" | "dr_grpo" | "bnpo" | "dapo"
+        # epsilon = 0.2, epsilon_high = 0.28, delta = 1.5   # DAPO / two-sided knobs
+        # mask_truncated_completions = True
+    ),
+)
+trainer.train()
+```
+
+Notes:
+- `num_generations` is the group size — bigger gives a better reward signal but costs generation time
+  and VRAM.
+- Advanced objectives (DAPO, Dr. GRPO) are `loss_type` + `epsilon`/`delta` flags on `GRPOConfig` —
+  verify the current names on the RL guide.
+- Reasoning traces need context headroom: budget `max_prompt_length + max_completion_length` under
+  `max_seq_length`.
+- Export the result exactly like an SFT run — see [export.md](export.md).