@zenera/cli 1.1.24 → 1.1.25

package/templates/editor/.github/skills/zen-finetune/SKILL.md

@@ -1,6 +1,6 @@
 ---
 name: zen-finetune
-description: Fine-tune an agent project against a training set in two stages — first WITHOUT memory, then WITH it. Turn a file of example queries in any format into a cached, classified, complexity-annotated dataset with rubrics; sample a batch uniformly across classes and complexities preferring graded samples. Stage 1 runs the batch cold with `zen run batch` against an empty memory, grades every trajectory with `zen inspect graph`, `zen inspect node` and `zen inspect ask` for rubric compliance, optimality, memory hygiene, fork use and delegation, generalises the findings into prompts, skills and new `agents/<topic>-policy-instructions.md` files, and re-runs cold until every query holds and what the runs write to memory is generalisable. Stage 2 then freezes the prose, merges the last good batch's memory, re-runs the same cases with every item given its own writable copy of that graph, and tunes the memory policy until recall makes the batch dramatically faster and cheaper without changing a verdict. Load before evaluating an agent project against example queries, before acting on "it gets this wrong", when asked to improve prompts or instructions from evidence rather than taste, when building an eval or regression set out of a specification, when optimising memory use or run cost, or whenever a batch of runs has to be graded rather than merely executed.
+description: Fine-tune an agent project against a training set, a batch at a time, and every batch twice over — first with NO memory, then WITH the memory it just built. Read the whole training set, in any format, into a classified, complexity-rated dataset with rubrics; choose up to a limit of cases from it evenly by class, rubric and complex cases first. Batch 1 is a smoke test of 3 cases; each batch that goes smoothly doubles the next, up to a maximum, and each is built when it is needed from new cases plus rechecks of earlier ones — difficult ones first, then complex ones. Each batch runs with an empty memory via `zen run batch`; every trajectory is graded with `zen inspect graph`, `node` and `ask` for rubric compliance, llm-call count, fork use, assumptions, what it committed to memory and delegation; the proposals of every case are merged into edits of prompts, skills, house rules and `agents/<topic>-policy-instructions.md` files, and the same cases run again until they pass. Then what the passing run's cases committed is merged with the last good memory into a candidate, every case runs again from its own copy of it, recall and re-commits are graded the same way, the memory policy is tuned until recall makes the work dramatically cheaper without changing a verdict or re-committing what memory already held — and only then does the candidate become the last good memory and the next batch start. Cases that cause the most trouble go on a difficult list and keep coming back until they are fixed or declared stuck. The agent writes `finetune/README.md` — the plan, a mermaid map of every step, and the progress — before the first run and patches it after every step; `scripts/next.mjs` reads the state off disk so an interrupted session can resume. Load before evaluating an agent project against example queries, before acting on "it gets this wrong", when asked to improve prompts or instructions from evidence rather than taste, when building an eval or regression set out of a specification, when optimising memory use or run cost, or whenever a batch of runs has to be graded rather than merely executed.
 ---
 
 # Fine-tuning an agent project
@@ -10,150 +10,606 @@ made of — the agent prompts, the skills, the house rules in `agents/` — and
 it is tuned against is a dataset of real queries and the trajectories they
 produce. The model is fixed; the instructions around it are the parameters.
 
-The reason it is a loop rather than a review is that an instruction's effect is
-not legible from reading it. A rule that reads as obvious gets ignored; a rule
-nobody expected to matter turns out to be load-bearing. The only way to know
-which is which is to run the queries, read the graphs, change one thing, and run
-them again — which is what everything below is about.
-
-It is two loops, not one: a first stage run entirely **without memory**, which
-tunes the prose until the agent is correct from nothing, and a second stage run
-**with memory**, which tunes recall until the same work costs a fraction of what
-it did. They run in that order and never at the same time.
+It is a loop rather than a review because an instruction's effect is not
+legible from reading it. A rule that reads as obvious gets ignored; a rule nobody
+expected to matter turns out to be load-bearing. The only way to know which is
+which is to run the queries, read the graphs, change one thing, and run them
+again — which is what everything below is about.
+
+## Words
+
+One word per idea, used everywhere — in findings, in the README, in reports.
+When two words mean one thing, readers assume they mean two.
+
+| Word                  | Means                                                                                                                 |
+| --------------------- | --------------------------------------------------------------------------------------------------------------------- |
+| **case**              | one query from the training set, with its rubric if it has one. `zen run batch` calls it an _item_                    |
+| **dataset**           | every case in the training set — `finetune/dataset.json`. Never cut down. Its size is written **T**                   |
+| **limit** (N)         | how many cases this tuning uses — equal to T unless the user named a smaller number                                   |
+| **selection**         | those N cases, chosen evenly by class, rubric and complex cases first — `finetune/selection.json`                     |
+| **batch**             | the cases worked on together until they pass without memory and then with it. See [What a batch is](#what-a-batch-is) |
+| **batch size**        | how many new cases a batch takes: `minBatch` for batch 1, doubling after each smooth batch, up to `maxBatch`          |
+| **recheck cases** (R) | cases from earlier batches, run again in a later one — difficult ones first, then complex ones                        |
+| **run**               | one execution of a batch's cases — a no-memory run or a with-memory run                                               |
+| **no-memory run**     | every case starts from an empty memory, and writes its own; tunes the instructions                                    |
+| **with-memory run**   | every case starts from its own copy of the candidate memory; tunes the memory policy                                  |
+| **proposal**          | one improvement a case's grading suggests — merged with the others before anything is edited                          |
+| **passed**            | a run where every case is right AND its review found nothing left to cut                                              |
+| **concurrency** (C)   | cases executing at the same time inside one run — 16 at most, halved on OOM, never below 4                            |
+| **trajectory**        | what one case did in one run — its graph                                                                              |
+| **difficult case**    | a case that caused real trouble — on `finetune/difficult.json`, retested until `fixed` or `stuck`                     |
+| **candidate memory**  | the last good memory plus what the batch's passing no-memory run committed — `memory.mjs path --candidate`            |
+| **last good memory**  | every passed batch's candidate, in turn — `memory.mjs path`, always whole on disk                                     |
+
+A run lives in `finetune/runs/batch<NN>-<nomem|mem>-run<N>/` —
+`batch03-nomem-run2` is batch 3's second run without memory, `batch03-mem-run1`
+its first run with memory. Inside it, `batch/` is `zen run batch`'s own output
+for that run: the command's name, not this method's.
 
 ```mermaid
 flowchart TD
-    A[training set, any format] --> B[dataset.json<br/>classified · rated · rubrics]
-    B --> C[sample N<br/>uniform across strata]
-
-    subgraph S1["STAGE 1 — no memory: tune the prose"]
-        D[zen run batch<br/>cold, empty memory, still writes]
-        E[grade trajectories<br/>zen inspect]
-        F{every sample holds<br/>and writes generalise?}
-        G[generalise findings<br/>prompts · skills · house rules]
-        D --> E --> F
-        F -- no --> G
-        G --> D
+    A[training set, any format] --> B["dataset.json<br/>every case · classified · rated · rubrics"]
+    B --> C["selection.json<br/>N cases, evenly by class<br/>rubric and complex first"]
+    C --> P["README.md<br/>the plan and the map"]
+    P --> E
+
+    E["batch.mjs next — build batch n<br/>3 new for batch 1, doubling while smooth<br/>+ R recheck from earlier batches"]
+    D[("difficult.json<br/>wrong · regressed · flaky · memory · costly")]
+
+    subgraph NM["1. WITHOUT memory: fix the instructions"]
+        F["zen run batch --memory finetune/empty<br/>every case starts from an EMPTY graph<br/>and writes its own"]
+        G["grade every case: right? what did it cost?<br/>what did it commit? ask where it went wrong"]
+        H{"all right, and nothing<br/>left to cut?"}
+        I["merge every case's proposals<br/>edit prompts · skills · rules<br/>then the SAME cases.json again"]
+        F --> G --> H
+        H -->|"no — at most 4 runs"| I --> F
     end
 
-    subgraph S2["STAGE 2 — with memory: tune recall"]
-        H[merge memory from<br/>the last good cold batch]
-        I[same batch, each item<br/>its own writable copy]
-        J{shorter, still correct,<br/>and did it re-commit?}
-        K[generalise findings<br/>memory-policy-instructions.md]
-        H --> I --> J
-        J -- no --> K
-        K --> I
+    CAND["memory.mjs candidate<br/>last good + what this run's cases committed"]
+
+    subgraph WM["2. WITH memory: make it cheaper"]
+        Y["zen run batch --memory (the candidate)<br/>one COPY per case: it recalls from the copy<br/>and commits into the copy"]
+        N["grade memory use: recalled? cheaper?<br/>re-committed what it knew? ask why"]
+        Q{"still right, much cheaper,<br/>nothing re-committed?"}
+        O["merge the proposals<br/>edit the memory policy<br/>then the SAME cases.json again"]
+        Y --> N --> Q
+        Q -->|"no — at most 4 runs"| O --> Y
     end
 
-    C --> D
-    F -- yes --> H
-    J -- yes --> L[done: commit dataset,<br/>findings and changes]
+    K["memory.mjs checkpoint<br/>the candidate becomes the last good memory"]
+
+    E --> F
+    H -->|yes| CAND --> Y
+    Q -->|yes| K
+    G -.->|"add · fix · stuck"| D
+    N -.-> D
+    D -.->|"open ones are the FIRST<br/>rechecks of every later batch"| E
+    K -->|"cases unused, or difficult open"| E
+    K -->|"every case used, none open"| V["final check<br/>the whole selection at once, last good memory"]
 ```
 
 The dataset is built once and cached. Everything after it runs many times.
 
-## The two stages
+## What a batch is
+
+The selection is not run all at once, for two reasons. After forty trajectories
+nobody reads the fortieth carefully. And after one big round of edits nobody can
+say which edit helped and which hurt. A batch keeps both small enough to
+handle: a few cases, read in full, fixed, and **confirmed** before anything else
+starts.
+
+### How big a batch is
+
+Not fixed. The first batch is **3 cases** (`minBatch`): a smoke test of the
+project, the sandbox, the scripts and the grading, on cases cheap enough to read
+twice. A broken project, a missing key or a wrong rubric costs three trajectories
+to find, not forty.
+
+After that, `batch.mjs next` sizes each batch from how the one before went:
+
+| The previous batch                                                                                      | This batch                           |
+| ------------------------------------------------------------------------------------------------------- | ------------------------------------ |
+| went smoothly — at most 2 no-memory runs, at most 2 with-memory runs, no case put on the difficult list | **twice** its size, up to `maxBatch` |
+| needed more runs, or left a difficult case open                                                         | the **same** size                    |
 
-Fine-tuning is two stages, run in order and never interleaved. They optimise
-different things, they are graded on different evidence, and running them
-together makes both unreadable.
+So a project the prose already fits climbs 3 → 6 → 10 in three batches, and one
+that fights every rule stays small, where each run is cheap and each edit
+attributable. `-m <n>` overrides it for one batch; say why in the README.
 
-|                    | **Stage 1 — without memory**                                                      | **Stage 2 — with memory**                                                                       |
-| ------------------ | --------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- |
-| Phases             | 1 to 7                                                                            | 8                                                                                               |
-| What is tuned      | agent prompts, skills, `agents/instructions.md`, new policy files                 | the memory policy: what is committed, and when it is recalled                                   |
-| Memory at run time | `--memory .finetune/empty` — every item starts empty and **writes its own graph** | `--memory <merged>` — every item starts from **its own copy** of one graph and writes into that |
-| Graded on          | the eight criteria; the trajectory must be right _from nothing_                   | whether recall shortened the trajectory, and what was committed on top of a graph that knew     |
-| The win            | correctness — the agent does the right work for the right reason                  | speed and cost — the same verdicts for dramatically fewer tokens and less wall clock            |
-| Done when          | every sample holds cold, and what each run _wrote_ to memory is generalisable     | every sample still holds warm, materially cheaper, and no recall replaced a live reading        |
+### What is in a batch
 
-**Both stages run the same tool surface, and that is the point.** A batch given
-a memory directory copies it per item and lets the item write — cold from an
-empty directory, warm from the merged graph — so the agent sees the same tools
+There is no plan of batches drawn up in advance. `batch.mjs next` decides each
+one when it is needed, from what has happened so far:
+
+```mermaid
+flowchart LR
+    SEL[("selection.json<br/>class by class<br/>rubric and complex first")]
+    DIF[("difficult.json<br/>open · fixed · stuck")]
+    USED[("cases used by<br/>earlier batches")]
+    NEW["new cases, as many as the size<br/>not used by any batch yet"]
+    RC["up to R recheck cases<br/>taken in this order until R is full"]
+    CJ["cases.json + plan.json<br/>batch n"]
+    RUNS["every run of the batch<br/>no memory and with memory<br/>a COPY of that cases.json"]
+
+    SEL -->|"the next unused, in selection order"| NEW
+    DIF -->|"1. open — worst reason first"| RC
+    DIF -->|"2. fixed — what broke once breaks first"| RC
+    USED -->|"3. complex cases, seeded order"| RC
+    USED -->|"4. the rest, seeded order"| RC
+    DIF -.->|"selection used up: the new<br/>are open difficult cases too"| NEW
+    NEW --> CJ
+    RC --> CJ
+    CJ --> RUNS
+```
+
+A `stuck` case is on none of those paths: it is out of the draw for good.
+
+The selection is already ordered with classes taking turns, and inside a class
+cases with a rubric first and the most complex first, so the first, smallest
+batches are the hard, graded cases of every class — the ones that show the most
+per trajectory read — and the large late batches are the simple ones. With 40
+cases, a recheck of 3, and one case turning difficult in batch 3:
+
+```
+batch 1    3 new                     smoke test; nothing to recheck yet
+batch 2    6 new  + 3 recheck        batch 1 was smooth: doubled; complex first
+batch 3   10 new  + 3 recheck        doubled again, to maxBatch
+batch 4   10 new  + 3 recheck        the difficult case first; size held at 10
+batch 5   10 new  + 3 recheck        ...still first until it is fixed
+batch 6    1 new  + 3 recheck        selection used up after this one
+batch 7   open difficult cases only  + 3 recheck, if any are still open
+```
+
+`batch.mjs` with no arguments says where that stands: cases used, cases left,
+the next size and why, difficult cases open, and the least number of batches
+still ahead. `batch.mjs next -o <run>/cases.json` also writes `plan.json`
+beside it — the size, why, which cases are new and which are rechecks — because
+`cases.json` itself may hold nothing but cases.
+
+The recheck cases are there to **fail**. A rule written for batch 3 is read by
+every agent on every query, and the case it breaks is almost never in batch 3.
+Without rechecks that breakage is found at the final check, five batches and a
+dozen rules later, when nobody can say which rule did it. With them it is found
+in the next batch, when the suspect is the handful of edits since. Complex
+cases come before the rest because they touch the most rules.
+
+A batch is built **once**, for its first no-memory run. Every later run — no
+memory or with memory — gets a **copy** of that `cases.json`, never a rebuild:
+the rechecks depend on the difficult list, which moves, and the copy is what
+makes one run comparable to the next.
+
+### What happens in a batch
+
+Every case of the batch goes through the same fifteen steps, in two halves. The
+first half fixes the instructions with memory out of the picture; the second
+gives back the memory the first half wrote and makes the same work cheaper.
+
+**Without memory** — runs `batch<NN>-nomem-run<N>`:
+
+1. **Run** the batch with no memory: every case starts from an empty graph.
+2. **Analyze** every trajectory: is it right, what did it cost, where is it
+   suboptimal. At each critical point, `zen inspect ask` the model why.
+3. **Read what it committed** to memory: durable facts and pointers, or live
+   readings dressed up as facts?
+4. **Propose** improvements, per case, each citing nodes and its `why` line.
+5. **Merge** the proposals of every case into patterns — one rule per pattern,
+   never one per case.
+6. **Update** the prompts, skills, house rules and policy files — create the
+   ones that do not exist yet.
+7. **Run again**, no memory, the same cases.
+8. **See** whether the edits worked: everything fixed, nothing broken, cheaper.
+   If not, back to 2. At most 4 no-memory runs.
+
+**With memory** — runs `batch<NN>-mem-run<N>`:
+
+9. **Build the candidate memory** from the run that passed step 8: the last good
+   memory, plus what each of its cases committed, merged — `zen memory merge`
+   folds what the graph already held, so the same fact from two cases is one node.
+10. **Run** the batch from the candidate. Every case gets **its own copy** of it,
+    recalls from the copy and commits into the copy — nothing one case writes
+    reaches another, or the candidate.
+11. **Analyze the memory use**, against the no-memory run that passed: did it
+    recall, did recall replace discovery, did it go cheaper, did it re-commit
+    what it had just recalled? `zen inspect ask` where it did not.
+12. **Merge** the proposals into patterns.
+13. **Update** the memory policy — `agents/memory-policy-instructions.md`, and
+    the memory paragraphs of a prompt or skill where a `why` line names one.
+14. **Run again** from the same candidate, the same cases. Back to 11 until it
+    passes. At most 4 with-memory runs.
+15. **Keep it and move on**: the candidate becomes the last good memory, the
+    difficult and complex cases are tracked for rechecks, and the next batch is
+    built.
+
+```mermaid
+flowchart LR
+    N1["nomem run 1"] --> G1{"right? cheap?<br/>commits sound?"}
+    G1 -->|"no"| FIX["merge proposals<br/>edit prose"] --> N2["nomem run 2…4<br/>the SAME cases.json"] --> G1
+    G1 -->|"yes — PASSED"| CAND["candidate =<br/>last good + its commits"]
+    CAND --> M1["mem run 1<br/>a copy per case"] --> G2{"same verdicts?<br/>much cheaper?<br/>nothing re-committed?"}
+    G2 -->|"no"| MFIX["merge proposals<br/>edit memory policy"] --> M2["mem run 2…4<br/>the SAME candidate"] --> G2
+    G2 -->|"yes — PASSED"| CPT["candidate becomes<br/>last good memory"] --> NEXT["next batch"]
+```
+
+What each review looks at, and what a run must show to pass:
+
+- **Correctness first, without memory.** A case that is wrong is the only work
+  that run has: fix it, and grade the cost after the next run.
+- **Cost next, in the same run** once every case is right: llm calls per case,
+  missed forks, discovery it did not need. This is where most of the tuning
+  happens. A batch right on run 1 has **not** passed — it has reached the cost
+  work.
+- **Memory last, with memory.** The same verdicts, a large saving, recall
+  replacing discovery and never a live reading, and no commit of what memory
+  already held.
+- **Each next run is the identical `cases.json`.** The cases did not change, the
+  machine did not change, the memory did not change; only the prose did. So the
+  difference between two runs of one kind _is_ the edit — the one controlled
+  experiment the whole method has.
+- **Passed** is the first run of a kind where every case is right, new and
+  recheck, **and** its review finds nothing more worth changing. That run applies
+  no edits, so nothing unconfirmed is carried forward.
+
+Each kind gets **at most 4 runs** per batch. By then either the rules are wrong
+rather than under-worded, or the batch is as cheap as prose will make it. Stop,
+revert what the last run did not confirm, put every case still wrong on the
+difficult list, and pass it.
+
+### Difficult cases, and why the number of batches is not fixed
+
+A few cases cause most of the trouble, and passing their batch does not make
+them easy. So they go on a list, and the list shapes every later batch.
+
+**What makes a case difficult** — decided mechanically, after grading every run,
+and recorded with `difficult.mjs add <id> --why <reason> --run <run>`:
+
+| Reason      | When                                                                                       |
+| ----------- | ------------------------------------------------------------------------------------------ |
+| `wrong`     | still wrong after a run that tried to fix it — not merely wrong on run 1                   |
+| `regressed` | a recheck case that failed                                                                 |
+| `flaky`     | right in one run, wrong in a later run of the same cases, no rule aimed at it              |
+| `memory`    | with memory: wrong, no cheaper, or re-committing what it recalled — after a run that tried |
+| `costly`    | over 2x the batch's median llm calls in its passing run                                    |
+
+**How they come back.** An open difficult case is the first recheck in every
+later batch until it is fixed; with more open than R, the worst reason goes
+first (`wrong`, `regressed`, `flaky`, `memory`, `costly`), then the fewest
+retries. Once the selection is used up, a batch is made of open difficult cases
+alone, up to the batch size, plus rechecks. A difficult case right in a
+`PASSED` run is marked `difficult.mjs fix <id> --run <run>`, and stays ahead of
+the other rechecks after that — what was hard once breaks again first. A case
+added in a batch also keeps the next batch from growing.
+
+**Stuck.** A case still open after `maxRetries` batches have held it since it was
+added (default 3) is `stuck` — the model's ceiling, a wrong rubric, or a missing
+tool. It is no longer picked, and it is reported at the end with what was tried.
+
+**When the tuning ends.** When every case in the selection has been used and no
+difficult case is open. So it takes at least limit ÷ `maxBatch` batches, plus
+the small ones it starts with, plus as many as its difficult cases need — and
+every difficult case ends `fixed` or `stuck`, so it always ends.
+
+Before grading a batch that holds a difficult case, read its notes
+(`difficult.mjs`) and every earlier `findings.md` that names it: the rules that
+were already tried and did not work are the most useful thing known about it.
+
+## The memory: candidate and last good
+
+No-memory runs still write memory — each case into its own empty graph — and
+what the passing no-memory run of a batch wrote is exactly what its with-memory
+runs should be able to reuse. So when a no-memory run passes, it becomes a
+**candidate**:
+
+```sh
+.github/skills/zen-finetune/scripts/memory.mjs candidate batch03-nomem-run2
+```
+
+That copies the last good memory into a new directory,
+`finetune/memory/candidate-batch03/`, and folds in what each of that run's
+cases committed with `zen memory merge`. Merge drops what the graph already
+holds, so the same endpoint learned by four cases is one node, and a fact an
+earlier batch already knew is not added twice. The candidate is what every
+with-memory run of the batch starts from: `zen run batch --memory` gives **every
+case its own copy**, so a case recalls from its copy, commits into its copy, and
+nothing it writes reaches another case or the candidate. Every with-memory run
+of the batch starts from the same bytes.
+
+When a with-memory run passes, the candidate is kept:
+
+```sh
+.github/skills/zen-finetune/scripts/memory.mjs checkpoint batch03-mem-run1
+```
+
+It renames the candidate to `finetune/memory/after-batch03/` and only then
+repoints `finetune/memory/last-good.json` at it. A merge writes its files one
+after another, so a merge killed half-way can leave a graph whose files
+disagree — but only ever a candidate, never the graph the pointer names. The
+last three graphs are kept to roll back to.
+
+```mermaid
+flowchart TD
+    subgraph P1["the PASSED no-memory run — each case wrote its own graph"]
+        MA["case a<br/>batch/a/memory"]
+        MB["case b<br/>batch/b/memory"]
+        MC["case c — committed nothing,<br/>so there is no directory"]
+    end
+
+    BAD["no-memory runs that did not pass"] -->|"committed under prose<br/>that was then changed"| NIL(["dropped, never merged"])
+
+    LG0[("last good memory<br/>after batch n-1")]
+    MRG["memory.mjs candidate — copy it,<br/>then zen memory merge each case's graph in<br/>what it already held is folded, not added"]
+    CAND[("candidate<br/>batch n")]
+
+    MA --> MRG
+    MB --> MRG
+    LG0 --> MRG --> CAND
+
+    subgraph P2["the with-memory runs read it, and never write to it"]
+        CA["case a: its own copy"]
+        CB["case b: its own copy"]
+        WR["what a case commits stays in ITS copy:<br/>evidence for grading, never merged back"]
+        CA --> WR
+        CB --> WR
+    end
+
+    CAND -->|"zen run batch --memory (the candidate)"| CA
+    CAND --> CB
+    PASS{"a with-memory<br/>run PASSED"}
+    CAND --> PASS
+    PASS -->|"memory.mjs checkpoint"| LG1[("last good memory<br/>after batch n")]
+    LG1 -->|"tuning stops for good"| PROM["promote into the project's memory/"]
+```
+
+`next.mjs` will not let a with-memory run start without a candidate built from
+the run that passed, nor the next batch be built until the candidate is kept.
+So at any moment:
+
+```sh
+.github/skills/zen-finetune/scripts/memory.mjs                    # both, and what built them
+.github/skills/zen-finetune/scripts/memory.mjs path               # the last good memory
+.github/skills/zen-finetune/scripts/memory.mjs path --candidate   # the batch's candidate
+```
+
+names a whole graph holding everything the passed batches learned. If the
+tuning stops for good it is the graph to promote into `memory/`. Only passing
+runs go in: an earlier run of the same batch committed under prose that was then
+changed, often while getting the answer wrong. And a with-memory run's copies
+are never merged back — each holds the whole graph again, and what a case
+committed there is graded, not kept.
+
+A candidate with **0 nodes** means nothing was ever committed: the with-memory
+run would measure nothing. That is a commit-policy finding for the no-memory
+runs, and `memory.mjs candidate` says so.
+
+### What may be compared to what
+
+`report.mjs compare` will diff any two runs, so this is the rule most easily
+broken:
+
+| Comparison                                               | Valid   | Why                                                  |
+| -------------------------------------------------------- | ------- | ---------------------------------------------------- |
+| `batch03-nomem-run2` against `batch03-nomem-run1`        | **yes** | same cases, no memory, prose moved — the measurement |
+| `batch03-mem-run2` against `batch03-mem-run1`            | **yes** | same cases, same candidate, memory policy moved      |
+| `batch03-mem-run1` against the no-memory run that passed | **yes** | same cases, same prose, memory added — what it saves |
+| `batch03-nomem-run1` against `batch02-nomem-run2`        | **no**  | different cases _and_ different prose                |
+
+Never compare two different batches. If a number has to cover the whole tuning,
+it is the per-batch table in the README, read as a trend, not an experiment.
+
+## Why no memory first, then memory — in every batch
+
+The two halves optimise different things and are graded on different evidence,
+so they are never run together. But they are run **batch by batch**, not as two
+long stages: the memory a batch builds is tested while the cases that built it,
+and the rules that shaped it, are still in front of you.
+
+|                    | **No-memory runs**                                                                 | **With-memory runs**                                                                        |
+| ------------------ | ---------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- |
+| What is tuned      | agent prompts, skills, `agents/instructions.md`, policy files — and what to commit | the memory policy: when to recall, when to trust it, what not to commit again               |
+| Memory at run time | `--memory finetune/empty` — every case starts empty and **writes its own**         | `--memory "$(memory.mjs path --candidate)"` — every case starts from **its own copy**       |
+| Graded on          | the eight criteria; the trajectory must be right _from nothing_                    | whether recall shortened the trajectory, and what was committed on top of a graph that knew |
+| The win            | correctness — the right work for the right reason — then cost                      | speed and cost — the same verdicts for dramatically fewer tokens and less wall clock        |
+
+**Both run the same tool surface, and that is the point.** A run given a memory
+directory copies it per case and lets the case write — from an empty directory
+without memory, from the candidate with it — so the agent sees the same tools
 and the same house rules in both. `--memory-read-only` would share one graph
-across the batch instead, but it also clamps every agent's `access` to `read`,
-which removes `memory_commit` from the schema and changes the memory prose the
-model reads. That is two variables moving at once, and a warm round has only one
-number to explain them with. Keep the flag for production-shaped runs where many
-processes genuinely share one graph; do not tune with it.
-
-**Stage 1 writes memory but never reads it.** That is not a side effect to be
-tolerated — it is half of what stage 1 grades. Each cold item commits into its
-own empty graph, so the commits are a clean record of what that run _thought was
-worth keeping_, uncontaminated by anything a previous run left behind. Criterion
-3 grades exactly that: a run that cached today's inbox as a fact has poisoned
-every future run, and a run that learned a new API's shape and committed nothing
-has thrown the round away. Fixing those is stage 1 work, because the merged graph
-stage 2 measures against is built out of these commits. Tuning recall on top of
-commits you have not graded is tuning on sand.
-
-**Stage 2 never reads a cold run's prose problem as a memory problem.** If a
-sample is still failing cold, memory can hide it — the warm run passes, the graph
-is rebuilt one day, and the failure returns with no trace of why. So stage 2 does
-not begin until stage 1 is green. When it does begin, the prompts, skills and
-house rules are frozen: the only artefact stage 2 edits is
-`agents/memory-policy-instructions.md`. Changing both layers at once gives two
-causes for one number.
-
-The expected shape of a successful stage 2 is a large drop, not a small one.
-Cold, every run rediscovers the same endpoints, the same schema, the same layout
-of the same system, every time. Warm, it should recall them and go straight to
-work — fewer llm calls, fewer tool calls, less wall clock, same verdicts. A warm
-round that is only a few percent cheaper has not used memory; it has merely
-carried it.
+across the run instead, but it also clamps every agent's `access` to `read`,
+which removes `memory_commit` from the schema and changes the memory prose. That
+is two variables moving at once. Keep the flag for production-shaped runs; do
+not tune with it.
+
+**No-memory runs write memory but never read it.** That is half of what they
+grade. Each case commits into its own empty graph, so the commits are a clean
+record of what that run _thought was worth keeping_. Criterion 3 grades exactly
+that: a run that cached today's inbox as a fact has poisoned every future run,
+and a run that learned a new API's shape and committed nothing has wasted the
+lesson. Fix those in the no-memory runs, because the candidate is built out of
+these commits.
+
+**With-memory runs never fix a no-memory problem.** A case that is wrong with no
+memory is fixed before the batch reaches memory at all — memory would hide it,
+and it would come back the day the graph is rebuilt. So a with-memory run edits
+only the memory policy. A with-memory finding that needs any other prose is
+noted, its cases go on the difficult list, and the next batch's no-memory runs
+fix it, with those cases as rechecks.
+
+A successful with-memory run is a large drop, not a small one. With no memory,
+every run rediscovers the same endpoints, the same schema, the same layout,
+every time. With memory it should recall them and go straight to work. A
+with-memory run that is only a few percent cheaper has not used memory; it has
+merely carried it.
+
+## The settings
+
+| Setting             | Is                                           | Chosen for                                                           |
+| ------------------- | -------------------------------------------- | -------------------------------------------------------------------- |
+| dataset size (T)    | every case in the training set               | nothing — it is whatever the training set holds                      |
+| **limit** (N)       | cases this tuning uses, `N = T`              | nothing — the whole dataset, unless the user gives a number          |
+| **minBatch**        | new cases in batch 1                         | a smoke test: 3                                                      |
+| **maxBatch**        | the most new cases a batch grows to          | how many trajectories one grading pass can actually read; 8–12       |
+| **recheck** (R)     | earlier cases re-run in each batch           | the regression guard and the difficult-case retest; 2–4              |
+| **concurrency** (C) | the most cases running at once inside a run  | `min(maxBatch + recheck, 16)`; halved on an out-of-memory run, min 4 |
+| **seed**            | makes the recheck order repeatable           | anything; just never change it mid-way                               |
+| **maxRetries**      | batches a difficult case gets before `stuck` | 3                                                                    |
+
+**Batch size is not concurrency**, and they are confused constantly because
+both answer "how many at once". Batch size is how much evidence is collected
+before prose may be edited; concurrency is how many of those cases run at one
+time. A batch of 12 run 4 at a time is perfectly ordinary, and a batch of 3 at a
+concurrency of 13 simply runs all three at once.
+
+**Concurrency is not sized, it is backed off.** Start at
+`min(maxBatch + recheck, 16)`. When a run is out of memory — `report.mjs oom`
+finds an exit 137 — halve it, `C = max(4, floor(C / 2))`, write it to
+`config.json` and run the same cases again. Nothing else changes it: no
+preflight, no measuring, no raising it back. Out of memory at 4 means the
+machine cannot run the batch — stop and tell the user.
+
+Write them down once, in `finetune/config.json`, so every script and every
+resumed session agrees:
 
-## What fine-tuning may change
+```json
+{
+    "datasetSize": 64,
+    "limit": 64,
+    "minBatch": 3,
+    "maxBatch": 10,
+    "recheck": 3,
+    "concurrency": 13,
+    "seed": 1,
+    "maxRetries": 3
+}
+```
 
-| Artefact                                            | Stage  | Changed by this loop                                                  |
-| --------------------------------------------------- | ------ | --------------------------------------------------------------------- |
-| `agents/prompts/<name>.md`                          | 1      | Yes — the usual place a per-agent finding lands                       |
-| `agents/skills/<name>/SKILL.md`                     | 1      | Yes — when the fix is knowledge, not standing policy                  |
-| `agents/instructions.md`                            | 1      | Yes — when the fix binds every agent in the project                   |
-| `agents/<topic>-policy-instructions.md`             | 1      | Yes — new files are created here, one topic per file                  |
-| `agents/memory-policy-instructions.md`              | 1 or 2 | Yes — commit policy in stage 1, recall policy in stage 2              |
-| `agents.yaml`                                       | 1      | Rarely — only when the finding is structural (wrong tools, wrong fan) |
-| `agents/memory-instructions.md` and the other three | —      | **Never.** They are `zen`'s copies; `zen check --fix` overwrites them |
-| The dataset                                         | —      | Only to add samples or fix a wrong rubric — never to make a run pass  |
+`limit`, `minBatch`, `maxBatch`, `recheck` and `seed` decide which cases go into
+which batch. Changing any of them after batch 1 has run changes every batch
+built after it — so fix them first and leave them alone. `concurrency` only ever
+goes down, by halving, after an out-of-memory run; it is the machine, not the
+experiment.
+
+## The README
+
+`finetune/README.md` is the one page a person opens to understand the tuning.
+**It is written by you, not generated**, and it is written for someone who has
+never seen this project: plain words, real numbers, no jargon that is not in the
+[Words](#words) table.
+
+Start from [references/readme-template.md](references/readme-template.md). It has
+every section, an example of each, and a comment saying when that section is
+patched.
+
+### Before the first run: write the plan
+
+Once the settings are fixed and the cases selected, write the README whole:
+
+- **What this is** — one paragraph: what is being tuned, against what, why.
+- **Where we are** — one bold line: batch, run (no memory or with memory), and
+  what is happening.
+- **The plan** — the dataset by class (in the dataset / chosen / with a rubric,
+  from `select.mjs`'s output); what a batch is, how it grows, and its two
+  halves, in short paragraphs and the small loop diagram; how difficult cases
+  come back; the memory; the settings, each with a one-line meaning.
+- **Map** — a mermaid flowchart from the dataset to the final check: one node
+  per batch, each with its two halves — `no memory` and `with memory` — and the
+  last good memory it feeds; draw the batches the growth rule gives if every
+  batch is smooth (3, 6, 10, 10, …). Four classes: `done`, `now`, `ahead`,
+  `failed`. Before the first run, the dataset and selection are `done`, batch 1's
+  no-memory half is `now`, everything else `ahead`. Batches the plan did not
+  foresee — a batch that stayed small, the ones difficult cases add — are
+  drawn when they are built, and a line under the map says what is left: unused
+  cases, open difficult cases.
+- **Batches**, **Difficult cases**, **Log**, **Results so far** — empty tables
+  and headings.
+- **Next** — the first run, in words, then the command.
+
+### After every step: patch it
+
+A step is: a run finishing, a run graded, edits applied, a candidate built, a
+batch passing. After each, patch — do not rewrite — these parts:
+
+| Part                | Patch                                                                                                                                                                                                                     |
+| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **Where we are**    | always                                                                                                                                                                                                                    |
+| **Map**             | always — move node ids between the `class` lines; update labels (`▶ run 2`, `✔ passed · 2 runs`, `✘ 1 unfixed`); add a node for each batch the plan did not foresee and fix the sizes drawn ahead; update the "left" line |
+| **Batches**         | the current batch's row: size, new and recheck ids, runs so far of each kind, status, cases right (`7/8 + 3/3`), llm calls and tokens run to run, and with memory against without                                         |
+| **Difficult cases** | whenever `difficult.mjs` changes: the list as it prints, with each case's reason and what has been tried                                                                                                                  |
+| **Log**             | a new entry, newest first: what ran, what was found, what changed, links to `findings.md` and `changes.md`                                                                                                                |
+| **Results so far**  | when a batch passes: cases used and passing out of N, instruction files changed, memory size and saving, open problems                                                                                                    |
+| **Next**            | always — what `next.mjs` says, in words, then the command                                                                                                                                                                 |
+
+The map must always have **exactly one** `now` node, everything before it
+`done` or `failed`, everything after it `ahead`. A batch that passed with a case
+the prose could not fix is `failed`, not `done`, and its label says how many.
+
+`next.mjs` checks the README's age: if a run's `batch.json`, `findings.md` or
+`changes.md`, or a memory pointer, is newer than the README, its first line says
+`first: patch finetune/README.md`. Do that before anything else. A README that is
+only right at the end is one nobody trusts in the middle, which is the only time
+it is useful.
+
+The Plan section changes only if a setting changes — and then the Log says so
+and why.
 
-Moving a failing sample's rubric to match what the agent did is not tuning. It
-is deleting the test. If a rubric was wrong, say so in the round's notes and fix
-it before the round, not after seeing the result.
+## What fine-tuning may change
 
-## Phase 1 — Build the dataset (shared)
+| File                                                | Run kind    | Changed by this loop                                                  |
+| --------------------------------------------------- | ----------- | --------------------------------------------------------------------- |
+| `agents/prompts/<name>.md`                          | no memory   | Yes — the usual place a per-agent finding lands                       |
+| `agents/skills/<name>/SKILL.md`                     | no memory   | Yes — when the fix is knowledge, not standing policy                  |
+| `agents/instructions.md`                            | no memory   | Yes — when the fix binds every agent in the project                   |
+| `agents/<topic>-policy-instructions.md`             | no memory   | Yes — new files are created here, one topic per file                  |
+| `agents/memory-policy-instructions.md`              | both        | Yes — what to commit without memory; recall and re-commit with it     |
+| the memory paragraph of a prompt or skill           | with memory | Only where an ask's `why` line names it                               |
+| `agents.yaml`                                       | —           | No — a structural finding is an open problem in the README            |
+| `agents/memory-instructions.md` and the other three | —           | **Never.** They are `zen`'s copies; `zen check --fix` overwrites them |
+| The dataset                                         | —           | Only to add cases or fix a wrong rubric — never to make a run pass    |
+
+Moving a failing case's rubric to match what the agent did is not tuning. It is
+deleting the test. If a rubric was wrong, say so in the run's findings and fix it
+before the next run, not after seeing the result.
+
+## Phase 1 — Build the dataset: every case
+
+### All of it
+
+Read **the whole training set** and write **every case** into the dataset. No
+limit, no sampling, no "the first twenty to get going". The dataset is the
+record of everything the project is meant to handle, and it is cached and reused
+by every tuning after this one; cutting it here loses cases for good. How many
+of them this tuning _uses_ is the limit, and the limit is applied in phase 2,
+by `select.mjs`, evenly across classes — never here.
 
 ### The input is whatever the user has
 
 A training set arrives as a section of `SPECIFICATION.md`, a markdown file of
 headed examples, a transcript, a spreadsheet exported to CSV, a folder of
-screenshots with captions. There is no format to parse, so do not try to write a
+screenshots with captions. There is no format to parse, so do not write a
 parser. Read the file, understand which parts are queries, and write out the
 structure yourself. This is the one step in the loop that is a model's job and
-not a script's.
-
-What to look for:
+not a script's. Large sources are read in parts; the dataset is appended to as
+you go, and the count is checked against the source at the end.
 
 | In the source                                                    | In the dataset                                                       |
 | ---------------------------------------------------------------- | -------------------------------------------------------------------- |
-| A heading like `## Query:` and a quoted string                   | One sample, `input` is the string                                    |
+| A heading like `## Query:` and a quoted string                   | One case, `input` is the string                                      |
 | A bullet list under `### Rubric:`                                | `rubric` — **verbatim**, one entry per bullet                        |
 | An embedded image, a `![](...)` link, an audio or PDF attachment | A media part in `input`, path relative to the dataset file           |
 | A paragraph of context before the query                          | `notes`, not `input` — unless the agent is meant to see it           |
 | The expected answer                                              | `expected`, one string; it is context for grading, not a diff target |
-| Nothing but a query                                              | A sample with no `rubric`. Still useful, ranked lower                |
+| Nothing but a query                                              | A case with no `rubric`. Still useful, selected after graded ones    |
 
 Preserve the rubric word for word. A paraphrased rubric grades the paraphrase.
 
 ### The shape
 
-Write `.finetune/dataset.json`:
+Write `finetune/dataset.json`:
 
 ```json
 {
     "version": 1,
     "source": ["SPECIFICATION.md", "docs/examples/day-planning.md"],
     "extractedAt": "2026-03-04T10:12:00Z",
-    "samples": [
+    "cases": [
         {
             "id": "planning-organize-day",
             "class": "planning",
@@ -192,15 +648,15 @@ underscore only, unique across the dataset, and worth making readable:
 
 ### Classify only what the source already tells you
 
-The instruction is "if obvious from the input" and it should be taken literally.
 A class is a distinction the source makes — by section heading, by which agent or
-API the example is about, by task type. Do not invent a taxonomy; six classes
-over forty samples is useful, twenty classes over forty samples is one sample per
-class and stratified sampling degenerates into taking everything. When a sample
-fits nowhere, leave `class` out: the sampler groups those as `unclassified` and
-they still get their share.
+API the example is about, by task type. Do not invent a taxonomy. The limit is
+spread evenly by class, so the classes decide what "a fair share" means: six
+classes over sixty cases is useful, twenty classes over sixty is three cases each
+and the limit spreads too thin to say anything about any of them. A case that
+fits nowhere gets no `class`; it is selected as `unclassified` and still gets its
+share.
 
-### Annotate complexity
+### Rate complexity
 
 Three levels, and rate the _trajectory_ the query demands, not the sentence:
 
@@ -210,138 +666,214 @@ Three levels, and rate the _trajectory_ the query demands, not the sentence:
 | `medium`  | Several calls that depend on each other, or one delegation, or one artefact |
 | `complex` | Fan-out, multi-agent, ambiguity to resolve, or a plan that can go wrong     |
 
-"organize my day" is three words and complex. Complexity is where the interesting
-failures are, which is why the sampler spreads across it rather than letting the
-easy majority set the score.
+"organize my day" is three words and complex. The selection puts the complex
+cases of each class first, so a rating that is too low hides a case in the late
+batches; when unsure, rate up.
 
 ### It is a cache
 
-Extraction is expensive and non-deterministic; re-doing it per round would make
-two rounds incomparable. So `.finetune/dataset.json` is written once and read
-thereafter. Re-extract only when the source files change, and when you do, keep
-the existing ids stable — an id that moves breaks every previous round's
-findings. Commit the dataset. It is the project's eval set from then on.
+Extraction is expensive and non-deterministic; re-doing it per tuning would make
+two tunings incomparable. So `finetune/dataset.json` is written once and read
+thereafter. Re-extract only when the source changes, and keep existing ids
+stable — an id that moves breaks every earlier run's findings. Commit it. It is
+the project's eval set from then on. When done, report the count per class and
+confirm it matches the source.
 
-## Phase 2 — Sample (shared)
+## Phase 2 — Select the cases, write the README
 
-Input is limited: a batch of 100 complex runs is a lot of tokens and a long wait,
-and nothing is learned from the eightieth one that the twentieth did not already
-say. Pick a budget — 10 to 25 is a working round — and spend it evenly.
+### Select
 
 ```sh
-.github/skills/zen-finetune/scripts/sample.sh -n 16 -o .finetune/rounds/r1/cases.json
+.github/skills/zen-finetune/scripts/select.mjs -o finetune/selection.json
 ```
 
-| Flag                   | Effect                                                    |
-| ---------------------- | --------------------------------------------------------- |
-| `-n <N>`               | Budget. Without it, everything, still in stratified order |
-| `--class <name>`       | One class only — for a targeted round after a class fails |
-| `--complexity <level>` | One level only                                            |
-| `--rubric-only`        | Drop samples with no rubric                               |
-| `--seed <n>`           | Deterministic. The same seed picks the same batch         |
-| `-o <file>`            | Where to write; default stdout                            |
-
-The sampler groups the dataset into strata of one class × one complexity, sorts
-rubric-bearing samples to the front of each, and then takes one from each stratum
-in turn until the budget runs out. Three consequences worth knowing:
+| Flag                   | Effect                                                             |
+| ---------------------- | ------------------------------------------------------------------ |
+| `-n`, `--limit <n>`    | How many cases. Default `config.json` `limit`, else all            |
+| `--class <name>`       | One class only — for a targeted pass after a class fails           |
+| `--complexity <level>` | One level only                                                     |
+| `--rubric-only`        | Only cases with a rubric                                           |
+| `--seed <n>`           | Default `config.json` `seed`. The same seed chooses the same cases |
+| `-o <file>`            | Where to write; default stdout                                     |
+
+It chooses from **the whole dataset**, evenly by class:
+
+1. Cases are grouped by class.
+2. Inside a class, cases with a rubric come first, and among those the most
+   complex first — complex, medium, simple — then the cases with no rubric in
+   the same order.
+3. Classes take turns: one case from each class, then a second from each, and so
+   on until the limit is spent.
+
+So a limit of 12 over 4 classes is 3 of each, however lopsided the dataset is;
+a class with fewer cases than its share gives all it has and the others carry on.
+Small classes are therefore over-represented relative to their share of the
+dataset. That is deliberate: a class with three cases is the one most likely to
+be under-instructed. It prints chosen / available per class — put that table in
+the README.
+
+`limit` is `datasetSize` — every case is tuned against. A smaller limit is only
+ever the user's choice: write one only when they give a number, never to save
+cost or time on your own judgement, and then say in the README which classes
+lost the most. A limit below the dataset silently drops cases the project is
+meant to handle, and nothing downstream will notice.
+
+The order `select.mjs` writes is also the order batches take new cases in, so
+every batch is already a mix of classes — and the first, small batches hold the
+graded, complex case of each class. A graded case says _why_ it failed; a
+complex one exercises the most rules per trajectory read. That is where the
+first edits should come from. The simple cases arrive last, in the large
+batches, when the prose has settled and they mostly confirm it.
+
+### Write the README
+
+Now, before anything runs — see [The README](#the-readme).
+
+## Phase 3 — Build a batch, run it without memory (step 1)
 
-- A budget smaller than the dataset still buys a cross-section of it, not the
-  front of the file.
-- Rare classes are over-represented relative to their share. That is deliberate.
-  A class with two samples is the one most likely to be under-instructed.
-- Graded samples come first, so a small budget is spent almost entirely on
-  samples that can say _why_ they failed rather than only _what_ happened.
-
-Keep the seed fixed across the rounds of one tuning session. Changing the sample
-and the instructions at the same time means a difference in the score has two
-possible causes and you cannot tell which.
+```sh
+.github/skills/zen-finetune/scripts/batch.mjs            # progress: used, left, next size, difficult open
+.github/skills/zen-finetune/scripts/batch.mjs next -o finetune/runs/batch01-nomem-run1/cases.json
+```
 
-## Phase 3 — Stage 1: run cold
+| Flag         | Effect                                                        |
+| ------------ | ------------------------------------------------------------- |
+| _(none)_     | Progress: cases used, left, next size and why, difficult open |
+| `next`       | Build the next batch; with `-o`, also `plan.json` beside it   |
+| `-m <size>`  | New cases in this batch, overriding the growth rule — say why |
+| `-r <count>` | Recheck cases; default `config.json` `recheck`                |
+| `-o <file>`  | Where to write; default stdout                                |
+| `--seed <n>` | Default `config.json` `seed`                                  |
 
-**Check the machine before the first cold batch, and again after any change to
-`--concurrency`, to the indexes, or to the podman VM:**
+`next` prints the size and why, which cases are new, which are rechecks and what
+is left after it — put that in the README's batch row. It is built **once**, for
+the first no-memory run; every later run of the batch copies that file.
 
 ```sh
-.github/skills/zen-sandbox-capacity/scripts/preflight_sandbox.sh --concurrency 8 \
-    && zen run batch …
+RUN=finetune/runs/batch01-nomem-run1
+C=$(node -p 'require("./finetune/config.json").concurrency')
+.github/skills/zen-finetune/scripts/batch.mjs next -o "$RUN/cases.json"
+zen run batch \
+    --input "$RUN/cases.json" \
+    --batch-dir "$RUN/batch" \
+    --memory finetune/empty \
+    --concurrency "$C"
 ```
 
-It takes seconds, starts nothing you have to clean up, and exits non-zero when
-the batch cannot fit. Skipping it is how five rounds get spent grading prose
-that was never the cause. Load `zen-sandbox-capacity` if it refuses.
+Then patch the README.
 
-```sh
-cd <project>
-mkdir -p .finetune/rounds/r1
-zen run batch \
-    --input .finetune/rounds/r1/cases.json \
-    --batch-dir .finetune/rounds/r1/batch \
-    --memory .finetune/empty \
-    --concurrency 8
-```
-
-`--memory` naming a directory that does not exist is how a batch is made cold:
-each item gets its own empty graph, the project's real `memory/` is not read and
-not touched, and every item still writes, so what it _chose_ to remember is
-evidence. Both halves matter, and both are graded in this stage: the trajectory
-has to be right starting from nothing, and the commits have to be worth the
-next run's attention. Never point a tuning batch at the project's live memory —
-a warm graph means a run can succeed by recall instead of by instruction, which
-is precisely the thing stage 1 exists to rule out.
-
-Concurrency has two ceilings and the provider is only one of them. The default
-is 16 and the cap is 32, but rate-limit errors turn a graded run into a retried
-one — and the machine's ceiling is usually the lower of the two, because every
-item is a container and the podman VM is not the host. Whichever binds first,
-`--concurrency` is the dial; see `zen-sandbox-capacity` for the other ceiling.
-`stdout` is the batch directory alone, so `BATCH="$(zen run batch …)"` is safe
-to script. `<batch-dir>/README.md` is a live dashboard — open it while the
-batch runs.
+`--memory` naming a directory that does not exist is how a run gets no memory:
+each case gets its own empty graph, the project's real `memory/` is not read and
+not touched, and every case still writes, so what it _chose_ to remember is
+evidence. Never point a tuning run at the project's live memory — a graph with
+content means a run can succeed by recall instead of by instruction, which is
+exactly what a no-memory run exists to rule out.
+
+Concurrency always comes from `config.json` — read it fresh for every run.
+`zen run batch`'s own default is 16 and its cap 32; the tuning never goes above 16. `stdout` is the batch directory
+alone, so `BATCH="$(zen run batch …)"` is safe to script. `<batch-dir>/README.md`
+is a live dashboard of the run — open it while it runs.
 
 What lands:
 
 ```
-.finetune/rounds/r1/batch/
-    batch.json                   roster, counts, timings, memory mode, tokens per model
-    README.md                    the dashboard
-    <id>/output.json             the envelope: run.dir, run.graph, usage, output
-    <id>/workspace/              anything the item wrote
-    <id>/memory/                 what the item committed, if it committed anything
+finetune/runs/batch01-nomem-run1/
+    cases.json                   new + recheck; built once here, copied for every later run
+    plan.json                    size, why, which are new, which are rechecks
+    batch/
+        batch.json               roster, counts, timings, memory mode, tokens per model
+        README.md                the batch's own live dashboard
+        <id>/output.json         the envelope: run.dir, run.graph, usage, output
+        <id>/workspace/          anything the case wrote
+        <id>/memory/             what the case committed, if anything
 ```
 
-What a round cost is in both: `README.md` draws a `Tokens by model` table, and
-`batch.json` carries the same split under `batch.models`. Read it per model — a
-round that moved work onto a cheaper agent shows up there and nowhere else.
+`batch.json` carries tokens per model under `batch.models`. Read it per model — a
+run that moved work onto a cheaper agent shows up there and nowhere else.
 
-An item that fails is data, not an outage: its `output.json` holds
-`{ "ok": false, "id", "error" }` and the rest of the batch still runs. Grade the
-failures too — a crash is a finding.
+A case that fails is data, not an outage: its `output.json` holds
+`{ "ok": false, "id", "error" }` and the rest of the run still goes on. Grade the
+failures too — a crash is a finding. In batch 1 a crash is most likely the
+project, the sandbox or a key, not the prose: that is what the smoke test is
+for. Fix it and run the three cases again before anything is graded.
 
-## Phase 4 — Stage 1: grade the trajectories
+## Phase 4 — Grade without memory (steps 2–4)
+
+### A run with any exit 137 is void, not graded
+
+Before grading anything, check how the run died:
 
 ```sh
-.github/skills/zen-finetune/scripts/collect.sh -d .finetune/rounds/r1/batch
-.github/skills/zen-finetune/scripts/collect.sh -d .finetune/rounds/r1/batch graphs > /tmp/r1.mmd
+.github/skills/zen-finetune/scripts/report.mjs -d "$RUN/batch" oom
 ```
 
-`collect.sh index` prints one line per item — verdict, agent, stop reason,
-tokens, duration, whether it has a rubric, and its run directory.
-`collect.sh graphs` concatenates every item's `graph.mmd` with its id and rubric
-as a header, which is one read instead of N invocations of `zen inspect graph`.
-`collect.sh compare` is the per-sample cost table the round is written up with,
-and it is the last thing this phase does rather than the first — read the
-trajectories before the numbers, or the numbers decide what you look at.
+A case killed at 137 ran out of memory; a case at 124 hit a timeout, which under
+memory pressure is the same fault wearing a different number. Neither produced
+evidence about the prompt. **Do not grade the run, do not change one word of
+instruction on its basis, and do not compare its tokens to anything**. Write
+`## VOID` in its `findings.md`, halve `concurrency` in `finetune/config.json`
+(never below 4), and run a copy of its `cases.json` as the next run. A void run
+does not count toward the run cap. `next.mjs` prints the exact value to set. Out
+of memory at concurrency 4 is the one machine failure that stops the tuning:
+stop and tell the user.
+
+A 124 with no 137 is not void by itself; if timeouts repeat in the next run,
+halve the concurrency the same way.
 
-Then work item by item. Load **zen-inspect** for the mechanics; this is what to
-look for.
+`failures` will not catch this. A case whose command was killed usually
+recovers, answers anyway and is recorded `ok`; a run can read `16 items, 16 ok,
+0 failed` with six of them OOM-killed inside. The kill is in the graph, which is
+why `oom` is a separate mode and runs first.
+
+### Read
+
+```sh
+.github/skills/zen-finetune/scripts/report.mjs -d "$RUN/batch"
+.github/skills/zen-finetune/scripts/report.mjs -d "$RUN/batch" answers
+.github/skills/zen-finetune/scripts/report.mjs -d "$RUN/batch" graphs > /tmp/batch01-run1.mmd
+```
+
+`report.mjs` with no mode prints one line per case — ok, agent, stop reason,
+tokens, seconds, rubric lines. `answers` prints each case's final answer under
+its rubric. `graphs` concatenates every case's `graph.mmd`
+with its id and rubric above it: one read instead of one `zen inspect graph` per
+case. `compare` is the per-case cost table the findings open with, and it is the
+last thing this phase does — read the trajectories before the numbers, or the
+numbers decide what you look at.
+
+**Every question about a run has a command.** Never open `output.json`,
+`batch.json`, `state.json` or a memory's `graph.json` with `node -e`,
+`python -c` or `jq` — the formats are the CLI's, and a guess at one costs a
+probe call to find out it was wrong.
+
+| Question                                               | Command                                                    |
+| ------------------------------------------------------ | ---------------------------------------------------------- |
+| A case's trajectory directory                          | `report.mjs -d "$RUN/batch" paths <id>`                    |
+| What it answered, under its rubric                     | `report.mjs -d "$RUN/batch" answers <id>`                  |
+| What it committed (against the candidate, with memory) | `report.mjs -d "$RUN/batch" commits <id>`                  |
+| Every recall and memory tool step, top recalls         | `report.mjs -d "$RUN/batch" recalls <id>`                  |
+| One memory node in full, with its links                | `zen memory show <node-id> --dir "$RUN/batch/<id>/memory"` |
+| A whole memory                                         | `zen memory ls --dir <dir> --limit 200`                    |
+| What one memory holds that another did not             | `zen memory diff <base> --dir <dir>`                       |
+| One step of a trajectory                               | `zen inspect node <nN> --dir <trajectory dir>`             |
+
+Then work case by case. Load **zen-inspect** for the mechanics,
+**zen-analyze-run** for how to read one trajectory's decisions,
+**zen-topology** for criteria 4 and 5 below, and
+**zen-inspect-ask** before the first `zen inspect ask` - which node, what to
+rule out first, how to word the question. The audit checklist of
+zen-analyze-run (memory used and saved, delegation, forking, tool use, cost) is
+what to look for under criteria 2-8 below, and its section on `zen inspect ask`
+says when to question the model at a critical point. Do not write its full
+report per case - `findings.md` is the report here.
 
 ### The three depths
 
-| Depth                                                           | Answers                                           | Cost           |
-| --------------------------------------------------------------- | ------------------------------------------------- | -------------- |
-| The graph (`graph.mmd`, or `zen inspect graph --dir <run.dir>`) | What happened, in what order, and where it looped | Free           |
-| `zen inspect node n13 --dir <run.dir>`                          | What a step actually said, sent or got back       | Free           |
-| `zen inspect ask <llm_call-id> "…"`                             | Why the model chose this over that                | One model call |
+| Depth                                                           | Answers                                                  | Cost           |
+| --------------------------------------------------------------- | -------------------------------------------------------- | -------------- |
+| The graph (`graph.mmd`, or `zen inspect graph --dir <run.dir>`) | What happened, in what order, and where it looped        | Free           |
+| `zen inspect node n13 --dir <run.dir>`                          | What a step actually said, sent or got back              | Free           |
+| `zen inspect ask <llm_call-id> --question-file <f>`             | Why the model chose this over that - the sentence to fix | One model call |
 
 Stay at the top for as long as it works. The `%%` header rows on the graph carry
 `nodes`, `tokens`, `thinking`, `tools`, `branches`, `compacted` and `why` — and
@@ -349,57 +881,103 @@ the `tools` row is a loop detector: the same tool with the same arguments three
 times is a finding without opening a single node.
 
 Descend to `node` for evidence. A finding that does not cite node ids is an
-impression, and impressions are how a tuning round produces a rule nobody needed.
-
-`ask` is last, and it is worth the call in exactly one situation: the graph shows
-a choice you cannot explain and the node contents do not explain it either. It
-replays one `llm_call` with the same context, no tools, and writes nothing back —
-so the answer is that call's own reasoning, not a reconstruction.
+impression, and impressions are how a run produces a rule nobody needed.
+
+`ask` is for the **critical points** - the `llm_call` where the trajectory
+went wrong: research started without searching memory, recalled memory ignored,
+independent lookups run in series instead of forked, a handoff too early, a
+tool misused or a failing call repeated, a loaded skill not followed. The graph
+and the nodes show **what** it did there; only the replay says **which
+sentence** made it do it, and that sentence is what phase 5 will change.
+
+Load **zen-inspect-ask** before the first ask, then follow it for every ask:
+rule out the cheap causes first
+(tool not offered, instruction file absent, skill not loaded, context
+compacted), pick the `llm_call` that made the choice, and word the question for
+a model that sees only its own context - no node ids, "why didn't you ...", the
+fixed answer shape. Never ask every call. Across a batch, ask **once per
+pattern, not once per case**: the clearest case of the pattern, at the call
+that made the choice. A replay carries the whole conversation up to that call,
+so one ask at the right node covers the steps before it.
 
 ```sh
-RUN="$(jq -r .run.dir .finetune/rounds/r1/batch/planning-organize-day/output.json)"
-zen inspect graph --dir "$RUN"
-zen inspect node n12 n13 --dir "$RUN" --part request --full
-zen inspect ask n13 "why did you call /mail/list twice instead of paging the first result?" --dir "$RUN"
+DIR="$(.github/skills/zen-finetune/scripts/report.mjs \
+    -d finetune/runs/batch01-nomem-run1/batch paths planning-organize-day | cut -f2)"
+zen inspect graph --dir "$DIR"
+zen inspect node n12 n13 --dir "$DIR" --part request --full
+# .tmp/ask/planning-organize-day/n13-paging.md, written with your file tool:
+#   Why did you call /mail/list a second time instead of paging the first result?
+#   (then the answer shape from zen-inspect-ask)
+zen inspect ask n13 --question-file .tmp/ask/planning-organize-day/n13-paging.md --dir "$DIR"
 ```
 
-Treat the answer as testimony, not ground truth. It is the model's account of the
-model's choice; it is useful because it names which instruction it was reading,
-and that is the instruction you are about to change.
+Treat the answer as testimony, not ground truth. It is useful because it names
+which instruction the model was reading, and that is the instruction you are
+about to change. Grep every quote it gives in the node's `request` before using
+it; an answer whose quote is not there is discarded.
 
 ### The criteria
 
-Grade each sample against all of these. The first is the user's; the rest apply
+Grade each case against all of these. The first is the user's; the rest apply
 whether or not a rubric was written.
 
+They are listed in the order to act on them. **Correctness gates everything**: a
+run where a rubric line is missed, or where a recheck case regressed, has no
+optimisation work in it at all — fix the correctness and grade the cost next run.
+Once every case is right, criteria 2 and 4 become the objective **in that same
+run** — not in some later pass — and in that order: fewer llm calls first,
+because it is the number that moves most and is read straight off the graph
+header; then fan-out, because a fork turns a long trajectory into a wide one.
+The [cost review](#the-cost-review) below is how that is written down.
+
 **1. Rubric compliance.** Every line of the rubric, checked against the graph.
 Mark each `met` / `partial` / `missed` and cite the node. A rubric line naming an
 API call is met by the call appearing with sensible arguments — not by the answer
-claiming it was made.
-
-**2. Optimality.** Did it take the short path? Symptoms, all visible in the
-graph: the same read twice; a search that could have been one call made as four;
-a tool loop; thinking tokens spent re-deriving something already in context; a
-compaction that a shorter trajectory would not have needed. Record `nodes` and
-`tokens` from the header — they are the round-over-round number.
-
-**3. Memory hygiene.** Read `<id>/memory/` and the commit nodes. The rule is
+claiming it was made. **Recheck cases are graded exactly as hard as new ones.**
+One of them failing ends the batch's progress: a rule written since is the
+suspect, and it is reverted or narrowed before anything else is written.
+
+**2. Optimality — the short path, and the assumptions that shorten it.** Three
+separable faults, all visible in the graph:
+
+- _Repeated work._ The same read twice; a search that could have been one call
+  made as four; a tool loop; thinking tokens spent re-deriving something already
+  in context; a compaction a shorter trajectory would not have needed.
+- _Discovery it did not need._ A probe for something it had already been told —
+  it lists a directory to find a file the skill names, queries to learn a shape
+  the prompt states, or reads a config to confirm a default. This is the
+  assumption fault, and it is nearly always fixed in a skill rather than a
+  prompt: the agent guessed conservatively because nothing authoritative said
+  otherwise.
+- _Assumptions that were wrong._ The mirror image, and worse — it went straight
+  to an answer it had no grounds for. That is criterion 7, not this one.
+
+Record `nodes`, `llm` and `tokens` from the header. The llm-call count is the
+run-to-run number; tokens are the one that swings.
+
+**3. Memory hygiene — what it committed (step 3).** Start from
+`report.mjs -d "$RUN/batch" memory` — commits per case — then
+`report.mjs -d "$RUN/batch" commits` — every node each case committed, kind,
+audience and text — and the commit nodes. Every commit becomes part of the candidate
+the with-memory runs start from, so this is graded as hard as the answer. The rule is
 commit what _produces_ the answer, not the material it was made from: a file
-becomes a pointer, a call becomes the operation. So a run that stored the
+becomes a pointer, a call becomes the operation. A run that stored the
 _contents_ of `/mail/list` — today's inbox, a price, a status, a queue depth —
 has cached a live reading as if it were a fact, and every later run is wrong.
 What it should have stored is that the mail list is at that endpoint, what shape
 it returns and what the useful filter was. The carve-outs are narrow: an answer
 the agent assembled itself, and a source that genuinely cannot change. Also
-check the other direction — a run that committed nothing at all when it learned
-the shape of a new API has thrown the round away.
+check the other direction — a run that committed nothing when it learned the
+shape of a new API has wasted the lesson, and its with-memory run will have
+nothing to recall.
 
 **4. Fork.** Independent work should fan out; dependent work must not. Look for:
 serial calls with no data dependency between them (missed fork); branches that
 read each other's output (a fork that should have been a sequence); a fork with
 one branch (that is delegation, and should be written as delegation); branches
 that all do the same thing with the same inputs. `branches` in the header and the
-shape of the graph say all of it.
+shape of the graph say all of it. The signature table in **zen-topology** §7 says
+where each fix goes, and a missed fan-out is priced in elapsed time, not tokens.
 
 **5. Delegation.** Did it hand off to the right agent, and did it hand off at
 all? Both failures are common: doing a specialist's job inline because the prompt
@@ -419,70 +997,86 @@ be wrong by luck next time.
 **8. Failure handling.** When a tool returned an error, did it adapt, or did it
 retry the identical call? The `tools` header row finds this in one line.
 
-### Write it down
+### Write it down: findings.md
 
-One file per round, `.finetune/rounds/r1/findings.md`. It opens with a **verdict
-block** — a plain-language answer to "are we making progress", what was changed
-to get there, and the per-sample table that either supports that or does not —
-and only then the spec and the per-sample grading.
+One file per run, `finetune/runs/batch01-nomem-run1/findings.md`. It opens
+with a **verdict block** — a plain answer to "are we making progress", what was
+changed to get there, and the per-case table that supports it or does not — and
+only then the per-case grading.
 
-Write the verdict block last and put it first. Somebody who has not read the
-earlier rounds should learn from the top of the page whether the work is moving;
-somebody who has should be able to stop after it.
+Write the verdict block last and put it first. Someone who has not read the
+earlier runs should learn from the top of the page whether the work is moving;
+someone who has should be able to stop after it.
 
-The table is not typed by hand. `compare` builds it against the previous round
-and prints markdown ready to paste:
+The table is not typed by hand. `compare` builds it against the previous run and
+prints markdown ready to paste:
 
 ```sh
-.github/skills/zen-finetune/scripts/collect.sh \
-    -d .finetune/rounds/r2/batch -p .finetune/rounds/r1/batch compare
+.github/skills/zen-finetune/scripts/report.mjs \
+    -d finetune/runs/batch03-nomem-run2/batch \
+    -p finetune/runs/batch03-nomem-run1/batch compare
 ```
 
 Every column comes from the run itself — tokens from `output.json`, llm calls,
-tool calls and forks from the graph's own `%% nodes` header row, time from
-`durationMs`. The one column it leaves as `?` is **verdict**, because whether a
-sample is right is the judgement the previous phase made and no script can read
-it off a trajectory. Fill that column in from the rubric grading below.
-
-Above the table it prints a `memory:` line for this round and, when comparing, a
-`prev memory:` line for the other. Both belong in `findings.md` as printed. They
-are not a footnote: the same numbers mean opposite things cold and warm, and a
-round whose report does not say which cannot be compared to any other round
-later. `zen run batch --memory .finetune/empty` records in `batch.json` as mode
-`copied` exactly like a warm run does, so do not read the mode out of the JSON
-by eye — `collect.sh` decides it by whether the source is a real graph.
+tool calls and forks from the graph's own `%% nodes` header row, memory reads
+and commits from its `%% tools` row, the `known` count from what the commit
+results said, time from `durationMs`. The one column it leaves as `?` is
+**verdict**: whether a case is right is the judgement grading made, and no
+script can read it off a trajectory.
+
+Above the table it prints a `memory:` line for this run and, when comparing, a
+`prev memory:` line for the other. Paste both. The same numbers mean opposite
+things with and without memory, and `batch.json` records
+`--memory finetune/empty` as mode `copied` exactly like a with-memory run — so
+do not read the mode out of the JSON by eye; `report.mjs` decides it by whether
+the source is a real graph.
 
 ```md
-# r2 — COLD, --concurrency 7
+# batch03-nomem-run2 — NO MEMORY, --concurrency 7
 
-## PROGRESS — one sample fixed, none broken
+## PROGRESS — one case fixed, none broken
 
-memory: COLD — every item started from an empty graph and wrote its own (.finetune/empty)
-prev memory: COLD — every item started from an empty graph and wrote its own (.finetune/empty)
+memory: NO MEMORY — every case started from an empty graph and wrote its own (finetune/empty)
+prev memory: NO MEMORY — every case started from an empty graph and wrote its own (finetune/empty)
 
-**Changed since r1:** one paragraph in `agents/skills/rag_search/SKILL.md`
+**Changed since run 1:** one paragraph in `agents/skills/rag_search/SKILL.md`
 (+11 lines) — a question naming two products is two lookups, and the worst
 verdict decides.
 
-| sample                           | verdict | tokens                   | llm calls | tool calls | forks   | time               |
-| -------------------------------- | ------- | ------------------------ | --------- | ---------- | ------- | ------------------ |
-| info-upgrade-pair-4100-4110      | ✘ → ✔   | 923k → 2.04M (+121%)     | 23 → 49   | 24 → 47    | 1 (=)   | 74s → 128s (+73%)  |
-| info-intel-4120-compat-420       | ✔ → ✔   | 676k → 1.09M (+61%)      | 24 → 33   | 21 → 30    | 1 (=)   | 70s → 79s (+13%)   |
-| info-malware-profiles-medium-412 | ✘ → ✘   | 3.40M → 1.00M (−70%)     | 65 → 41   | 61 → 36    | 2 (=)   | 222s → 107s (−52%) |
-| code-can-vms-talk                | ✔ → ✔   | 1.20M → 4.80M (+302%)    | 35 → 124  | 32 → 113   | 1 → 5   | 93s → 463s (+398%) |
-| **total**                        | 14 → 15 | **19.7M → 24.3M (+23%)** | 585 → 717 | 517 → 637  | 27 → 31 | 1602s → 1922s      |
+| case                             | verdict | tokens                   | llm calls | tool calls | forks   | memory reads | commits (known) | time               |
+| -------------------------------- | ------- | ------------------------ | --------- | ---------- | ------- | ------------ | --------------- | ------------------ |
+| info-upgrade-pair-4100-4110      | ✘ → ✔   | 923k → 2.04M (+121%)     | 23 → 49   | 24 → 47    | 1 (=)   | 1 (=)        | 1 → 2           | 74s → 128s (+73%)  |
+| info-intel-4120-compat-420 _(r)_ | ✔ → ✔   | 676k → 1.09M (+61%)      | 24 → 33   | 21 → 30    | 1 (=)   | 1 (=)        | 1 → 1           | 70s → 79s (+13%)   |
+| info-malware-profiles-medium-412 | ✘ → ✘   | 3.40M → 1.00M (−70%)     | 65 → 41   | 61 → 36    | 2 (=)   | 2 → 1 (−1)   | 3 → 1           | 222s → 107s (−52%) |
+| code-can-vms-talk _(r)_          | ✔ → ✔   | 1.20M → 4.80M (+302%)    | 35 → 124  | 32 → 113   | 1 → 5   | 1 (=)        | 0 → 0           | 93s → 463s (+398%) |
+| **total**                        | 14 → 15 | **19.7M → 24.3M (+23%)** | 585 → 717 | 517 → 637  | 27 → 31 | 18 → 17      | 21 → 19         | 1602s → 1922s      |
+
+_(r)_ marks a recheck case. Both still pass, so the rule broke nothing earlier —
+which is the first thing this table has to answer.
 
 **Effect:** `info-upgrade-pair-4100-4110` is fixed and nothing regressed. The
-+121% on it is the second lookup the rule asks for, which is the cost of the
-fix and not a surprise.
++121% on it is the second lookup the rule asks for — the cost of the fix.
 
 **Not the rule's doing:** `code-can-vms-talk` alone is +3.6 M of the +4.5 M, and
 124 llm calls against 35 is a loop, not variance — the rule cannot reach that
-sample. Excluding it the round moves +5%, inside the noise floor.
+case. Without it the run moves +5%, inside the noise floor.
+
+**Verdict:** keep. One case fixed for eleven lines. Fix the loop next run.
 
-**Verdict:** keep. One sample fixed for eleven lines. Open the loop next round.
+machine: concurrency 8 · vm 12.0 GiB / 6 cpus · host 36 GiB
 
-machine: vm 12.0 GiB / 6 cpus · container 4096 MiB · index docs 3.2 GiB · host 36 GiB
+## Cost review
+
+llm calls per case: median 33; above 1.5x: `code-can-vms-talk` 124 — loop, see below.
+
+- repeated work: `code-can-vms-talk` n40–n118 re-runs `rag_search` with the same
+  query 27 times after an empty result; `info-malware-profiles-medium-412` n12
+  and n19 are the same lookup. Pattern (2 cases + 1 in batch 1): no rule says
+  what to do on an empty result → rule for the rag_search skill.
+- missed forks: `info-upgrade-pair-4100-4110` n8–n31 looks the two products up
+  one after the other; they are independent → fork rule in the researcher prompt.
+- unneeded discovery: none — checked all four; every listing (n5, n6, n4, n7) is
+  of a directory the skill does not name.
 
 ## planning-organize-day — partial
 
@@ -497,504 +1091,701 @@ machine: vm 12.0 GiB / 6 cpus · container 4096 MiB · index docs 3.2 GiB · hos
 - memory: committed the body of the 09:00 stand-up invite as a fact (3) — n18
 - fork: n9 and n7 are independent and ran in sequence (4)
 - delegation: none needed
+- why (ask n6, fork): steered-by [system prompt: planner.md › Method]
+  "Work through the request one step at a time" — verified in the request
+- proposals:
+    - P1 (4) planner.md › Method: narrow "one step at a time" to dependent steps;
+      independent reads go out together
+    - P2 (3) memory-policy: commit where a listing lives and how it is filtered,
+      never what it listed
 ```
 
-Cite the criterion number. The next phase reads down the column.
-
-The heading carries the verdict in one word, so a folder of rounds shows the
-shape of the work without anything being opened. The round's own title carries
-`COLD` or `WARM` for the same reason — `# r3 — WARM, --concurrency 8`. Use
-exactly one of:
-
-| Heading      | Means                                                              |
-| ------------ | ------------------------------------------------------------------ |
-| `PROGRESS`   | at least one sample fixed, none broken                             |
-| `NO CHANGE`  | the round ran clean and moved nothing — the change did not work    |
-| `REGRESSION` | a sample that passed now fails, whatever else improved             |
-| `MIXED`      | something fixed and something broken in the same round             |
-| `VOID`       | any exit 137, or the machine spec changed — nothing here is graded |
-| `VERIFIED`   | a confirmation round after both stages ended — nothing was changed |
-
-Five rules keep the block honest, and each is a round that was wasted once:
-
-- **Say whether memory was used, in the heading and in the `memory:` line.** A
-  warm round graded as a cold one reads as an agent that knew things it was
-  never told, and a cold round compared against a warm one attributes to a
-  prompt edit what was really recall. Both lines are printed by `compare`; paste
-  them. If `compare` warns that the two rounds ran on different memory, either
-  that _is_ the measurement (phase 8) and the heading must say so, or the wrong
-  `-p` was given.
+Cite the criterion number. The next phase reads down the column. A `why` line
+is the answer of an `ask` at a critical point: the node asked, the finding it
+explains, and the `steered-by` / `should-have-applied` / `missing` citation,
+marked verified or not.
+
+Each case ends with its **proposals** (step 4): what that case alone says
+should change, each with the criterion, the file and the sentence its `why` line
+named. They are not edits. Phase 5 merges them across every case of the run —
+and the earlier runs' — into patterns, and only a pattern becomes an edit.
+
+The `##` heading carries the verdict in one word, so a folder of runs shows the
+shape of the work without anything being opened. The `#` title carries the run
+name and `NO MEMORY` or `WITH MEMORY`. Use exactly one of:
+
+| Heading      | Means                                                                         |
+| ------------ | ----------------------------------------------------------------------------- |
+| `PROGRESS`   | at least one case fixed, none broken                                          |
+| `NO CHANGE`  | the run was clean and moved nothing — the change did not work                 |
+| `REGRESSION` | a case that passed now fails, whatever else improved                          |
+| `MIXED`      | something fixed and something broken in the same run                          |
+| `CORRECT`    | every case right, but the review found something to cut                       |
+| `PASSED`     | **this half of the batch has passed** — every case right, nothing left to cut |
+| `VOID`       | any exit 137 — halve concurrency, re-run the same cases; not graded           |
+| `VERIFIED`   | the final check: nothing regressed and nothing was changed                    |
+
+`PASSED` is the one the tooling reads: a no-memory run that says `PASSED` sends
+the batch to its candidate memory, and a with-memory run that says it sends the
+batch on. Write it only when every case in the batch is right, recheck cases
+included, **and** the run's review found nothing worth changing — the
+`## Cost review` without memory, the `## Memory review` with it — so the run
+that says `PASSED` applies no edits. `next.mjs` refuses a `PASSED` from a run
+that has a `changes.md` (its edits are unconfirmed) or no review section. A run
+that is all correct with something to cut is `CORRECT`, not `PASSED`.
+
+### The cost review
+
+Every run where all cases are right has a `## Cost review` section, straight
+after the verdict block, as in the example above. It is what makes the cost work
+happen rather than be skipped, so it has a fixed shape:
+
+- **llm calls per case**: the median, and every case above 1.5x it with the
+  reason it cost that much, citing nodes.
+- **repeated work**: the same call with the same arguments twice, a loop, a
+  search made as four calls that could have been one — the pattern, or `none`.
+- **missed forks**: independent calls or sub-tasks run one after another — the
+  pattern, or `none`.
+- **unneeded discovery**: listing, probing or reading to learn something a prompt
+  or skill could simply have stated — the pattern, or `none`.
+
+`none` needs its evidence: which trajectories were checked and what was found in
+place of the fault. A batch averaging 40+ llm calls per case almost never has
+`none` on all three, and "every case was right" is not evidence about cost. Each
+pattern found is generalised and applied exactly like a correctness finding
+(phases 5 and 6), and the next run confirms it.
+
+Five rules keep the block honest; each is a run that was wasted once:
+
+- **Say which memory the run had, in the title and in the `memory:` line.** A
+  with-memory run graded as a no-memory one reads as an agent that knew things it
+  was never told. If `compare` warns that the two runs had different memory,
+  either that _is_ the measurement (a with-memory run against the no-memory run
+  that passed) and the title says so, or the wrong `-p` was given.
 - **The verdict column decides, not the token column.** Tokens swing ±2× between
   identical runs of this kind of project. Say what noise floor you measured and
   treat anything under it as unchanged — an aggregate move of 20% is usually
   nothing.
-- **Localise an aggregate change before attributing it to the edit.** That is
-  what the per-sample rows are for. Growth concentrated in the samples the rule
-  touches is the price of the rule; growth spread evenly across all of them is
-  prose leaking into every agent, which is a regression even when every sample
-  still passes.
-- **Read the call columns, not only the tokens.** They are the ones that say
-  _why_: tool calls up with tokens is more work, tokens up with tool calls flat
-  is a longer prefix, and llm calls at three times the round before is a loop
-  that no amount of token arithmetic will name.
-- **Name the fixed and broken samples.** "14 → 15" is not actionable and
-  `info-upgrade-pair-4100-4110` is. A round that fixes one and breaks one is
+- **Localise an aggregate change before attributing it to the edit.** Growth
+  concentrated in the cases the rule touches is the price of the rule; growth
+  spread evenly across all of them is prose leaking into every agent, which is a
+  regression even when every case still passes.
+- **Read the call columns, not only the tokens.** Tool calls up with tokens is
+  more work; tokens up with tool calls flat is a longer prefix; llm calls at
+  three times the previous run is a loop that no token arithmetic will name.
+- **Name the fixed and broken cases.** "14 → 15" is not actionable and
+  `info-upgrade-pair-4100-4110` is. A run that fixes one and breaks one is
   `MIXED`, not a draw — the broken one is the more valuable finding, because it
   is almost always the new instruction.
 
-`Verdict:` ends in keep or revert, and the ratio to judge it by is prose bought
-per sample fixed. A round that added forty lines and fixed nothing is reverted,
-not built on.
-
-For a **warm** round (phase 8) the same command compares against the cold round
-the memory was merged from, and the heading answers the only question that phase
-asks — whether memory shortened the work:
-
-```sh
-.github/skills/zen-finetune/scripts/collect.sh \
-    -d .finetune/rounds/r3/batch -p .finetune/rounds/r2/batch compare
-```
-
-A warm round that costs _more_ than its cold round is `NO CHANGE` at best:
-memory is being loaded and not used, and the fix is an instruction about when to
-recall. A warm round that costs dramatically less has one thing left to check
-before it counts — that the saving is recall replacing _discovery_ and not
-recall replacing a live read. Open a sample's trajectory and confirm the calls
-that must hit the live system still did.
+`Verdict:` ends in keep or revert, judged by prose added per case fixed. A run
+that added forty lines and fixed nothing is reverted, not built on.
 
-The `machine:` line is not decoration. Token counts and wall clocks are only
-comparable between rounds **measured on the same spec** — a resized VM
-invalidates a baseline exactly as a prompt edit does, and a round with no spec
-recorded cannot be compared to anything later. `preflight_sandbox.sh --json`
-prints every figure on that line.
+The `machine:` line is not decoration. Tokens and wall clock are only comparable
+between runs **on the same spec** — a resized VM invalidates a baseline exactly
+as a prompt edit does. Concurrency is `batch.concurrency` in the run's
+`batch.json`; `podman info` gives the VM figures.
 
-### A round with any exit 137 is void, not graded
+### Update the difficult list
 
-Before grading anything, check how the round died:
+After every graded run, before anything else is written, apply the table in
+[Difficult cases, and why the number of batches is not fixed](#difficult-cases-and-why-the-number-of-batches-is-not-fixed) to every
+case in it — mechanically, not by feel:
 
 ```sh
-.github/skills/zen-finetune/scripts/collect.sh <batch-dir> oom
+D=.github/skills/zen-finetune/scripts/difficult.mjs
+$D add planning-organize-day --why wrong --run batch03-nomem-run2 \
+    --note "lists mail and calendar, never proposes a schedule; rule in planner.md did not move it"
+$D add code-can-vms-talk --why costly --run batch03-nomem-run2 --note "124 llm calls, rag_search loop"
+$D add info-intel-4120-compat-420 --why memory --run batch03-mem-run2 \
+    --note "recalls the endpoint, then lists the directory anyway; recall rule did not move it"
+$D fix info-upgrade-pair-4100-4110 --run batch05-nomem-run2
+$D
 ```
 
-An item killed at 137 ran out of memory; an item at 124 hit a timeout, which
-under memory pressure is the same fault wearing a different number. Neither
-produced evidence about the prompt. **Do not grade the round, do not change one
-word of instruction on its basis, and do not compare its tokens to anything** —
-fix the machine, then run it again. A round scored on OOM debris will send the
-next round after a defect that does not exist.
-
-`failures` will not catch this and neither will the dashboard. An item whose
-command was killed usually recovers, answers anyway and is recorded `ok`; a
-round can read `16 items, 16 ok, 0 failed` with six of them OOM-killed inside.
-The kill is in the graph, not in the verdict, which is why this is a separate
-mode and why it runs before grading rather than after.
-
-## Phase 5 — Stage 1: generalise
-
-This is where a tuning round is won or lost. Collect every finding from every
-item into one table before writing a single word of instruction, because the
-unit of change is not a finding — it is a _pattern across findings_.
-
-**One item is an anecdote.** A rule written from a single sample's failure fixes
-that sample and adds a line every other run has to read. Three items missing the
-same beat is a rule. Two items and a plausible mechanism is a judgement call.
-One item is a note in `findings.md` and nothing else, unless the failure was
-catastrophic.
-
-Then, for each pattern, ask what the _generalisable_ form is. "Call
-`/calendar/list` before answering a scheduling question" is not it — that is the
-dataset written into the prompt, and it will not transfer to the next query. The
-generalisable form is one level up: "when a request depends on the current state
-of an external system, read that state before proposing anything that acts on
-it." Same fix, and it covers the queries not in the dataset.
-
-Two traps, both of which have cost whole rounds:
+The note is for the batch that will face the case again: say what goes
+wrong and what has already been tried. `add` on a case that was fixed puts it
+back on the list — it was not fixed.
+
+Then patch the README.
+
+## Phase 5 — Merge the proposals (step 5)
+
+**First, the recheck cases.** If one failed, this phase does not happen yet. A
+case that passed in an earlier batch and fails now was broken by a rule written
+since, and the fix is to revert or narrow that rule — not to add another on top.
+Find it in the earlier batches' `changes.md`, undo or qualify it, note the
+over-reach in this run's findings, and go straight to the next run.
+
+Otherwise, collect every case's proposals into one table before writing a
+single word of instruction — the correctness findings and the cost review's
+patterns alike — because the unit of change is not a proposal — it is a _pattern
+across proposals_. Two cases proposing the same sentence in different words are
+one pattern; two cases proposing opposite edits to one sentence are a conflict,
+and the conflict is the finding. Write the merged table into `findings.md`,
+under `## Proposals, merged`: one row per pattern, the cases and nodes behind
+it, the file, and the edit. A cost pattern becomes a rule the same way a
+correctness one does: three cases listing a directory to find a file the skill
+could name is a line in that skill; three cases running independent lookups in
+sequence is a fan-out rule in the prompt.
+
+**One case is an anecdote.** A rule written from a single failure fixes that
+case and adds a line every other run has to read. Three cases missing the same
+beat is a rule. Two cases and a plausible mechanism is a judgement call. One case
+is a note in `findings.md` and nothing else, unless the failure was catastrophic.
+
+The three do not have to be in this batch. A batch is often too small to show
+a pattern on its own, which is why the notes matter: one case here plus two in
+earlier batches is still three. Grep the earlier runs' `findings.md` before
+deciding something is an anecdote.
+
+Then, for each pattern, find the _generalisable_ form. Start from the pattern's
+`why` line: a verified `steered-by` quote is the sentence to narrow or delete,
+a `should-have-applied` quote is a rule that exists but was not reached in time,
+and `no instruction — my own default` or `missing` is a rule to add. A pattern
+with no `why` line yet gets one before it gets a rule - ask at its clearest
+case's critical point, as in phase 4. "Call `/calendar/list`
+before answering a scheduling question" is not it — that is the dataset written
+into the prompt, and it will not transfer. The generalisable form is one level
+up: "when a request depends on the current state of an external system, read
+that state before proposing anything that acts on it." Same fix, and it covers
+the queries not in the dataset.
+
+Two traps, both of which have cost whole batches:
 
 - **A prohibition with a self-judged exemption is a permission.** "Do not do X,
-  except in rare cases where it is clearly warranted" is read by the model as
-  "do X when you want to", because every case is clearly warranted from inside
-  it. Either forbid it or do not.
+  except in rare cases where it is clearly warranted" is read as "do X when you
+  want to", because every case is clearly warranted from inside it. Either forbid
+  it or do not.
 - **Before rewriting a rule the agent keeps ignoring, grep the whole assembled
   prefix for the rule's own text.** House rules, the prompt and every preloaded
-  skill are concatenated. A rule that is obeyed nowhere usually has a second,
-  softer copy somewhere the agent reads more often. The contradiction is the
-  bug; better adjectives are not the fix.
-
-Prefer mechanical wording. Rules of the form "put X before Y", "if the path does
-not end in `.py`, stop", "copy this literal verbatim" are followed. Rules that
-require judgement — "be thorough", "generalise appropriately" — are followed
-about half the time and are not worth their tokens.
-
-## Phase 6 — Stage 1: apply
-
-Route each generalised rule by what kind of thing it is.
-
-| The finding is…                               | It goes in                                          |
-| --------------------------------------------- | --------------------------------------------------- |
-| One agent's behaviour, one agent's job        | `agents/prompts/<agent>.md`                         |
-| A fact, a format, an API shape, a procedure   | A skill — extend one, or create one                 |
-| A standing rule every agent must obey         | `agents/instructions.md`                            |
-| A standing rule about one capability          | `agents/<topic>-policy-instructions.md` — new file  |
-| The wrong agent had the tool, or no agent did | `agents.yaml`, and say so loudly — it is structural |
-| The dataset was wrong                         | `.finetune/dataset.json`, before the next round     |
-
-New policy files are the usual output of a first round: `memory-policy-`,
+  skill are concatenated. A rule obeyed nowhere usually has a second, softer copy
+  somewhere the agent reads more often. The contradiction is the bug.
+
+Prefer mechanical wording. "Put X before Y", "if the path does not end in `.py`,
+stop", "copy this literal verbatim" are followed. "Be thorough", "generalise
+appropriately" are followed about half the time and are not worth their tokens.
+
+## Phase 6 — Apply (step 6)
+
+Route each rule by what kind of thing it is. When the pattern's `why` line
+cites a source - a prompt heading, a skill, a tool description, a house rule -
+that file is where the change goes, whatever the table below would have
+guessed; find it the way **zen-analyze-run** finds its sources of trouble, by
+grepping the quote in the request and then in `agents.yaml` and `agents/`.
+
+| The finding is…                               | It goes in                                                                   |
+| --------------------------------------------- | ---------------------------------------------------------------------------- |
+| One agent's behaviour, one agent's job        | `agents/prompts/<agent>.md`                                                  |
+| A fact, a format, an API shape, a procedure   | A skill — extend one, or create one                                          |
+| A standing rule every agent must obey         | `agents/instructions.md`                                                     |
+| A standing rule about one capability          | `agents/<topic>-policy-instructions.md` — new file                           |
+| The wrong agent had the tool, or no agent did | not changed — structural: an open problem in the README, reported at the end |
+| The dataset was wrong                         | `finetune/dataset.json`, before the next run                                 |
+
+New policy files are the usual output of a first batch: `memory-policy-`,
 `fork-policy-`, `tools-policy-`, `files-policy-instructions.md`. They sit beside
 `zen`'s own four, they are yours, and `zen check --fix` does not touch them. Load
-**zen-instructions** before writing one — in particular for two things it is easy
-to get wrong: `requires:` is _not_ inherited, so a policy file must repeat the
-condition of the file it sits next to (`requires: memory` beside
-`memory-instructions.md`), and the files are prepended in filename order, so
-`memory-policy-` reads after `memory-instructions.md`, which is what you want.
+**zen-instructions** before writing one — in particular: `requires:` is _not_
+inherited, so a policy file must repeat the condition of the file it sits next to
+(`requires: memory` beside `memory-instructions.md`), and the files are prepended
+in filename order, so `memory-policy-` reads after `memory-instructions.md`.
 
 Never edit `agents/memory-instructions.md`, `agents/fork-instructions.md`,
 `agents/tools-instructions.md` or `agents/files-instructions.md`. They are copies
-of the runtime templates; `zen check` reports them stale when they drift and
-`zen check --fix` replaces them wholesale, taking your edit with it. If a finding
-genuinely belongs in one of those — that is, it is true of every Zenera project
-and not just this one — change the master upstream in the CLI templates instead,
-and note it in the round.
+of the runtime templates; `zen check --fix` replaces them wholesale. If a finding
+genuinely belongs in one — true of every Zenera project, not just this one —
+change the master upstream in the CLI templates, and note it in the run.
 
-Record what changed and why in `.finetune/rounds/r1/changes.md`, one entry per
-edit, each naming the finding numbers that motivated it. That file is what makes
-the next round's diff interpretable, and it is what you read when a score goes
-_down_.
+Record what changed and why in the run's own `changes.md` —
+`finetune/runs/batch01-nomem-run1/changes.md` — one entry per edit, each
+naming the merged pattern and the proposals behind it. It is what makes the next
+run's difference interpretable, and it is what you read when a number goes
+_down_. `next.mjs` looks for it to tell whether a graded run is waiting to be
+confirmed.
 
-Then, before re-running:
+Then:
 
 ```sh
 zen check
 .github/skills/zen-review/scripts/check-paths.sh
 ```
 
-A broken project fails every item for one reason, and a round spent discovering
-that is a round wasted.
+and patch the README. A broken project fails every case for one reason.
 
-## Phase 7 — Stage 1: re-run cold, and leave the stage
+## Phase 7 — Confirm without memory, build the candidate (steps 7–9)
 
-Same cases file, same seed, new round directory, still no memory.
+Run the **same batch** again as the next run — a copy of its cases file,
+never a rebuild — still with no memory:
 
 ```sh
-mkdir -p .finetune/rounds/r2
-zen run batch --input .finetune/rounds/r1/cases.json \
-    --batch-dir .finetune/rounds/r2/batch --memory .finetune/empty --concurrency 8
+RUN=finetune/runs/batch01-nomem-run2
+mkdir -p "$RUN" && cp finetune/runs/batch01-nomem-run1/cases.json "$RUN/"
+C=$(node -p 'require("./finetune/config.json").concurrency')
+zen run batch --input "$RUN/cases.json" \
+    --batch-dir "$RUN/batch" --memory finetune/empty --concurrency "$C"
+.github/skills/zen-finetune/scripts/report.mjs \
+    -d "$RUN/batch" -p finetune/runs/batch01-nomem-run1/batch compare
 ```
 
-Re-use `r1/cases.json` rather than re-sampling — the point of the round is the
-difference, and the difference only exists if the queries are identical. Compare
-per sample and on the aggregate: verdicts, nodes, tokens.
+Same cases, same machine, same memory, one thing changed. Read it in this order:
+
+1. **The recheck cases.** They passed before. If one now fails, the batch's new
+   rule broke it — revert or narrow it and run again. Nothing else in the table
+   matters until this row is clean.
+2. **The new cases the run set out to fix.** Did the rule do what it was written
+   to do?
+3. **The new cases it was not about.** A rule aimed at one agent is read by all
+   of them; something getting worse here is the same regression arriving early.
+4. **The cost columns**, localised, as in phase 4. A cost edit that did not cut
+   llm calls or tokens in the cases it was aimed at by more than the noise floor
+   is reverted — prose that does not pay is a cost on every future run. A cost
+   edit that made any case wrong is reverted whatever it saved.
+5. **The commits.** A commit-policy edit should show in what the cases wrote:
+   fewer live readings, the durable shape committed where it was missing.
 
-Three things to watch for:
+Three things to watch across the whole tuning:
 
-- **Regression.** A sample that passed in r1 and fails in r2 is the most valuable
-  result in the round, and it is almost always the new instruction. Prose added
-  for one agent is read by all of them.
 - **The score going up for the wrong reason.** If the new rule names the
-  dataset's own endpoints or entities, the batch passes and nothing generalises.
-  Sample a query the dataset does not contain and try it by hand.
-- **Prose growth.** If the instruction files have grown by a third and the score
-  moved by one sample, the round was net negative. Delete something.
-
-Loop until every sample in the batch behaves. Then widen: sample a _larger_ N
-from the dataset with the same seed — the earlier items recur, so the previous
-round's samples act as a regression set — and go again. Stop when a round
-produces no generalisable pattern. That is the signal that the remaining failures
-are the model's, not the prose's.
-
-### Leaving stage 1
-
-Stage 1 is over when **both** of these hold on the same cold round, not one of
-them:
-
-1. **Every sample holds.** Rubric compliance, and the other seven criteria, with
-   no regression against the round before.
-2. **What the runs wrote to memory is worth keeping.** Read `<id>/memory/` across
-   the whole batch, not per item. Every commit should be an operation, a pointer,
-   a shape or a rule — something that will still be true next week — and none
-   should be a live reading dressed up as a fact. A run that learned something
-   durable and committed nothing fails this as squarely as one that cached the
-   inbox.
-
-The second criterion is the one that gets skipped, and skipping it makes stage 2
-unreadable: the merged graph is built from exactly these commits, so a loose
-commit policy shows up there as forty near-duplicates or as a stale fact that
-makes a warm sample _worse_. Fix commit policy here, in
-`agents/memory-policy-instructions.md`, while the runs are still cold and the
-cause is still attributable.
-
-When both hold, freeze the prompts, the skills and the house rules. Stage 2
-measures recall against a fixed prose baseline; editing that baseline mid-stage
-means a changed number has two causes.
-
-## Phase 8 — Stage 2: tune memory
-
-A separate stage, with its own rounds, its own findings files and its own exit
-criterion. It starts only once stage 1 is green, and what it optimises is not
-correctness — stage 1 settled that — but **cost and speed through reuse**.
-
-The premise is that a cold run rediscovers, every single time, things that do not
-change: where an endpoint is, what shape it returns, which filter was the useful
-one, how a repository is laid out, which agent owns what. Paid once and recalled
-thereafter, that work disappears from the trajectory. A warm batch should
-therefore be _dramatically_ cheaper than the cold batch it is compared against —
-fewer llm calls, fewer tool calls, less wall clock — while every verdict stays
-exactly where it was. Anything less than that is memory being carried and not
-used, and it is a finding.
-
-Starting early is the classic mistake. Memory that compensates for a missing
-instruction is a bug that looks like a feature: the warm run passes, the prose
-defect is invisible, and the failure comes back the day the graph is rebuilt.
-
-Note the division of labour, because it is the part most often confused:
-
-- **Stage 1 grades what an agent writes into nothing.** Each item has its own
-  empty graph, so criterion 3 is about whether the commit was worth making at
-  all.
-- **Stage 2 grades what an agent reads, and what it writes on top of a graph
-  that already knew.** Every item starts from an identical copy of the merged
-  graph, so the input is the same for all of them and the comparison is clean —
-  and because the copies are per item, nothing one item commits can reach
-  another.
-
-That second half is a question stage 1 cannot ask. Cold, every graph is empty,
-so there is nothing to duplicate and nothing to supersede. Warm, an item that
-commits a near-copy of a node it just recalled is growing the graph for no
-information, and an item that learns the old fact was wrong and does not
-supersede it leaves the next run reading the wrong thing. Both are recall-policy
-findings and both are visible only here.
-
-The only artefact stage 2 edits is `agents/memory-policy-instructions.md` — rules
-about _when_ to recall, when to trust a recalled fact, when to go and look
-anyway, and what to commit when memory already holds a version of it. If a
-stage-2 finding cannot be written as a recall or re-commit rule, it is a stage-1
-finding that escaped: note it, finish the stage, and re-open stage 1 rather than
-quietly editing a prompt here.
-
-### The round
-
-Merge the last good cold round's memory into one graph, then run the same cases
-against a copy of it:
+  dataset's own endpoints or entities, the batch passes and nothing
+  generalises. Try a query the dataset does not contain, by hand.
+- **Prose growth.** If the instruction files grew by a third and one case moved,
+  the batch was net negative. Delete something.
+- **A batch that will not pass.** Each kind of run gets at most 4 per batch.
+  After the 4th no-memory run, stop: revert what it did not confirm, put every
+  case still wrong on the difficult list, and write `PASSED` on it. If a case is
+  still wrong its map node is `failed`, not `done` — the case comes back first in
+  the next batches.
+
+Then grade the run exactly as in phase 4, cost review and commits included. When
+every case — new and recheck — is right and the cost review finds nothing more
+worth changing, write `PASSED` at the top of that run's `findings.md` and build
+the candidate from it, before anything else:
 
 ```sh
-zen memory merge .finetune/rounds/r2/batch/*/memory --dir .finetune/rounds/r3/memory --yes
-zen memory stats --dir .finetune/rounds/r3/memory
+.github/skills/zen-finetune/scripts/memory.mjs candidate batch01-nomem-run2
+zen memory stats --dir "$(.github/skills/zen-finetune/scripts/memory.mjs path --candidate)"
+```
 
-mkdir -p .finetune/rounds/r3
-zen run batch --input .finetune/rounds/r1/cases.json \
-    --batch-dir .finetune/rounds/r3/batch \
-    --memory .finetune/rounds/r3/memory --concurrency 8
+Read the stats before the first with-memory run. A graph with near-duplicate
+nodes or live readings in it says the commit policy is too loose — and the
+candidate is built from the run that just passed, so that is a commit-policy
+finding the no-memory grading missed. Note it; it is the first thing the
+with-memory grading will confirm or rule out.
+
+Then patch the README (the batch's no-memory half `done`, its with-memory half
+`now`) and go straight on. Do not stop to ask between halves or between
+batches. The only stops are a run out of memory at concurrency 4, or the user
+asking for a pause.
+
+## Phase 8 — With memory (steps 10–15)
+
+What the with-memory runs optimise is not correctness — the no-memory runs
+settled that — but **cost and speed through reuse**. A run with no memory
+rediscovers, every single time, things that do not change: where an endpoint
+is, what shape it returns, which filter was useful, how a repository is laid
+out, which agent owns what. Learned once and recalled thereafter, that work
+disappears from the trajectory. A with-memory run should therefore be
+_dramatically_ cheaper than the no-memory run that passed the same batch —
+fewer llm calls, fewer tool calls, less wall clock — with every verdict where it
+was.
+
+The division of labour, because it is the part most often confused:
+
+- **No-memory runs grade what an agent writes into nothing.** Each case has its
+  own empty graph, so criterion 3 asks whether the commit was worth making at all.
+- **With-memory runs grade what an agent reads, and what it writes on top of a
+  graph that already knew.** Every case starts from an identical copy of the
+  candidate, so the comparison is clean — and because the copies are per case,
+  nothing one case commits can reach another.
+
+That second half cannot be seen without memory. There every graph is empty, so
+there is nothing to duplicate and nothing to supersede. Here, a case that
+commits a near-copy of a node it just recalled is spending a call to grow the
+graph by nothing, and a case that learns an old fact was wrong and does not
+supersede it leaves the next run reading the wrong thing.
+
+### Run from the candidate (step 10)
+
+```sh
+RUN=finetune/runs/batch01-mem-run1
+mkdir -p "$RUN" && cp finetune/runs/batch01-nomem-run1/cases.json "$RUN/"
+C=$(node -p 'require("./finetune/config.json").concurrency')
+zen run batch --input "$RUN/cases.json" \
+    --batch-dir "$RUN/batch" \
+    --memory "$(.github/skills/zen-finetune/scripts/memory.mjs path --candidate)" \
+    --concurrency "$C"
 ```
 
-Same cases file, same seed, same machine as the cold round it is compared to,
-and — crucially — the same flags. The cold round's `--memory .finetune/empty`
-and this round's `--memory .finetune/rounds/r3/memory` differ in one thing only:
-whether the directory being copied holds a graph. Everything else the agent sees
-is identical, which is what makes the two rounds subtractable.
+The no-memory run's `--memory finetune/empty` and this run's
+`--memory "$(memory.mjs path --candidate)"` differ in one thing only: whether
+the directory being copied holds a graph. Same cases, same seed, same machine,
+same flags.
+
+The new cases of the batch are recalling what they themselves committed in the
+run that passed — the easiest recall there is, and so the first test. The
+recheck cases recall what an earlier batch committed, and are where a stale
+fact, or recall replacing a live reading, shows up first.
 
 Five things follow from copying rather than sharing:
 
-- **The source is never written.** Each item copies it, writes into the copy,
-  and leaves the copy behind at `<id>/memory/`. The merged graph is the same
-  bytes at the end of the stage as at the start.
-- **`<id>/memory/` existing means nothing warm.** Cold, a directory is there
-  only if the item committed — a memory with no manifest is deleted after the
-  run. Warm, every copy arrives with the merged graph's manifest, so every item
-  has one whether it wrote or not. What an item actually added is in its
-  trajectory, as the `memory_op` nodes; read those, not the directory listing.
-- **Never merge a warm round's copies back into the source.** The glob that was
-  right for a cold round — `r2/batch/*/memory` — matches every item of a warm
-  one and would fold the whole merged graph back in N times. Merge once, at the
-  top of the stage, from the last good _cold_ round, and never again.
+- **The candidate is never written.** Each case copies it, writes into the copy,
+  and leaves the copy at `<id>/memory/`. It is the same bytes after every
+  with-memory run as before the first, so run 2 starts where run 1 did.
+- **`<id>/memory/` existing means nothing here.** Without memory a directory is
+  there only if the case committed. With it every copy arrives with the
+  candidate's manifest, so every case has one. What a case actually added is
+  `report.mjs commits`: it diffs each copy against the candidate the batch
+  started from (`zen memory diff`), marks every commit the candidate already
+  held, and lists the nodes `memory_load` read.
+- **Never fold a with-memory run's copies back in.** Each copy holds the whole
+  graph again; `memory.mjs candidate` refuses a with-memory run for exactly that
+  reason.
 - **Copies cost disk and a pause.** They are made serially, before the first
-  model call, one per item, and they are kept: check `zen memory stats` and
-  multiply by the item count before starting a round on a large graph.
+  model call, one per case, and kept: check `zen memory stats` and multiply by
+  the case count before running on a large graph.
 - **The source must not be locked.** `zen run batch` refuses a memory another
   process holds, so close any `zen memory` session first.
 
-The merge glob above is safe because it reads the _cold_ round: an item that
-committed nothing has its memory directory deleted, so `*/memory` there only
-ever matches real graphs.
-
-Read `zen memory stats` before the run. A merged graph with forty near-duplicate
-nodes says the commit policy is too loose, and that is a stage-1 fix before it is
-a stage-2 measurement.
+### Analyze the memory use (step 11)
 
-Grade with the same tools and write `findings.md` the same way, comparing against
-the cold round the memory came from. The heading carries `WARM`:
+With the same tools as phase 4, and write `findings.md` the same way — the title
+says `WITH MEMORY` — comparing against **the no-memory run that passed the same
+batch**, and from run 2 on also against the previous with-memory run:
 
 ```sh
-.github/skills/zen-finetune/scripts/collect.sh \
-    -d .finetune/rounds/r3/batch -p .finetune/rounds/r2/batch compare
+R=finetune/runs
+.github/skills/zen-finetune/scripts/report.mjs -d $R/batch01-mem-run1/batch oom
+.github/skills/zen-finetune/scripts/report.mjs -d $R/batch01-mem-run1/batch memory
+.github/skills/zen-finetune/scripts/report.mjs -d $R/batch01-mem-run1/batch recalls
+.github/skills/zen-finetune/scripts/report.mjs -d $R/batch01-mem-run1/batch commits
+.github/skills/zen-finetune/scripts/report.mjs \
+    -d $R/batch01-mem-run1/batch -p $R/batch01-nomem-run2/batch compare
 ```
 
-### What "memory used correctly" looks like
+That is the one comparison where the two runs are _meant_ to differ in memory,
+and `compare` says so in the note under it. Grade correctness first, exactly as
+in phase 4: a case that was right without memory and is wrong with it is the
+worst finding a with-memory run can have.
 
 | Good                                                                    | Bad                                                                  |
 | ----------------------------------------------------------------------- | -------------------------------------------------------------------- |
-| Far fewer nodes and tokens than the cold run, same verdict              | Same trajectory — memory was never consulted                         |
+| Far fewer nodes and tokens than without memory, same verdict            | Same trajectory — memory was never consulted                         |
 | Recall replaces a discovery step: it knows the endpoint without probing | Recall replaces a _reading_: it reports yesterday's inbox as today's |
 | One recall near the top, then straight to work                          | Repeated searches of memory mid-run, finding nothing                 |
 | Recalled facts are checked when they are cheap to check                 | A recalled fact contradicts a fresh reading and the recall wins      |
-| Nothing is committed that memory already held                           | A near-copy of a node it recalled this run, committed again          |
+| Nothing is committed that memory already held — `known` is 0            | A near-copy of a node it recalled this run, committed again          |
 | A fact found to be wrong is superseded, not merely added to             | The correction sits beside the stale node with no SUPERSEDES         |
 | The answer is at least as good                                          | The answer got worse — a stale fact short-circuited the work         |
 
-The whole point is the second column of row one. Memory should shorten the
-trajectory. If the warm batch costs more tokens than the cold one, memory is
-being loaded and not used, and the fix is an instruction about _when_ to recall,
-not more memory.
+**Re-committing is a cost finding, not a hygiene nicety.** Every
+`memory_commit` of something the graph held is an llm turn and a tool call that
+bought nothing — `zen memory merge` would have folded it anyway. The `commits
+(known)` column and `report.mjs memory` count them per case, and
+`report.mjs commits` names them — `ALREADY HELD as <id>`, with the nearest
+candidate node and its score beside every other commit; the target with
+memory is **zero known**, and commits only of what the run genuinely learned
+that the graph did not have. A case that recalled a node and then committed the
+same fact in its own words is the pattern to ask about.
+
+If a with-memory run costs more than the no-memory run, memory is being loaded
+and not used; the fix is an instruction about _when_ to recall, not more memory.
+It is `NO CHANGE` at best. And a large saving is only good news once explained:
+open at least one cheap case's trajectory and confirm the calls that must hit
+the live system still did.
+
+The bad column is where to ask. A trajectory that never consulted memory,
+recalled and then redid the work, or committed what it had recalled, has a
+critical point at the first research call, the call right after the recall, or
+the call that chose to commit: ask it why, following the memory rows of
+**zen-analyze-run** §5 and the wording rules of **zen-inspect-ask**, before
+writing a word of policy.
+
+Every with-memory `findings.md` has a `## Memory review` straight after the
+verdict block — `next.mjs` refuses a `PASSED` without one — with a fixed shape:
+
+- **saving**: llm calls, tool calls, tokens and wall clock against the no-memory
+  run that passed, per case and in total, and the noise floor it is read against.
+- **recall**: per case, whether recall replaced discovery, citing the recall node
+  and the discovery it saved — or `none`, with the trajectory checked.
+- **live readings**: any recalled fact used in place of a reading that must be
+  live — the pattern, or `none`.
+- **re-commits**: commits of what the graph held (`known`), and near-copies of
+  recalled nodes — the pattern, or `none`.
+- **corrections**: facts found wrong and whether they were superseded — or `none`.
+
+Then the per-case grading, each case ending in its proposals, as in phase 4.
+
+### Merge, update the memory policy, run again (steps 12–14)
+
+Merge the proposals into patterns exactly as in phase 5, and write the merged
+table under `## Proposals, merged`. Then apply them — to
+`agents/memory-policy-instructions.md`, or to the memory paragraph of a prompt
+or skill where an ask's `why` line names one: rules about _when_ to recall, when
+to trust a recalled fact, when to go and look anyway, and when **not** to
+commit because memory already holds it. Record them in this run's `changes.md`
+and run `zen check`.
+
+A pattern that cannot be written as one of those is a no-memory finding that
+escaped: the with-memory runs do not fix it. Write it under "Open problems" in
+the README, put its cases on the difficult list, and leave the prose alone — the
+next batch's no-memory runs, with those cases as rechecks, are where it is
+fixed. A pattern about what is _in_ the candidate — a live reading committed as
+a fact — is a commit-policy rule in `agents/memory-policy-instructions.md`, and
+the offending nodes may be removed from the candidate with
+`zen memory forget --dir "$(memory.mjs path --candidate)" <id>…`, recorded in
+`changes.md`; nothing else edits the candidate by hand.
+
+Then run **the same batch** again from **the same candidate**:
+`batch01-mem-run2`, a copy of the batch's cases. Read it against the previous
+with-memory run (the effect of the policy edit) and against the no-memory run
+that passed (the saving). A with-memory run passes on `PASSED` exactly as a
+no-memory one does, with at most 4 runs, and the difficult list is updated after
+every graded run — a case memory makes wrong or no cheaper, still, after a run
+that tried to fix it, is `--why memory`.
+
+### Keep it, and move on (step 15)
+
+A with-memory run passes when, against the no-memory run that passed:
 
-And a large saving is only good news once it has been explained. Open at least
-one cheap sample's trajectory and confirm the calls that must hit the live system
-still did: recall replacing discovery is the win, recall replacing a live reading
-is a correctness regression wearing a speed-up's numbers.
+1. no verdict regressed;
+2. the run is materially cheaper — a saving obvious next to the noise floor;
+3. every large saving is traced to recall replacing discovery, not a live
+   reading;
+4. what the cases committed is either nothing or a genuine addition — `known`
+   is 0, no near-copies of what they recalled, every correction linked with
+   `SUPERSEDES`.
 
-Compare warm against cold per sample:
+Write `PASSED`, then keep the candidate before anything else:
 
 ```sh
-for id in $(jq -r '.batch_results[].id' .finetune/rounds/r2/batch/batch.json); do
-    cold=$(jq -r '.usage.inputTokens + .usage.outputTokens' ".finetune/rounds/r2/batch/$id/output.json")
-    warm=$(jq -r '.usage.inputTokens + .usage.outputTokens' ".finetune/rounds/r3/batch/$id/output.json")
-    printf '%-32s cold %8s  warm %8s\n' "$id" "$cold" "$warm"
-done
+.github/skills/zen-finetune/scripts/memory.mjs checkpoint batch01-mem-run1
 ```
 
-### The stage-2 loop, and leaving it
+Then patch the README (the batch `done`, the next one `now`), mark any difficult
+case right in both passing runs `fixed`, and build the next batch as `next.mjs`
+says — straight away. It goes on while the selection has unused cases or the
+difficult list has open ones; then comes the final check.
 
-Any sample that got longer, or got worse, is a finding, and so is any item that
-re-committed what it had just recalled. Generalise it exactly as in phase 5 — a
-pattern across samples, not one sample — write the rule into
-`agents/memory-policy-instructions.md`, and re-run from **the same merged graph**
-with the same cases into a new round. Re-merging between rounds, or merging a
-warm round's own copies back into the source, changes the memory and the prose at
-once; merge once per stage-2 session and keep that graph byte-identical until the
-stage ends.
+## The final check
 
-Stage 2 is over when, against the cold baseline:
+When every case in the selection has been used and no difficult case is open,
+run **the whole selection** as one batch — the last good memory, the same flags,
+the same machine — into `finetune/runs/final-check/`. It is the only run that
+measures every case under the final prose and the final memory at once: the
+first batches were tuned against prose that later batches changed, and only
+their rechecks have seen it since.
 
-1. no verdict regressed;
-2. the batch is materially cheaper — a saving that is obvious next to the noise
-   floor, not inside it;
-3. every large saving has been traced to recall replacing discovery, not recall
-   replacing a live reading;
-4. what the warm items committed is either nothing or a genuine addition — no
-   duplicates of what they recalled, and every correction linked with
-   `SUPERSEDES` rather than left beside the node it contradicts.
-
-If the project will ship with a warm memory, the merged graph from the last good
-round is the one to promote into `memory/`. Read **zen-memory-warmup** first —
-seeding a project's memory deliberately is that skill's whole subject, and it has
-the rules about what belongs in a shipped graph.
+```sh
+RUN=finetune/runs/final-check
+mkdir -p "$RUN"
+cp finetune/selection.json "$RUN/cases.json"
+C=$(node -p 'require("./finetune/config.json").concurrency')
+zen run batch --input "$RUN/cases.json" --batch-dir "$RUN/batch" \
+    --memory "$(.github/skills/zen-finetune/scripts/memory.mjs path)" --concurrency "$C"
+```
+
+Grade it, and head `findings.md` with `VERIFIED` if no verdict regressed and the
+saving holds. Change nothing. If something regressed, that is a real finding:
+write it under "Open problems" with the batch whose rules are the suspect, and
+report it. Then patch the README into its
+final form: every map node `done` or `failed`, "Results so far" as the summary of
+the whole tuning, and every `stuck` case with what was tried.
+
+If the project will ship with a memory, `memory.mjs path` is the graph to promote
+into `memory/` — and it is that graph whenever the tuning stops, finished or not.
+Read **zen-memory-warmup** first — it has the rules about what belongs in a
+shipped graph.
 
 ## The state directory
 
 ```
-.finetune/
-    dataset.json            built once, cached, committed
-    rounds/
-        r1/                 stage 1, cold
-            cases.json      the sample; re-used verbatim by every later round, both stages
+finetune/
+    dataset.json            every case in the training set; built once, cached, committed
+    config.json             limit, minBatch, maxBatch, recheck, concurrency, seed, maxRetries
+    selection.json          the cases up to the limit, evenly by class, rubric and complex first
+    difficult.json          the difficult cases: why, when, notes, fixed — written by difficult.mjs
+    README.md               the plan, the map and the progress — written by you, patched every step
+    USAGE.md                tokens per model, batch, run and stage — written by usage.mjs, never by hand
+    usage.json              the same numbers, for a script
+    usage/
+        ledger.jsonl        one line per meta agent call, run, batch and inspect ask — appended by zen
+    empty/                  never created; naming it is what gives a run no memory
+    memory/
+        candidate.json      the batch being tuned: its candidate, and the run it was built from
+        candidate-batch<NN>/    last good + that run's commits; what the with-memory runs copy
+        last-good.json      which graph is last good, and which batches built it
+        after-batch<NN>/    a kept candidate, one per passed batch; the last three are kept
+    runs/
+        batch01-nomem-run1/
+            cases.json      new + recheck; built once, copied for every later run of the batch
+            plan.json       the size and why, which cases are new, which are rechecks
             batch/          zen run batch output
-            findings.md     the verdict block, then per-sample grading with node citations
-            changes.md      what was edited and which findings motivated it
-        r2/ …               stage 1, cold — until the stage-1 exit criteria hold
-        r3/                 stage 2, warm
-            memory/         the merged graph; built once, copied per item, never written to
-            batch/ findings.md changes.md
-        r4/ …               stage 2, warm — same merged graph, recall policy edited
+            findings.md     the verdict block, the cost review, per-case grading, proposals merged
+            changes.md      what was edited and which merged proposals motivated it
+        batch01-nomem-run2/     same cases, new prose. PASSED here builds the candidate
+        batch01-mem-run1/       same cases, from the candidate; findings.md has a memory review
+        batch01-mem-run2/       same candidate, new memory policy. PASSED here keeps it
+        batch02-nomem-run1/     the next batch: twice as many new cases + rechecks
+        batch07-nomem-run1/     past the selection: open difficult cases only
+        final-check/            the whole selection at once, with the last good memory
 ```
 
-Commit `dataset.json`, `cases.json`, `findings.md` and `changes.md` — they are the
-project's eval history and the reason a later reader can tell why an instruction
-exists. The `batch/` directories are large, contain whole workspaces, and may
-contain live API responses: ignore them. A warm round's is larger still — it
-holds one copy of the merged graph per item. Add to `.gitignore`:
+The `batch<NN>-<nomem|mem>-run<N>` naming is not cosmetic: `next.mjs` parses it
+to find which batch a run belongs to, which half it is in and whether it has
+passed, and `batch.mjs` and `difficult.mjs` read which cases each batch held
+from its first no-memory run. Any other directory under `runs/` is ignored.
+
+Commit `dataset.json`, `config.json`, `selection.json`, `difficult.json`,
+`README.md`, `USAGE.md`, `usage.json`, every `cases.json`, `plan.json`,
+`findings.md` and `changes.md` —
+they are the
+project's eval history. The `batch/` directories are large, contain whole
+workspaces, and may contain live API responses: ignore them, and the memory
+graphs built from them, and the raw usage ledger. Add to `.gitignore`:
 
 ```
-.finetune/rounds/*/batch/
-.finetune/empty/
+finetune/runs/*/batch/
+finetune/memory/
+finetune/empty/
+finetune/usage/
 ```
 
+## The usage report
+
+`finetune/USAGE.md` says what the tuning has cost in tokens — in total, by
+model, by batch, by run, the costliest cases, and the meta
+agent's own spend by stage and session. Three things spend tokens, and only one
+is the project under test: the **project agents** (every case of every run, read
+off `batch.json`, so runs from before the ledger count too), the **meta agent**
+doing the tuning, and the **`zen inspect ask`** calls of the grading.
+
+Nobody writes it by hand: `next.mjs` regenerates it on every call, and a running
+`zen meta` does every five minutes and when it exits. `zen meta`, `zen run`,
+`zen run batch` and `zen inspect ask` append to `finetune/usage/ledger.jsonl`
+whenever the project has a `finetune/`. Link it from the README instead of
+copying its numbers; `usage.json` holds the same numbers for a script.
+
 ## Scripts this skill ships
 
-| Script               | What it does                                                                                                                                                         |
-| -------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `scripts/sample.sh`  | `dataset.json` → `cases.json`, stratified, rubric-first, deterministic                                                                                               |
-| `scripts/collect.sh` | A batch directory → an index, the concatenated graphs, the run paths, the per-sample comparison against the previous round, or whether the round is gradeable at all |
+| Script                  | What it does                                                                                                                                                                                                                                         |
+| ----------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `scripts/select.mjs`    | `dataset.json` → `selection.json`: up to the limit, evenly by class, rubric then complex first — the order batches take new cases in                                                                                                                 |
+| `scripts/batch.mjs`     | builds the next batch's `cases.json` and `plan.json`: its size by the growth rule, new cases from the selection + rechecks, difficult then complex first; only difficult ones once the selection is used. No command: progress                       |
+| `scripts/report.mjs`    | one run's `batch/` → an index, the concatenated graphs, the trajectory paths, the failures, the OOM check, memory use per case, each case's commits (against the candidate, with memory), recalls and answers, or the comparison against another run |
+| `scripts/difficult.mjs` | the difficult list: `add` / `fix` a case with its reason and run; with no command, the list with `open` / `fixed` / `stuck` and retry counts                                                                                                         |
+| `scripts/memory.mjs`    | `candidate <run>` builds a batch's candidate from a passed no-memory run; `checkpoint <run>` keeps it after a passed with-memory run; `path [--candidate]` names a graph; no command says what built them                                            |
+| `scripts/next.mjs`      | reads `finetune/` and prints the next step — including when a candidate or checkpoint is missing and when the final check is due — and whether the README has fallen behind; refreshes `USAGE.md` on the way                                         |
+| `scripts/usage.mjs`     | the ledger and every run's `batch.json` → `USAGE.md` and `usage.json`: tokens by who spent them, model, batch, run and stage; `--stdout` prints it instead                                                                                           |
 
-Before the batch, `zen-sandbox-capacity` ships a third:
-`.github/skills/zen-sandbox-capacity/scripts/preflight_sandbox.sh`, which
-answers whether this machine can run the round you are about to start.
+The README template is `references/readme-template.md`.
 
-All find the project root from their own location and can be run from anywhere.
-All are copies: `zen init` and `zen open` rewrite the whole `.github/` tree, so
-an edit made here is gone at the next one. Change them upstream, in the CLI's
-`templates/editor/.github/skills/` — that is the only place a change survives.
+All seven are Node programs and run on the Node that `zen` itself requires — no
+`jq`, and they work the same on Windows. `memory.mjs` also needs `zen` on PATH. They find the project root from their
+own location and can be run from anywhere. They are copies: `zen init` and
+`zen open` rewrite the whole `.github/` tree, so an edit made here is gone at the
+next one. Change them upstream, in the CLI's `templates/editor/.github/skills/`.
 
 ## Rules that are easy to get wrong
 
-1. **Never tune against a warm memory in stage 1.** A cold run is the only one
+1. **Never give a no-memory run memory.** A run with no memory is the only one
    whose result is attributable to the prose.
-2. **Never start stage 2 while a sample still fails cold.** Memory will paper
-   over it, the warm round will pass, and the defect returns the day the graph
-   is rebuilt.
-3. **Never edit a prompt, a skill or a house rule during stage 2.** The prose is
-   the fixed baseline recall is measured against; stage 2 edits
-   `agents/memory-policy-instructions.md` and nothing else.
+2. **Never run a batch with memory before it has passed without it.** Memory
+   will paper over the defect, and it returns the day the graph is rebuilt.
+3. **Never edit anything but the memory policy in a with-memory run.**
+   `agents/memory-policy-instructions.md`, or the memory paragraph a `why` line
+   names. Anything else goes to "Open problems" and the next batch's no-memory
+   runs.
 4. **Never tune with `--memory-read-only`.** It clamps `access` to `read`, which
-   takes `memory_commit` out of the schema and changes the memory prose — so a
-   warm round run that way differs from its cold baseline in two ways, not one.
-   Give `--memory` a real graph and let each item copy it.
-5. **Never merge a warm round's copies back into the stage's graph.** The merged
-   graph is built once, from the last good cold round, and stays byte-identical
-   until the stage ends.
-6. **Never change the sample and the instructions in the same round.** Two
-   variables, one number, no conclusion.
-7. **Never write a rule from one failing sample** unless the failure was
-   catastrophic. Note it and wait for the pattern.
-8. **Never edit `zen`'s four instruction files.** `zen check --fix` overwrites
-   them. Project policy goes in a `-policy-` file beside them.
-9. **Never adjust a rubric after seeing the run.** Fix a wrong rubric before the
-   round and say so, or leave it.
-10. **Never cite a finding without a node id.** If you cannot point at it in the
+   takes `memory_commit` out of the schema and changes the memory prose — two
+   differences from the no-memory baseline, not one.
+5. **Never build a candidate from anything but a PASSED no-memory run, and never
+   fold a with-memory run's copies into it.** Each copy holds the whole graph
+   again. Every case of a with-memory run gets its own copy of the candidate;
+   the candidate itself is never written by a run.
+6. **Never change the cases and the instructions in the same run.** Two
+   variables, one number, no conclusion. Edits go in after a run is graded, and
+   the next run re-runs a **copy** of the batch's `cases.json`.
+7. **Never compare two different batches.** Only run to run within a batch and
+   kind, and a with-memory run against the no-memory run that passed the same
+   batch.
+8. **Never treat batch size as concurrency.** Batch size is chosen so every
+   trajectory in the batch gets read; concurrency is
+   `min(maxBatch + recheck, 16)`, halved on each out-of-memory run, never below 4.
+9. **Never let a recheck case's failure be outranked.** A new case failing is
+   work to do; a recheck case failing means a recent rule broke something that
+   worked. Revert or narrow it before writing anything new.
+10. **Never change `limit`, `minBatch`, `maxBatch`, `recheck` or `seed` once
+    batch 1 has run, and never rebuild a batch for a later run — copy its
+    `cases.json`.** What goes into a batch depends on all five and on the
+    difficult list at the time it was built. `concurrency` only goes down, by
+    halving after an OOM. A batch's size comes from `batch.mjs`, not from you —
+    `-m` only with a reason in the README.
+11. **Never write `PASSED` on a run that is only correct.** It means every case is
+    right, recheck cases included, **and** the run's review — cost without
+    memory, memory with it — found nothing to cut, so the run applies no edits.
+    All right with something to cut is `CORRECT`.
+12. **Never cut the dataset, and never set `limit` below `datasetSize` on your
+    own.** Every case in the training set goes into `dataset.json`, and every
+    case is tuned against unless the user named a smaller limit.
+13. **Patch the README after every step, by hand.** Exactly one `now` node on the
+    map. When `next.mjs` says `first: patch finetune/README.md`, do that first.
+14. **Never edit from one case's proposal.** Merge every case's proposals first;
+    a rule needs a pattern, unless the failure was catastrophic.
+15. **Never edit `zen`'s four instruction files.** `zen check --fix` overwrites
+    them. Project policy goes in a `-policy-` file beside them.
+16. **Never adjust a rubric after seeing the run.** Fix a wrong rubric before the
+    run and say so, or leave it.
+17. **Never cite a finding without a node id.** If you cannot point at it in the
     graph, it did not happen.
-11. **Never let a rule name the dataset.** Endpoints and entity names from the
+18. **Never let a rule name the dataset.** Endpoints and entity names from the
     training set inside an instruction means the eval passes and the product does
     not.
-12. **Re-run `zen check` after every edit round.** A load error fails every item
+19. **Run `zen check` after every round of edits.** A load error fails every case
     identically and looks exactly like a catastrophic regression.
-13. **Never grade a round that was OOM-killed.** `collect.sh oom` before
-    anything else. An exit 137 is the machine, not the prose, and the items
-    still report `ok`.
-14. **Never compare two rounds measured on different machines**, or a warm round
-    against anything but the cold round its memory was merged from. Tokens and
-    wall clock are only comparable within one spec and one memory mode.
+20. **Never grade a run that was OOM-killed.** `report.mjs oom` before anything
+    else. An exit 137 is the machine, not the prose, and the cases still report
+    `ok`. Halve the concurrency and run the same cases again.
+21. **Never compare two runs measured on different machines.** Tokens and wall
+    clock are only comparable within one spec.
+22. **Never build the next batch before `memory.mjs checkpoint`.** A passed
+    batch's candidate becomes the last good memory before anything else, or a
+    stop at the wrong moment loses it. `next.mjs` insists.
+23. **Never let a difficult case go unrecorded.** Wrong after a fix, a failed
+    recheck, flaky, made worse or no cheaper by memory, or over 2x the median:
+    `difficult.mjs add`, with a note saying what was tried. A case forgotten when
+    its batch passes is the one that fails in production.
+24. **Never retry a `stuck` case again.** It had `maxRetries` batches. Report it
+    with what was tried; more prose will not move the model's ceiling.
+25. **Never pass a with-memory run that re-commits what it recalled.** `known`
+    above 0 is a call spent on nothing, in every future run. It is a finding for
+    the memory policy, not noise.
 
 ## When not to fine-tune
 
-- **The project does not load.** `zen check` first; a round on a broken project
+- **The project does not load.** `zen check` first; a run on a broken project
   measures nothing. Load **zen-review**.
-- **The last round was OOM-killed.** Nothing about the prose is in question yet.
-  Load **zen-sandbox-capacity**, size the machine, re-run the same round.
+- **The last run was OOM-killed.** Nothing about the prose is in question yet.
+  Halve the concurrency (never below 4) and run the same cases again; at 4, stop
+  and tell the user.
 - **There is no training set.** Fewer than about eight queries is a debugging
   session, not a tuning loop — run them singly with `zen run` and read the graphs.
-- **One run is behaving strangely.** That is diagnosis. Load **zen-inspect** and
-  read that run.
+- **One run is behaving strangely.** That is diagnosis. Load **zen-analyze-run**
+  and analyze that run.
 - **The specification changed.** Reconciling prose with intent is
   **zen-spec-sync**'s job; tune after it, against what the project is meant to be.
-- **The failure is the model's.** If three rounds of clear, mechanical
-  instructions do not move a sample, the ceiling is the model. Change the model
-  in `agents.yaml` and re-run the same batch — that is a legitimate experiment
-  and the dataset is already set up for it.
+- **The failure is the model's.** If three runs of clear, mechanical instructions
+  do not move a case, the ceiling is the model. Change the model in `agents.yaml`
+  and run the same batch again — a legitimate experiment, and the cases are
+  already set up for it.
 
 ## Related
 
 - **zen-cli** — command surface, flags, exit codes, and what `--json` emits
 - **zen-inspect** — the graph/node/ask loop, node kinds, and the symptom table
+- **zen-analyze-run** — auditing one trajectory, and asking the model why at its critical points
 - **zen-instructions** — house rules, `requires:`, filename order, `zen check`
 - **zen-memory** — the graph model, commit rules, audiences, `merge` and `stats`
 - **zen-memory-warmup** — building a memory deliberately, and shipping one
-- **zen-review** — the mechanical checks to run before and after every round
-- **zen-sandbox-capacity** — whether the machine can run the batch; read it
-  before the first cold round, and whenever one dies at 137 or 124
+- **zen-review** — the mechanical checks to run before and after every batch
 - **zen-spec-sync** — reconciling the project with its specification, before tuning