paperlint 10.2.0 → 10.4.0

package/docs/design/study-modes.md

@@ -0,0 +1,247 @@
+# Study modes: decide first, or exploratory — design
+
+## Why
+
+A study's outcomes are computed by an automated runner in the same workspace where the people
+deciding its inputs work: which runs count, which units are excluded, whether a veto applies, whether
+the drawn sample is approved. If those decisions can be made — or changed — after an outcome has been
+seen, the decision can follow the result, and nothing in the repository says which came first.
+
+What is needed:
+
+1. every measurement says how its outcomes are kept from the decisions on its inputs, and none can
+   say nothing;
+2. the default needs no key and no outside service: the decision is recorded first, the run starts
+   after, and anyone with the repository can re-check the order;
+3. a study that cannot work that way is declared exploratory, with its reason, and reported as such;
+4. the paper's Methods and its reviewers read each study's mode, decision commit and first output
+   from one generated place, not from prose someone keeps in sync by hand.
+
+## What is decided
+
+### 1. One mode per measurement, in its plan
+
+A study's `plan.json` holds one `mode`, a union on its `kind` (`src/domain/study-mode.ts`):
+
+```json
+{ "mode": { "kind": "decide_before_run",
+            "outputs": ["out/**", "RESULTS.md"],
+            "decision": { "file": "repro/P1-S03-x/decision.md", "note": "exclusions and veto" } } }
+
+{ "mode": { "kind": "exploratory", "reason": "a first look at the logs, to find the question" } }
+```
+
+- **Exactly one, by construction.** The modes are one discriminated union, each variant closed: a
+  second mode's key in the same object is an unknown key, and an unknown `kind` is refused. This is
+  the parser's (`src/adapters/study-registry/`), not a later check.
+- **None is not ready.** The parser is told the study's `kind` (from its row). A `measurement`'s plan
+  without a mode is a **draft** naming `mode` among its empty texts, like an empty `question`: legal
+  while the study is `designing`, reported by `repro/study-registered` (`bad-plan`, `unfilled`) once
+  it runs, and a parse error when the author's check is already on it (the author approved an
+  incomplete plan). A spike, a tool, a data set, an artifact or a salvage may omit it.
+- **The default is written for you.** `paperlint studies new` writes a measurement's plan with
+  `decide_before_run`, its decision file named (`repro/<folder>/decision.md`), its outputs and note
+  left to fill — a draft until they are. The decision file itself is NOT created: an empty
+  placeholder that got committed would open the gate on nothing.
+- **Paths.** `outputs` are globs relative to the study's folder, inside it, never the study's
+  `README.md`. A decision `file` is relative to the **paper's** folder, so it may live beside the
+  study or anywhere in the paper (`research/decisions.md`); it may not leave the paper, and it is a
+  file's path — names joined by `/`, none empty, `.` or `..` — so `.` (the paper itself) or a path
+  ending in `/` cannot be written. A name that is a directory on disk is the gate's to refuse and the
+  rule's to report (`decision-not-file`): git answers for a directory with the commits of anything
+  under it. Whether a
+  decision inside the study's folder is matched by an output glob needs the folder, which the plan
+  does not hold: the rule reports it (`decision-is-output`) and the gate refuses it.
+- **Empty texts of a mode** — no outputs yet, an empty decision file or note, an empty reason — are a
+  draft's empty texts, named field by field (`mode.outputs`, `mode.decision.file`, …).
+
+### 2. `paperlint studies gate <paper-dir> <id>`
+
+Exit 0 only if the decision file exists and is committed as it stands (git's last commit for it, and
+nothing changed since), printing the commit and its date; exit 1 saying why otherwise (not
+committed, changed since its commit, committed and since deleted — not in HEAD, though `git log -1`
+names the deleting commit and the tree is clean — git unable to answer, no decision file named, a
+decision that is a directory, a decision its own outputs match). A runner calls it first:
+
+```sh
+npx paperlint studies gate papers/my-paper P1-S03 && ./run.sh
+```
+
+For an exploratory study it exits 0 and says it has no gate. A study that declares no mode, or whose
+plan is still a draft, is refused.
+
+### 3. `repro/outputs-before-decision` — the proof of order a reviewer can re-run
+
+An error, reported on `PIPELINE-STATUS.md`. For each decide-first study: every output whose first
+commit is not a strict descendant of the decision file's first commit (earlier, on another line of
+history, or in the same commit) is a finding; deleted outputs count. An output in the tree or in history while the
+decision was never committed is a finding, and so is a decision committed and since deleted
+(`decision-deleted`): it is not in HEAD, so no order is judged against it. A shallow clone, and a
+history git cannot read, are
+findings that say so — never a pass. The study's folder is walked without following a symbolic link
+(`ListEntries`, `src/ports/list-entries.ts`): a link is one entry, as git stores it, so a link to the
+folder itself cannot loop the walk and one outside brings no foreign file into the study.
+
+**Why a lint rule may ask git here.** Rules in `eslint-rules/` may not run git
+(`eslint.config.mjs`, `GIT_IN_A_RULE`): a pinned sha is deleted by a squash or a gc, and a shallow CI
+checkout has none. This rule pins no sha; it asks the history the checkout has for an order. A squash
+merge keeps that order when the decision was merged before the run — which is the decide-first
+workflow — and a shallow checkout is named by the rule (`fetch-depth: 0`).
+
+### 4. The index carries each study's mode
+
+`repro/README.md`'s study index (`paperlint studies index`, docs/design/study-registry.md) has three
+columns for this (`src/domain/index-columns.ts`):
+
+| column         | cell                                                                                                                         |
+| -------------- | ---------------------------------------------------------------------------------------------------------------------------- |
+| `mode`         | `decide_before_run`, `exploratory: <reason>`, or `—` for a study whose plan has none                                         |
+| `decision`     | the decision file's first commit (12 characters) and its date (`, deleted since` when it is not in HEAD), or `not committed` |
+| `first output` | the first commit of the oldest output and its date, or `none`                                                                |
+
+A sample row:
+
+```markdown
+| id     | title   | status | result   | in paper | mode              | decision                | first output            | folder                             |
+| ------ | ------- | ------ | -------- | -------- | ----------------- | ----------------------- | ----------------------- | ---------------------------------- |
+| P1-S01 | Outcome | done   | 41 of 60 | yes      | decide_before_run | 3f2a9c1d7e40 2026-10-14 | 8b51e0aa42c9 2026-10-15 | [P1-S01-outcome/](P1-S01-outcome/) |
+```
+
+- **The mode is compared, the commits are not.** The mode comes from the plan, which is in the tree,
+  so `repro/study-registered` reports an index whose mode column is not the plan's. The two commits
+  are what git's history said when `studies index` last ran: a squash merge or a rebase gives the
+  same files new commits, and a shallow CI clone has none to read, so the lint never compares those
+  two columns — comparing them would turn every squash merge red. `studies index` rewrites the README
+  whenever any column moved, and in a shallow clone writes `unknown (shallow clone)`. The order
+  itself is `repro/outputs-before-decision`'s to judge, not the table's.
+- **The skills read the table.** `draft-paper` writes the Methods sentence from it (decided before
+  the run, the decision's commit and date; exploratory studies labelled with their reason);
+  `harden-paper` and `pc-panel-review` check that every confirmatory claim rests on a
+  `decide_before_run` study whose decision commit precedes its first output, and flag a
+  "pre-registered" or "decided before" claim the table does not support; `tighten-paper` keeps that
+  sentence; `osf-artifact-upload` and `camera-ready` ship each plan, its decision file and the index,
+  with the whole history. Each regenerates the index first, so the commits are this checkout's.
+
+### 5. Reads stay inside the paper
+
+**What these checks guard against.** paperlint's checks guard against accidents and honest
+mistakes — a run started before the decision was committed, a decision edited or deleted afterwards,
+a path typed wrong — not against an author who controls the repository's history and sets out to
+forge it. The path rules below exist to keep every read inside the paper, so a mistyped row cannot
+make a command or a rule read, judge or write files that belong to nothing in it.
+
+Every command and rule that reads a study's folder — `studies gate`, `studies index` and `new`,
+`repro/outputs-before-decision`, and `repro/study-registered`'s reads of plans and result files —
+reads a row only through one predicate, `readableFolders` (`src/study-registry-rule.ts`): its
+`folder` is one folder directly inside `repro/` (`isFolderName`: not empty, not `.` or `..`, no
+separator or NUL) and not a symbolic link (`ListEntries`, read without following links). Such a row
+is never read through: `studies index` and `new` refuse the registry, the gate refuses the study,
+the order rule treats it as no study, and `repro/study-registered` reads its plan and results as
+missing. A plan's path is held inside its folder by the registry rule's own `isInside`, the same on
+every platform (a backslash is refused outright); the decision file is held inside the paper by the
+schema (`staysInside`) and must be a regular file on disk — a directory or a symbolic link is refused
+by the gate and reported by the rule (`decision-not-file`). Only a **ready** plan's mode decides
+anything: the gate refuses a draft, naming its empty fields, and the order rule judges no draft — a
+draft may name no outputs, or an empty decision file.
+
+### 6. Legacy plans
+
+A plan still named `prereg.json` is parsed by its loose legacy schema, which never asks for a mode,
+and `repro/study-plan-name` keeps warning that it moves to `plan.json` — where the strict parser asks
+for one. So an old paper keeps linting as before until it renames the file. A `plan.json` written by
+the release before modes existed, for a measurement, reads as a draft whose empty text is `mode`:
+silent while the study is `designing`, reported once it runs, and an unreadable plan when the
+author's check is already on it (the message names `mode`). Adding one line — usually the
+decide-first mode — is the migration. An index generated before the mode columns existed is out of
+sync until `paperlint studies index` regenerates it.
+
+### 7. Why: confirmatory vs exploratory, and venues that review the plan first
+
+Read for this note (2026-10-16):
+
+- Nosek, Ebersole, DeHaven, Mellor, "The preregistration revolution", _PNAS_ 115(11):2600–2606,
+  2018 (metadata and abstract from Crossref, doi:10.1073/pnas.1708274114). The abstract, verbatim:
+
+  > Progress in science relies in part on generating hypotheses with existing observations and
+  > testing hypotheses with new observations. This distinction between postdiction and prediction
+  > is appreciated conceptually but is not respected in practice. Mistaking generation of
+  > postdictions with testing of predictions reduces the credibility of research findings. […] An
+  > effective solution is to define the research questions and analysis plan before observing the
+  > research outcomes—a process called preregistration. Preregistration distinguishes analyses and
+  > outcomes that result from predictions from those that result from postdictions.
+
+  This is the distinction the modes encode: decided first is prediction; exploratory is postdiction,
+  legitimate when it is labelled.
+
+- The MSR (Mining Software Repositories) Registered Reports track, 2020 edition
+  (https://2020.msrconf.org/track/msr-2020-Registered-Reports; the 2021, 2025 and 2026 editions have a
+  track page at the same address, the 2019 site has none there). Quoted from the 2020 page:
+
+  > Methods and proposed analyses are pre-registered and reviewed prior to research being conducted.
+
+  > (MSR 2020) Phase 1: Introduction, Methods (including proposed analyses), and Pilot Data (where
+  > applicable).
+
+  > (EMSE) Phase 2: full study, after data collection and analysis.
+
+  > Additional exploratory analyses in Phase 2 may be conducted, if they are justified.
+
+  > Any deviation from the protocol must be justified and logged in detail to ensure replicability.
+
+  A venue of this field already reviews the plan before the data, and accepts exploratory analyses
+  beside it only when they are justified. The modes, the order rule and the index give a paper the
+  same evidence in its own repository: what was fixed before the run, with its commit, and what was
+  exploratory, with its reason.
+
+## Alternatives considered
+
+- **Encrypting the outputs until the decision (age, to a key the author holds): rejected.** It needs
+  an external binary that `paperlint toolchain` does not install, and a key whose custody has no good
+  home: a session in the cloud has nowhere private to keep the identity, and a key in a CI secret is
+  readable by any workflow that can be edited. Decide first already proves the order from git with no
+  key at all, so encryption would add a dependency and a secret to guard to buy what the history
+  shows. Blind analysis in physics (Klein & Roodman, _Annu. Rev. Nucl. Part. Sci._ 55, 2005) and
+  MacCoun & Perlmutter, "Hide results to seek the truth" (_Nature_ 526, 2015) — both cited from
+  memory, not re-read for this note — are where the idea comes from; what is kept is that the
+  decision is fixed and recorded before the outcome is seen.
+- **Paragraph links (`% study: <id>` comments in `paper.tex`) checked by lint rules: rejected.** Such
+  rules check only the paragraphs that carry a convention authors do not write, so an unlinked
+  overclaim passes silently. The review skills check the Methods sentence and every confirmatory
+  claim against the index instead, which covers the whole text.
+- **A separate `studies methods` command printing the modes and commits: rejected.** The same facts
+  are three columns of the `studies index` table the registry already generates; a second command
+  would be a second rendering of one set of facts.
+- **Comparing the decision and first-output columns in the lint: rejected.** A squash merge renames
+  the commits, so the index would go stale on every merge, and a shallow CI clone cannot read them;
+  the order rule judges the history directly instead (§4).
+- **An outside pre-registration (OSF, AsPredicted) as the record: not required.** A timestamped
+  record before the outcome is what they give; here the repository's history is that record,
+  re-checkable by anyone who clones it. A paper may still register outside and cite it.
+
+## Where each part lives
+
+| part                                                          | file                                  | layer   |
+| ------------------------------------------------------------- | ------------------------------------- | ------- |
+| the modes, what is an output, which decision a mode waits on  | `src/domain/study-mode.ts`            | domain  |
+| the order judge and its messages                              | `src/domain/decide-first.ts`          | domain  |
+| the index's mode columns and the history they show            | `src/domain/index-columns.ts`         | domain  |
+| the index, which columns the lint compares, `isFolderName`    | `src/domain/study-registry.ts`        | domain  |
+| the schema of `mode`; readiness by kind                       | `src/adapters/study-registry/`        | adapter |
+| git: committed state, first commits, ancestry, shallow        | `src/adapters/git/`                   | adapter |
+| what the app asks of git                                      | `src/ports/git-history.ts`            | port    |
+| a directory's entries, links not followed                     | `src/ports/list-entries.ts`           | port    |
+| the studies with a mode, and the files under a study's folder | `src/study-modes.ts`                  | app     |
+| the order rule                                                | `src/outputs-before-decision-rule.ts` | app     |
+| the plans and histories `studies index` and `new` write from  | `src/study-index-input.ts`            | app     |
+| `gate`; `new` and `index`                                     | `src/study-gate.ts`, `src/studies.ts` | app     |
+| wiring: git with PATH and HOME, the clock                     | `src/cli.ts`                          | root    |
+
+## What it does not check
+
+- What a decision says. The gate is that it is committed, not its content.
+- A decision amended after the outputs: `repro/outputs-before-decision` orders FIRST commits; later
+  edits of the decision are in its history but not reported.
+- That nobody ran a decide-first study before its decision and kept the outputs out of git. The rule
+  keeps early outputs out of the repository's history, not out of a person's hands.
+- Files outside the study's folder as outputs: outputs are relative to it by construction.
+- Whether the paper's text matches the index: that is the review skills' job, not a lint rule's.

package/docs/design/study-registry.md

@@ -47,9 +47,13 @@ two commands that write it.
    | `superseded_by` | optional; an id in the registry. Required when `superseded`               |
    | `in_paper`      | `yes`, `no`, `registry` (listed only in the registry, not in the text)    |
 
-   **A row may carry more fields**, and they are kept: a paper merges its own measurement metadata
-   into the rows (`role`, `frame`, `method`, `outcome`, …). The top level is closed: a misspelt
-   `prefix` is an error, not an ignored key.
+   | `measurement` | optional; the id of the measurement the study belongs to (6) |
+   | `plan` | optional; the study's plan, a path inside its folder: `plan.json` (7) |
+
+   **A row may carry more fields**, and they are kept: a paper merges its own metadata into the rows
+   (`frame`, `method`, `outcome`, …). The top level is closed — `prefix`, `studies`, and the optional
+   `author`, `requirements` and `measurements` (6) — so a misspelt key is an error, not an ignored
+   key.
 
 2. **The rule `repro/study-registered`, an error,** reported on the first line of
    `PIPELINE-STATUS.md` like `repro/allowed-files-only`, once per finding:
@@ -74,8 +78,9 @@ two commands that write it.
 3. **Two commands**, in `src/studies.ts`, wired in `src/cli.ts`:
 
    - `paperlint studies new <paper-dir> "<title>" [--kind measurement]` takes the next id, creates
-     `repro/<id>-<slug>/README.md` (frontmatter `id:` and `title:`), appends the row with
-     `status: designing`, `in_paper: no` and today's date, and regenerates the index block. It refuses a
+     `repro/<id>-<slug>/README.md` (frontmatter `id:` and `title:`) and the plan's template,
+     `plan.json` (7), appends the row with `status: designing`, `in_paper: no`, today's date and
+     `plan: "plan.json"`, and regenerates the index block. It refuses a
      title over 8 words, a title with no letter or digit to make a slug from, an unknown kind, a folder
      that already exists, and a paper with no `studies.json` (the prefix is the author's choice, so the
      refusal names the file to write and does not guess one).
@@ -93,19 +98,133 @@ two commands that write it.
    `paperlint studies new`, and update its row (status, result, result file, in paper) when it ends,
    then run `paperlint studies index`.
 
+6. **Measurements above studies, and the author's requirements.** One measurement (a census, a
+   benchmark) spans several study folders: a pilot, the main run, a replication. Before this, a paper
+   put the measurement's fields on the row of its first study under prefixed names
+   (`measurement_title`, `measurement_status`) and the other rows carried `measurement: <id>`, which
+   nothing checked; the author's verbatim requirements went to a separate file nothing read. Now:
+
+   ```json
+   {
+     "prefix": "P1",
+     "author": "A. Author",
+     "requirements": [
+       {
+         "id": "R1",
+         "text": "Every sample is a random draw from all rules found.",
+         "date": "2026-10-10",
+         "source": "chat"
+       }
+     ],
+     "measurements": [
+       {
+         "id": "M1",
+         "title": "Rule census",
+         "rq": "RQ1",
+         "role": "headline",
+         "approval": {
+           "by": "A. Author",
+           "date": "2026-10-12",
+           "quote": "This is the one."
+         },
+         "requirements": ["R1"]
+       }
+     ],
+     "studies": [
+       { "id": "P1-S01", "measurement": "M1", "plan": "plan.json", "…": "…" }
+     ]
+   }
+   ```
+
+   `role` is `headline` (the paper's claim rests on it) or `baseline`. A measurement keeps extra
+   fields like a row; a requirement and an approval are exactly their fields. The rule reports, each
+   finding its own message id: a row naming an undeclared measurement (`unknown-measurement`), a
+   measurement no row names (`empty-measurement`), a measurement naming an undeclared requirement
+   (`unknown-requirement`), and a headline without `approval`, or approved by anyone but `author`
+   when the registry names one (`headline-approval`). These cross-references are the rule's, not the
+   parser's: one dangling id must not fail the whole parse and hide every other finding. **These checks run only when the registry has
+   `measurements`**: a registry written before them, whose rows already carry `measurement` as a field
+   of their own, stays valid.
+
+7. **The study's plan, `plan.json`.** Each study folder held a hand-written `prereg.json`, its
+   fields differing study to study, read only by the experimental `study/declare-sample`. The name
+   misled — it is not an outside pre-registration — so the plan is `plan.json`, written as a
+   template by `paperlint studies new` beside the README, and named in the row's `plan`:
+
+   ```json
+   {
+     "question": "",
+     "population": {
+       "source": "",
+       "draw": { "kind": "all", "method": "" }
+     },
+     "arms": [],
+     "primary_outcome": "",
+     "author_check": null
+   }
+   ```
+
+   The template's draw is of all the population (`"kind": "all"`, a census); a sampled study changes
+   it to `{ "kind": "seeded", "method": "random", "seed": 7 }`. `arms` is empty for a census;
+   `author_check` becomes `{ "by", "date", "quote" }` once the author has seen the drawn units.
+   **Parse, don't validate:**
+
+   - **Closed at every level.** A misspelt key is an error; the paper's own keys (hypotheses,
+     models, budget) go in the one optional key `extra`, an object of anything.
+   - **One spelling of "none".** `author_check` is required, and `null` is "not yet"; an absent key
+     is an error, never a second way to say the same.
+   - **No contradictory state.** Each of these is a parse error, not a later finding:
+     - an `author_check` on a draft — the author approved a plan whose texts are empty;
+     - a draw that is not one of two kinds: `seeded`, whose `seed` is required and not null, or
+       `all` (a census), which has no seed;
+     - two arms of one name;
+     - a key of `extra` that is a field of the plan, which would say one thing twice.
+
+     The population's size is not stored: it is counted from the source, and a stored copy could
+     only contradict it. Nothing downstream read it.
+
+   - **Complete or not is a type.** The parser returns a plan `ready` (every required text filled)
+     or a `draft` with the list of its empty texts, or why it is no plan. The rule never looks at a
+     text again: a study that has run on a `draft` is `bad-plan`, with the parser's list.
+   - **The schemas are the domain's types.** The domain's types are written by hand (the domain
+     imports no zod); the adapter holds a compile-time proof that each schema's output — registry,
+     study, measurement, requirement, approval, author check, plan, ready-or-draft — and its domain type are the
+     same type both ways, so a field added to one alone stops the build.
+
+   The schemas live beside the registry's, in `src/adapters/study-registry/`. The rule reports a
+   `plan` outside the folder, missing, or not of the schema, and — once the study has run (`running`,
+   `done`, `stopped`, `superseded`) — a draft (`bad-plan`); a study of a headline measurement
+   that has run with no `author_check` in its plan, no plan, or a check by anyone but `author`
+   (`author-check`). A `designing` study's template is clean, so `studies new` never leaves the lint
+   red.
+
+   **Compatibility.** A row may name `prereg.json` as its plan: it is parsed by a loose schema of
+   its own — keys of its own accepted at every level, those at the top kept in `extra`, a seed makes
+   the draw `seeded`, none (or `null`) makes it `all`; a size is not read; an absent check is
+   `null` — and judged as the plan, the same contradictions refused; a second rule, `repro/study-plan-name`, warns that it moves to `plan.json` — a warning cannot
+   come from an error rule, since ESLint sets the level per rule. A `prereg.json` no row names is not
+   read, so no existing paper turns red.
+
+8. **A study's mode.** A measurement's plan declares how its outcomes are kept from the decisions
+   on its inputs: `decide_before_run` (the default `studies new` writes) or `exploratory` with its
+   reason. `paperlint studies gate` and the rule `repro/outputs-before-decision` hold it, and the
+   index carries it in three columns: the mode, the decision's first commit and date, and the first
+   output's commit. Its own note: [`study-modes.md`](study-modes.md).
+
 ## Where each part lives
 
-| part                                    | file                                | layer   |
-| --------------------------------------- | ----------------------------------- | ------- |
-| the types, the judge, the index, ids    | `src/domain/study-registry.ts`      | domain  |
-| the schema (zod), parsing the text      | `src/adapters/study-registry/`      | adapter |
-| what the app asks of a parser           | `src/ports/study-registry-codec.ts` | port    |
-| the rule: list the folder, read, report | `src/study-registry-rule.ts`        | app     |
-| the two commands                        | `src/studies.ts`                    | app     |
-| wiring, the clock for today's date      | `src/cli.ts`                        | root    |
+| part                                   | file                                | layer   |
+| -------------------------------------- | ----------------------------------- | ------- |
+| the types, the judge, the index, ids   | `src/domain/study-registry.ts`      | domain  |
+| the plan's type and template           | `src/domain/study-registry.ts`      | domain  |
+| the schemas (zod) of registry and plan | `src/adapters/study-registry/`      | adapter |
+| what the app asks of a parser          | `src/ports/study-registry-codec.ts` | port    |
+| both rules: list, read, report         | `src/study-registry-rule.ts`        | app     |
+| the two commands                       | `src/studies.ts`                    | app     |
+| wiring, the clock for today's date     | `src/cli.ts`                        | root    |
 
 The judge is pure: it takes the folder names, the parsed registry (or why it is absent or unreadable),
-the README's text, and which result files exist, and returns findings. The rule and the commands are
+the README's text, which result files exist, and each named plan parsed, and returns findings. The rule and the commands are
 the only places that touch the disk, through the `Files` and `ListDir` ports.
 
 ## Generated index, and prettier
@@ -127,4 +246,7 @@ a cell is escaped and a line break becomes a space.
 
 - Whether the `question` is one sentence, or the `result` one line, beyond their being strings.
 - What a study's `README.md` says, or whether its frontmatter matches the row.
-- Files inside a study folder other than `result_file`.
+- Files inside a study folder other than `result_file` and the `plan` its row names.
+- `study/declare-sample` still reads only `prereg.json`, and asks for different fields
+  (`strata`, `screens`, `coverage`, and an `author_check` of `who`, `when`, `units_seen`,
+  `units_drawn`, `verdict`); merging the two shapes is its own decision.

package/docs/e2e.md

@@ -8,10 +8,14 @@ or CI:
 | -------------------------- | ------------- | -------------------------------------------- | --------------------------- |
 | `npm run test:e2e:install` | `e2e-install` | npm and pnpm, and the network for `npm pack` | `test/e2e/install/*.e2e.ts` |
 | `npm run test:e2e:tex`     | `e2e-tex`     | TeX Live from `paperlint toolchain`, banal   | `test/e2e/tex/*.e2e.ts`     |
-| `npm run test:e2e`         | both          |                                              |                             |
-
-Both areas are part of `npm run check`, so nobody has to remember to call them. CI runs each in the
-job that has its environment: `test:e2e:install` in `gates`, `test:e2e:tex` in `build-e2e`.
+| `npm run test:e2e:studies` | `e2e-studies` | git                                          | `test/e2e/studies/*.e2e.ts` |
+| `npm run test:e2e`         | all three     |                                              |                             |
+
+All three areas are part of `npm run check`, so nobody has to remember to call them. CI runs each in
+the job that has its environment: `test:e2e:install` and `test:e2e:studies` in `gates`, `test:e2e:tex`
+in `build-e2e`. The studies run drives a study decided before its run through the real binary in a
+real git repository: the gate, the order the lint reads, a shallow clone named, and the two commits
+`studies index` writes into the index.
 
 This page says what they prove, what they deliberately do not, and when a change owes a new one.
 

package/docs/layout.md

@@ -18,8 +18,8 @@ flowchart TB
   subgraph S3["3 · study: repro/#lt;id#gt;-#lt;slug#gt;/"]
     direction TB
     D["<b>design</b> · build-benchmark<br/>paperlint studies new<br/>writes: the study's folder, README.md,<br/>its row in repro/studies.json<br/>checked by: repro/allowed-files-only,<br/>repro/study-registered"]
-    P["<b>pre-register</b> · build-benchmark<br/>writes: prereg.json, before the draw<br/>checked by: study/declare-sample (experimental, off)"]
-    A["<b>author check</b> · you see the drawn units<br/>writes: author_check in prereg.json<br/>checked by: study/declare-sample (experimental, off)"]
+    P["<b>plan</b> · build-benchmark<br/>fills: plan.json, before the draw<br/>checked by: repro/study-registered,<br/>once the study runs"]
+    A["<b>author check</b> · you see the drawn units<br/>writes: author_check in plan.json<br/>checked by: repro/study-registered,<br/>for a headline measurement"]
     R["<b>run</b> · your scripts<br/>writes: the study folder,<br/>repro/tools/, repro/data/<br/>checked by: repro/allowed-files-only"]
     L["<b>labels</b> · build-label-page<br/>a phone page; the answers go<br/>into the study's data/"]
     Y["<b>analysis</b> · build-benchmark<br/>writes: the numbers, NOTES.md,<br/>the study's row: status, result<br/>then paperlint studies index<br/>checked by: repro/study-registered"]
@@ -110,7 +110,9 @@ Both are explained in [`venues.md`](venues.md).
 | `repro/`, its top                    | 3 study          | `build-benchmark`, your scripts                                                                                                                                                                  | [`repro/allowed-files-only`](rules/repro/allowed-files-only.md); the hook `paper-folder-layout` refuses a new loose entry                                                |
 | `repro/studies.json`                 | 3 study          | `paperlint studies new` appends a row; you edit a row when its study ends; `paperlint studies index` regenerates the index in `repro/README.md`                                                  | [`repro/study-registered`](rules/repro/study-registered.md): a row per study folder, its fields, the index in sync                                                       |
 | `repro/<id>-<slug>/`                 | 3 study          | `paperlint studies new` (the folder and its `README.md`), then `build-benchmark` (the design, in its `README.md`)                                                                                | [`repro/allowed-files-only`](rules/repro/allowed-files-only.md): the name, and a `README.md` in it; [`repro/study-registered`](rules/repro/study-registered.md): its row |
-| `repro/<id>-<slug>/prereg.json`      | 3 study          | `build-benchmark`, before the sample is drawn; you add the author's check                                                                                                                        | [`study/declare-sample`](rules/study/declare-sample.md)                                                                                                                  |
+| `repro/<id>-<slug>/plan.json`        | 3 study          | `paperlint studies new` writes the template; `build-benchmark` fills it before the sample is drawn; you add the author's check                                                                   | [`repro/study-registered`](rules/repro/study-registered.md): its fields, filled once the study runs, and `author_check` in a headline measurement                        |
+| `repro/<id>-<slug>/prereg.json`      | 3 study          | the plan's old name, still read when the row names it                                                                                                                                            | [`repro/study-plan-name`](rules/repro/study-plan-name.md) (warning); [`study/declare-sample`](rules/study/declare-sample.md), off by default                             |
+| `repro/<id>-<slug>/decision.md`      | 3 study          | you: the decisions on the inputs of a study decided before its run, committed before the run (`paperlint studies gate`); the plan names it, and it may live elsewhere in the paper               | [`repro/outputs-before-decision`](rules/repro/outputs-before-decision.md)                                                                                                |
 | `repro/<id>-<slug>/data/`            | 3 study          | `build-label-page` exports the labels here                                                                                                                                                       | nothing                                                                                                                                                                  |
 | `repro/tools/`, `repro/data/`        | 3 study          | your scripts; `paperlint build` keeps `repro/data/references-cache.json`                                                                                                                         | nothing inside them                                                                                                                                                      |
 | `NOTES.md`                           | 3, then 4        | you: each number → the data it comes from; `build-label-page` adds a line per label file                                                                                                         | nothing                                                                                                                                                                  |
@@ -134,24 +136,27 @@ A check runs at one of four moments:
   The lint-only job skips what needs a build (pages, fonts, references).
 - **On demand** — a skill, when you ask for what it does. A skill stops nothing; it writes.
 
-| what                                                                                        | kind  | when it runs                                                | what it stops                                                                                                                           |
-| ------------------------------------------------------------------------------------------- | ----- | ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
-| `paper-folder-layout`                                                                       | hook  | before an agent creates a file (Edit, Write)                | a new entry the layout does not allow, at a paper's root or the top of its `repro/`; never an existing file                             |
-| `paper-edit-guard`                                                                          | hook  | before an agent's shell command                             | a shell command that writes or moves a paper's source, which would skip the hooks that run after an edit                                |
-| `paper-skills-nudge`                                                                        | hook  | after an agent edits a paper's source                       | nothing: it shows the agent the pre-submit checklist                                                                                    |
-| `paper-status-gates`                                                                        | hook  | after an agent edits a paper's source                       | nothing: it prints the paper's readiness and its unrun gates, for a person reading the transcript                                       |
-| [`paper/allowed-files-only`](rules/paper/allowed-files-only.md)                             | rule  | every `paperlint lint`, and CI                              | a stray file or folder at a paper's root (error)                                                                                        |
-| [`repro/allowed-files-only`](rules/repro/allowed-files-only.md)                             | rule  | every `paperlint lint`, and CI                              | a loose entry at the top of `repro/`, a folder that is not a study's, a study folder without `README.md` (error)                        |
-| [`repro/study-registered`](rules/repro/study-registered.md)                                 | rule  | every `paperlint lint`, and CI                              | a study folder without its row in `repro/studies.json`, a row that does not match, an index out of sync (error)                         |
-| [`study/declare-sample`](rules/study/declare-sample.md)                                     | rule  | every `paperlint lint`, and CI, once a paper turns it on    | a study's `prereg.json` without its population, strata, screens, coverage or the author's check (experimental, off by default)          |
-| [`paper/folder-venue-leftover`](rules/paper/folder-venue-leftover.md)                       | rule  | every `paperlint lint`, and CI                              | nothing: it warns when the folder is named after another venue                                                                          |
-| `review/frontmatter`, `sibling/frontmatter`                                                 | rule  | every `paperlint lint`, and CI                              | a review whose findings break the schema (error); a sibling card that does not say how much was read (warning)                          |
-| `paper/stages`, `paper/source`                                                              | rule  | every `paperlint lint`, and CI                              | a stage whose frozen PDF or source is missing or changed (error)                                                                        |
-| the rules on the source (`tex/*`, `paper/*`, `bib/*`)                                       | rule  | every `paperlint lint`, and CI                              | what each one's [row](rules.md) says                                                                                                    |
-| the rules on what `paperlint build` measured (`pdf/*`, `format/page-limit`, the references) | rule  | every `paperlint lint` after a build; CI's full job         | a PDF over the page limit, in the wrong font or format, or with a bad reference                                                         |
-| `study-accepted-papers`                                                                     | skill | when you ask; stage 2 (modes A, B) and stage 5 (modes C, D) | nothing: it fills the venue's corpus, and writes the strong-accept scorecard in `PIPELINE-STATUS.md`                                    |
-| `build-benchmark`                                                                           | skill | when you ask; stage 3                                       | nothing: it registers the study with `paperlint studies new`, writes its design and its `prereg.json`, and updates its row when it ends |
-| `build-label-page`                                                                          | skill | when you ask; stage 3                                       | nothing: it builds the labelling page, and exports the answers into the study's `data/`                                                 |
+| what                                                                                        | kind    | when it runs                                                | what it stops                                                                                                                                             |
+| ------------------------------------------------------------------------------------------- | ------- | ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `paper-folder-layout`                                                                       | hook    | before an agent creates a file (Edit, Write)                | a new entry the layout does not allow, at a paper's root or the top of its `repro/`; never an existing file                                               |
+| `paper-edit-guard`                                                                          | hook    | before an agent's shell command                             | a shell command that writes or moves a paper's source, which would skip the hooks that run after an edit                                                  |
+| `paper-skills-nudge`                                                                        | hook    | after an agent edits a paper's source                       | nothing: it shows the agent the pre-submit checklist                                                                                                      |
+| `paper-status-gates`                                                                        | hook    | after an agent edits a paper's source                       | nothing: it prints the paper's readiness and its unrun gates, for a person reading the transcript                                                         |
+| [`paper/allowed-files-only`](rules/paper/allowed-files-only.md)                             | rule    | every `paperlint lint`, and CI                              | a stray file or folder at a paper's root (error)                                                                                                          |
+| [`repro/allowed-files-only`](rules/repro/allowed-files-only.md)                             | rule    | every `paperlint lint`, and CI                              | a loose entry at the top of `repro/`, a folder that is not a study's, a study folder without `README.md` (error)                                          |
+| [`repro/study-registered`](rules/repro/study-registered.md)                                 | rule    | every `paperlint lint`, and CI                              | a study folder without its row in `repro/studies.json`, a row that does not match, a measurement or plan that does not hold, an index out of sync (error) |
+| [`repro/study-plan-name`](rules/repro/study-plan-name.md)                                   | rule    | every `paperlint lint`, and CI                              | nothing: it warns when a study's plan is still named `prereg.json`                                                                                        |
+| [`repro/outputs-before-decision`](rules/repro/outputs-before-decision.md)                   | rule    | every `paperlint lint`, and CI (with the whole history)     | an output of a study decided before its run that git shows was not made after the decision                                                                |
+| `paperlint studies gate`                                                                    | command | before a decide-first study's run                           | the run, until the decision is committed                                                                                                                  |
+| [`study/declare-sample`](rules/study/declare-sample.md)                                     | rule    | every `paperlint lint`, and CI, once a paper turns it on    | a study's `prereg.json` without its population, strata, screens, coverage or the author's check (experimental, off by default)                            |
+| [`paper/folder-venue-leftover`](rules/paper/folder-venue-leftover.md)                       | rule    | every `paperlint lint`, and CI                              | nothing: it warns when the folder is named after another venue                                                                                            |
+| `review/frontmatter`, `sibling/frontmatter`                                                 | rule    | every `paperlint lint`, and CI                              | a review whose findings break the schema (error); a sibling card that does not say how much was read (warning)                                            |
+| `paper/stages`, `paper/source`                                                              | rule    | every `paperlint lint`, and CI                              | a stage whose frozen PDF or source is missing or changed (error)                                                                                          |
+| the rules on the source (`tex/*`, `paper/*`, `bib/*`)                                       | rule    | every `paperlint lint`, and CI                              | what each one's [row](rules.md) says                                                                                                                      |
+| the rules on what `paperlint build` measured (`pdf/*`, `format/page-limit`, the references) | rule    | every `paperlint lint` after a build; CI's full job         | a PDF over the page limit, in the wrong font or format, or with a bad reference                                                                           |
+| `study-accepted-papers`                                                                     | skill   | when you ask; stage 2 (modes A, B) and stage 5 (modes C, D) | nothing: it fills the venue's corpus, and writes the strong-accept scorecard in `PIPELINE-STATUS.md`                                                      |
+| `build-benchmark`                                                                           | skill   | when you ask; stage 3                                       | nothing: it registers the study with `paperlint studies new`, writes its design, fills its `plan.json`, and updates its row when it ends                  |
+| `build-label-page`                                                                          | skill   | when you ask; stage 3                                       | nothing: it builds the labelling page, and exports the answers into the study's `data/`                                                                   |
 
 The hooks are wired by `paperlint init` ([`install.md`](install.md)). A rule's level and options:
 [`rules.md`](rules.md).