paperlint 10.1.1 → 10.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -0
- package/dist/adapters/study-registry/index.d.ts +6 -0
- package/dist/adapters/study-registry/index.d.ts.map +1 -0
- package/dist/adapters/study-registry/index.js +58 -0
- package/dist/adapters/study-registry/index.js.map +1 -0
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +40 -2
- package/dist/cli.js.map +1 -1
- package/dist/domain/repro-layout.d.ts +12 -4
- package/dist/domain/repro-layout.d.ts.map +1 -1
- package/dist/domain/repro-layout.js +49 -18
- package/dist/domain/repro-layout.js.map +1 -1
- package/dist/domain/scaffold-layout.d.ts +4 -3
- package/dist/domain/scaffold-layout.d.ts.map +1 -1
- package/dist/domain/scaffold-layout.js +7 -4
- package/dist/domain/scaffold-layout.js.map +1 -1
- package/dist/domain/study-registry.d.ts +176 -0
- package/dist/domain/study-registry.d.ts.map +1 -0
- package/dist/domain/study-registry.js +371 -0
- package/dist/domain/study-registry.js.map +1 -0
- package/dist/ports/study-registry-codec.d.ts +11 -0
- package/dist/ports/study-registry-codec.d.ts.map +1 -0
- package/dist/ports/study-registry-codec.js +2 -0
- package/dist/ports/study-registry-codec.js.map +1 -0
- package/dist/repro-layout-rule.d.ts.map +1 -1
- package/dist/repro-layout-rule.js +4 -3
- package/dist/repro-layout-rule.js.map +1 -1
- package/dist/studies.d.ts +27 -0
- package/dist/studies.d.ts.map +1 -0
- package/dist/studies.js +160 -0
- package/dist/studies.js.map +1 -0
- package/dist/study-registry-rule.d.ts +17 -0
- package/dist/study-registry-rule.d.ts.map +1 -0
- package/dist/study-registry-rule.js +115 -0
- package/dist/study-registry-rule.js.map +1 -0
- package/docs/design/study-registry.md +130 -0
- package/docs/layout.md +47 -44
- package/docs/rules/paper/allowed-files-only.md +3 -2
- package/docs/rules/repro/allowed-files-only.md +40 -30
- package/docs/rules/repro/study-registered.md +162 -0
- package/docs/rules/study/declare-sample.md +1 -1
- package/docs/rules.md +2 -1
- package/package.json +1 -1
- package/skills/build-benchmark/SKILL.md +20 -1
- package/skills/build-benchmark/SKILL.md.spec.ts +19 -0
- package/skills/build-label-page/SKILL.md +7 -4
- package/skills/build-label-page/SKILL.md.spec.ts +6 -3
- package/skills/paper-pipeline/references/artifact-checklist.md +2 -1
- package/skills/paper-pipeline/references/pipeline-status-template.md +5 -3
- package/skills/run-agent-grid/SKILL.md +8 -1
- package/skills/run-agent-grid/SKILL.md.spec.ts +7 -0
- package/src/adapters/study-registry/index.test.ts +144 -0
- package/src/adapters/study-registry/index.ts +70 -0
- package/src/cli.ts +51 -2
- package/src/domain/paper-layout.test.ts +1 -1
- package/src/domain/repro-layout.test.ts +18 -9
- package/src/domain/repro-layout.ts +58 -18
- package/src/domain/scaffold-layout.ts +7 -4
- package/src/domain/study-registry.test.ts +657 -0
- package/src/domain/study-registry.ts +598 -0
- package/src/folder-layout-hook.test.ts +10 -0
- package/src/new-paper.test.ts +28 -2
- package/src/ports/study-registry-codec.ts +11 -0
- package/src/repro-layout-rule.ts +4 -3
- package/src/rule-cases.test.ts +41 -0
- package/src/studies-cli.test.ts +113 -0
- package/src/studies.test.ts +353 -0
- package/src/studies.ts +259 -0
- package/src/study-registry-rule.test.ts +317 -0
- package/src/study-registry-rule.ts +172 -0
package/docs/layout.md
CHANGED
|
@@ -15,14 +15,14 @@ flowchart TB
|
|
|
15
15
|
S1["<b>1 · idea</b><br/>research-ideate · map-prior-work<br/>analyze-sibling-paper<br/>writes: siblings/<br/>checked by: sibling/frontmatter"]
|
|
16
16
|
S2["<b>2 · venue</b><br/>find-venue · plan-paper-timeline<br/>study-accepted-papers, mode A or B<br/>writes: paperlint.json, a report in reviews/<br/>checked by: review/frontmatter"]
|
|
17
17
|
V[("<b>#lt;papers-root#gt;/venues/#lt;venue#gt;/</b><br/>beside the papers, not in one:<br/>the venue's accepted papers,<br/>shared by every paper aimed at it<br/>checked by: nothing")]
|
|
18
|
-
subgraph S3["3 · study: repro/#lt;slug#gt
|
|
18
|
+
subgraph S3["3 · study: repro/#lt;id#gt;-#lt;slug#gt;/"]
|
|
19
19
|
direction TB
|
|
20
|
-
D["<b>design</b> · build-benchmark<br/>writes: the study's README.md<br/>checked by: repro/allowed-files-only"]
|
|
20
|
+
D["<b>design</b> · build-benchmark<br/>paperlint studies new<br/>writes: the study's folder, README.md,<br/>its row in repro/studies.json<br/>checked by: repro/allowed-files-only,<br/>repro/study-registered"]
|
|
21
21
|
P["<b>pre-register</b> · build-benchmark<br/>writes: prereg.json, before the draw<br/>checked by: study/declare-sample (experimental, off)"]
|
|
22
22
|
A["<b>author check</b> · you see the drawn units<br/>writes: author_check in prereg.json<br/>checked by: study/declare-sample (experimental, off)"]
|
|
23
23
|
R["<b>run</b> · your scripts<br/>writes: the study folder,<br/>repro/tools/, repro/data/<br/>checked by: repro/allowed-files-only"]
|
|
24
24
|
L["<b>labels</b> · build-label-page<br/>a phone page; the answers go<br/>into the study's data/"]
|
|
25
|
-
Y["<b>analysis</b> · build-benchmark<br/>writes: the numbers, NOTES.md"]
|
|
25
|
+
Y["<b>analysis</b> · build-benchmark<br/>writes: the numbers, NOTES.md,<br/>the study's row: status, result<br/>then paperlint studies index<br/>checked by: repro/study-registered"]
|
|
26
26
|
D --> P --> A --> R --> L --> Y
|
|
27
27
|
end
|
|
28
28
|
S4["<b>4 · draft</b><br/>draft-paper · argument-arc · tighten-paper<br/>writes: paper.tex, sections/, figures/<br/>checked by: the tex/* rules<br/>hook paper-edit-guard: no shell writes"]
|
|
@@ -69,11 +69,12 @@ creates one.
|
|
|
69
69
|
├── siblings/ ← the cards of competing papers
|
|
70
70
|
├── research/ ← dated research and decision records
|
|
71
71
|
├── repro/ ← the reproduction artifact
|
|
72
|
-
│ ├── README.md ← the index: one line per study
|
|
72
|
+
│ ├── README.md ← the index: one line per study, generated from studies.json
|
|
73
|
+
│ ├── studies.json ← the study registry: id, title, status and result of each study
|
|
73
74
|
│ ├── .gitignore ← keeps the scripts' byproducts out of git
|
|
74
75
|
│ ├── tools/ ← scripts used across the paper (numbers, figures, anonymity check, harnesses)
|
|
75
76
|
│ ├── data/ ← shared data (tables, caches)
|
|
76
|
-
│ └── <slug
|
|
77
|
+
│ └── <id>-<slug>/ ← one folder per study or spike, named by its row in studies.json (P1-S03-baseline)
|
|
77
78
|
│ └── README.md ← required: what it asks, how to run it, what it found
|
|
78
79
|
├── figures/ ← graphics and their sources
|
|
79
80
|
├── sections/ ← the body, when it is split
|
|
@@ -97,28 +98,29 @@ Both are explained in [`venues.md`](venues.md).
|
|
|
97
98
|
"Stage" is the stage of [the pipeline](#the-flow) that writes the place first.
|
|
98
99
|
"Checked by" is the rule `paperlint lint` runs over it; "nothing" means no rule reads its content.
|
|
99
100
|
|
|
100
|
-
| place
|
|
101
|
-
|
|
|
102
|
-
| `<papers-root>/venues/<venue>/`
|
|
103
|
-
| the paper folder's root
|
|
104
|
-
| `PIPELINE-STATUS.md`
|
|
105
|
-
| `paperlint.json`
|
|
106
|
-
| `siblings/`
|
|
107
|
-
| `reviews/`
|
|
108
|
-
| `research/`
|
|
109
|
-
| `repro/`, its top
|
|
110
|
-
| `repro
|
|
111
|
-
| `repro/<slug
|
|
112
|
-
| `repro/<slug
|
|
113
|
-
| `repro
|
|
114
|
-
| `
|
|
115
|
-
| `
|
|
116
|
-
| `
|
|
117
|
-
| `
|
|
118
|
-
| `
|
|
119
|
-
| `
|
|
120
|
-
| `
|
|
121
|
-
| `
|
|
101
|
+
| place | stage | written by | checked by |
|
|
102
|
+
| ------------------------------------ | ---------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
103
|
+
| `<papers-root>/venues/<venue>/` | 2 venue | `study-accepted-papers`, its first run for that venue (mode A, usually) | nothing ([why](venues.md#what-checks-it)) |
|
|
104
|
+
| the paper folder's root | before 1 | `paperlint new`: the three files, and an empty `.gitkeep` folder for `reviews/`, `research/`, `figures/`, `versions/`, `submission/`, `repro/tools/`, `repro/data/` (`repro/` gets a stub index) | [`paper/allowed-files-only`](rules/paper/allowed-files-only.md); the hook `paper-folder-layout` refuses a new stray entry |
|
|
105
|
+
| `PIPELINE-STATUS.md` | every stage | `paperlint new`, then `paper-pipeline`, `paper-status`; `study-accepted-papers` (the scorecard) | `paper/stages`, `paper/source`, [`cycle/stage`](rules/cycle/stage.md) ([rules](rules.md)) |
|
|
106
|
+
| `paperlint.json` | before 1, then 2 | `paperlint new`; `find-venue` names the venue | `paperlint lint` refuses an unknown key; [`cycle/record`](rules/cycle/record.md) |
|
|
107
|
+
| `siblings/` | 1 idea | `map-prior-work`, `analyze-sibling-paper` | `sibling/frontmatter` ([records](configuration.md#records-in-frontmatter-checked-by-shipped-json-schemas)) |
|
|
108
|
+
| `reviews/` | 2, then 4–5 | `study-accepted-papers` (its report), `grade-paper-writing`, `cold-read-diff`, the review skills | `review/frontmatter` ([records](configuration.md#records-in-frontmatter-checked-by-shipped-json-schemas)) |
|
|
109
|
+
| `research/` | any | you and the skills: dated research and decision records | nothing |
|
|
110
|
+
| `repro/`, its top | 3 study | `build-benchmark`, your scripts | [`repro/allowed-files-only`](rules/repro/allowed-files-only.md); the hook `paper-folder-layout` refuses a new loose entry |
|
|
111
|
+
| `repro/studies.json` | 3 study | `paperlint studies new` appends a row; you edit a row when its study ends; `paperlint studies index` regenerates the index in `repro/README.md` | [`repro/study-registered`](rules/repro/study-registered.md): a row per study folder, its fields, the index in sync |
|
|
112
|
+
| `repro/<id>-<slug>/` | 3 study | `paperlint studies new` (the folder and its `README.md`), then `build-benchmark` (the design, in its `README.md`) | [`repro/allowed-files-only`](rules/repro/allowed-files-only.md): the name, and a `README.md` in it; [`repro/study-registered`](rules/repro/study-registered.md): its row |
|
|
113
|
+
| `repro/<id>-<slug>/prereg.json` | 3 study | `build-benchmark`, before the sample is drawn; you add the author's check | [`study/declare-sample`](rules/study/declare-sample.md) |
|
|
114
|
+
| `repro/<id>-<slug>/data/` | 3 study | `build-label-page` exports the labels here | nothing |
|
|
115
|
+
| `repro/tools/`, `repro/data/` | 3 study | your scripts; `paperlint build` keeps `repro/data/references-cache.json` | nothing inside them |
|
|
116
|
+
| `NOTES.md` | 3, then 4 | you: each number → the data it comes from; `build-label-page` adds a line per label file | nothing |
|
|
117
|
+
| `paper.tex`, `sections/`, `*.bib` | 4 draft | `draft-paper`, `argument-arc`, `tighten-paper` | the `tex/*`, `paper/*` and `bib/*` rules ([rules](rules.md)); the hook `paper-edit-guard` refuses a shell write |
|
|
118
|
+
| `CLAIMS.md`, `figures/` | 4 draft | you | nothing |
|
|
119
|
+
| `paper.pdf`, `_build/` | 6 submit | `paperlint build` | `pdf/*`, [`format/page-limit`](rules/format/page-limit.md), [`anonymity/identity`](rules/anonymity/identity.md), the reference rules |
|
|
120
|
+
| `versions/` | 6, then 7 | you, at submission and at camera-ready (`submit-paper` says how): the frozen PDF and source of each stage | `paper/stages`, `paper/source` |
|
|
121
|
+
| `submission/`, `SUBMIT-CHECKLIST.md` | 6 submit | you, while `submit-paper` and `submission-portal` do the submitting: forms, portal records, mail with the venue | [`cycle/evidence`](rules/cycle/evidence.md), for the files a closed cycle names |
|
|
122
|
+
| `talk/` | 7 camera-ready | you | the `talk/*` rules ([talk](talk.md)) |
|
|
123
|
+
| `extra/` | any | you | nothing, ever |
|
|
122
124
|
|
|
123
125
|
## What runs when
|
|
124
126
|
|
|
@@ -132,23 +134,24 @@ A check runs at one of four moments:
|
|
|
132
134
|
The lint-only job skips what needs a build (pages, fonts, references).
|
|
133
135
|
- **On demand** — a skill, when you ask for what it does. A skill stops nothing; it writes.
|
|
134
136
|
|
|
135
|
-
| what | kind | when it runs | what it stops
|
|
136
|
-
| ------------------------------------------------------------------------------------------- | ----- | ----------------------------------------------------------- |
|
|
137
|
-
| `paper-folder-layout` | hook | before an agent creates a file (Edit, Write) | a new entry the layout does not allow, at a paper's root or the top of its `repro/`; never an existing file
|
|
138
|
-
| `paper-edit-guard` | hook | before an agent's shell command | a shell command that writes or moves a paper's source, which would skip the hooks that run after an edit
|
|
139
|
-
| `paper-skills-nudge` | hook | after an agent edits a paper's source | nothing: it shows the agent the pre-submit checklist
|
|
140
|
-
| `paper-status-gates` | hook | after an agent edits a paper's source | nothing: it prints the paper's readiness and its unrun gates, for a person reading the transcript
|
|
141
|
-
| [`paper/allowed-files-only`](rules/paper/allowed-files-only.md) | rule | every `paperlint lint`, and CI | a stray file or folder at a paper's root (error)
|
|
142
|
-
| [`repro/allowed-files-only`](rules/repro/allowed-files-only.md) | rule | every `paperlint lint`, and CI | a loose entry at the top of `repro/`,
|
|
143
|
-
| [`study
|
|
144
|
-
| [`
|
|
145
|
-
| `
|
|
146
|
-
| `
|
|
147
|
-
|
|
|
148
|
-
| the rules on
|
|
149
|
-
| `
|
|
150
|
-
| `
|
|
151
|
-
| `build-
|
|
137
|
+
| what | kind | when it runs | what it stops |
|
|
138
|
+
| ------------------------------------------------------------------------------------------- | ----- | ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
|
|
139
|
+
| `paper-folder-layout` | hook | before an agent creates a file (Edit, Write) | a new entry the layout does not allow, at a paper's root or the top of its `repro/`; never an existing file |
|
|
140
|
+
| `paper-edit-guard` | hook | before an agent's shell command | a shell command that writes or moves a paper's source, which would skip the hooks that run after an edit |
|
|
141
|
+
| `paper-skills-nudge` | hook | after an agent edits a paper's source | nothing: it shows the agent the pre-submit checklist |
|
|
142
|
+
| `paper-status-gates` | hook | after an agent edits a paper's source | nothing: it prints the paper's readiness and its unrun gates, for a person reading the transcript |
|
|
143
|
+
| [`paper/allowed-files-only`](rules/paper/allowed-files-only.md) | rule | every `paperlint lint`, and CI | a stray file or folder at a paper's root (error) |
|
|
144
|
+
| [`repro/allowed-files-only`](rules/repro/allowed-files-only.md) | rule | every `paperlint lint`, and CI | a loose entry at the top of `repro/`, a folder that is not a study's, a study folder without `README.md` (error) |
|
|
145
|
+
| [`repro/study-registered`](rules/repro/study-registered.md) | rule | every `paperlint lint`, and CI | a study folder without its row in `repro/studies.json`, a row that does not match, an index out of sync (error) |
|
|
146
|
+
| [`study/declare-sample`](rules/study/declare-sample.md) | rule | every `paperlint lint`, and CI, once a paper turns it on | a study's `prereg.json` without its population, strata, screens, coverage or the author's check (experimental, off by default) |
|
|
147
|
+
| [`paper/folder-venue-leftover`](rules/paper/folder-venue-leftover.md) | rule | every `paperlint lint`, and CI | nothing: it warns when the folder is named after another venue |
|
|
148
|
+
| `review/frontmatter`, `sibling/frontmatter` | rule | every `paperlint lint`, and CI | a review whose findings break the schema (error); a sibling card that does not say how much was read (warning) |
|
|
149
|
+
| `paper/stages`, `paper/source` | rule | every `paperlint lint`, and CI | a stage whose frozen PDF or source is missing or changed (error) |
|
|
150
|
+
| the rules on the source (`tex/*`, `paper/*`, `bib/*`) | rule | every `paperlint lint`, and CI | what each one's [row](rules.md) says |
|
|
151
|
+
| the rules on what `paperlint build` measured (`pdf/*`, `format/page-limit`, the references) | rule | every `paperlint lint` after a build; CI's full job | a PDF over the page limit, in the wrong font or format, or with a bad reference |
|
|
152
|
+
| `study-accepted-papers` | skill | when you ask; stage 2 (modes A, B) and stage 5 (modes C, D) | nothing: it fills the venue's corpus, and writes the strong-accept scorecard in `PIPELINE-STATUS.md` |
|
|
153
|
+
| `build-benchmark` | skill | when you ask; stage 3 | nothing: it registers the study with `paperlint studies new`, writes its design and its `prereg.json`, and updates its row when it ends |
|
|
154
|
+
| `build-label-page` | skill | when you ask; stage 3 | nothing: it builds the labelling page, and exports the answers into the study's `data/` |
|
|
152
155
|
|
|
153
156
|
The hooks are wired by `paperlint init` ([`install.md`](install.md)). A rule's level and options:
|
|
154
157
|
[`rules.md`](rules.md).
|
|
@@ -34,11 +34,12 @@ it: [the map](../../layout.md).
|
|
|
34
34
|
├── siblings/ ← the cards of competing papers
|
|
35
35
|
├── research/ ← dated research and decision records
|
|
36
36
|
├── repro/ ← the reproduction artifact
|
|
37
|
-
│ ├── README.md ← the index: one line per study
|
|
37
|
+
│ ├── README.md ← the index: one line per study, generated from studies.json
|
|
38
|
+
│ ├── studies.json ← the study registry: id, title, status and result of each study
|
|
38
39
|
│ ├── .gitignore ← keeps the scripts' byproducts out of git
|
|
39
40
|
│ ├── tools/ ← scripts used across the paper (numbers, figures, anonymity check, harnesses)
|
|
40
41
|
│ ├── data/ ← shared data (tables, caches)
|
|
41
|
-
│ └── <slug
|
|
42
|
+
│ └── <id>-<slug>/ ← one folder per study or spike, named by its row in studies.json (P1-S03-baseline)
|
|
42
43
|
│ └── README.md ← required: what it asks, how to run it, what it found
|
|
43
44
|
├── figures/ ← graphics and their sources
|
|
44
45
|
├── sections/ ← the body, when it is split
|
|
@@ -7,8 +7,8 @@ per misplaced entry
|
|
|
7
7
|
## What it catches
|
|
8
8
|
|
|
9
9
|
An entry at the top of a paper's `repro/` that is not part of its layout, and a study folder there
|
|
10
|
-
without a `README.md`. `repro/` holds an index, the tools and data the whole
|
|
11
|
-
|
|
10
|
+
without a `README.md`. `repro/` holds an index, the study registry, the tools and data the whole
|
|
11
|
+
paper shares, and one folder per study. Stage 3 of the pipeline, the study, writes here; the rule runs on every
|
|
12
12
|
`paperlint lint` and in CI, and a hook refuses a new loose entry before it is written. The whole map:
|
|
13
13
|
[`layout.md`](../../layout.md).
|
|
14
14
|
|
|
@@ -16,12 +16,13 @@ dated folder per study. Stage 3 of the pipeline, the study, writes here; the rul
|
|
|
16
16
|
|
|
17
17
|
```text
|
|
18
18
|
repro/
|
|
19
|
-
├── README.md
|
|
20
|
-
├── .
|
|
21
|
-
├──
|
|
22
|
-
├──
|
|
23
|
-
|
|
24
|
-
|
|
19
|
+
├── README.md ← the index: one line per study, generated from studies.json
|
|
20
|
+
├── studies.json ← the study registry: id, title, status and result of each study
|
|
21
|
+
├── .gitignore ← keeps the scripts' byproducts out of git
|
|
22
|
+
├── tools/ ← scripts used across the paper (numbers, figures, anonymity check, harnesses)
|
|
23
|
+
├── data/ ← shared data (tables, caches)
|
|
24
|
+
└── <id>-<slug>/ ← one folder per study or spike, named by its row in studies.json (P1-S03-baseline)
|
|
25
|
+
└── README.md ← required: what it asks, how to run it, what it found
|
|
25
26
|
```
|
|
26
27
|
|
|
27
28
|
<!-- layout:end -->
|
|
@@ -33,10 +34,16 @@ above, and it already holds the index.
|
|
|
33
34
|
A loose note belongs in the folder of the study it is about, or in the paper's `research/` when it is
|
|
34
35
|
research or a decision rather than the record of a run.
|
|
35
36
|
|
|
36
|
-
**
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
37
|
+
**A study's name** is `<id>-<slug>`: the id of its row in `repro/studies.json` — the paper's prefix,
|
|
38
|
+
`-S`, two or more digits — then a slug, lowercase letters and digits in words joined by single
|
|
39
|
+
hyphens: `P1-S03-baseline`. `npx paperlint studies new` makes the folder, its `README.md` and its row.
|
|
40
|
+
This rule accepts the shape; whether the row exists, and matches, is
|
|
41
|
+
[`repro/study-registered`](study-registered.md).
|
|
42
|
+
|
|
43
|
+
**The dated form, `<slug>-YYYY-MM-DD`**, is accepted too, so a paper that keeps its dated folders and
|
|
44
|
+
turns `repro/study-registered` off is not broken: `ablation-2026-10-12`, the day the study began. A
|
|
45
|
+
month is accepted as well, `human-labels-2026-10`, for a study that ran over weeks. The date must be a
|
|
46
|
+
real one — no month 13, no 30 February.
|
|
40
47
|
|
|
41
48
|
Never reported: what interpreters, package managers, test runners and the operating system leave in
|
|
42
49
|
the folder they run in — `__pycache__/`, `.pytest_cache/`, `.mypy_cache/`, `.ruff_cache/`,
|
|
@@ -55,14 +62,15 @@ or where a new run goes. Each kind of material now has one place, and each study
|
|
|
55
62
|
Failing: a script loose at the top.
|
|
56
63
|
|
|
57
64
|
```text
|
|
58
|
-
papers/
|
|
65
|
+
papers/example/PIPELINE-STATUS.md
|
|
59
66
|
1:1 error `repro/paper_numbers.py` is loose at the top of the paper's repro/ folder, which holds
|
|
60
|
-
only an index, the shared tools and data, and one
|
|
61
|
-
goes: repro/tools/ — the scripts used across the paper (numbers, figures, the
|
|
62
|
-
check, harnesses); a script only one study runs goes in that study's folder
|
|
67
|
+
only an index, the study registry, the shared tools and data, and one folder per study.
|
|
68
|
+
Where it goes: repro/tools/ — the scripts used across the paper (numbers, figures, the
|
|
69
|
+
anonymity check, harnesses); a script only one study runs goes in that study's folder
|
|
70
|
+
instead.
|
|
63
71
|
What repro/ holds (what interpreters and package managers leave there, like __pycache__/ and node_modules/, is never reported):
|
|
64
72
|
repro/
|
|
65
|
-
├── README.md
|
|
73
|
+
├── README.md ← the index: one line per study, generated from studies.json
|
|
66
74
|
…
|
|
67
75
|
Rule repro/allowed-files-only: https://github.com/zernie/paperlint/blob/main/docs/rules/repro/allowed-files-only.md repro/allowed-files-only
|
|
68
76
|
```
|
|
@@ -70,14 +78,14 @@ Rule repro/allowed-files-only: https://github.com/zernie/paperlint/blob/main/doc
|
|
|
70
78
|
Each message names the entry and where it goes, and draws the layout above, so it reads on its own. Where
|
|
71
79
|
an entry goes depends on what it is:
|
|
72
80
|
|
|
73
|
-
| entry | where it goes
|
|
74
|
-
| ------------------------------------------------------------------------------------------------------------------------------ |
|
|
75
|
-
| a script (`.py`, `.ipynb`, `.sh`, `.js`, `.mjs`, `.ts`, `.r`, `.jl`, `.sql` …), `Makefile`, `requirements.txt`, `package.json` | `repro/tools/`, or the folder of the one study that runs it
|
|
76
|
-
| data (`.tsv`, `.csv`, `.json`, `.jsonl`, `.yaml`, `.txt`, `.log`, `.parquet`, `.zip` …) | `repro/data/`, or the folder of the one study that produced it
|
|
77
|
-
| a note (`.md`) | the
|
|
78
|
-
| a folder whose name is
|
|
79
|
-
| a study folder without `README.md` | add one: what the study asks, how to run it, what it found; and
|
|
80
|
-
| anything else | by what it is, as above
|
|
81
|
+
| entry | where it goes |
|
|
82
|
+
| ------------------------------------------------------------------------------------------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
83
|
+
| a script (`.py`, `.ipynb`, `.sh`, `.js`, `.mjs`, `.ts`, `.r`, `.jl`, `.sql` …), `Makefile`, `requirements.txt`, `package.json` | `repro/tools/`, or the folder of the one study that runs it |
|
|
84
|
+
| data (`.tsv`, `.csv`, `.json`, `.jsonl`, `.yaml`, `.txt`, `.log`, `.parquet`, `.zip` …) | `repro/data/`, or the folder of the one study that produced it |
|
|
85
|
+
| a note (`.md`) | the folder of the study it is about, or the paper's `research/` |
|
|
86
|
+
| a folder whose name is neither `<id>-<slug>` nor `<slug>-YYYY-MM-DD` | rename it to `<id>-<slug>` with its row in `repro/studies.json` and a `README.md`; with `repro/study-registered` off, `<slug>-YYYY-MM-DD`, dated by its first commit |
|
|
87
|
+
| a study folder without `README.md` | add one: what the study asks, how to run it, what it found; and the study's row in `repro/studies.json` |
|
|
88
|
+
| anything else | by what it is, as above |
|
|
81
89
|
|
|
82
90
|
Three files paperlint itself once kept at the top of `repro/` are named with their new place and the
|
|
83
91
|
command that moves them: `references-cache.json` → `repro/data/` (read by `paperlint build`),
|
|
@@ -98,7 +106,8 @@ No rule options.
|
|
|
98
106
|
## What it does not check
|
|
99
107
|
|
|
100
108
|
- What is inside `tools/`, `data/` and the study folders.
|
|
101
|
-
- Whether
|
|
109
|
+
- Whether every study folder has its row in `repro/studies.json`, and whether `repro/README.md`'s index
|
|
110
|
+
is in sync: that is [`repro/study-registered`](study-registered.md).
|
|
102
111
|
- What a study's `README.md` says.
|
|
103
112
|
- `repro/` itself: whether the paper has one is `paper/allowed-files-only`'s concern, and that rule never
|
|
104
113
|
looks inside it. Each entry is judged by exactly one of the two rules.
|
|
@@ -106,18 +115,19 @@ No rule options.
|
|
|
106
115
|
## The hook
|
|
107
116
|
|
|
108
117
|
The hook `paper-folder-layout` refuses, before it is written, a new file that would add an entry
|
|
109
|
-
this rule reports — a loose file at the top of `repro/`,
|
|
118
|
+
this rule reports — a loose file at the top of `repro/`, a folder that is not a study's, or a study folder whose
|
|
110
119
|
first file is not its `README.md` — with this rule's message as the reason. It never refuses a file
|
|
111
120
|
that already exists, and it cannot see a shell `mv` or `cp`: the lint is the guarantee. What it does
|
|
112
121
|
and does not do is on [`paper/allowed-files-only`'s page](../paper/allowed-files-only.md#the-hook-a-new-stray-entry-is-refused-before-it-is-written).
|
|
113
122
|
|
|
114
123
|
## How to fix
|
|
115
124
|
|
|
116
|
-
Move or rename the entry where its message says, and update whatever names its path
|
|
125
|
+
Move or rename the entry where its message says, and update whatever names its path; a renamed study
|
|
126
|
+
folder gets its row in `repro/studies.json` in the same commit:
|
|
117
127
|
|
|
118
128
|
```sh
|
|
119
|
-
git mv papers/
|
|
120
|
-
git mv papers/
|
|
129
|
+
git mv papers/example/repro/paper_numbers.py papers/example/repro/tools/
|
|
130
|
+
git mv papers/example/repro/baseline papers/example/repro/P1-S04-baseline
|
|
121
131
|
```
|
|
122
132
|
|
|
123
133
|
To switch the rule off for one paper — say, one whose `repro/` predates the layout — set it in that
|
|
@@ -0,0 +1,162 @@
|
|
|
1
|
+
# repro/study-registered
|
|
2
|
+
|
|
3
|
+
**Level:** error · **Reads:** the listing of the paper's `repro/`, `repro/studies.json`,
|
|
4
|
+
`repro/README.md`, and each `result_file` the registry names · **Reported on:** the first line of the
|
|
5
|
+
paper's `PIPELINE-STATUS.md`, once per finding
|
|
6
|
+
|
|
7
|
+
## What it catches
|
|
8
|
+
|
|
9
|
+
A study that is not in the paper's study registry, a row that does not match the folders, and an
|
|
10
|
+
index that does not match the registry. The registry is `repro/studies.json`: one row per study,
|
|
11
|
+
with a stable id, a plain name, its status and its result. Every study folder at the top of `repro/`
|
|
12
|
+
is named `<id>-<slug>` after its row — `P1-S03-baseline`. The design: [`study-registry.md`](../../design/study-registry.md).
|
|
13
|
+
|
|
14
|
+
```json
|
|
15
|
+
{
|
|
16
|
+
"prefix": "P1",
|
|
17
|
+
"studies": [
|
|
18
|
+
{
|
|
19
|
+
"id": "P1-S01",
|
|
20
|
+
"folder": "P1-S01-baseline-pass-rate",
|
|
21
|
+
"title": "Baseline pass rate",
|
|
22
|
+
"question": "How often does the unmodified agent pass the task?",
|
|
23
|
+
"kind": "measurement",
|
|
24
|
+
"status": "done",
|
|
25
|
+
"date": "2026-10-15",
|
|
26
|
+
"result": "41 of 60 runs pass",
|
|
27
|
+
"result_file": "results.tsv",
|
|
28
|
+
"in_paper": "yes"
|
|
29
|
+
}
|
|
30
|
+
]
|
|
31
|
+
}
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
| field | what it holds |
|
|
35
|
+
| --------------- | -------------------------------------------------------------------------------- |
|
|
36
|
+
| `prefix` | the paper's prefix: a capital letter, then capitals or digits (`P1`) |
|
|
37
|
+
| `id` | `<prefix>-S` and two or more digits; unique |
|
|
38
|
+
| `folder` | `<id>-<slug>`, the study's folder at the top of `repro/` |
|
|
39
|
+
| `title` | a plain name of at most 8 words |
|
|
40
|
+
| `question` | one sentence: what the study asks |
|
|
41
|
+
| `kind` | `measurement`, `spike`, `tooling`, `data`, `artifact` or `salvage` |
|
|
42
|
+
| `status` | `designing`, `running`, `done`, `stopped`, `superseded` or `dropped` |
|
|
43
|
+
| `date` | `YYYY-MM-DD`, the day the study was registered |
|
|
44
|
+
| `result` | optional: the result in one line; required when `done` |
|
|
45
|
+
| `result_file` | optional: the file inside the folder the result is read from; required when done |
|
|
46
|
+
| `superseded_by` | optional: the id of the study that replaced it; required when `superseded` |
|
|
47
|
+
| `in_paper` | `yes`, `no`, or `registry` (listed in the registry, not in the paper's text) |
|
|
48
|
+
|
|
49
|
+
A row may carry more fields — a paper's own measurement metadata (`role`, `frame`, `method`,
|
|
50
|
+
`outcome`, …) — and `paperlint studies new` keeps them when it rewrites the file. The top level holds
|
|
51
|
+
only `prefix` and `studies`.
|
|
52
|
+
|
|
53
|
+
The rule reports:
|
|
54
|
+
|
|
55
|
+
- a folder at the top of `repro/` with no row (`tools/`, `data/` and what interpreters and package
|
|
56
|
+
managers leave there, like `__pycache__/` and `node_modules/`, are not studies);
|
|
57
|
+
- a row whose folder does not exist, and a folder named by more than one row;
|
|
58
|
+
- an id that is not `<prefix>-S<digits>`, an id on more than one row, and a folder that is not
|
|
59
|
+
`<id>-<slug>`;
|
|
60
|
+
- a title over 8 words;
|
|
61
|
+
- `done` without `result` or `result_file`, and a `result_file` that is not a file inside the folder;
|
|
62
|
+
- `superseded` without `superseded_by`, or naming an id that is not in the registry;
|
|
63
|
+
- an index out of sync: the block of `repro/README.md` between the markers below must equal the
|
|
64
|
+
text the registry generates, character for character;
|
|
65
|
+
|
|
66
|
+
```text
|
|
67
|
+
<!-- studies:begin — generated from studies.json by `paperlint studies index`; do not edit -->
|
|
68
|
+
<!-- studies:end -->
|
|
69
|
+
```
|
|
70
|
+
|
|
71
|
+
- a `repro/` with study folders and no `repro/studies.json` (one finding naming the file), and a
|
|
72
|
+
`studies.json` that is not JSON or not of the shape above (one finding with the reasons).
|
|
73
|
+
|
|
74
|
+
## Why
|
|
75
|
+
|
|
76
|
+
A paper's `repro/` grew to 79 study folders named `<slug>-YYYY-MM-DD`, and nobody could tell which was
|
|
77
|
+
which, which had ended, or which one a number in the paper came from. A date and a slug say when a
|
|
78
|
+
study began, not what it is. The registry gives each study a number that does not change and a name a
|
|
79
|
+
person can read, and says in one place where every study stands. A rule holds it, not a guideline: a
|
|
80
|
+
row the rule does not check is a row that drifts.
|
|
81
|
+
|
|
82
|
+
The index in `repro/README.md` is generated, so it is never edited by hand and never parsed: the rule
|
|
83
|
+
compares strings. It is a table padded the way prettier writes one, so formatting the README leaves
|
|
84
|
+
it in sync.
|
|
85
|
+
|
|
86
|
+
## Examples
|
|
87
|
+
|
|
88
|
+
Failing: a folder in the dated form beside a registered study.
|
|
89
|
+
|
|
90
|
+
```text
|
|
91
|
+
papers/example/PIPELINE-STATUS.md
|
|
92
|
+
1:1 error `repro/baseline-2026-10-02/` has no row in `repro/studies.json`, and every study folder
|
|
93
|
+
has exactly one. What to do: add its row with the next free id, `P1-S02`, and rename the
|
|
94
|
+
folder to `P1-S02-<slug>`; a new study is created with
|
|
95
|
+
`npx paperlint studies new <paper-dir> "<title>"`, which does both.
|
|
96
|
+
Rule repro/study-registered: https://github.com/zernie/paperlint/blob/main/docs/rules/repro/study-registered.md repro/study-registered
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
Failing: a row set to `done` by hand, with no result, and the index not regenerated — two findings.
|
|
100
|
+
|
|
101
|
+
Passing: a paper with no study folder; and a `repro/` made only by `paperlint studies new`, its rows
|
|
102
|
+
edited as the studies end and its index regenerated:
|
|
103
|
+
|
|
104
|
+
```text
|
|
105
|
+
repro/
|
|
106
|
+
├── README.md
|
|
107
|
+
├── studies.json
|
|
108
|
+
├── P1-S01-baseline-pass-rate/
|
|
109
|
+
│ ├── README.md
|
|
110
|
+
│ └── results.tsv
|
|
111
|
+
├── tools/
|
|
112
|
+
└── data/
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
```markdown
|
|
116
|
+
<!-- studies:begin — generated from studies.json by `paperlint studies index`; do not edit -->
|
|
117
|
+
|
|
118
|
+
| id | title | status | result | in paper | folder |
|
|
119
|
+
| ------ | ------------------ | ------ | ------------------ | -------- | -------------------------------------------------------- |
|
|
120
|
+
| P1-S01 | Baseline pass rate | done | 41 of 60 runs pass | yes | [P1-S01-baseline-pass-rate/](P1-S01-baseline-pass-rate/) |
|
|
121
|
+
|
|
122
|
+
<!-- studies:end -->
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
## Options / preset fields
|
|
126
|
+
|
|
127
|
+
No rule options.
|
|
128
|
+
|
|
129
|
+
## What it does not check
|
|
130
|
+
|
|
131
|
+
- Whether `question` is one sentence and `result` one line, beyond their being text.
|
|
132
|
+
- What a study's `README.md` says, or whether its frontmatter matches the row.
|
|
133
|
+
- Whether the paper cites a study: `in_paper` is declared, not detected.
|
|
134
|
+
- What may stand at the top of `repro/` besides the study folders, and whether a study folder has a
|
|
135
|
+
`README.md`: that is [`repro/allowed-files-only`](allowed-files-only.md).
|
|
136
|
+
|
|
137
|
+
## How to fix
|
|
138
|
+
|
|
139
|
+
Create every new study with the command, which takes the next id, makes the folder and its
|
|
140
|
+
`README.md`, appends the row and regenerates the index:
|
|
141
|
+
|
|
142
|
+
```sh
|
|
143
|
+
npx paperlint studies new papers/example "Baseline pass rate" --kind measurement
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
When a study ends, edit its row in `repro/studies.json` — `status`, `result`, `result_file`,
|
|
147
|
+
`in_paper`, and `superseded_by` when another study replaced it — then regenerate the index:
|
|
148
|
+
|
|
149
|
+
```sh
|
|
150
|
+
npx paperlint studies index papers/example
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
A paper without a registry writes `repro/studies.json` once, with its prefix:
|
|
154
|
+
`{ "prefix": "P1", "studies": [] }`. A folder that predates the registry gets a row by hand and is
|
|
155
|
+
renamed to `<id>-<slug>` with `git mv`.
|
|
156
|
+
|
|
157
|
+
To switch the rule off for one paper — one that keeps its dated `<slug>-YYYY-MM-DD` folders —
|
|
158
|
+
set it in that paper's `paperlint.json`; `repro/allowed-files-only` still accepts the dated form:
|
|
159
|
+
|
|
160
|
+
```json
|
|
161
|
+
{ "rules": { "repro/study-registered": "off" } }
|
|
162
|
+
```
|
|
@@ -12,7 +12,7 @@ its name) · **Reported on:** the first line of the paper's
|
|
|
12
12
|
## What it catches
|
|
13
13
|
|
|
14
14
|
A study's pre-registration that does not say how its sample relates to the population the paper
|
|
15
|
-
talks about. Every `prereg.json` in a study folder (`repro/<slug
|
|
15
|
+
talks about. Every `prereg.json` in a study folder (`repro/<id>-<slug>/`) must carry five
|
|
16
16
|
fields; every other key in the file is the author's and is not read.
|
|
17
17
|
|
|
18
18
|
| field | holds |
|
package/docs/rules.md
CHANGED
|
@@ -14,7 +14,8 @@ reads and when it fails. Errors fail `paperlint lint`; warnings print and do not
|
|
|
14
14
|
| [`cycle/port-done`](rules/cycle/port-done.md) | error | `paper.tex` → `paperlint.json` (`cycles`) → the preset's `template` | the current cycle still declares `"phase": "porting"` (which silences the venue's format rules) while the `\documentclass` is already the venue's template: the port is done, drop the phase — [page](rules/cycle/port-done.md) |
|
|
15
15
|
| [`paper/folder-venue-leftover`](rules/paper/folder-venue-leftover.md) | warn | the folder name → `paperlint.json` | the paper's folder is named after another shipped venue than the one it extends (`papers/aisec-2026/` for a paper now set for AIDC); reported on `PIPELINE-STATUS.md` — [page](rules/paper/folder-venue-leftover.md) |
|
|
16
16
|
| [`paper/allowed-files-only`](rules/paper/allowed-files-only.md) | error | the paper folder's root listing → `paperlint.json` | a file or folder at the paper's root is not part of the layout (`PLAN.md`, `archive/`, a stray PDF); reported on `PIPELINE-STATUS.md`, once per entry, with where it goes; `extra/` takes anything — [page](rules/paper/allowed-files-only.md) |
|
|
17
|
-
| [`repro/allowed-files-only`](rules/repro/allowed-files-only.md) | error | the listing of the paper's `repro/` | an entry at the top of `repro/` is not `README.md`, `.gitignore`, `tools/`, `data/` or a study folder named `<slug
|
|
17
|
+
| [`repro/allowed-files-only`](rules/repro/allowed-files-only.md) | error | the listing of the paper's `repro/` | an entry at the top of `repro/` is not `README.md`, `studies.json`, `.gitignore`, `tools/`, `data/` or a study folder named `<id>-<slug>` (or, in a paper without the registry, `<slug>-YYYY-MM-DD`), or a study folder has no `README.md`; reported on `PIPELINE-STATUS.md`, once per entry, with where it goes by kind (a script → `repro/tools/`, data → `repro/data/`, a note → its study or `research/`) — [page](rules/repro/allowed-files-only.md) |
|
|
18
|
+
| [`repro/study-registered`](rules/repro/study-registered.md) | error | the listing of the paper's `repro/`, `repro/studies.json`, `repro/README.md` | a study folder with no row in `repro/studies.json`, a row whose folder is missing or not `<id>-<slug>`, a bad or repeated id, a title over 8 words, `done` without a result and an existing result file, `superseded` without an existing successor, or the index in `repro/README.md` out of sync with the registry; `paperlint studies new` and `paperlint studies index` write both — [page](rules/repro/study-registered.md) |
|
|
18
19
|
| [`paper/sources-fresh`](rules/paper/sources-fresh.md) | warn | `paper.tex` → `_build/sources.json` | the paper has not been built (no record of the files TeX read), or a file changed since the last build (its SHA-256 differs) — run `npx paperlint build` |
|
|
19
20
|
| `paper/author-list` | error | `paper.tex` → `_build/references.json`, `_build/sources.json` | an entry's authors are not those of the version it cites — typically the arXiv preprint's list under a published venue. Checked online by `paperlint build` (DBLP), reported on the entry's own line (or at the top of `paper.tex`, naming the entry's file); silent without a current build record |
|
|
20
21
|
| `paper/section-word` | warn | `paper.tex` and its body files | `§` or `\S\ref` instead of the word Section. **Fixable:** `--fix` writes `Section~\ref{…}` / `Section 5` |
|
package/package.json
CHANGED
|
@@ -5,7 +5,7 @@ context: fork
|
|
|
5
5
|
allowed-tools: [Read, Write, Edit, Grep, Glob, Bash, Agent]
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
<!-- vigiles:sha256:
|
|
8
|
+
<!-- vigiles:sha256:93d83b56c6c1baea compiled from skills/build-benchmark/SKILL.md.spec.ts -->
|
|
9
9
|
|
|
10
10
|
# build-benchmark — the study + the artifact reviewers can run
|
|
11
11
|
|
|
@@ -36,6 +36,25 @@ about *preventing* something, write the one sentence saying **what physically st
|
|
|
36
36
|
the experimental arm**. "There is different text in it" means you are measuring persuasion, not
|
|
37
37
|
prevention, and the design is wrong before a single run.
|
|
38
38
|
|
|
39
|
+
## 0.5 Register the study before its first file
|
|
40
|
+
|
|
41
|
+
Every study gets its folder, its id and its row from one command, never from a folder made by hand:
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
npx paperlint studies new <paper-dir> "<plain title, at most 8 words>" --kind measurement
|
|
45
|
+
```
|
|
46
|
+
|
|
47
|
+
It takes the next id in `repro/studies.json`, creates `repro/<id>-<slug>/README.md`, appends the row
|
|
48
|
+
(`status: designing`) and regenerates the index in `repro/README.md`. The kinds: `measurement`,
|
|
49
|
+
`spike`, `tooling`, `data`, `artifact`, `salvage`. Write the study's one-sentence `question` into its
|
|
50
|
+
row at once.
|
|
51
|
+
|
|
52
|
+
As the study moves, edit its row in `repro/studies.json`: `running` when the first counted run starts;
|
|
53
|
+
when it ends, `done` with `result` (one line) and `result_file` (the file inside the folder the result
|
|
54
|
+
is read from) — or `stopped`, `dropped`, or `superseded` with `superseded_by` — and `in_paper`. Then
|
|
55
|
+
run `npx paperlint studies index <paper-dir>`. The rule `repro/study-registered` fails `paperlint lint`
|
|
56
|
+
on a folder without a row, a `done` row without its result, and an index out of sync.
|
|
57
|
+
|
|
39
58
|
## 1. Design that inverts the pitfall you're critiquing
|
|
40
59
|
|
|
41
60
|
The strongest measurement papers are a corrected version of the mistake they name. Pick the design that
|
|
@@ -42,6 +42,25 @@ about *preventing* something, write the one sentence saying **what physically st
|
|
|
42
42
|
the experimental arm**. "There is different text in it" means you are measuring persuasion, not
|
|
43
43
|
prevention, and the design is wrong before a single run.
|
|
44
44
|
|
|
45
|
+
## 0.5 Register the study before its first file
|
|
46
|
+
|
|
47
|
+
Every study gets its folder, its id and its row from one command, never from a folder made by hand:
|
|
48
|
+
|
|
49
|
+
\`\`\`bash
|
|
50
|
+
npx paperlint studies new <paper-dir> "<plain title, at most 8 words>" --kind measurement
|
|
51
|
+
\`\`\`
|
|
52
|
+
|
|
53
|
+
It takes the next id in \`repro/studies.json\`, creates \`repro/<id>-<slug>/README.md\`, appends the row
|
|
54
|
+
(\`status: designing\`) and regenerates the index in \`repro/README.md\`. The kinds: \`measurement\`,
|
|
55
|
+
\`spike\`, \`tooling\`, \`data\`, \`artifact\`, \`salvage\`. Write the study's one-sentence \`question\` into its
|
|
56
|
+
row at once.
|
|
57
|
+
|
|
58
|
+
As the study moves, edit its row in \`repro/studies.json\`: \`running\` when the first counted run starts;
|
|
59
|
+
when it ends, \`done\` with \`result\` (one line) and \`result_file\` (the file inside the folder the result
|
|
60
|
+
is read from) — or \`stopped\`, \`dropped\`, or \`superseded\` with \`superseded_by\` — and \`in_paper\`. Then
|
|
61
|
+
run \`npx paperlint studies index <paper-dir>\`. The rule \`repro/study-registered\` fails \`paperlint lint\`
|
|
62
|
+
on a folder without a row, a \`done\` row without its result, and an index out of sync.
|
|
63
|
+
|
|
45
64
|
## 1. Design that inverts the pitfall you're critiquing
|
|
46
65
|
|
|
47
66
|
The strongest measurement papers are a corrected version of the mistake they name. Pick the design that
|