paperlint 10.3.0 → 10.4.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/workflows/ci.yml +15 -4
- package/README.md +1 -0
- package/dist/adapters/git/index.d.ts +10 -0
- package/dist/adapters/git/index.d.ts.map +1 -0
- package/dist/adapters/git/index.js +151 -0
- package/dist/adapters/git/index.js.map +1 -0
- package/dist/adapters/memory/index.d.ts +6 -0
- package/dist/adapters/memory/index.d.ts.map +1 -1
- package/dist/adapters/memory/index.js +8 -0
- package/dist/adapters/memory/index.js.map +1 -1
- package/dist/adapters/node/index.d.ts +1 -0
- package/dist/adapters/node/index.d.ts.map +1 -1
- package/dist/adapters/node/index.js +1 -0
- package/dist/adapters/node/index.js.map +1 -1
- package/dist/adapters/node/list-entries.io.d.ts +3 -0
- package/dist/adapters/node/list-entries.io.d.ts.map +1 -0
- package/dist/adapters/node/list-entries.io.js +24 -0
- package/dist/adapters/node/list-entries.io.js.map +1 -0
- package/dist/adapters/study-registry/index.d.ts +70 -10
- package/dist/adapters/study-registry/index.d.ts.map +1 -1
- package/dist/adapters/study-registry/index.js +84 -10
- package/dist/adapters/study-registry/index.js.map +1 -1
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +33 -3
- package/dist/cli.js.map +1 -1
- package/dist/domain/decide-first.d.ts +100 -0
- package/dist/domain/decide-first.d.ts.map +1 -0
- package/dist/domain/decide-first.js +102 -0
- package/dist/domain/decide-first.js.map +1 -0
- package/dist/domain/index-columns.d.ts +45 -0
- package/dist/domain/index-columns.d.ts.map +1 -0
- package/dist/domain/index-columns.js +62 -0
- package/dist/domain/index-columns.js.map +1 -0
- package/dist/domain/study-mode.d.ts +37 -0
- package/dist/domain/study-mode.d.ts.map +1 -0
- package/dist/domain/study-mode.js +45 -0
- package/dist/domain/study-mode.js.map +1 -0
- package/dist/domain/study-registry.d.ts +57 -10
- package/dist/domain/study-registry.d.ts.map +1 -1
- package/dist/domain/study-registry.js +106 -30
- package/dist/domain/study-registry.js.map +1 -1
- package/dist/outputs-before-decision-rule.d.ts +17 -0
- package/dist/outputs-before-decision-rule.d.ts.map +1 -0
- package/dist/outputs-before-decision-rule.js +90 -0
- package/dist/outputs-before-decision-rule.js.map +1 -0
- package/dist/ports/git-history.d.ts +44 -0
- package/dist/ports/git-history.d.ts.map +1 -0
- package/dist/ports/git-history.js +2 -0
- package/dist/ports/git-history.js.map +1 -0
- package/dist/ports/list-entries.d.ts +17 -0
- package/dist/ports/list-entries.d.ts.map +1 -0
- package/dist/ports/list-entries.js +2 -0
- package/dist/ports/list-entries.js.map +1 -0
- package/dist/ports/study-registry-codec.d.ts +6 -3
- package/dist/ports/study-registry-codec.d.ts.map +1 -1
- package/dist/studies.d.ts +4 -0
- package/dist/studies.d.ts.map +1 -1
- package/dist/studies.js +50 -22
- package/dist/studies.js.map +1 -1
- package/dist/study-gate.d.ts +14 -0
- package/dist/study-gate.d.ts.map +1 -0
- package/dist/study-gate.js +95 -0
- package/dist/study-gate.js.map +1 -0
- package/dist/study-index-input.d.ts +5 -0
- package/dist/study-index-input.d.ts.map +1 -0
- package/dist/study-index-input.js +13 -0
- package/dist/study-index-input.js.map +1 -0
- package/dist/study-modes.d.ts +45 -0
- package/dist/study-modes.d.ts.map +1 -0
- package/dist/study-modes.js +70 -0
- package/dist/study-modes.js.map +1 -0
- package/dist/study-registry-rule.d.ts +17 -1
- package/dist/study-registry-rule.d.ts.map +1 -1
- package/dist/study-registry-rule.js +34 -13
- package/dist/study-registry-rule.js.map +1 -1
- package/dist/study-run.d.ts +19 -0
- package/dist/study-run.d.ts.map +1 -0
- package/dist/study-run.js +9 -0
- package/dist/study-run.js.map +1 -0
- package/docs/design/study-modes.md +247 -0
- package/docs/design/study-registry.md +6 -0
- package/docs/e2e.md +8 -4
- package/docs/layout.md +22 -19
- package/docs/rules/repro/outputs-before-decision.md +119 -0
- package/docs/rules/repro/study-registered.md +47 -14
- package/docs/rules.md +54 -53
- package/docs/testing.md +1 -0
- package/package.json +2 -1
- package/scripts/check.ts +6 -0
- package/skills/camera-ready/SKILL.md +6 -1
- package/skills/camera-ready/SKILL.md.spec.ts +5 -0
- package/skills/draft-paper/SKILL.md +11 -3
- package/skills/draft-paper/SKILL.md.spec.ts +10 -2
- package/skills/harden-paper/SKILL.md +12 -2
- package/skills/harden-paper/SKILL.md.spec.ts +11 -1
- package/skills/osf-artifact-upload/SKILL.md +7 -1
- package/skills/osf-artifact-upload/SKILL.md.spec.ts +6 -0
- package/skills/pc-panel-review/SKILL.md +10 -3
- package/skills/pc-panel-review/SKILL.md.spec.ts +9 -2
- package/skills/tighten-paper/SKILL.md +5 -1
- package/skills/tighten-paper/SKILL.md.spec.ts +4 -0
- package/src/adapters/git/history.test.ts +121 -0
- package/src/adapters/git/index.test.ts +268 -0
- package/src/adapters/git/index.ts +227 -0
- package/src/adapters/memory/index.test.ts +15 -1
- package/src/adapters/memory/index.ts +13 -0
- package/src/adapters/node/index.ts +1 -0
- package/src/adapters/node/list-entries.io.ts +26 -0
- package/src/adapters/node/list-entries.test.ts +43 -0
- package/src/adapters/study-registry/index.test.ts +12 -9
- package/src/adapters/study-registry/index.ts +130 -24
- package/src/adapters/study-registry/mode.test.ts +162 -0
- package/src/cli.ts +47 -3
- package/src/domain/decide-first.test.ts +258 -0
- package/src/domain/decide-first.ts +243 -0
- package/src/domain/index-columns.test.ts +216 -0
- package/src/domain/index-columns.ts +105 -0
- package/src/domain/study-measurements.test.ts +9 -5
- package/src/domain/study-mode.test.ts +55 -0
- package/src/domain/study-mode.ts +78 -0
- package/src/domain/study-registry.test.ts +38 -31
- package/src/domain/study-registry.ts +170 -31
- package/src/new-paper.test.ts +5 -1
- package/src/outputs-before-decision-rule.test.ts +206 -0
- package/src/outputs-before-decision-rule.ts +124 -0
- package/src/ports/git-history.ts +44 -0
- package/src/ports/list-entries.ts +18 -0
- package/src/ports/study-registry-codec.ts +10 -3
- package/src/rule-cases.test.ts +78 -3
- package/src/studies-index-modes.test.ts +129 -0
- package/src/studies.test.ts +35 -11
- package/src/studies.ts +88 -27
- package/src/study-gate.test.ts +316 -0
- package/src/study-gate.ts +161 -0
- package/src/study-index-input.ts +23 -0
- package/src/study-modes.ts +120 -0
- package/src/study-registry-rule.test.ts +198 -11
- package/src/study-registry-rule.ts +51 -14
- package/src/study-run.ts +21 -0
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"study-registry-rule.js","sourceRoot":"","sources":["../src/study-registry-rule.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;GAkBG;AACH,OAAO,EAAE,QAAQ,EAAE,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,MAAM,WAAW,CAAC;AAC3D,OAAO,EAAE,UAAU,EAAE,MAAM,kBAAkB,CAAC;AAC9C,OAAO,EACL,MAAM,EACN,gBAAgB,EAChB,SAAS,EACT,UAAU,GACX,MAAM,0BAA0B,CAAC;AAClC,OAAO,EACL,gBAAgB,EAChB,kBAAkB,EAClB,YAAY,EACZ,YAAY,EACZ,QAAQ,EACR,YAAY,GAIb,MAAM,4BAA4B,CAAC;
|
|
1
|
+
{"version":3,"file":"study-registry-rule.js","sourceRoot":"","sources":["../src/study-registry-rule.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;;GAkBG;AACH,OAAO,EAAE,QAAQ,EAAE,OAAO,EAAE,IAAI,EAAE,KAAK,EAAE,MAAM,WAAW,CAAC;AAC3D,OAAO,EAAE,UAAU,EAAE,MAAM,kBAAkB,CAAC;AAC9C,OAAO,EACL,MAAM,EACN,gBAAgB,EAChB,SAAS,EACT,UAAU,GACX,MAAM,0BAA0B,CAAC;AAClC,OAAO,EACL,gBAAgB,EAChB,kBAAkB,EAClB,YAAY,EACZ,YAAY,EACZ,YAAY,EACZ,QAAQ,EACR,YAAY,GAIb,MAAM,4BAA4B,CAAC;AASpC,OAAO,EAAE,cAAc,EAAE,MAAM,wBAAwB,CAAC;AACxD,OAAO,EAAE,WAAW,EAAE,MAAM,kBAAkB,CAAC;AAE/C,MAAM,WAAW,GAAG,oBAAoB,CAAC;AASzC,sEAAsE;AACtE,MAAM,OAAO,GAAG,CACd,GAAW,EACX,IAAwC,EACnB,EAAE,CACvB,IAAI,GAAG,CACL,IAAI;KACD,OAAO,CAAC,UAAU,CAAC,IAAI,CAAC,GAAG,EAAE,SAAS,CAAC,CAAC,CAAC;KACzC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,KAAK,SAAS,CAAC;KACnC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CACtB,CAAC;AAEJ;;;GAGG;AACH,MAAM,CAAC,MAAM,eAAe,GAAG,CAC7B,GAAW,EACX,IAAwC,EACT,EAAE;IACjC,MAAM,KAAK,GAAG,OAAO,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC;IACjC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,YAAY,CAAC,MAAM,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC;AAChE,CAAC,CAAC;AAEF,uFAAuF;AACvF,MAAM,CAAC,MAAM,MAAM,GAAG,CAAC,KAAY,EAAE,CAAS,EAAiB,EAAE;IAC/D,MAAM,CAAC,GAAG,KAAK,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,SAAS,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC;IAC9E,OAAO,CAAC,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,WAAW,EAAE,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC;AACzD,CAAC,CAAC;AAEF,oFAAoF;AACpF,MAAM,UAAU,UAAU,CACxB,GAAW,EACX,IAAgD;IAEhD,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,GAAG,EAAE,SAAS,EAAE,YAAY,CAAC,CAAC,CAAC;IACpE,IAAI,IAAI,KAAK,IAAI;QAAE,OAAO,EAAE,IAAI,EAAE,QAAQ,EAAE,CAAC;IAC7C,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IACtC,OAAO,MAAM,CAAC,EAAE;QACd,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,EAAE,QAAQ,EAAE,MAAM,CAAC,QAAQ,EAAE;QAC7C,CAAC,CAAC,EAAE,IAAI,EAAE,YAAY,EAAE,GAAG,EAAE,MAAM,CAAC,GAAG,EAAE,CAAC;AAC9C,CAAC;AAED,2GAA2G;AAC3G,MAAM,YAAY,GAAG,CAAC,GAAW,EAAE,IAAsB,EAAqB,EAAE,CAC9E,cAAc,CAAC,GAAG,EAAE,IAAI,CAAC;KACtB,MAAM,CACL,CAAC,CAAC,EAAE,EAAE,CACJ,CAAC,CAAC,IAAI,KAAK,KAAK;IAChB,CAAC,UAAU,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC;IAC5B,CAAC,gBAAgB,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,CACrC;KACA,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;AAExB,2DAA2D;AAC3D,MAAM,eAAe,GAAG,CACtB,GAAW,EACX,CAAgB,EAChB,KAAY,EACZ,QAAqC,EAChB,EAAE,CACvB,IAAI,GAAG,CACL,CAAC,CAAC,CAAC,IAAI,KAAK,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC;IAC3C,sFAAsF;KACrF,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,QAAQ,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC;KACjC,OAAO,CAAC,CAAC,CAAC,EAAE,EAAE,CACb,CAAC,CAAC,WAAW,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,MAAM,IAAI,CAAC,CAAC,WAAW,EAAE,CAAC,CACpE;IACD,2FAA2F;KAC1F,MAAM,CACL,CAAC,CAAC,EAAE,EAAE,CACJ,CAAC,CAAC,CAAC,QAAQ,CAAC,IAAI,CAAC;IACjB,KAAK,CAAC,MAAM,CAAC,UAAU,CAAC,IAAI,CAAC,GAAG,EAAE,SAAS,EAAE,CAAC,CAAC,CAAC,CAAC,CACpD,CACJ,CAAC;AAEJ;;;GAGG;AACH,MAAM,CAAC,MAAM,OAAO,GAAG,CACrB,GAAW,EACX,CAAgB,EAChB,IAAgD,EAChD,QAAqC,EACL,EAAE,CAClC,IAAI,GAAG,CACL,CAAC,CAAC,CAAC,IAAI,KAAK,MAAM,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;IAC7C,8FAA8F;IAC9F,+CAA+C;KAC9C,MAAM,CAAC,CAAC,EAAE,GAAG,EAAE,MAAM,EAAE,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,QAAQ,CAAC,IAAI,CAAC,IAAI,QAAQ,CAAC,MAAM,CAAC,CAAC;KACpE,OAAO,CAAC,CAAC,EAAE,GAAG,EAAE,IAAI,EAAE,EAA6C,EAAE;IACpE,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,GAAG,EAAE,SAAS,EAAE,GAAG,CAAC,CAAC,CAAC;IAC3D,IAAI,IAAI,KAAK,IAAI;QAAE,OAAO,EAAE,CAAC;IAC7B,OAAO;QACL;YACE,GAAG;YACH,KAAK,CAAC,QAAQ,CAAC,GAAG,CAAC,KAAK,gBAAgB;gBACtC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,eAAe,CAAC,IAAI,CAAC;gBAClC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,SAAS,CAAC,IAAI,EAAE,IAAI,CAAC;SACrC;KACF,CAAC;AACJ,CAAC,CAAC,CACL,CAAC;AAEJ,iGAAiG;AACjG,MAAM,UAAU,aAAa,CAC3B,GAAW,EACX,IAAuB;IAEvB,MAAM,QAAQ,GAAG,eAAe,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC;IAC5C,MAAM,QAAQ,GAAG,UAAU,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC;IACvC,OAAO,YAAY,CAAC;QAClB,OAAO,EAAE,YAAY,CAAC,GAAG,EAAE,IAAI,CAAC;QAChC,QAAQ;QACR,MAAM,EAAE,MAAM,CAAC,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,GAAG,EAAE,SAAS,EAAE,MAAM,CAAC,CAAC;QACxD,eAAe,EAAE,eAAe,CAAC,GAAG,EAAE,QAAQ,EAAE,IAAI,CAAC,KAAK,EAAE,QAAQ,CAAC;QACrE,KAAK,EAAE,OAAO,CAAC,GAAG,EAAE,QAAQ,EAAE,IAAI,EAAE,QAAQ,CAAC;KAC9C,CAAC,CAAC;AACL,CAAC;AAED,0GAA0G;AAC1G,MAAM,QAAQ,GAAuC;IACnD,aAAa;IACb,YAAY;IACZ,cAAc;IACd,WAAW;IACX,UAAU;IACV,QAAQ;IACR,cAAc;IACd,aAAa;IACb,YAAY;IACZ,WAAW;IACX,aAAa;IACb,cAAc;IACd,aAAa;IACb,qBAAqB;IACrB,mBAAmB;IACnB,qBAAqB;IACrB,mBAAmB;IACnB,UAAU;IACV,cAAc;CACf,CAAC;AAEF,kGAAkG;AAClG,MAAM,MAAM,GAAG,CACb,IAAY,EACZ,WAAmB,EACnB,QAA4C,EAClB,EAAE,CAAC,CAAC;IAC9B,IAAI,EAAE,YAAY;IAClB,IAAI,EAAE,EAAE,WAAW,EAAE,GAAG,EAAE,WAAW,CAAC,IAAI,CAAC,EAAE;IAC7C,MAAM,EAAE,EAAE;IACV,QAAQ,EAAE,MAAM,CAAC,WAAW,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,UAAU,CAAC,CAAC,CAAC;CACnE,CAAC,CAAC;AAEH,yFAAyF;AACzF,MAAM,MAAM,GAAG,CACb,IAA8B,EAC9B,QAA4C,EAC5C,IAAuB,EACL,EAAE,CAAC,CAAC;IACtB,IAAI;IACJ,MAAM,CAAC,OAA0B;QAC/B,IAAI,QAAQ,CAAC,OAAO,CAAC,QAAQ,CAAC,KAAK,WAAW;YAAE,OAAO,EAAE,CAAC;QAC1D,OAAO;YACL,IAAI;gBACF,MAAM,EAAE,GAAG,OAAO,CAAC,UAAU,CAAC,eAAe,CAAC,CAAC,CAAC,CAAC;gBACjD,aAAa,CAAC,OAAO,CAAC,OAAO,CAAC,QAAQ,CAAC,EAAE,IAAI,CAAC;qBAC3C,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,QAAQ,CAAC,QAAQ,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC;qBAC3C,OAAO,CAAC,CAAC,CAAC,EAAE,EAAE;oBACb,OAAO,CAAC,MAAM,CAAC;wBACb,GAAG,EAAE,EAAE,KAAK,EAAE,EAAE,EAAE,GAAG,EAAE,EAAE,EAAE;wBAC3B,SAAS,EAAE,CAAC,CAAC,OAAO;wBACpB,IAAI,EAAE,EAAE,IAAI,EAAE,YAAY,CAAC,CAAC,CAAC,EAAE;qBAChC,CAAC,CAAC;gBACL,CAAC,CAAC,CAAC;YACP,CAAC;SACF,CAAC;IACJ,CAAC;CACF,CAAC,CAAC;AAEH;;;;GAIG;AACH,MAAM,UAAU,kBAAkB,CAChC,IAAuB;IAEvB,OAAO;QACL,kBAAkB,EAAE,MAAM,CACxB,MAAM,CACJ,wBAAwB,EACxB,8OAA8O,EAC9O,QAAQ,CACT,EACD,QAAQ,EACR,IAAI,CACL;QACD,iBAAiB,EAAE,MAAM,CACvB,MAAM,CACJ,uBAAuB,EACvB,oDAAoD,EACpD,kBAAkB,CACnB,EACD,kBAAkB,EAClB,IAAI,CACL;KACF,CAAC;AACJ,CAAC;AAED,mGAAmG;AACnG,MAAM,CAAC,MAAM,0BAA0B,GAGlC,EAAE,wBAAwB,EAAE,OAAO,EAAE,uBAAuB,EAAE,MAAM,EAAE,CAAC"}
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
/** What a `paperlint studies` command on one study comes to: the lines it prints, or its refusal. */
|
|
2
|
+
/** A refusal: the exit code, and the line that explains it. */
|
|
3
|
+
export interface StudyRefusal {
|
|
4
|
+
readonly code: 1 | 2;
|
|
5
|
+
readonly line: string;
|
|
6
|
+
}
|
|
7
|
+
/** What a run came to: the lines it printed, or the refusal. */
|
|
8
|
+
export type StudyRun = {
|
|
9
|
+
readonly ok: true;
|
|
10
|
+
readonly lines: readonly string[];
|
|
11
|
+
} | {
|
|
12
|
+
readonly ok: false;
|
|
13
|
+
readonly refusal: StudyRefusal;
|
|
14
|
+
};
|
|
15
|
+
/** A refusal for the paper's state: exit 1. */
|
|
16
|
+
export declare const refusal: (line: string) => StudyRefusal;
|
|
17
|
+
/** The run that is `r`. */
|
|
18
|
+
export declare const refused: (r: StudyRefusal) => StudyRun;
|
|
19
|
+
//# sourceMappingURL=study-run.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"study-run.d.ts","sourceRoot":"","sources":["../src/study-run.ts"],"names":[],"mappings":"AAAA,qGAAqG;AAErG,+DAA+D;AAC/D,MAAM,WAAW,YAAY;IAC3B,QAAQ,CAAC,IAAI,EAAE,CAAC,GAAG,CAAC,CAAC;IACrB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;CACvB;AAED,gEAAgE;AAChE,MAAM,MAAM,QAAQ,GAChB;IAAE,QAAQ,CAAC,EAAE,EAAE,IAAI,CAAC;IAAC,QAAQ,CAAC,KAAK,EAAE,SAAS,MAAM,EAAE,CAAA;CAAE,GACxD;IAAE,QAAQ,CAAC,EAAE,EAAE,KAAK,CAAC;IAAC,QAAQ,CAAC,OAAO,EAAE,YAAY,CAAA;CAAE,CAAC;AAE3D,+CAA+C;AAC/C,eAAO,MAAM,OAAO,GAAI,MAAM,MAAM,KAAG,YAAmC,CAAC;AAE3E,2BAA2B;AAC3B,eAAO,MAAM,OAAO,GAAI,GAAG,YAAY,KAAG,QAGxC,CAAC"}
|
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
/** What a `paperlint studies` command on one study comes to: the lines it prints, or its refusal. */
|
|
2
|
+
/** A refusal for the paper's state: exit 1. */
|
|
3
|
+
export const refusal = (line) => ({ code: 1, line });
|
|
4
|
+
/** The run that is `r`. */
|
|
5
|
+
export const refused = (r) => ({
|
|
6
|
+
ok: false,
|
|
7
|
+
refusal: r,
|
|
8
|
+
});
|
|
9
|
+
//# sourceMappingURL=study-run.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"study-run.js","sourceRoot":"","sources":["../src/study-run.ts"],"names":[],"mappings":"AAAA,qGAAqG;AAarG,+CAA+C;AAC/C,MAAM,CAAC,MAAM,OAAO,GAAG,CAAC,IAAY,EAAgB,EAAE,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC,EAAE,IAAI,EAAE,CAAC,CAAC;AAE3E,2BAA2B;AAC3B,MAAM,CAAC,MAAM,OAAO,GAAG,CAAC,CAAe,EAAY,EAAE,CAAC,CAAC;IACrD,EAAE,EAAE,KAAK;IACT,OAAO,EAAE,CAAC;CACX,CAAC,CAAC"}
|
|
@@ -0,0 +1,247 @@
|
|
|
1
|
+
# Study modes: decide first, or exploratory — design
|
|
2
|
+
|
|
3
|
+
## Why
|
|
4
|
+
|
|
5
|
+
A study's outcomes are computed by an automated runner in the same workspace where the people
|
|
6
|
+
deciding its inputs work: which runs count, which units are excluded, whether a veto applies, whether
|
|
7
|
+
the drawn sample is approved. If those decisions can be made — or changed — after an outcome has been
|
|
8
|
+
seen, the decision can follow the result, and nothing in the repository says which came first.
|
|
9
|
+
|
|
10
|
+
What is needed:
|
|
11
|
+
|
|
12
|
+
1. every measurement says how its outcomes are kept from the decisions on its inputs, and none can
|
|
13
|
+
say nothing;
|
|
14
|
+
2. the default needs no key and no outside service: the decision is recorded first, the run starts
|
|
15
|
+
after, and anyone with the repository can re-check the order;
|
|
16
|
+
3. a study that cannot work that way is declared exploratory, with its reason, and reported as such;
|
|
17
|
+
4. the paper's Methods and its reviewers read each study's mode, decision commit and first output
|
|
18
|
+
from one generated place, not from prose someone keeps in sync by hand.
|
|
19
|
+
|
|
20
|
+
## What is decided
|
|
21
|
+
|
|
22
|
+
### 1. One mode per measurement, in its plan
|
|
23
|
+
|
|
24
|
+
A study's `plan.json` holds one `mode`, a union on its `kind` (`src/domain/study-mode.ts`):
|
|
25
|
+
|
|
26
|
+
```json
|
|
27
|
+
{ "mode": { "kind": "decide_before_run",
|
|
28
|
+
"outputs": ["out/**", "RESULTS.md"],
|
|
29
|
+
"decision": { "file": "repro/P1-S03-x/decision.md", "note": "exclusions and veto" } } }
|
|
30
|
+
|
|
31
|
+
{ "mode": { "kind": "exploratory", "reason": "a first look at the logs, to find the question" } }
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
- **Exactly one, by construction.** The modes are one discriminated union, each variant closed: a
|
|
35
|
+
second mode's key in the same object is an unknown key, and an unknown `kind` is refused. This is
|
|
36
|
+
the parser's (`src/adapters/study-registry/`), not a later check.
|
|
37
|
+
- **None is not ready.** The parser is told the study's `kind` (from its row). A `measurement`'s plan
|
|
38
|
+
without a mode is a **draft** naming `mode` among its empty texts, like an empty `question`: legal
|
|
39
|
+
while the study is `designing`, reported by `repro/study-registered` (`bad-plan`, `unfilled`) once
|
|
40
|
+
it runs, and a parse error when the author's check is already on it (the author approved an
|
|
41
|
+
incomplete plan). A spike, a tool, a data set, an artifact or a salvage may omit it.
|
|
42
|
+
- **The default is written for you.** `paperlint studies new` writes a measurement's plan with
|
|
43
|
+
`decide_before_run`, its decision file named (`repro/<folder>/decision.md`), its outputs and note
|
|
44
|
+
left to fill — a draft until they are. The decision file itself is NOT created: an empty
|
|
45
|
+
placeholder that got committed would open the gate on nothing.
|
|
46
|
+
- **Paths.** `outputs` are globs relative to the study's folder, inside it, never the study's
|
|
47
|
+
`README.md`. A decision `file` is relative to the **paper's** folder, so it may live beside the
|
|
48
|
+
study or anywhere in the paper (`research/decisions.md`); it may not leave the paper, and it is a
|
|
49
|
+
file's path — names joined by `/`, none empty, `.` or `..` — so `.` (the paper itself) or a path
|
|
50
|
+
ending in `/` cannot be written. A name that is a directory on disk is the gate's to refuse and the
|
|
51
|
+
rule's to report (`decision-not-file`): git answers for a directory with the commits of anything
|
|
52
|
+
under it. Whether a
|
|
53
|
+
decision inside the study's folder is matched by an output glob needs the folder, which the plan
|
|
54
|
+
does not hold: the rule reports it (`decision-is-output`) and the gate refuses it.
|
|
55
|
+
- **Empty texts of a mode** — no outputs yet, an empty decision file or note, an empty reason — are a
|
|
56
|
+
draft's empty texts, named field by field (`mode.outputs`, `mode.decision.file`, …).
|
|
57
|
+
|
|
58
|
+
### 2. `paperlint studies gate <paper-dir> <id>`
|
|
59
|
+
|
|
60
|
+
Exit 0 only if the decision file exists and is committed as it stands (git's last commit for it, and
|
|
61
|
+
nothing changed since), printing the commit and its date; exit 1 saying why otherwise (not
|
|
62
|
+
committed, changed since its commit, committed and since deleted — not in HEAD, though `git log -1`
|
|
63
|
+
names the deleting commit and the tree is clean — git unable to answer, no decision file named, a
|
|
64
|
+
decision that is a directory, a decision its own outputs match). A runner calls it first:
|
|
65
|
+
|
|
66
|
+
```sh
|
|
67
|
+
npx paperlint studies gate papers/my-paper P1-S03 && ./run.sh
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
For an exploratory study it exits 0 and says it has no gate. A study that declares no mode, or whose
|
|
71
|
+
plan is still a draft, is refused.
|
|
72
|
+
|
|
73
|
+
### 3. `repro/outputs-before-decision` — the proof of order a reviewer can re-run
|
|
74
|
+
|
|
75
|
+
An error, reported on `PIPELINE-STATUS.md`. For each decide-first study: every output whose first
|
|
76
|
+
commit is not a strict descendant of the decision file's first commit (earlier, on another line of
|
|
77
|
+
history, or in the same commit) is a finding; deleted outputs count. An output in the tree or in history while the
|
|
78
|
+
decision was never committed is a finding, and so is a decision committed and since deleted
|
|
79
|
+
(`decision-deleted`): it is not in HEAD, so no order is judged against it. A shallow clone, and a
|
|
80
|
+
history git cannot read, are
|
|
81
|
+
findings that say so — never a pass. The study's folder is walked without following a symbolic link
|
|
82
|
+
(`ListEntries`, `src/ports/list-entries.ts`): a link is one entry, as git stores it, so a link to the
|
|
83
|
+
folder itself cannot loop the walk and one outside brings no foreign file into the study.
|
|
84
|
+
|
|
85
|
+
**Why a lint rule may ask git here.** Rules in `eslint-rules/` may not run git
|
|
86
|
+
(`eslint.config.mjs`, `GIT_IN_A_RULE`): a pinned sha is deleted by a squash or a gc, and a shallow CI
|
|
87
|
+
checkout has none. This rule pins no sha; it asks the history the checkout has for an order. A squash
|
|
88
|
+
merge keeps that order when the decision was merged before the run — which is the decide-first
|
|
89
|
+
workflow — and a shallow checkout is named by the rule (`fetch-depth: 0`).
|
|
90
|
+
|
|
91
|
+
### 4. The index carries each study's mode
|
|
92
|
+
|
|
93
|
+
`repro/README.md`'s study index (`paperlint studies index`, docs/design/study-registry.md) has three
|
|
94
|
+
columns for this (`src/domain/index-columns.ts`):
|
|
95
|
+
|
|
96
|
+
| column | cell |
|
|
97
|
+
| -------------- | ---------------------------------------------------------------------------------------------------------------------------- |
|
|
98
|
+
| `mode` | `decide_before_run`, `exploratory: <reason>`, or `—` for a study whose plan has none |
|
|
99
|
+
| `decision` | the decision file's first commit (12 characters) and its date (`, deleted since` when it is not in HEAD), or `not committed` |
|
|
100
|
+
| `first output` | the first commit of the oldest output and its date, or `none` |
|
|
101
|
+
|
|
102
|
+
A sample row:
|
|
103
|
+
|
|
104
|
+
```markdown
|
|
105
|
+
| id | title | status | result | in paper | mode | decision | first output | folder |
|
|
106
|
+
| ------ | ------- | ------ | -------- | -------- | ----------------- | ----------------------- | ----------------------- | ---------------------------------- |
|
|
107
|
+
| P1-S01 | Outcome | done | 41 of 60 | yes | decide_before_run | 3f2a9c1d7e40 2026-10-14 | 8b51e0aa42c9 2026-10-15 | [P1-S01-outcome/](P1-S01-outcome/) |
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
- **The mode is compared, the commits are not.** The mode comes from the plan, which is in the tree,
|
|
111
|
+
so `repro/study-registered` reports an index whose mode column is not the plan's. The two commits
|
|
112
|
+
are what git's history said when `studies index` last ran: a squash merge or a rebase gives the
|
|
113
|
+
same files new commits, and a shallow CI clone has none to read, so the lint never compares those
|
|
114
|
+
two columns — comparing them would turn every squash merge red. `studies index` rewrites the README
|
|
115
|
+
whenever any column moved, and in a shallow clone writes `unknown (shallow clone)`. The order
|
|
116
|
+
itself is `repro/outputs-before-decision`'s to judge, not the table's.
|
|
117
|
+
- **The skills read the table.** `draft-paper` writes the Methods sentence from it (decided before
|
|
118
|
+
the run, the decision's commit and date; exploratory studies labelled with their reason);
|
|
119
|
+
`harden-paper` and `pc-panel-review` check that every confirmatory claim rests on a
|
|
120
|
+
`decide_before_run` study whose decision commit precedes its first output, and flag a
|
|
121
|
+
"pre-registered" or "decided before" claim the table does not support; `tighten-paper` keeps that
|
|
122
|
+
sentence; `osf-artifact-upload` and `camera-ready` ship each plan, its decision file and the index,
|
|
123
|
+
with the whole history. Each regenerates the index first, so the commits are this checkout's.
|
|
124
|
+
|
|
125
|
+
### 5. Reads stay inside the paper
|
|
126
|
+
|
|
127
|
+
**What these checks guard against.** paperlint's checks guard against accidents and honest
|
|
128
|
+
mistakes — a run started before the decision was committed, a decision edited or deleted afterwards,
|
|
129
|
+
a path typed wrong — not against an author who controls the repository's history and sets out to
|
|
130
|
+
forge it. The path rules below exist to keep every read inside the paper, so a mistyped row cannot
|
|
131
|
+
make a command or a rule read, judge or write files that belong to nothing in it.
|
|
132
|
+
|
|
133
|
+
Every command and rule that reads a study's folder — `studies gate`, `studies index` and `new`,
|
|
134
|
+
`repro/outputs-before-decision`, and `repro/study-registered`'s reads of plans and result files —
|
|
135
|
+
reads a row only through one predicate, `readableFolders` (`src/study-registry-rule.ts`): its
|
|
136
|
+
`folder` is one folder directly inside `repro/` (`isFolderName`: not empty, not `.` or `..`, no
|
|
137
|
+
separator or NUL) and not a symbolic link (`ListEntries`, read without following links). Such a row
|
|
138
|
+
is never read through: `studies index` and `new` refuse the registry, the gate refuses the study,
|
|
139
|
+
the order rule treats it as no study, and `repro/study-registered` reads its plan and results as
|
|
140
|
+
missing. A plan's path is held inside its folder by the registry rule's own `isInside`, the same on
|
|
141
|
+
every platform (a backslash is refused outright); the decision file is held inside the paper by the
|
|
142
|
+
schema (`staysInside`) and must be a regular file on disk — a directory or a symbolic link is refused
|
|
143
|
+
by the gate and reported by the rule (`decision-not-file`). Only a **ready** plan's mode decides
|
|
144
|
+
anything: the gate refuses a draft, naming its empty fields, and the order rule judges no draft — a
|
|
145
|
+
draft may name no outputs, or an empty decision file.
|
|
146
|
+
|
|
147
|
+
### 6. Legacy plans
|
|
148
|
+
|
|
149
|
+
A plan still named `prereg.json` is parsed by its loose legacy schema, which never asks for a mode,
|
|
150
|
+
and `repro/study-plan-name` keeps warning that it moves to `plan.json` — where the strict parser asks
|
|
151
|
+
for one. So an old paper keeps linting as before until it renames the file. A `plan.json` written by
|
|
152
|
+
the release before modes existed, for a measurement, reads as a draft whose empty text is `mode`:
|
|
153
|
+
silent while the study is `designing`, reported once it runs, and an unreadable plan when the
|
|
154
|
+
author's check is already on it (the message names `mode`). Adding one line — usually the
|
|
155
|
+
decide-first mode — is the migration. An index generated before the mode columns existed is out of
|
|
156
|
+
sync until `paperlint studies index` regenerates it.
|
|
157
|
+
|
|
158
|
+
### 7. Why: confirmatory vs exploratory, and venues that review the plan first
|
|
159
|
+
|
|
160
|
+
Read for this note (2026-10-16):
|
|
161
|
+
|
|
162
|
+
- Nosek, Ebersole, DeHaven, Mellor, "The preregistration revolution", _PNAS_ 115(11):2600–2606,
|
|
163
|
+
2018 (metadata and abstract from Crossref, doi:10.1073/pnas.1708274114). The abstract, verbatim:
|
|
164
|
+
|
|
165
|
+
> Progress in science relies in part on generating hypotheses with existing observations and
|
|
166
|
+
> testing hypotheses with new observations. This distinction between postdiction and prediction
|
|
167
|
+
> is appreciated conceptually but is not respected in practice. Mistaking generation of
|
|
168
|
+
> postdictions with testing of predictions reduces the credibility of research findings. […] An
|
|
169
|
+
> effective solution is to define the research questions and analysis plan before observing the
|
|
170
|
+
> research outcomes—a process called preregistration. Preregistration distinguishes analyses and
|
|
171
|
+
> outcomes that result from predictions from those that result from postdictions.
|
|
172
|
+
|
|
173
|
+
This is the distinction the modes encode: decided first is prediction; exploratory is postdiction,
|
|
174
|
+
legitimate when it is labelled.
|
|
175
|
+
|
|
176
|
+
- The MSR (Mining Software Repositories) Registered Reports track, 2020 edition
|
|
177
|
+
(https://2020.msrconf.org/track/msr-2020-Registered-Reports; the 2021, 2025 and 2026 editions have a
|
|
178
|
+
track page at the same address, the 2019 site has none there). Quoted from the 2020 page:
|
|
179
|
+
|
|
180
|
+
> Methods and proposed analyses are pre-registered and reviewed prior to research being conducted.
|
|
181
|
+
|
|
182
|
+
> (MSR 2020) Phase 1: Introduction, Methods (including proposed analyses), and Pilot Data (where
|
|
183
|
+
> applicable).
|
|
184
|
+
|
|
185
|
+
> (EMSE) Phase 2: full study, after data collection and analysis.
|
|
186
|
+
|
|
187
|
+
> Additional exploratory analyses in Phase 2 may be conducted, if they are justified.
|
|
188
|
+
|
|
189
|
+
> Any deviation from the protocol must be justified and logged in detail to ensure replicability.
|
|
190
|
+
|
|
191
|
+
A venue of this field already reviews the plan before the data, and accepts exploratory analyses
|
|
192
|
+
beside it only when they are justified. The modes, the order rule and the index give a paper the
|
|
193
|
+
same evidence in its own repository: what was fixed before the run, with its commit, and what was
|
|
194
|
+
exploratory, with its reason.
|
|
195
|
+
|
|
196
|
+
## Alternatives considered
|
|
197
|
+
|
|
198
|
+
- **Encrypting the outputs until the decision (age, to a key the author holds): rejected.** It needs
|
|
199
|
+
an external binary that `paperlint toolchain` does not install, and a key whose custody has no good
|
|
200
|
+
home: a session in the cloud has nowhere private to keep the identity, and a key in a CI secret is
|
|
201
|
+
readable by any workflow that can be edited. Decide first already proves the order from git with no
|
|
202
|
+
key at all, so encryption would add a dependency and a secret to guard to buy what the history
|
|
203
|
+
shows. Blind analysis in physics (Klein & Roodman, _Annu. Rev. Nucl. Part. Sci._ 55, 2005) and
|
|
204
|
+
MacCoun & Perlmutter, "Hide results to seek the truth" (_Nature_ 526, 2015) — both cited from
|
|
205
|
+
memory, not re-read for this note — are where the idea comes from; what is kept is that the
|
|
206
|
+
decision is fixed and recorded before the outcome is seen.
|
|
207
|
+
- **Paragraph links (`% study: <id>` comments in `paper.tex`) checked by lint rules: rejected.** Such
|
|
208
|
+
rules check only the paragraphs that carry a convention authors do not write, so an unlinked
|
|
209
|
+
overclaim passes silently. The review skills check the Methods sentence and every confirmatory
|
|
210
|
+
claim against the index instead, which covers the whole text.
|
|
211
|
+
- **A separate `studies methods` command printing the modes and commits: rejected.** The same facts
|
|
212
|
+
are three columns of the `studies index` table the registry already generates; a second command
|
|
213
|
+
would be a second rendering of one set of facts.
|
|
214
|
+
- **Comparing the decision and first-output columns in the lint: rejected.** A squash merge renames
|
|
215
|
+
the commits, so the index would go stale on every merge, and a shallow CI clone cannot read them;
|
|
216
|
+
the order rule judges the history directly instead (§4).
|
|
217
|
+
- **An outside pre-registration (OSF, AsPredicted) as the record: not required.** A timestamped
|
|
218
|
+
record before the outcome is what they give; here the repository's history is that record,
|
|
219
|
+
re-checkable by anyone who clones it. A paper may still register outside and cite it.
|
|
220
|
+
|
|
221
|
+
## Where each part lives
|
|
222
|
+
|
|
223
|
+
| part | file | layer |
|
|
224
|
+
| ------------------------------------------------------------- | ------------------------------------- | ------- |
|
|
225
|
+
| the modes, what is an output, which decision a mode waits on | `src/domain/study-mode.ts` | domain |
|
|
226
|
+
| the order judge and its messages | `src/domain/decide-first.ts` | domain |
|
|
227
|
+
| the index's mode columns and the history they show | `src/domain/index-columns.ts` | domain |
|
|
228
|
+
| the index, which columns the lint compares, `isFolderName` | `src/domain/study-registry.ts` | domain |
|
|
229
|
+
| the schema of `mode`; readiness by kind | `src/adapters/study-registry/` | adapter |
|
|
230
|
+
| git: committed state, first commits, ancestry, shallow | `src/adapters/git/` | adapter |
|
|
231
|
+
| what the app asks of git | `src/ports/git-history.ts` | port |
|
|
232
|
+
| a directory's entries, links not followed | `src/ports/list-entries.ts` | port |
|
|
233
|
+
| the studies with a mode, and the files under a study's folder | `src/study-modes.ts` | app |
|
|
234
|
+
| the order rule | `src/outputs-before-decision-rule.ts` | app |
|
|
235
|
+
| the plans and histories `studies index` and `new` write from | `src/study-index-input.ts` | app |
|
|
236
|
+
| `gate`; `new` and `index` | `src/study-gate.ts`, `src/studies.ts` | app |
|
|
237
|
+
| wiring: git with PATH and HOME, the clock | `src/cli.ts` | root |
|
|
238
|
+
|
|
239
|
+
## What it does not check
|
|
240
|
+
|
|
241
|
+
- What a decision says. The gate is that it is committed, not its content.
|
|
242
|
+
- A decision amended after the outputs: `repro/outputs-before-decision` orders FIRST commits; later
|
|
243
|
+
edits of the decision are in its history but not reported.
|
|
244
|
+
- That nobody ran a decide-first study before its decision and kept the outputs out of git. The rule
|
|
245
|
+
keeps early outputs out of the repository's history, not out of a person's hands.
|
|
246
|
+
- Files outside the study's folder as outputs: outputs are relative to it by construction.
|
|
247
|
+
- Whether the paper's text matches the index: that is the review skills' job, not a lint rule's.
|
|
@@ -205,6 +205,12 @@ two commands that write it.
|
|
|
205
205
|
come from an error rule, since ESLint sets the level per rule. A `prereg.json` no row names is not
|
|
206
206
|
read, so no existing paper turns red.
|
|
207
207
|
|
|
208
|
+
8. **A study's mode.** A measurement's plan declares how its outcomes are kept from the decisions
|
|
209
|
+
on its inputs: `decide_before_run` (the default `studies new` writes) or `exploratory` with its
|
|
210
|
+
reason. `paperlint studies gate` and the rule `repro/outputs-before-decision` hold it, and the
|
|
211
|
+
index carries it in three columns: the mode, the decision's first commit and date, and the first
|
|
212
|
+
output's commit. Its own note: [`study-modes.md`](study-modes.md).
|
|
213
|
+
|
|
208
214
|
## Where each part lives
|
|
209
215
|
|
|
210
216
|
| part | file | layer |
|
package/docs/e2e.md
CHANGED
|
@@ -8,10 +8,14 @@ or CI:
|
|
|
8
8
|
| -------------------------- | ------------- | -------------------------------------------- | --------------------------- |
|
|
9
9
|
| `npm run test:e2e:install` | `e2e-install` | npm and pnpm, and the network for `npm pack` | `test/e2e/install/*.e2e.ts` |
|
|
10
10
|
| `npm run test:e2e:tex` | `e2e-tex` | TeX Live from `paperlint toolchain`, banal | `test/e2e/tex/*.e2e.ts` |
|
|
11
|
-
| `npm run test:e2e`
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
11
|
+
| `npm run test:e2e:studies` | `e2e-studies` | git | `test/e2e/studies/*.e2e.ts` |
|
|
12
|
+
| `npm run test:e2e` | all three | | |
|
|
13
|
+
|
|
14
|
+
All three areas are part of `npm run check`, so nobody has to remember to call them. CI runs each in
|
|
15
|
+
the job that has its environment: `test:e2e:install` and `test:e2e:studies` in `gates`, `test:e2e:tex`
|
|
16
|
+
in `build-e2e`. The studies run drives a study decided before its run through the real binary in a
|
|
17
|
+
real git repository: the gate, the order the lint reads, a shallow clone named, and the two commits
|
|
18
|
+
`studies index` writes into the index.
|
|
15
19
|
|
|
16
20
|
This page says what they prove, what they deliberately do not, and when a change owes a new one.
|
|
17
21
|
|
package/docs/layout.md
CHANGED
|
@@ -112,6 +112,7 @@ Both are explained in [`venues.md`](venues.md).
|
|
|
112
112
|
| `repro/<id>-<slug>/` | 3 study | `paperlint studies new` (the folder and its `README.md`), then `build-benchmark` (the design, in its `README.md`) | [`repro/allowed-files-only`](rules/repro/allowed-files-only.md): the name, and a `README.md` in it; [`repro/study-registered`](rules/repro/study-registered.md): its row |
|
|
113
113
|
| `repro/<id>-<slug>/plan.json` | 3 study | `paperlint studies new` writes the template; `build-benchmark` fills it before the sample is drawn; you add the author's check | [`repro/study-registered`](rules/repro/study-registered.md): its fields, filled once the study runs, and `author_check` in a headline measurement |
|
|
114
114
|
| `repro/<id>-<slug>/prereg.json` | 3 study | the plan's old name, still read when the row names it | [`repro/study-plan-name`](rules/repro/study-plan-name.md) (warning); [`study/declare-sample`](rules/study/declare-sample.md), off by default |
|
|
115
|
+
| `repro/<id>-<slug>/decision.md` | 3 study | you: the decisions on the inputs of a study decided before its run, committed before the run (`paperlint studies gate`); the plan names it, and it may live elsewhere in the paper | [`repro/outputs-before-decision`](rules/repro/outputs-before-decision.md) |
|
|
115
116
|
| `repro/<id>-<slug>/data/` | 3 study | `build-label-page` exports the labels here | nothing |
|
|
116
117
|
| `repro/tools/`, `repro/data/` | 3 study | your scripts; `paperlint build` keeps `repro/data/references-cache.json` | nothing inside them |
|
|
117
118
|
| `NOTES.md` | 3, then 4 | you: each number → the data it comes from; `build-label-page` adds a line per label file | nothing |
|
|
@@ -135,25 +136,27 @@ A check runs at one of four moments:
|
|
|
135
136
|
The lint-only job skips what needs a build (pages, fonts, references).
|
|
136
137
|
- **On demand** — a skill, when you ask for what it does. A skill stops nothing; it writes.
|
|
137
138
|
|
|
138
|
-
| what | kind
|
|
139
|
-
| ------------------------------------------------------------------------------------------- |
|
|
140
|
-
| `paper-folder-layout` | hook
|
|
141
|
-
| `paper-edit-guard` | hook
|
|
142
|
-
| `paper-skills-nudge` | hook
|
|
143
|
-
| `paper-status-gates` | hook
|
|
144
|
-
| [`paper/allowed-files-only`](rules/paper/allowed-files-only.md) | rule
|
|
145
|
-
| [`repro/allowed-files-only`](rules/repro/allowed-files-only.md) | rule
|
|
146
|
-
| [`repro/study-registered`](rules/repro/study-registered.md) | rule
|
|
147
|
-
| [`repro/study-plan-name`](rules/repro/study-plan-name.md) | rule
|
|
148
|
-
| [`
|
|
149
|
-
|
|
|
150
|
-
| `
|
|
151
|
-
| `paper/
|
|
152
|
-
|
|
|
153
|
-
|
|
|
154
|
-
|
|
|
155
|
-
| `build
|
|
156
|
-
| `
|
|
139
|
+
| what | kind | when it runs | what it stops |
|
|
140
|
+
| ------------------------------------------------------------------------------------------- | ------- | ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
141
|
+
| `paper-folder-layout` | hook | before an agent creates a file (Edit, Write) | a new entry the layout does not allow, at a paper's root or the top of its `repro/`; never an existing file |
|
|
142
|
+
| `paper-edit-guard` | hook | before an agent's shell command | a shell command that writes or moves a paper's source, which would skip the hooks that run after an edit |
|
|
143
|
+
| `paper-skills-nudge` | hook | after an agent edits a paper's source | nothing: it shows the agent the pre-submit checklist |
|
|
144
|
+
| `paper-status-gates` | hook | after an agent edits a paper's source | nothing: it prints the paper's readiness and its unrun gates, for a person reading the transcript |
|
|
145
|
+
| [`paper/allowed-files-only`](rules/paper/allowed-files-only.md) | rule | every `paperlint lint`, and CI | a stray file or folder at a paper's root (error) |
|
|
146
|
+
| [`repro/allowed-files-only`](rules/repro/allowed-files-only.md) | rule | every `paperlint lint`, and CI | a loose entry at the top of `repro/`, a folder that is not a study's, a study folder without `README.md` (error) |
|
|
147
|
+
| [`repro/study-registered`](rules/repro/study-registered.md) | rule | every `paperlint lint`, and CI | a study folder without its row in `repro/studies.json`, a row that does not match, a measurement or plan that does not hold, an index out of sync (error) |
|
|
148
|
+
| [`repro/study-plan-name`](rules/repro/study-plan-name.md) | rule | every `paperlint lint`, and CI | nothing: it warns when a study's plan is still named `prereg.json` |
|
|
149
|
+
| [`repro/outputs-before-decision`](rules/repro/outputs-before-decision.md) | rule | every `paperlint lint`, and CI (with the whole history) | an output of a study decided before its run that git shows was not made after the decision |
|
|
150
|
+
| `paperlint studies gate` | command | before a decide-first study's run | the run, until the decision is committed |
|
|
151
|
+
| [`study/declare-sample`](rules/study/declare-sample.md) | rule | every `paperlint lint`, and CI, once a paper turns it on | a study's `prereg.json` without its population, strata, screens, coverage or the author's check (experimental, off by default) |
|
|
152
|
+
| [`paper/folder-venue-leftover`](rules/paper/folder-venue-leftover.md) | rule | every `paperlint lint`, and CI | nothing: it warns when the folder is named after another venue |
|
|
153
|
+
| `review/frontmatter`, `sibling/frontmatter` | rule | every `paperlint lint`, and CI | a review whose findings break the schema (error); a sibling card that does not say how much was read (warning) |
|
|
154
|
+
| `paper/stages`, `paper/source` | rule | every `paperlint lint`, and CI | a stage whose frozen PDF or source is missing or changed (error) |
|
|
155
|
+
| the rules on the source (`tex/*`, `paper/*`, `bib/*`) | rule | every `paperlint lint`, and CI | what each one's [row](rules.md) says |
|
|
156
|
+
| the rules on what `paperlint build` measured (`pdf/*`, `format/page-limit`, the references) | rule | every `paperlint lint` after a build; CI's full job | a PDF over the page limit, in the wrong font or format, or with a bad reference |
|
|
157
|
+
| `study-accepted-papers` | skill | when you ask; stage 2 (modes A, B) and stage 5 (modes C, D) | nothing: it fills the venue's corpus, and writes the strong-accept scorecard in `PIPELINE-STATUS.md` |
|
|
158
|
+
| `build-benchmark` | skill | when you ask; stage 3 | nothing: it registers the study with `paperlint studies new`, writes its design, fills its `plan.json`, and updates its row when it ends |
|
|
159
|
+
| `build-label-page` | skill | when you ask; stage 3 | nothing: it builds the labelling page, and exports the answers into the study's `data/` |
|
|
157
160
|
|
|
158
161
|
The hooks are wired by `paperlint init` ([`install.md`](install.md)). A rule's level and options:
|
|
159
162
|
[`rules.md`](rules.md).
|
|
@@ -0,0 +1,119 @@
|
|
|
1
|
+
# repro/outputs-before-decision
|
|
2
|
+
|
|
3
|
+
**Level:** error · **Reads:** `repro/studies.json`, each study's `plan.json`, the files under a
|
|
4
|
+
decide-first study's folder, and git's history of them · **Reported on:** the first line of the
|
|
5
|
+
paper's `PIPELINE-STATUS.md`, once per finding
|
|
6
|
+
|
|
7
|
+
## What it catches
|
|
8
|
+
|
|
9
|
+
A study decided before its run — its plan's `mode` is `decide_before_run`, the default
|
|
10
|
+
`paperlint studies new` writes for a measurement — whose history does not show that order:
|
|
11
|
+
|
|
12
|
+
```json
|
|
13
|
+
{
|
|
14
|
+
"mode": {
|
|
15
|
+
"kind": "decide_before_run",
|
|
16
|
+
"outputs": ["out/**", "RESULTS.md"],
|
|
17
|
+
"decision": {
|
|
18
|
+
"file": "repro/P1-S01-outcome/decision.md",
|
|
19
|
+
"note": "exclusions, veto, and the sample approved"
|
|
20
|
+
}
|
|
21
|
+
}
|
|
22
|
+
}
|
|
23
|
+
```
|
|
24
|
+
|
|
25
|
+
- **`output-before-decision`** — an output (a file the `outputs` globs match, under the study's
|
|
26
|
+
folder) whose FIRST commit is not a strict descendant of the decision file's FIRST commit: earlier,
|
|
27
|
+
on a line of history the decision is not on, or in the very same commit, which says nothing about
|
|
28
|
+
which came first. An output committed and later deleted still counts.
|
|
29
|
+
- **`undecided-output`** — an output in the tree, or one ever committed and since deleted, while the
|
|
30
|
+
decision file has never been committed.
|
|
31
|
+
- **`decision-is-output`** — the decision file lies in the study's folder and an output glob matches
|
|
32
|
+
it: the decision would be a file its own run produces.
|
|
33
|
+
- **`decision-deleted`** — the decision file was committed and later deleted: it is not in HEAD, so
|
|
34
|
+
the study has no decision its outputs could follow, and the order is not judged against it.
|
|
35
|
+
- **`decision-not-file`** — the decision path names a directory or a symbolic link: git answers for
|
|
36
|
+
a directory with the commits of anything under it, and records a link rather than the file it
|
|
37
|
+
points to, so no order is read from either. (A path that names the paper's folder
|
|
38
|
+
itself, `.`, or ends in `/` is refused when the plan is parsed.)
|
|
39
|
+
- **`shallow`** — the clone is shallow, so every file's first commit is the clone's: no order can be
|
|
40
|
+
read, and the rule says so instead of judging.
|
|
41
|
+
- **`no-history`** — git could not answer (not installed, not a repository).
|
|
42
|
+
|
|
43
|
+
The study's own files — `plan.json`, `README.md` and the decision file — are never outputs. A symbolic
|
|
44
|
+
link in the study's folder is one entry, as git stores it, and is never followed: a link to the folder
|
|
45
|
+
itself cannot loop the walk, and one to a tree outside brings none of its files into the study.
|
|
46
|
+
|
|
47
|
+
## Why
|
|
48
|
+
|
|
49
|
+
A measurement's inputs — which runs count, which units are excluded, whether the sample is approved
|
|
50
|
+
— must be decided before anyone has seen an outcome; otherwise the decision can follow the result.
|
|
51
|
+
"Decide first, then compute" needs no key and no encryption: the decision is committed, the run
|
|
52
|
+
starts only after (`paperlint studies gate` exits 0 only then), and git's history is the proof of
|
|
53
|
+
order. This rule re-reads that proof on every lint, so a reviewer who clones the repository can run
|
|
54
|
+
the same check and get the same answer. The design is
|
|
55
|
+
[`docs/design/study-modes.md`](../../design/study-modes.md).
|
|
56
|
+
|
|
57
|
+
**It asks git for an order, never pins a sha**, which is why it can live as a lint rule (rules in
|
|
58
|
+
`eslint-rules/` may not run git: a pinned sha is deleted by a squash or a gc). A squash merge keeps
|
|
59
|
+
the order when the decision was merged before the run — the decide-first workflow — and a shallow
|
|
60
|
+
checkout is named, not read as a violation.
|
|
61
|
+
|
|
62
|
+
## Examples
|
|
63
|
+
|
|
64
|
+
Failing:
|
|
65
|
+
|
|
66
|
+
```text
|
|
67
|
+
papers/example/PIPELINE-STATUS.md
|
|
68
|
+
1:1 error `repro/P1-S01-outcome/RESULTS.md`, an output of `P1-S01`, entered history in
|
|
69
|
+
4f2a9c0e1b7d (2026-10-16T10:00:00+05:00), the very commit that first recorded its
|
|
70
|
+
decision, 4f2a9c0e1b7d (2026-10-16T10:00:00+05:00). The study is declared decided
|
|
71
|
+
before its run, and its history does not show that order. What to do: if the decision
|
|
72
|
+
really came first, nothing in this history can prove it — declare the study
|
|
73
|
+
`exploratory` with the reason, or run it again after the decision; never rewrite the
|
|
74
|
+
history to make it pass.
|
|
75
|
+
Rule repro/outputs-before-decision: https://github.com/zernie/paperlint/blob/main/docs/rules/repro/outputs-before-decision.md repro/outputs-before-decision
|
|
76
|
+
```
|
|
77
|
+
|
|
78
|
+
Passing: `decision.md` committed in one commit; `RESULTS.md` and `out/` committed in a later commit
|
|
79
|
+
that descends from it. An `exploratory` study is not this rule's.
|
|
80
|
+
|
|
81
|
+
## Options / preset fields
|
|
82
|
+
|
|
83
|
+
No rule options. The study declares its mode in its plan.
|
|
84
|
+
|
|
85
|
+
## What it does not check
|
|
86
|
+
|
|
87
|
+
- What the decision says. The gate is that it is committed, not its content.
|
|
88
|
+
- **A decision amended after the outputs**: the order is between FIRST commits. A later edit of the
|
|
89
|
+
decision file is visible in its history (`git log -- <file>`) but not reported.
|
|
90
|
+
- That nobody ran the study before the decision and kept the outputs out of git. History records
|
|
91
|
+
what was committed; a run on a machine that never committed it leaves no trace.
|
|
92
|
+
- A study whose row names a folder that is not one inside `repro/` (`../x`), or a folder that is a
|
|
93
|
+
symbolic link: nothing is read through it, so it is no study here;
|
|
94
|
+
[`repro/study-registered`](study-registered.md) reports the row or reads its plan as missing.
|
|
95
|
+
- A study whose plan is still a draft: only a ready plan's mode decides anything, and
|
|
96
|
+
[`repro/study-registered`](study-registered.md) reports an unfilled plan once the study runs.
|
|
97
|
+
- Commit dates. They are shown in the message, but the order is ancestry, which a rebase can
|
|
98
|
+
rewrite only by making a new history that a reviewer comparing against the published one sees.
|
|
99
|
+
|
|
100
|
+
## How to fix
|
|
101
|
+
|
|
102
|
+
Commit the decision first, then let the run start:
|
|
103
|
+
|
|
104
|
+
```sh
|
|
105
|
+
git add papers/example/repro/P1-S01-outcome/decision.md && git commit -m "P1-S01: decision"
|
|
106
|
+
npx paperlint studies gate papers/example P1-S01 && ./run.sh
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
In CI, check out the whole history: `actions/checkout` with `fetch-depth: 0`. A study whose outputs
|
|
110
|
+
were made before its decision cannot be repaired by editing history; declare it exploratory:
|
|
111
|
+
|
|
112
|
+
```json
|
|
113
|
+
{
|
|
114
|
+
"mode": {
|
|
115
|
+
"kind": "exploratory",
|
|
116
|
+
"reason": "outputs existed before the exclusions were fixed"
|
|
117
|
+
}
|
|
118
|
+
}
|
|
119
|
+
```
|