paperlint 9.0.0 → 10.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.github/CLAUDE.md +22 -0
- package/.github/CLAUDE.md.spec.ts +20 -0
- package/CLAUDE.md +59 -515
- package/CONTRIBUTING.md +2 -2
- package/README.md +68 -128
- package/dist/CLAUDE.md.spec.d.ts +3 -0
- package/dist/CLAUDE.md.spec.d.ts.map +1 -0
- package/dist/CLAUDE.md.spec.js +62 -0
- package/dist/CLAUDE.md.spec.js.map +1 -0
- package/dist/build.d.ts +1 -1
- package/dist/build.js +2 -2
- package/dist/cli.d.ts +5 -1
- package/dist/cli.d.ts.map +1 -1
- package/dist/cli.js +52 -6
- package/dist/cli.js.map +1 -1
- package/dist/domain/layout-tree.d.ts +19 -0
- package/dist/domain/layout-tree.d.ts.map +1 -0
- package/dist/domain/layout-tree.js +32 -0
- package/dist/domain/layout-tree.js.map +1 -0
- package/dist/domain/lookup-cache.d.ts +2 -2
- package/dist/domain/lookup-cache.js +2 -2
- package/dist/domain/paper-layout.d.ts +81 -0
- package/dist/domain/paper-layout.d.ts.map +1 -0
- package/dist/domain/paper-layout.js +259 -0
- package/dist/domain/paper-layout.js.map +1 -0
- package/dist/domain/repro-layout.d.ts +80 -0
- package/dist/domain/repro-layout.d.ts.map +1 -0
- package/dist/domain/repro-layout.js +247 -0
- package/dist/domain/repro-layout.js.map +1 -0
- package/dist/domain/scaffold-layout.d.ts +28 -0
- package/dist/domain/scaffold-layout.d.ts.map +1 -0
- package/dist/domain/scaffold-layout.js +81 -0
- package/dist/domain/scaffold-layout.js.map +1 -0
- package/dist/eslint-rules/CLAUDE.md.spec.d.ts +3 -0
- package/dist/eslint-rules/CLAUDE.md.spec.d.ts.map +1 -0
- package/dist/eslint-rules/CLAUDE.md.spec.js +39 -0
- package/dist/eslint-rules/CLAUDE.md.spec.js.map +1 -0
- package/dist/eslint-rules/install-path-literals.d.ts +1 -1
- package/dist/eslint-rules/install-path-literals.js +4 -4
- package/dist/eslint-rules/install-path-literals.js.map +1 -1
- package/dist/eslint-rules/pdf-last-page-balance.js +1 -1
- package/dist/eslint-rules/prereg-coverage.d.ts +98 -0
- package/dist/eslint-rules/prereg-coverage.d.ts.map +1 -0
- package/dist/eslint-rules/prereg-coverage.js +274 -0
- package/dist/eslint-rules/prereg-coverage.js.map +1 -0
- package/dist/folder-layout-hook.d.ts +21 -0
- package/dist/folder-layout-hook.d.ts.map +1 -0
- package/dist/folder-layout-hook.js +117 -0
- package/dist/folder-layout-hook.js.map +1 -0
- package/dist/folder-layout-rule.d.ts +22 -0
- package/dist/folder-layout-rule.d.ts.map +1 -0
- package/dist/folder-layout-rule.js +107 -0
- package/dist/folder-layout-rule.js.map +1 -0
- package/dist/init.js +1 -1
- package/dist/init.js.map +1 -1
- package/dist/lib/agent-cli-version.js +1 -1
- package/dist/lib/agent-cli-version.js.map +1 -1
- package/dist/new-paper.d.ts +4 -1
- package/dist/new-paper.d.ts.map +1 -1
- package/dist/new-paper.js +40 -6
- package/dist/new-paper.js.map +1 -1
- package/dist/references.d.ts +1 -1
- package/dist/references.d.ts.map +1 -1
- package/dist/references.js +2 -2
- package/dist/references.js.map +1 -1
- package/dist/repro-layout-rule.d.ts +16 -0
- package/dist/repro-layout-rule.d.ts.map +1 -0
- package/dist/repro-layout-rule.js +81 -0
- package/dist/repro-layout-rule.js.map +1 -0
- package/dist/tex-requirements.js +1 -1
- package/dist/toolchain.js +1 -1
- package/docs/ci.md +60 -0
- package/docs/configuration.md +2 -0
- package/docs/e2e.md +3 -3
- package/docs/incidents.md +5 -5
- package/docs/install.md +20 -16
- package/docs/layout.md +154 -0
- package/docs/optional-rules.md +5 -4
- package/docs/references.md +5 -2
- package/docs/rules/paper/allowed-files-only.md +192 -0
- package/docs/rules/repro/allowed-files-only.md +128 -0
- package/docs/rules/study/declare-sample.md +146 -0
- package/docs/rules.md +53 -50
- package/docs/skills.md +6 -3
- package/docs/venues.md +94 -0
- package/eslint-rules/CLAUDE.md +44 -0
- package/eslint-rules/CLAUDE.md.spec.ts +43 -0
- package/eslint-rules/install-path-literals.harness.mjs +1 -1
- package/eslint-rules/install-path-literals.ts +4 -4
- package/eslint-rules/pdf-last-page-balance.ts +1 -1
- package/eslint-rules/prereg-coverage.test.ts +448 -0
- package/eslint-rules/prereg-coverage.ts +419 -0
- package/eslint.config.mjs +6 -3
- package/hooks/CLAUDE.md +34 -0
- package/hooks/CLAUDE.md.spec.ts +30 -0
- package/lib/agent-cli-version.ts +1 -1
- package/lib/skill-trigger-cases.harness.mjs +51 -5
- package/lib/skill-trigger-cases.mjs +53 -15
- package/package.json +2 -2
- package/plugin/CLAUDE.md +22 -0
- package/plugin/CLAUDE.md.spec.ts +22 -0
- package/plugin/hooks/hooks.json +9 -0
- package/scripts/coverage-config.test.ts +2 -1
- package/skills/CLAUDE.md +19 -0
- package/skills/CLAUDE.md.spec.ts +18 -0
- package/skills/analyze-sibling-paper/SKILL.md +2 -2
- package/skills/analyze-sibling-paper/SKILL.md.spec.ts +1 -1
- package/skills/build-benchmark/SKILL.md +14 -1
- package/skills/build-benchmark/SKILL.md.spec.ts +13 -0
- package/skills/build-label-page/SKILL.md +118 -0
- package/skills/build-label-page/SKILL.md.spec.ts +129 -0
- package/skills/build-label-page/build-label-page.eval.mjs +15 -0
- package/skills/build-label-page/build-label-page.test.ts +959 -0
- package/skills/build-label-page/references/checklist.md +148 -0
- package/skills/build-label-page/references/example-generator.py +803 -0
- package/skills/find-venue/SKILL.md +5 -3
- package/skills/find-venue/SKILL.md.spec.ts +4 -2
- package/skills/grade-paper-writing/SKILL.md +7 -7
- package/skills/grade-paper-writing/SKILL.md.spec.ts +6 -6
- package/skills/map-prior-work/SKILL.md +2 -2
- package/skills/map-prior-work/SKILL.md.spec.ts +1 -1
- package/skills/paper-pipeline/SKILL.md +8 -6
- package/skills/paper-pipeline/SKILL.md.spec.ts +7 -5
- package/skills/paper-pipeline/grade-paper-writing-ablation.eval.mjs +19 -9
- package/skills/paper-pipeline/references/acceptance-gate.md +19 -19
- package/skills/paper-pipeline/references/artifact-checklist.md +5 -0
- package/skills/paper-pipeline/references/pipeline-status-template.md +30 -1
- package/skills/paper-pipeline/repro/2026-08-07-fork-check.log +1 -1
- package/skills/paper-pipeline/repro/2026-08-07-fork-check2.log +1 -1
- package/skills/paper-pipeline/repro/2026-08-07-language-eval-pilot.log +1 -1
- package/skills/paper-pipeline/repro/2026-08-07-language-eval-raw.log +1 -1
- package/skills/paper-pipeline/repro/2026-08-08-framing-vs-vocabulary-oracle.log +48 -48
- package/skills/paper-pipeline/repro/2026-08-08-grade-paper-writing-ablation-A6-oracle.log +8 -8
- package/skills/paper-pipeline/repro/2026-08-08-grade-paper-writing-ablation-oracle.log +48 -48
- package/skills/paper-pipeline/repro/2026-08-08-parent-replication-gpw.log +2 -2
- package/skills/paper-pipeline/repro/2026-08-08-parent-replication.mjs +3 -3
- package/skills/paper-pipeline/repro/2026-10-13-grade-paper-writing-ablation-preflight-after.log +81 -0
- package/skills/paper-pipeline/repro/2026-10-13-grade-paper-writing-ablation-preflight-before.log +81 -0
- package/skills/paper-pipeline/repro/2026-10-13-grade-paper-writing-ablation-setupdiff.log +59 -0
- package/skills/paper-pipeline/repro/2026-10-13-grade-paper-writing-trigger.jsonl +1 -0
- package/skills/paper-pipeline/repro/2026-10-13-grade-paper-writing-trigger.log +22 -0
- package/skills/paper-pipeline/repro/README.md +4 -0
- package/skills/paper-pipeline/scripts/extract-ref-facts.harness.mjs +2 -2
- package/skills/paper-pipeline/scripts/extract-ref-facts.mjs +2 -2
- package/skills/paper-pipeline/scripts/extract-ref-facts.test.mjs +4 -2
- package/skills/paper-pipeline/scripts/generated-code.harness.mjs +7 -2
- package/skills/paper-pipeline/scripts/generated-code.mjs +8 -4
- package/skills/paper-pipeline/scripts/generated-code.test.ts +15 -1
- package/skills/paper-status/SKILL.md +2 -2
- package/skills/paper-status/SKILL.md.spec.ts +1 -1
- package/skills/plan-paper-timeline/plan-paper-timeline.effects.harness.mjs +21 -102
- package/skills/study-accepted-papers/SKILL.md +163 -161
- package/skills/study-accepted-papers/SKILL.md.spec.ts +167 -165
- package/skills/study-accepted-papers/references/base-rates.md +69 -0
- package/skills/study-accepted-papers/references/mechanisms.md +181 -0
- package/skills/verify-citations/SKILL.md +2 -2
- package/skills/verify-citations/SKILL.md.spec.ts +1 -1
- package/src/CLAUDE.md +23 -1
- package/src/CLAUDE.md.spec.ts +72 -0
- package/src/build.ts +2 -2
- package/src/cli.harness.mjs +42 -0
- package/src/cli.test.ts +90 -6
- package/src/cli.ts +78 -5
- package/src/domain/layout-tree.test.ts +32 -0
- package/src/domain/layout-tree.ts +47 -0
- package/src/domain/lookup-cache.ts +2 -2
- package/src/domain/paper-layout.test.ts +264 -0
- package/src/domain/paper-layout.ts +336 -0
- package/src/domain/repro-layout.test.ts +222 -0
- package/src/domain/repro-layout.ts +311 -0
- package/src/domain/scaffold-layout.ts +104 -0
- package/src/folder-layout-hook.test.ts +214 -0
- package/src/folder-layout-hook.ts +141 -0
- package/src/folder-layout-rule.test.ts +235 -0
- package/src/folder-layout-rule.ts +151 -0
- package/src/hooks-settings.harness.ts +9 -8
- package/src/init.ts +1 -1
- package/src/layout-docs.test.ts +52 -0
- package/src/new-paper.harness.mjs +23 -1
- package/src/new-paper.test.ts +299 -6
- package/src/new-paper.ts +51 -5
- package/src/paper-files.test.ts +9 -6
- package/src/references.test.ts +5 -5
- package/src/references.ts +2 -2
- package/src/repro-layout-rule.test.ts +114 -0
- package/src/repro-layout-rule.ts +121 -0
- package/src/rule-cases.test.ts +98 -0
- package/src/tex-requirements.ts +1 -1
- package/src/toolchain.ts +1 -1
- package/templates/paper/PIPELINE-STATUS.md +41 -7
|
@@ -1,174 +1,176 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: study-accepted-papers
|
|
3
|
-
description: "
|
|
3
|
+
description: "Learn what a target venue rewards from its own accepted papers and award winners, and turn it into requirements for your study or a diagnosis of your draft. Use EARLY, in SETUP right after find-venue and before the study is designed, so what the venue rewards shapes the experiment (mode A idea, mode B design); and LATE, once text exists (mode C draft: score the paper against the strong-accept mechanisms M1–M10 with line references, blockers, owners and a plan; mode D final: the abstract and introduction, sentence by sentence). Always asks which mode, suggesting one from what exists, and asks the claim, breadth, depth (L0–L3) and budget up front, each with a default and a time estimate. Keeps one corpus per venue in <papers-root>/venues/<venue>/, shared by every paper that targets it; the venue's winners vs its ordinary papers and award papers on the topic at other venues are two corpora, labelled apart. Names the proxy for 'strong accept' and its base rate before any probability. Distinct from find-venue (picks WHERE), research-ideate (is the idea worth doing), map-prior-work and analyze-sibling-paper (your rivals, not what the venue rewards), and pc-panel-review / paper-adversarial-review (red-team your draft)."
|
|
4
4
|
context: fork
|
|
5
5
|
allowed-tools: [Read, Write, Edit, Grep, Glob, Bash, WebSearch, WebFetch, Agent]
|
|
6
6
|
---
|
|
7
7
|
|
|
8
|
-
<!-- vigiles:sha256:
|
|
9
|
-
|
|
10
|
-
# study-accepted-papers —
|
|
11
|
-
|
|
12
|
-
Reviewers
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
- `
|
|
21
|
-
-
|
|
22
|
-
|
|
23
|
-
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
**
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
Every
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
expensive lever is normally an **`extend-paper` task for the follow-on**, not a pre-deadline edit.
|
|
111
|
-
Say so explicitly; don't let a tempting expensive lever destabilize a shippable Accept.
|
|
112
|
-
|
|
113
|
-
## Output
|
|
114
|
-
|
|
115
|
-
1. **Venue-bar summary** — what this venue rewards, its track requirements, its PC's taste (2-4 lines,
|
|
116
|
-
each sourced).
|
|
117
|
-
2. **Strong-accepted-paper table** — `title | year | contribution | what made it STRONG`, with URLs.
|
|
118
|
-
3. **Ranked levers for your draft** — each: the gap, the strong-paper precedent it's drawn from,
|
|
119
|
-
CHEAP/EXPENSIVE tag, and the concrete edit (or the extend-paper deferral). Most-impactful first.
|
|
120
|
-
4. **Citation gaps** — specific real papers (arXiv id / DOI) a reviewer at this venue would expect and
|
|
121
|
-
ding you for missing, each with why. Hand these to `verify-citations`.
|
|
122
|
-
5. **Corpus-level distribution** — the denominators, each traceable to the measurement file: how many
|
|
123
|
-
papers have a figure and the median count, how many report confidence intervals, how many run a
|
|
124
|
-
statistical test, how many have a control or placebo arm, section-length norms, abstract-length
|
|
125
|
-
range. **These are what change decisions**, because they say whether your draft is normal, an
|
|
126
|
-
outlier, or quietly ahead.
|
|
127
|
-
|
|
128
|
-
## 🔴 Where the output lives — three homes, and all three are required
|
|
129
|
-
|
|
130
|
-
A run that returns only a write-up has destroyed its own evidence. Observed on `compile-rules-2026`
|
|
131
|
-
2026-08-03: the pass measured 34 PDFs and returned excellent prose, and what survived into the repo
|
|
132
|
-
was **two scripts, 16 KB**. The corpus, the extracted text and the measurement table were in a
|
|
133
|
-
scratchpad that gets wiped. Every denominator in the write-up was, an hour later, unverifiable.
|
|
134
|
-
|
|
135
|
-
| home | what goes there | why there |
|
|
8
|
+
<!-- vigiles:sha256:607eaf7930ecaf30 compiled from skills/study-accepted-papers/SKILL.md.spec.ts -->
|
|
9
|
+
|
|
10
|
+
# study-accepted-papers — what this venue rewards, measured on its own papers
|
|
11
|
+
|
|
12
|
+
Reviewers score against the papers that got in before. This skill makes that standard explicit, from
|
|
13
|
+
the venue's **accepted papers and award winners**, and turns it into something the author can act on:
|
|
14
|
+
requirements for a study not yet run, or a scored diagnosis of a draft.
|
|
15
|
+
|
|
16
|
+
It answers a different question from its neighbours:
|
|
17
|
+
- `find-venue`: *where* should this go?
|
|
18
|
+
- `research-ideate`: is the *idea* worth doing?
|
|
19
|
+
- `map-prior-work` / `analyze-sibling-paper`: who are the *rivals*, and is the delta real?
|
|
20
|
+
- `pc-panel-review` / `paper-adversarial-review`: what is wrong with *my draft*?
|
|
21
|
+
- **`study-accepted-papers`: what does it take to be *strong* at THIS venue, judged by its own papers?**
|
|
22
|
+
|
|
23
|
+
**Two entry points, one skill.** In `paper-pipeline` it sits in SETUP, right after `find-venue`: the
|
|
24
|
+
venue's standard is worth most before the study is designed, when it can still decide what the oracle is and
|
|
25
|
+
which belief the claim targets. It comes back once text exists, to score the draft. Run it only late
|
|
26
|
+
and its main findings arrive after they could change the design.
|
|
27
|
+
|
|
28
|
+
Two references beside this file, read before any level above L0:
|
|
29
|
+
- [`references/mechanisms.md`](references/mechanisms.md): the mechanisms M1–M10 that separated award
|
|
30
|
+
winners from ordinary accepted papers in the run this skill was rebuilt from, with what supports each
|
|
31
|
+
and what each is not. Form (figures, length, title shape) did **not** separate them.
|
|
32
|
+
- [`references/base-rates.md`](references/base-rates.md): the proxies for "strong accept" and the only
|
|
33
|
+
public score data found for software-engineering venues.
|
|
34
|
+
|
|
35
|
+
## Step 0 — ask before spending (always)
|
|
36
|
+
|
|
37
|
+
**Ask which mode. Always.** Look at what exists first, then suggest one; do not pick silently, and do
|
|
38
|
+
not start until the author answers (or the caller already passed a mode).
|
|
39
|
+
|
|
40
|
+
| mode | what exists | it compares | it produces | default depth |
|
|
41
|
+
|---|---|---|---|---|
|
|
42
|
+
| **A idea** | an idea and a target venue; no paper source | the venue's winners against its ordinary papers | what this venue rewards, written as **requirements for the study**: what the oracle is, which stated belief the claim targets, which factor must be a column | L1 (+ L0) |
|
|
43
|
+
| **B design** | a planned claim and measurements (the `frame` row, a design note in `research/`); no text | the planned claim and evidence against M1–M10 | **design gaps before anything runs** ("the headline rests on model labels", "no named belief") | L1 + the M1–M10 check of the plan |
|
|
44
|
+
| **C draft** | the paper's text | the draft against M1–M10 and the deep reads | the **scorecard**: pass / partial / fail per mechanism with `file:line`, blockers with cost and owner, a plan, probabilities against stated base rates | L2 + L3 |
|
|
45
|
+
| **D final** | near-final text, the deadline close | the abstract and introduction, sentence by sentence, against the closest winners | **wording**: the denominator in every claim sentence, the named belief, no drift between abstract and body | L3, wording only |
|
|
46
|
+
|
|
47
|
+
Suggest from the project: no `paper.tex` → A, or B if a claim is already written down; `paper.tex`
|
|
48
|
+
with a body → C; the current cycle's submission deadline within about a week → D. Say why you suggest it
|
|
49
|
+
("C: paper.tex has a full body, and the deadline is 20 days away"), and let the author override.
|
|
50
|
+
|
|
51
|
+
**Then the four questions, in one message, each with its default and its time.** The author may answer
|
|
52
|
+
"defaults" to all of them.
|
|
53
|
+
|
|
54
|
+
1. **The claim**, in one paragraph. Default: the `frame` row of `PIPELINE-STATUS.md`, else the
|
|
55
|
+
abstract. *Without it the deep-read selection is blind.* (Author: about 5 minutes.)
|
|
56
|
+
2. **Breadth.** Default: **two corpora, labelled apart**: (1) the venue's own award winners against its
|
|
57
|
+
ordinary accepted papers, over 5–7 years; (2) award papers on the author's topic at other venues.
|
|
58
|
+
Corpus 1 says what *this* committee rewards; corpus 2 says how strong papers on this topic make a
|
|
59
|
+
broad claim convincing, which matters when the venue has no winner on the topic. Never pool them.
|
|
60
|
+
3. **Depth**, L0–L3 (below). Default: the mode's row above.
|
|
61
|
+
4. **Budget**: agent time, money, and the author's own hours. Default: the sum of the chosen levels.
|
|
62
|
+
Ask again before going over it.
|
|
63
|
+
|
|
64
|
+
## The depth levels
|
|
65
|
+
|
|
66
|
+
| level | what it does | rough cost (from the run) | answers |
|
|
67
|
+
|---|---|---|---|
|
|
68
|
+
| **L0 form** | count body words, figures, tables, research questions, abstract length, statistics markers, artifact links across the venue's accepted papers | minutes | formal fit only. Say that form did **not** separate winners in the run (`references/mechanisms.md`) |
|
|
69
|
+
| **L1 winner corpus** | every award winner the venue lists over 5–7 years against the ordinary accepted papers of the same years; code form **and** substance per paper: the oracle under the headline (execution / developer action / human / model-only), a belief the field stated on record, the claim shape, a controlled design, the contribution sentence verbatim | about 30 min of agents | what does this venue reward? |
|
|
70
|
+
| **L2 deep reads** | a judge, briefed with the author's claim, picks 8–12 papers: winners closest to the claim plus **award / no-award pairs on the same question or dataset** (the pairs were the most informative part of the run); one agent per paper, full text, one shared reading brief | about 1 h of agents (parallel readers) | how do strong papers on this topic make a broad claim convincing? |
|
|
71
|
+
| **L3 diagnosis** | score the author's plan or draft on each mechanism; blockers with cost and owner; a plan; probabilities against the base rates | about 30 min | can this paper reach strong accept, and what stands in the way? |
|
|
72
|
+
|
|
73
|
+
Each level states **its proxy for "strong accept" and its base rate**, and which conclusions rest on
|
|
74
|
+
fewer than ten papers.
|
|
75
|
+
|
|
76
|
+
### L0 — measure all of the accepted set
|
|
77
|
+
|
|
78
|
+
The value of L0 is a **denominator** ("33 of 34 accepted papers have a figure"). A sample of 8–12 cannot
|
|
79
|
+
give one. A workshop's accepted volume is 30–60 papers: take all of them. For a main conference, take
|
|
80
|
+
the track or the topic, capped at about 100, and **state the cap and why**; a silently truncated corpus
|
|
81
|
+
gives denominators that are quietly wrong. Cross accepted papers with archival status before calibrating:
|
|
82
|
+
an oral that is non-archival is not the paper your archival submission competes with.
|
|
83
|
+
|
|
84
|
+
### L1 — the winner corpus
|
|
85
|
+
|
|
86
|
+
Take the venue's award list (its hall of fame, the awards page of each year's program) for 5–7 years,
|
|
87
|
+
not one year: a single year of five winners produced a "winners have many figures" pattern that vanished
|
|
88
|
+
over seven years. Code every winner and every ordinary paper of the same years with the same fields, so
|
|
89
|
+
each "winners do X" has an ordinary-paper count beside it. Report k/n for both groups and a test with a
|
|
90
|
+
correction for the number of features; a non-significant difference is reported as such.
|
|
91
|
+
|
|
92
|
+
### L2 — the deep reads
|
|
93
|
+
|
|
94
|
+
The selection is part of the result: save it with the reason for each pick, and say how many came from
|
|
95
|
+
corpus 2 and why ("the venue has no winner on this topic"). One agent per paper, full text, every page.
|
|
96
|
+
The shared brief asks three questions: what is the single claim and why does a reviewer believe it; what
|
|
97
|
+
makes it stronger than its ordinary neighbours; what in it maps onto the author's claim. Each reading
|
|
98
|
+
quotes verbatim with page numbers.
|
|
99
|
+
|
|
100
|
+
### L3 — the diagnosis
|
|
101
|
+
|
|
102
|
+
Score the paper against M1–M10 (`references/mechanisms.md`), with evidence as `file:line` in the paper
|
|
103
|
+
source. Every `fail` gets an owner and either a plan or an explicit **"accept ceiling"** with the
|
|
104
|
+
reason: a ceiling written down is a decision, a silent fail is a hole. Probabilities are a band, marked
|
|
105
|
+
as judgement, against the base rates of `references/base-rates.md`; simulated panels run hot.
|
|
106
|
+
|
|
107
|
+
## Where the results go — one home per fact
|
|
108
|
+
|
|
109
|
+
| what | where | why there |
|
|
136
110
|
|---|---|---|
|
|
137
|
-
|
|
|
138
|
-
|
|
|
139
|
-
|
|
|
111
|
+
| **the venue corpus** | `<papers-root>/venues/<venue>/` (the papers directory from the project's `paperlint.json`) | **shared by every paper that targets the venue**, collected once and reused by every later mode. `MANIFEST.md` (sources, years, what was capped, the proxy and its base rate), the winner and ordinary lists, `measurements.json` (L0/L1 codes, one row per paper), `text/` extractions, the scripts, `deep-reads/` (one file per paper, reusable by any paper). Corpus 2 goes in its own labelled subfolder. **PDFs are not committed**: record the IDs and the command. |
|
|
112
|
+
| **the scorecard** | the section **"Strong-accept scorecard (M1–M10)"** of the paper's `PIPELINE-STATUS.md` | a row per mechanism: status (pass / partial / fail), evidence (`file:line`), owner, plan or "accept ceiling". **The only place the verdicts live**; update it there. |
|
|
113
|
+
| **the reasoning** | `<paper>/reviews/<YYYY-MM-DD>-study-accepted-<venue>-<mode>.md` | a dated report per run, with frontmatter (below). It keeps the reasoning, the selection and the per-mechanism argument, and **points to the scorecard section; it never copies the table**. |
|
|
114
|
+
| **the stage** | the SETUP row with id `venuebar` in `PIPELINE-STATUS.md` (the id is kept for existing scorecards) | status, date, the mode and depth run, and a link to the report. One row, updated by every mode. |
|
|
115
|
+
| **the generic list** | `references/` beside this skill | M1–M10 and the base-rate sources. A mechanism your venue adds, or one that does not hold there, goes in the venue corpus's `MANIFEST.md`; the ids stay stable. |
|
|
140
116
|
|
|
141
|
-
|
|
117
|
+
The corpus, its modes and who reads it: `docs/venues.md` in the paperlint package; every place a stage
|
|
118
|
+
writes, and what checks it: `docs/layout.md`.
|
|
142
119
|
|
|
143
|
-
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
120
|
+
The paper folder has **no corpus folder of its own**: `paper/allowed-files-only` reports one, and the hook
|
|
121
|
+
refuses to create one. A venue's format facts (page limit, template, required sections) go
|
|
122
|
+
in its preset and venue card (`presets/<venue>.jsonc` and `.md` in the package, or the project's
|
|
123
|
+
own), with their source quote, not here.
|
|
124
|
+
|
|
125
|
+
The report's frontmatter (it lives under `reviews/`, so `review/frontmatter` validates it; a
|
|
126
|
+
`findings` list is optional and follows that schema):
|
|
127
|
+
|
|
128
|
+
```yaml
|
|
129
|
+
---
|
|
130
|
+
title: "<venue>: what it rewards, and where this paper stands (mode C, L2+L3)"
|
|
131
|
+
created: 2026-10-12
|
|
132
|
+
venue: <venue>
|
|
133
|
+
mode: C
|
|
134
|
+
depth: [L2, L3]
|
|
135
|
+
proxy: "distinguished-paper award, <k> of <n> accepted, <years>"
|
|
136
|
+
corpus: <papers-root>/venues/<venue>/
|
|
137
|
+
scorecard: PIPELINE-STATUS.md#strong-accept-scorecard-m1m10
|
|
138
|
+
---
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
## Before any level: pin the venue's stated bar
|
|
142
|
+
|
|
143
|
+
`WebFetch` the current call for papers and, if separate, the reviewer guidelines. Capture in the
|
|
144
|
+
venue's own words: the **track requirements** (a missing required element is a desk reject, never a
|
|
145
|
+
weak accept, so check these first); the **review criteria**; the **award criteria** if published; the
|
|
146
|
+
**PC chairs and members**, whose research areas are a findable signal of taste.
|
|
148
147
|
|
|
149
148
|
## Rules
|
|
150
|
-
|
|
151
|
-
- **
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
- **
|
|
156
|
-
- **
|
|
157
|
-
|
|
158
|
-
-
|
|
149
|
+
|
|
150
|
+
- **Fetch real papers.** Never fabricate a title, an author, or "what made it strong". Unverifiable →
|
|
151
|
+
drop it or mark VERIFY.
|
|
152
|
+
- **Evidence levels on every finding**: quoted from a paper or file · counted from the corpus · judgement.
|
|
153
|
+
A judgement is never written as a count.
|
|
154
|
+
- **Do not quote an acceptance rate** unless the venue published the number of submissions.
|
|
155
|
+
- **A citation this skill proposes is not written into the source until `verify-citations` confirms it.**
|
|
156
|
+
If added provisionally, mark `% VERIFY` at the cite; a surviving `% VERIFY` at submit is a bug.
|
|
157
|
+
- **"Accepted" is the reference, "strong" is the target.** Isolate what got in enthusiastically, against
|
|
158
|
+
ordinary papers of the same years, not what got in.
|
|
159
|
+
- **Venue-specific, not generic.** A lever counts only if this venue's papers, rubric or committee ground
|
|
160
|
+
it. Prose craft is `grade-paper-writing` and `tighten-paper`.
|
|
161
|
+
- **Cheap or expensive, and the deadline.** Tag every lever: CHEAP (framing, a scoped sentence, a cited
|
|
162
|
+
belief, a missing citation) or EXPENSIVE (a new oracle, a new column, a new corpus). In mode C or D an
|
|
163
|
+
expensive lever is normally an `extend-paper` task or an accepted ceiling, said explicitly; in modes A
|
|
164
|
+
and B it is exactly what the study should include.
|
|
165
|
+
- Convert any deadline that surfaces to the author's own zone and hand it to `plan-paper-timeline`.
|
|
159
166
|
|
|
160
167
|
## Compose with
|
|
161
|
-
|
|
162
|
-
- `
|
|
163
|
-
- `
|
|
164
|
-
- `
|
|
165
|
-
-
|
|
166
|
-
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
GateBench" paper sat at ~Accept (pc-panel ~0.72), and the task was to find what would push it toward a
|
|
171
|
-
strong accept without cramming new contributions before the 2026-07-24 deadline. The workflow that
|
|
172
|
-
produced the levers — pull AISec's accepted measurement/benchmark security papers, isolate the
|
|
173
|
-
adaptive-evaluation + threat-model + reusable-artifact pattern that the strong ones share, then split
|
|
174
|
-
fixes into cheap-prose vs expensive-experiment — is exactly what this skill encodes.
|
|
168
|
+
|
|
169
|
+
- `find-venue` (upstream) → this skill in mode A; `argument-arc` (frame mode) takes its requirements.
|
|
170
|
+
- `build-benchmark` — takes mode A/B requirements (the oracle, the column) into the study design.
|
|
171
|
+
- `harden-paper` — the pre-submit gate; this skill gives it the venue-specific content axis.
|
|
172
|
+
- `pc-panel-review` / `paper-adversarial-review` — red-team the draft; this skill says which bar the
|
|
173
|
+
red-team should hold it to.
|
|
174
|
+
- `analyze-sibling-paper` — a deep read that turns out to be a rival goes there for the scoop question.
|
|
175
|
+
- `verify-citations` — takes the citation gaps.
|
|
176
|
+
- `extend-paper` — the home of every expensive lever deferred past this submission.
|