paperlint 9.0.0 → 10.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (190) hide show
  1. package/.github/CLAUDE.md +22 -0
  2. package/.github/CLAUDE.md.spec.ts +20 -0
  3. package/CLAUDE.md +59 -515
  4. package/CONTRIBUTING.md +2 -2
  5. package/README.md +68 -128
  6. package/dist/CLAUDE.md.spec.d.ts +3 -0
  7. package/dist/CLAUDE.md.spec.d.ts.map +1 -0
  8. package/dist/CLAUDE.md.spec.js +62 -0
  9. package/dist/CLAUDE.md.spec.js.map +1 -0
  10. package/dist/build.d.ts +1 -1
  11. package/dist/build.js +2 -2
  12. package/dist/cli.d.ts +5 -1
  13. package/dist/cli.d.ts.map +1 -1
  14. package/dist/cli.js +52 -6
  15. package/dist/cli.js.map +1 -1
  16. package/dist/domain/layout-tree.d.ts +19 -0
  17. package/dist/domain/layout-tree.d.ts.map +1 -0
  18. package/dist/domain/layout-tree.js +32 -0
  19. package/dist/domain/layout-tree.js.map +1 -0
  20. package/dist/domain/lookup-cache.d.ts +2 -2
  21. package/dist/domain/lookup-cache.js +2 -2
  22. package/dist/domain/paper-layout.d.ts +81 -0
  23. package/dist/domain/paper-layout.d.ts.map +1 -0
  24. package/dist/domain/paper-layout.js +259 -0
  25. package/dist/domain/paper-layout.js.map +1 -0
  26. package/dist/domain/repro-layout.d.ts +80 -0
  27. package/dist/domain/repro-layout.d.ts.map +1 -0
  28. package/dist/domain/repro-layout.js +247 -0
  29. package/dist/domain/repro-layout.js.map +1 -0
  30. package/dist/domain/scaffold-layout.d.ts +28 -0
  31. package/dist/domain/scaffold-layout.d.ts.map +1 -0
  32. package/dist/domain/scaffold-layout.js +81 -0
  33. package/dist/domain/scaffold-layout.js.map +1 -0
  34. package/dist/eslint-rules/CLAUDE.md.spec.d.ts +3 -0
  35. package/dist/eslint-rules/CLAUDE.md.spec.d.ts.map +1 -0
  36. package/dist/eslint-rules/CLAUDE.md.spec.js +39 -0
  37. package/dist/eslint-rules/CLAUDE.md.spec.js.map +1 -0
  38. package/dist/eslint-rules/install-path-literals.d.ts +1 -1
  39. package/dist/eslint-rules/install-path-literals.js +4 -4
  40. package/dist/eslint-rules/install-path-literals.js.map +1 -1
  41. package/dist/eslint-rules/pdf-last-page-balance.js +1 -1
  42. package/dist/eslint-rules/prereg-coverage.d.ts +98 -0
  43. package/dist/eslint-rules/prereg-coverage.d.ts.map +1 -0
  44. package/dist/eslint-rules/prereg-coverage.js +274 -0
  45. package/dist/eslint-rules/prereg-coverage.js.map +1 -0
  46. package/dist/folder-layout-hook.d.ts +21 -0
  47. package/dist/folder-layout-hook.d.ts.map +1 -0
  48. package/dist/folder-layout-hook.js +117 -0
  49. package/dist/folder-layout-hook.js.map +1 -0
  50. package/dist/folder-layout-rule.d.ts +22 -0
  51. package/dist/folder-layout-rule.d.ts.map +1 -0
  52. package/dist/folder-layout-rule.js +107 -0
  53. package/dist/folder-layout-rule.js.map +1 -0
  54. package/dist/init.js +1 -1
  55. package/dist/init.js.map +1 -1
  56. package/dist/lib/agent-cli-version.js +1 -1
  57. package/dist/lib/agent-cli-version.js.map +1 -1
  58. package/dist/new-paper.d.ts +4 -1
  59. package/dist/new-paper.d.ts.map +1 -1
  60. package/dist/new-paper.js +40 -6
  61. package/dist/new-paper.js.map +1 -1
  62. package/dist/references.d.ts +1 -1
  63. package/dist/references.d.ts.map +1 -1
  64. package/dist/references.js +2 -2
  65. package/dist/references.js.map +1 -1
  66. package/dist/repro-layout-rule.d.ts +16 -0
  67. package/dist/repro-layout-rule.d.ts.map +1 -0
  68. package/dist/repro-layout-rule.js +81 -0
  69. package/dist/repro-layout-rule.js.map +1 -0
  70. package/dist/tex-requirements.js +1 -1
  71. package/dist/toolchain.js +1 -1
  72. package/docs/ci.md +60 -0
  73. package/docs/configuration.md +2 -0
  74. package/docs/e2e.md +3 -3
  75. package/docs/incidents.md +5 -5
  76. package/docs/install.md +20 -16
  77. package/docs/layout.md +154 -0
  78. package/docs/optional-rules.md +5 -4
  79. package/docs/references.md +5 -2
  80. package/docs/rules/paper/allowed-files-only.md +192 -0
  81. package/docs/rules/repro/allowed-files-only.md +128 -0
  82. package/docs/rules/study/declare-sample.md +146 -0
  83. package/docs/rules.md +53 -50
  84. package/docs/skills.md +6 -3
  85. package/docs/venues.md +94 -0
  86. package/eslint-rules/CLAUDE.md +44 -0
  87. package/eslint-rules/CLAUDE.md.spec.ts +43 -0
  88. package/eslint-rules/install-path-literals.harness.mjs +1 -1
  89. package/eslint-rules/install-path-literals.ts +4 -4
  90. package/eslint-rules/pdf-last-page-balance.ts +1 -1
  91. package/eslint-rules/prereg-coverage.test.ts +448 -0
  92. package/eslint-rules/prereg-coverage.ts +419 -0
  93. package/eslint.config.mjs +6 -3
  94. package/hooks/CLAUDE.md +34 -0
  95. package/hooks/CLAUDE.md.spec.ts +30 -0
  96. package/lib/agent-cli-version.ts +1 -1
  97. package/lib/skill-trigger-cases.harness.mjs +51 -5
  98. package/lib/skill-trigger-cases.mjs +53 -15
  99. package/package.json +2 -2
  100. package/plugin/CLAUDE.md +22 -0
  101. package/plugin/CLAUDE.md.spec.ts +22 -0
  102. package/plugin/hooks/hooks.json +9 -0
  103. package/scripts/coverage-config.test.ts +2 -1
  104. package/skills/CLAUDE.md +19 -0
  105. package/skills/CLAUDE.md.spec.ts +18 -0
  106. package/skills/analyze-sibling-paper/SKILL.md +2 -2
  107. package/skills/analyze-sibling-paper/SKILL.md.spec.ts +1 -1
  108. package/skills/build-benchmark/SKILL.md +14 -1
  109. package/skills/build-benchmark/SKILL.md.spec.ts +13 -0
  110. package/skills/build-label-page/SKILL.md +118 -0
  111. package/skills/build-label-page/SKILL.md.spec.ts +129 -0
  112. package/skills/build-label-page/build-label-page.eval.mjs +15 -0
  113. package/skills/build-label-page/build-label-page.test.ts +959 -0
  114. package/skills/build-label-page/references/checklist.md +148 -0
  115. package/skills/build-label-page/references/example-generator.py +803 -0
  116. package/skills/find-venue/SKILL.md +5 -3
  117. package/skills/find-venue/SKILL.md.spec.ts +4 -2
  118. package/skills/grade-paper-writing/SKILL.md +7 -7
  119. package/skills/grade-paper-writing/SKILL.md.spec.ts +6 -6
  120. package/skills/map-prior-work/SKILL.md +2 -2
  121. package/skills/map-prior-work/SKILL.md.spec.ts +1 -1
  122. package/skills/paper-pipeline/SKILL.md +8 -6
  123. package/skills/paper-pipeline/SKILL.md.spec.ts +7 -5
  124. package/skills/paper-pipeline/grade-paper-writing-ablation.eval.mjs +19 -9
  125. package/skills/paper-pipeline/references/acceptance-gate.md +19 -19
  126. package/skills/paper-pipeline/references/artifact-checklist.md +5 -0
  127. package/skills/paper-pipeline/references/pipeline-status-template.md +30 -1
  128. package/skills/paper-pipeline/repro/2026-08-07-fork-check.log +1 -1
  129. package/skills/paper-pipeline/repro/2026-08-07-fork-check2.log +1 -1
  130. package/skills/paper-pipeline/repro/2026-08-07-language-eval-pilot.log +1 -1
  131. package/skills/paper-pipeline/repro/2026-08-07-language-eval-raw.log +1 -1
  132. package/skills/paper-pipeline/repro/2026-08-08-framing-vs-vocabulary-oracle.log +48 -48
  133. package/skills/paper-pipeline/repro/2026-08-08-grade-paper-writing-ablation-A6-oracle.log +8 -8
  134. package/skills/paper-pipeline/repro/2026-08-08-grade-paper-writing-ablation-oracle.log +48 -48
  135. package/skills/paper-pipeline/repro/2026-08-08-parent-replication-gpw.log +2 -2
  136. package/skills/paper-pipeline/repro/2026-08-08-parent-replication.mjs +3 -3
  137. package/skills/paper-pipeline/repro/2026-10-13-grade-paper-writing-ablation-preflight-after.log +81 -0
  138. package/skills/paper-pipeline/repro/2026-10-13-grade-paper-writing-ablation-preflight-before.log +81 -0
  139. package/skills/paper-pipeline/repro/2026-10-13-grade-paper-writing-ablation-setupdiff.log +59 -0
  140. package/skills/paper-pipeline/repro/2026-10-13-grade-paper-writing-trigger.jsonl +1 -0
  141. package/skills/paper-pipeline/repro/2026-10-13-grade-paper-writing-trigger.log +22 -0
  142. package/skills/paper-pipeline/repro/README.md +4 -0
  143. package/skills/paper-pipeline/scripts/extract-ref-facts.harness.mjs +2 -2
  144. package/skills/paper-pipeline/scripts/extract-ref-facts.mjs +2 -2
  145. package/skills/paper-pipeline/scripts/extract-ref-facts.test.mjs +4 -2
  146. package/skills/paper-pipeline/scripts/generated-code.harness.mjs +7 -2
  147. package/skills/paper-pipeline/scripts/generated-code.mjs +8 -4
  148. package/skills/paper-pipeline/scripts/generated-code.test.ts +15 -1
  149. package/skills/paper-status/SKILL.md +2 -2
  150. package/skills/paper-status/SKILL.md.spec.ts +1 -1
  151. package/skills/plan-paper-timeline/plan-paper-timeline.effects.harness.mjs +21 -102
  152. package/skills/study-accepted-papers/SKILL.md +163 -161
  153. package/skills/study-accepted-papers/SKILL.md.spec.ts +167 -165
  154. package/skills/study-accepted-papers/references/base-rates.md +69 -0
  155. package/skills/study-accepted-papers/references/mechanisms.md +181 -0
  156. package/skills/verify-citations/SKILL.md +2 -2
  157. package/skills/verify-citations/SKILL.md.spec.ts +1 -1
  158. package/src/CLAUDE.md +23 -1
  159. package/src/CLAUDE.md.spec.ts +72 -0
  160. package/src/build.ts +2 -2
  161. package/src/cli.harness.mjs +42 -0
  162. package/src/cli.test.ts +90 -6
  163. package/src/cli.ts +78 -5
  164. package/src/domain/layout-tree.test.ts +32 -0
  165. package/src/domain/layout-tree.ts +47 -0
  166. package/src/domain/lookup-cache.ts +2 -2
  167. package/src/domain/paper-layout.test.ts +264 -0
  168. package/src/domain/paper-layout.ts +336 -0
  169. package/src/domain/repro-layout.test.ts +222 -0
  170. package/src/domain/repro-layout.ts +311 -0
  171. package/src/domain/scaffold-layout.ts +104 -0
  172. package/src/folder-layout-hook.test.ts +214 -0
  173. package/src/folder-layout-hook.ts +141 -0
  174. package/src/folder-layout-rule.test.ts +235 -0
  175. package/src/folder-layout-rule.ts +151 -0
  176. package/src/hooks-settings.harness.ts +9 -8
  177. package/src/init.ts +1 -1
  178. package/src/layout-docs.test.ts +52 -0
  179. package/src/new-paper.harness.mjs +23 -1
  180. package/src/new-paper.test.ts +299 -6
  181. package/src/new-paper.ts +51 -5
  182. package/src/paper-files.test.ts +9 -6
  183. package/src/references.test.ts +5 -5
  184. package/src/references.ts +2 -2
  185. package/src/repro-layout-rule.test.ts +114 -0
  186. package/src/repro-layout-rule.ts +121 -0
  187. package/src/rule-cases.test.ts +98 -0
  188. package/src/tex-requirements.ts +1 -1
  189. package/src/toolchain.ts +1 -1
  190. package/templates/paper/PIPELINE-STATUS.md +41 -7
@@ -1,174 +1,176 @@
1
1
  ---
2
2
  name: study-accepted-papers
3
- description: "Mine a target venue's ACCEPTED-paper corpus to learn what makes papers STRONG there, then turn that into concrete strengthening levers for your own draft — the move from a plain \"Accept\" toward \"Strong Accept\". Fetch 8-12 real recently-accepted papers at the venue (ACM DL / IEEE Xplore / venue program pages / arXiv), prioritizing the same paper-type and topic as yours (measurement/benchmark/SoK/security-critique), and for each extract WHAT MADE IT STRONG (adaptive evaluation, explicit threat model, reusable released artifact, real-world grounding, a memorable framing/coined handle, a released dataset, honest limitations). Cross-read the venue CFP + PC-chair research tastes + any best-paper criteria. Then diff your draft against those patterns and emit ranked levers split into CHEAP (framing/prose/citation, safe before a deadline) vs EXPENSIVE (new experiments/data). Use when a paper is drafted and you want venue-specific polish grounded in what actually lands there — NOT generic writing advice. Distinct from find-venue (picks WHERE), research-ideate (validates the idea), and pc-panel-review / paper-adversarial-review (red-team YOUR draft): this one studies the VENUE's own bar. Compose with those + harden-paper + verify-citations + extend-paper."
3
+ description: "Learn what a target venue rewards from its own accepted papers and award winners, and turn it into requirements for your study or a diagnosis of your draft. Use EARLY, in SETUP right after find-venue and before the study is designed, so what the venue rewards shapes the experiment (mode A idea, mode B design); and LATE, once text exists (mode C draft: score the paper against the strong-accept mechanisms M1–M10 with line references, blockers, owners and a plan; mode D final: the abstract and introduction, sentence by sentence). Always asks which mode, suggesting one from what exists, and asks the claim, breadth, depth (L0–L3) and budget up front, each with a default and a time estimate. Keeps one corpus per venue in <papers-root>/venues/<venue>/, shared by every paper that targets it; the venue's winners vs its ordinary papers and award papers on the topic at other venues are two corpora, labelled apart. Names the proxy for 'strong accept' and its base rate before any probability. Distinct from find-venue (picks WHERE), research-ideate (is the idea worth doing), map-prior-work and analyze-sibling-paper (your rivals, not what the venue rewards), and pc-panel-review / paper-adversarial-review (red-team your draft)."
4
4
  context: fork
5
5
  allowed-tools: [Read, Write, Edit, Grep, Glob, Bash, WebSearch, WebFetch, Agent]
6
6
  ---
7
7
 
8
- <!-- vigiles:sha256:b68bde442319a605 compiled from skills/study-accepted-papers/SKILL.md.spec.ts -->
9
-
10
- # study-accepted-papers — learn the venue's bar from its own accepted corpus, then lever your draft up
11
-
12
- Reviewers don't score in a vacuum — they score against the papers that got in last cycle. This skill
13
- makes that reference standard explicit: pull the venue's **actually-accepted** papers of your type,
14
- extract the concrete properties that separated the strong ones from the weak-accepts, and diff your
15
- draft against them to produce **specific, sourced levers** — not "add more rigor" but "AISec strong
16
- accepts in this lane all carry an adaptive-attacker evaluation; yours has static mutation only — here's
17
- the cheap version you can add and the expensive version you can't."
18
-
19
- It answers a different question than the neighboring skills:
20
- - `find-venue` → *where* should this go? (ranks venues)
21
- - `research-ideate` → is the *idea* worth doing? (go/no-go on the contribution)
22
- - `pc-panel-review` / `paper-adversarial-review` → what's wrong with *my draft*? (red-team your text)
23
- - **`study-accepted-papers` → what does it take to be *strong* at THIS venue, judged by its own accepted papers?**
24
-
25
- Grounding: the authorship criterion — why a *strong* accept at an indexed venue is
26
- worth more to the dossier than a borderline one — reviewer enthusiasm shows up in the acceptance and in
27
- letters), and the venue's card `presets/<venue>.md` in the paperlint package if one exists.
28
-
29
- ## Step 1 — pin the venue's stated bar (fetch, don't guess)
30
-
31
- `WebFetch` the current CFP and, if separate, the reviewer-guidelines / call-for-benchmark-papers page.
32
- Capture what the venue *says* it rewards, in its own words:
33
- - **Track requirements** — e.g. a benchmark track that demands a functional artifact link, a named
34
- reusable benchmark, a metric definition, a leaderboard. Missing a *required* element of your track is
35
- a desk-reject, never a weak-accept — check these first.
36
- - **Stated review criteria / scoring rubric** — novelty vs systematization vs reproducibility weighting.
37
- - **Best-paper / distinguished-artifact criteria** if published — these name the top-end bar directly.
38
- - **PC chairs + steering committee + recent PC** — look up their research areas. A PC heavy on
39
- adversarial-ML will punish a non-adaptive evaluation; a PC heavy on systems will want real deployment
40
- grounding. Taste is a real, findable signal.
41
-
42
- ## Step 2 — pull the corpus. **Measure ALL of it; deep-read 8-12.**
43
-
44
- 🔴 **Two tiers, and the first one is not optional.** The single most valuable output of this skill is
45
- a *denominator* — "33 of 34 accepted papers have at least one figure", "confidence intervals appear
46
- in 1 of 34", "control arms in 0 of 34". **A sample of 8-12 cannot produce a denominator.** It gives
47
- you impressions; the whole point of this skill is to replace impressions with the venue's own
48
- distribution.
49
-
50
- | tier | how many | what you get |
51
- |---|---|---|
52
- | **measure** | **all of them** | figure count · table count · page count · section word counts · greps for confidence intervals, statistical tests, control/placebo arms, artifact URLs · abstract length. Mechanical, scriptable, cheap. |
53
- | **deep-read** | **8-12**, chosen by relevance | contribution, what made it strong, framing, how limitations are handled. Judgement, expensive. |
54
-
55
- **How many is "all"?** A workshop's accepted volume is **30-60 papers** — download the lot, it costs
56
- minutes and a few hundred MB. For a main conference (1000+), "all" means *all in your track or topic*,
57
- capped at ~100, and **you must state what you capped and why** — a silently truncated corpus produces
58
- denominators that are quietly wrong.
59
-
60
- **Cross tier with archival status before calibrating.** Oral/spotlight/poster and archival/non-archival
61
- are different axes, and they interact: at REALM 2025 most orals were **non-archival**, so the honest
62
- "what a strong archival paper looks like" set was **six** papers, not eleven. Calibrating against the
63
- wrong subset is worse than not calibrating.
64
-
65
- **Do not quote an acceptance rate unless the venue publishes the submission count.** Accepted counts
66
- are public; submitted counts usually are not. An invented rate is the kind of number that gets
67
- repeated for weeks — it was, on `compile-rules-2026`, by this pipeline, until the corpus pass
68
- checked.
69
-
70
- 🔴 **Save the corpus WITH the paper, not in a scratchpad.** See "Where the output lives" below. A run
71
- that returns only prose has destroyed its own evidence.
72
-
73
- `WebSearch` / `WebFetch` the venue's recent proceedings (ACM DL, IEEE Xplore, the venue program pages,
74
- or arXiv copies). For the **deep-read** tier select **8-12 actually-accepted papers**, prioritizing:
75
- 1. Same **paper type** as yours (measurement / benchmark / SoK / empirical-critique / defense-eval).
76
- 2. Same **topic lane** (for a security venue: LLM/agent security, guardrail evaluation, jailbreak
77
- robustness, "X is security theater"-style critiques).
78
- 3. Any **best-paper / award / highly-cited** ones — these are the calibrated top of the distribution.
79
-
80
- For **each** paper, extract not just the contribution but **what made it strong** — the property a
81
- reviewer would have written in the "strengths" box:
82
- - adaptive / adversarial evaluation (an *adaptive attacker*, not just static perturbation);
83
- - an explicit, scoped **threat model** stated up front;
84
- - a **reusable, released artifact / dataset** (not a one-off script);
85
- - **real-world grounding** (real incidents, real deployed tools, a real corpus vs hand-built toys);
86
- - a **memorable framing or coined handle** that makes the paper travel (and get cited);
87
- - **baseline comparisons** against the obvious prior defenses/benchmarks;
88
- - **honest, self-aware limitations** that pre-empt the reviewer's objection.
89
-
90
- Record each as a row: `title | year | contribution | what made it STRONG`. Never invent a paper or a
91
- citation — if you can't verify it exists, drop it or mark it VERIFY.
92
-
93
- ## Step 3 — diff your draft against the pattern
94
-
95
- Lay your draft's properties beside the strong-accept pattern from Step 2. For every property the strong
96
- papers share that yours lacks or does weakly, that's a **lever**. Rank levers by how much *this venue's*
97
- PC weights the axis (from Step 1's chairs/rubric), not by generic importance. For a security venue,
98
- weight **adaptive evaluation** and **threat-model clarity** heavily — they are the usual difference
99
- between 4/5 and 5/5, and the usual reason an empirical security paper is held at weak-accept.
100
-
101
- ## Step 4 — split levers CHEAP vs EXPENSIVE (respect the deadline and the anti-cram rule)
102
-
103
- Every lever gets tagged:
104
- - **CHEAP** — framing, prose, a reordered abstract, a coined handle, an explicit threat-model paragraph,
105
- a missing-but-expected citation, promoting an existing number into the abstract, making an already-run
106
- robustness result read as "adaptive". Safe to land days before a deadline.
107
- - **EXPENSIVE** — needs new experiments, a bigger/external corpus, an actual adaptive-attacker study, a
108
- new baseline run. These usually **do NOT belong in a near-done paper under deadline pressure** — the
109
- anti-cram rule (two clean accepted works beat one overstuffed one) means an
110
- expensive lever is normally an **`extend-paper` task for the follow-on**, not a pre-deadline edit.
111
- Say so explicitly; don't let a tempting expensive lever destabilize a shippable Accept.
112
-
113
- ## Output
114
-
115
- 1. **Venue-bar summary** — what this venue rewards, its track requirements, its PC's taste (2-4 lines,
116
- each sourced).
117
- 2. **Strong-accepted-paper table** — `title | year | contribution | what made it STRONG`, with URLs.
118
- 3. **Ranked levers for your draft** — each: the gap, the strong-paper precedent it's drawn from,
119
- CHEAP/EXPENSIVE tag, and the concrete edit (or the extend-paper deferral). Most-impactful first.
120
- 4. **Citation gaps** — specific real papers (arXiv id / DOI) a reviewer at this venue would expect and
121
- ding you for missing, each with why. Hand these to `verify-citations`.
122
- 5. **Corpus-level distribution** — the denominators, each traceable to the measurement file: how many
123
- papers have a figure and the median count, how many report confidence intervals, how many run a
124
- statistical test, how many have a control or placebo arm, section-length norms, abstract-length
125
- range. **These are what change decisions**, because they say whether your draft is normal, an
126
- outlier, or quietly ahead.
127
-
128
- ## 🔴 Where the output lives — three homes, and all three are required
129
-
130
- A run that returns only a write-up has destroyed its own evidence. Observed on `compile-rules-2026`
131
- 2026-08-03: the pass measured 34 PDFs and returned excellent prose, and what survived into the repo
132
- was **two scripts, 16 KB**. The corpus, the extracted text and the measurement table were in a
133
- scratchpad that gets wiped. Every denominator in the write-up was, an hour later, unverifiable.
134
-
135
- | home | what goes there | why there |
8
+ <!-- vigiles:sha256:607eaf7930ecaf30 compiled from skills/study-accepted-papers/SKILL.md.spec.ts -->
9
+
10
+ # study-accepted-papers — what this venue rewards, measured on its own papers
11
+
12
+ Reviewers score against the papers that got in before. This skill makes that standard explicit, from
13
+ the venue's **accepted papers and award winners**, and turns it into something the author can act on:
14
+ requirements for a study not yet run, or a scored diagnosis of a draft.
15
+
16
+ It answers a different question from its neighbours:
17
+ - `find-venue`: *where* should this go?
18
+ - `research-ideate`: is the *idea* worth doing?
19
+ - `map-prior-work` / `analyze-sibling-paper`: who are the *rivals*, and is the delta real?
20
+ - `pc-panel-review` / `paper-adversarial-review`: what is wrong with *my draft*?
21
+ - **`study-accepted-papers`: what does it take to be *strong* at THIS venue, judged by its own papers?**
22
+
23
+ **Two entry points, one skill.** In `paper-pipeline` it sits in SETUP, right after `find-venue`: the
24
+ venue's standard is worth most before the study is designed, when it can still decide what the oracle is and
25
+ which belief the claim targets. It comes back once text exists, to score the draft. Run it only late
26
+ and its main findings arrive after they could change the design.
27
+
28
+ Two references beside this file, read before any level above L0:
29
+ - [`references/mechanisms.md`](references/mechanisms.md): the mechanisms M1–M10 that separated award
30
+ winners from ordinary accepted papers in the run this skill was rebuilt from, with what supports each
31
+ and what each is not. Form (figures, length, title shape) did **not** separate them.
32
+ - [`references/base-rates.md`](references/base-rates.md): the proxies for "strong accept" and the only
33
+ public score data found for software-engineering venues.
34
+
35
+ ## Step 0 — ask before spending (always)
36
+
37
+ **Ask which mode. Always.** Look at what exists first, then suggest one; do not pick silently, and do
38
+ not start until the author answers (or the caller already passed a mode).
39
+
40
+ | mode | what exists | it compares | it produces | default depth |
41
+ |---|---|---|---|---|
42
+ | **A idea** | an idea and a target venue; no paper source | the venue's winners against its ordinary papers | what this venue rewards, written as **requirements for the study**: what the oracle is, which stated belief the claim targets, which factor must be a column | L1 (+ L0) |
43
+ | **B design** | a planned claim and measurements (the `frame` row, a design note in `research/`); no text | the planned claim and evidence against M1–M10 | **design gaps before anything runs** ("the headline rests on model labels", "no named belief") | L1 + the M1–M10 check of the plan |
44
+ | **C draft** | the paper's text | the draft against M1–M10 and the deep reads | the **scorecard**: pass / partial / fail per mechanism with `file:line`, blockers with cost and owner, a plan, probabilities against stated base rates | L2 + L3 |
45
+ | **D final** | near-final text, the deadline close | the abstract and introduction, sentence by sentence, against the closest winners | **wording**: the denominator in every claim sentence, the named belief, no drift between abstract and body | L3, wording only |
46
+
47
+ Suggest from the project: no `paper.tex` → A, or B if a claim is already written down; `paper.tex`
48
+ with a body → C; the current cycle's submission deadline within about a week → D. Say why you suggest it
49
+ ("C: paper.tex has a full body, and the deadline is 20 days away"), and let the author override.
50
+
51
+ **Then the four questions, in one message, each with its default and its time.** The author may answer
52
+ "defaults" to all of them.
53
+
54
+ 1. **The claim**, in one paragraph. Default: the `frame` row of `PIPELINE-STATUS.md`, else the
55
+ abstract. *Without it the deep-read selection is blind.* (Author: about 5 minutes.)
56
+ 2. **Breadth.** Default: **two corpora, labelled apart**: (1) the venue's own award winners against its
57
+ ordinary accepted papers, over 5–7 years; (2) award papers on the author's topic at other venues.
58
+ Corpus 1 says what *this* committee rewards; corpus 2 says how strong papers on this topic make a
59
+ broad claim convincing, which matters when the venue has no winner on the topic. Never pool them.
60
+ 3. **Depth**, L0–L3 (below). Default: the mode's row above.
61
+ 4. **Budget**: agent time, money, and the author's own hours. Default: the sum of the chosen levels.
62
+ Ask again before going over it.
63
+
64
+ ## The depth levels
65
+
66
+ | level | what it does | rough cost (from the run) | answers |
67
+ |---|---|---|---|
68
+ | **L0 form** | count body words, figures, tables, research questions, abstract length, statistics markers, artifact links across the venue's accepted papers | minutes | formal fit only. Say that form did **not** separate winners in the run (`references/mechanisms.md`) |
69
+ | **L1 winner corpus** | every award winner the venue lists over 5–7 years against the ordinary accepted papers of the same years; code form **and** substance per paper: the oracle under the headline (execution / developer action / human / model-only), a belief the field stated on record, the claim shape, a controlled design, the contribution sentence verbatim | about 30 min of agents | what does this venue reward? |
70
+ | **L2 deep reads** | a judge, briefed with the author's claim, picks 8–12 papers: winners closest to the claim plus **award / no-award pairs on the same question or dataset** (the pairs were the most informative part of the run); one agent per paper, full text, one shared reading brief | about 1 h of agents (parallel readers) | how do strong papers on this topic make a broad claim convincing? |
71
+ | **L3 diagnosis** | score the author's plan or draft on each mechanism; blockers with cost and owner; a plan; probabilities against the base rates | about 30 min | can this paper reach strong accept, and what stands in the way? |
72
+
73
+ Each level states **its proxy for "strong accept" and its base rate**, and which conclusions rest on
74
+ fewer than ten papers.
75
+
76
+ ### L0 — measure all of the accepted set
77
+
78
+ The value of L0 is a **denominator** ("33 of 34 accepted papers have a figure"). A sample of 8–12 cannot
79
+ give one. A workshop's accepted volume is 30–60 papers: take all of them. For a main conference, take
80
+ the track or the topic, capped at about 100, and **state the cap and why**; a silently truncated corpus
81
+ gives denominators that are quietly wrong. Cross accepted papers with archival status before calibrating:
82
+ an oral that is non-archival is not the paper your archival submission competes with.
83
+
84
+ ### L1 — the winner corpus
85
+
86
+ Take the venue's award list (its hall of fame, the awards page of each year's program) for 5–7 years,
87
+ not one year: a single year of five winners produced a "winners have many figures" pattern that vanished
88
+ over seven years. Code every winner and every ordinary paper of the same years with the same fields, so
89
+ each "winners do X" has an ordinary-paper count beside it. Report k/n for both groups and a test with a
90
+ correction for the number of features; a non-significant difference is reported as such.
91
+
92
+ ### L2 — the deep reads
93
+
94
+ The selection is part of the result: save it with the reason for each pick, and say how many came from
95
+ corpus 2 and why ("the venue has no winner on this topic"). One agent per paper, full text, every page.
96
+ The shared brief asks three questions: what is the single claim and why does a reviewer believe it; what
97
+ makes it stronger than its ordinary neighbours; what in it maps onto the author's claim. Each reading
98
+ quotes verbatim with page numbers.
99
+
100
+ ### L3 — the diagnosis
101
+
102
+ Score the paper against M1–M10 (`references/mechanisms.md`), with evidence as `file:line` in the paper
103
+ source. Every `fail` gets an owner and either a plan or an explicit **"accept ceiling"** with the
104
+ reason: a ceiling written down is a decision, a silent fail is a hole. Probabilities are a band, marked
105
+ as judgement, against the base rates of `references/base-rates.md`; simulated panels run hot.
106
+
107
+ ## Where the results go — one home per fact
108
+
109
+ | what | where | why there |
136
110
  |---|---|---|
137
- | **`<paper-dir>/venue-corpus/`** | `measurements.json` (one row per paper — the denominators' source) · `text/*.txt` (extractions: the quotable primary source) · the fetch/measure scripts · `MANIFEST.md` | data. **PDFs are NOT committed** — they regenerate from the proceedings; record the IDs and the command instead. Text extractions ARE committed: a few MB, and they are what makes a quote re-checkable. |
138
- | **`<paper-dir>/reviews/<date>-study-accepted-<venue>.md`** | the full analysis: per-paper reading, the whole lever list, the reasoning | the record. **The EXPENSIVE levers are the `extend-paper` roadmap** and are worthless if they evaporate. |
139
- | 🔴 **`<paper-dir>/CLAUDE.md`** | **the 3-5 findings that change decisions while writing**, plus an explicit "what NOT to do" | **it auto-loads whenever anyone works on this paper.** A finding in `reviews/` is read when someone goes looking; a finding here is read *every time the draft is edited*. That is the difference between knowing the venue's bar and applying it. |
111
+ | **the venue corpus** | `<papers-root>/venues/<venue>/` (the papers directory from the project's `paperlint.json`) | **shared by every paper that targets the venue**, collected once and reused by every later mode. `MANIFEST.md` (sources, years, what was capped, the proxy and its base rate), the winner and ordinary lists, `measurements.json` (L0/L1 codes, one row per paper), `text/` extractions, the scripts, `deep-reads/` (one file per paper, reusable by any paper). Corpus 2 goes in its own labelled subfolder. **PDFs are not committed**: record the IDs and the command. |
112
+ | **the scorecard** | the section **"Strong-accept scorecard (M1–M10)"** of the paper's `PIPELINE-STATUS.md` | a row per mechanism: status (pass / partial / fail), evidence (`file:line`), owner, plan or "accept ceiling". **The only place the verdicts live**; update it there. |
113
+ | **the reasoning** | `<paper>/reviews/<YYYY-MM-DD>-study-accepted-<venue>-<mode>.md` | a dated report per run, with frontmatter (below). It keeps the reasoning, the selection and the per-mechanism argument, and **points to the scorecard section; it never copies the table**. |
114
+ | **the stage** | the SETUP row with id `venuebar` in `PIPELINE-STATUS.md` (the id is kept for existing scorecards) | status, date, the mode and depth run, and a link to the report. One row, updated by every mode. |
115
+ | **the generic list** | `references/` beside this skill | M1–M10 and the base-rate sources. A mechanism your venue adds, or one that does not hold there, goes in the venue corpus's `MANIFEST.md`; the ids stay stable. |
140
116
 
141
- Then link all three from `PIPELINE-STATUS.md` (**`venuebar`** row — "venue bar / levers", in SETUP).
117
+ The corpus, its modes and who reads it: `docs/venues.md` in the paperlint package; every place a stage
118
+ writes, and what checks it: `docs/layout.md`.
142
119
 
143
- **What belongs in `<paper-dir>/CLAUDE.md` and what does not.** It is not a summary of the analysis —
144
- it is the operative subset: the structural mismatch to fix, the strength being under-sold, the norm
145
- being violated, and the **things previously believed that the corpus disproved** (those are the most
146
- valuable lines in the file, because without them the pipeline re-derives the wrong belief). Keep it
147
- short enough that it survives being loaded into every session.
120
+ The paper folder has **no corpus folder of its own**: `paper/allowed-files-only` reports one, and the hook
121
+ refuses to create one. A venue's format facts (page limit, template, required sections) go
122
+ in its preset and venue card (`presets/<venue>.jsonc` and `.md` in the package, or the project's
123
+ own), with their source quote, not here.
124
+
125
+ The report's frontmatter (it lives under `reviews/`, so `review/frontmatter` validates it; a
126
+ `findings` list is optional and follows that schema):
127
+
128
+ ```yaml
129
+ ---
130
+ title: "<venue>: what it rewards, and where this paper stands (mode C, L2+L3)"
131
+ created: 2026-10-12
132
+ venue: <venue>
133
+ mode: C
134
+ depth: [L2, L3]
135
+ proxy: "distinguished-paper award, <k> of <n> accepted, <years>"
136
+ corpus: <papers-root>/venues/<venue>/
137
+ scorecard: PIPELINE-STATUS.md#strong-accept-scorecard-m1m10
138
+ ---
139
+ ```
140
+
141
+ ## Before any level: pin the venue's stated bar
142
+
143
+ `WebFetch` the current call for papers and, if separate, the reviewer guidelines. Capture in the
144
+ venue's own words: the **track requirements** (a missing required element is a desk reject, never a
145
+ weak accept, so check these first); the **review criteria**; the **award criteria** if published; the
146
+ **PC chairs and members**, whose research areas are a findable signal of taste.
148
147
 
149
148
  ## Rules
150
- - **Fetch real accepted papers.** Never fabricate a title, author, or "what made it strong". Unverifiable → drop or mark VERIFY.
151
- - **A lever-proposed citation is not written into the `.tex`/bib until `verify-citations` confirms it** (real
152
- arXiv-id/DOI + correct author/title/venue). If added provisionally, mark `% VERIFY` at the cite site; a
153
- surviving `% VERIFY` at submit is a bug. Order: propose → verify → then add — never propose → add.
154
- - **"Accepted" is the reference, but "strong" is the target** — don't just describe what got in; isolate what got in *enthusiastically*. Awards and heavy citation are your calibration for the top end.
155
- - **Venue-specific, not generic.** A lever only counts if it's grounded in this venue's rubric, PC taste, or its own accepted papers. Generic "tighten the writing" is out of scope (that's `grade-paper-writing` / `tighten-paper`).
156
- - **Security venues:** treat adaptive-attacker evaluation and an explicit threat model as first-class — they are the most common weak-accept→strong-accept axis and the most common reviewer complaint.
157
- - **Respect the anti-cram rule.** Flag expensive levers as extension material by default; protect a shippable Accept from deadline-driven destabilization.
158
- - Convert any newly-surfaced deadline to the author's own zone and hand it to `plan-paper-timeline`.
149
+
150
+ - **Fetch real papers.** Never fabricate a title, an author, or "what made it strong". Unverifiable →
151
+ drop it or mark VERIFY.
152
+ - **Evidence levels on every finding**: quoted from a paper or file · counted from the corpus · judgement.
153
+ A judgement is never written as a count.
154
+ - **Do not quote an acceptance rate** unless the venue published the number of submissions.
155
+ - **A citation this skill proposes is not written into the source until `verify-citations` confirms it.**
156
+ If added provisionally, mark `% VERIFY` at the cite; a surviving `% VERIFY` at submit is a bug.
157
+ - **"Accepted" is the reference, "strong" is the target.** Isolate what got in enthusiastically, against
158
+ ordinary papers of the same years, not what got in.
159
+ - **Venue-specific, not generic.** A lever counts only if this venue's papers, rubric or committee ground
160
+ it. Prose craft is `grade-paper-writing` and `tighten-paper`.
161
+ - **Cheap or expensive, and the deadline.** Tag every lever: CHEAP (framing, a scoped sentence, a cited
162
+ belief, a missing citation) or EXPENSIVE (a new oracle, a new column, a new corpus). In mode C or D an
163
+ expensive lever is normally an `extend-paper` task or an accepted ceiling, said explicitly; in modes A
164
+ and B it is exactly what the study should include.
165
+ - Convert any deadline that surfaces to the author's own zone and hand it to `plan-paper-timeline`.
159
166
 
160
167
  ## Compose with
161
- - `harden-paper` — the multi-axis pre-submit gate; this skill feeds it the venue-specific axis (what *this* PC rewards) that a generic hardening pass misses.
162
- - `pc-panel-review` / `paper-adversarial-review` — run those to red-team the draft; run this to learn the bar the red-team should hold it to. Complementary, not redundant.
163
- - `verify-citations` — consumes the Step-4 citation gaps.
164
- - `extend-paper` — the natural home for every EXPENSIVE lever this skill surfaces.
165
- - the venue card (`presets/<venue>.md` in the paperlint package, or the project's own `./venues/<name>.md` beside its preset) — save durable venue-bar findings there as data, not as a new skill per venue.
166
- - the paper's `<paper>/paperlint.json` (`{ "extends": "paperlint:<venue>", "kind": "<kind>" }`) — which venue preset the `pdf/*` rules check the built PDF against; a venue with no shipped preset extends a family (`paperlint:acm-sigconf`) or the project's own `./venues/<name>.jsonc`. Findings about the venue's page limit or format belong in that preset, with their source quote.
167
-
168
- ## Provenance
169
- Built from the **AISec 2026 @ ACM CCS** polish run (2026-07): the "Safety Theater in Agentic Coding /
170
- GateBench" paper sat at ~Accept (pc-panel ~0.72), and the task was to find what would push it toward a
171
- strong accept without cramming new contributions before the 2026-07-24 deadline. The workflow that
172
- produced the levers — pull AISec's accepted measurement/benchmark security papers, isolate the
173
- adaptive-evaluation + threat-model + reusable-artifact pattern that the strong ones share, then split
174
- fixes into cheap-prose vs expensive-experiment — is exactly what this skill encodes.
168
+
169
+ - `find-venue` (upstream) → this skill in mode A; `argument-arc` (frame mode) takes its requirements.
170
+ - `build-benchmark` — takes mode A/B requirements (the oracle, the column) into the study design.
171
+ - `harden-paper` — the pre-submit gate; this skill gives it the venue-specific content axis.
172
+ - `pc-panel-review` / `paper-adversarial-review` — red-team the draft; this skill says which bar the
173
+ red-team should hold it to.
174
+ - `analyze-sibling-paper` — a deep read that turns out to be a rival goes there for the scoop question.
175
+ - `verify-citations` — takes the citation gaps.
176
+ - `extend-paper` — the home of every expensive lever deferred past this submission.