paperlint 13.0.3 → 13.2.0

package/skills/build-label-page/references/checklist.md

@@ -1,12 +1,15 @@
 # Labelling-page checklist
 
-Seven rules for a page where a person labels items from a paper's study (gold labels, a blind review of
+Eight rules for a page where a person labels items from a paper's study (gold labels, a blind review of
 model outputs, "is this a real boundary case"). Each rule says what to do, why, and how to check it.
 They come from four such pages built in one week; each rule is a defect one of them shipped, or a
 failure the next answer would have hidden.
 
-The generator beside this file, `example-generator.py`, does rules 2 to 5 and 7 by construction. Rules
-1 and 6 are things the agent does. Read the rule before changing the part of the generator that
+The rules hold for every labelling page, generated or written by hand. The skill's generator,
+`scripts/label-page.ts`, does rules 2 to 5, 7 and the page half of 8 by construction
+([`closed-choice-generator.md`](closed-choice-generator.md)); a page written by hand meets each rule
+itself, and each rule's **Check** line is its acceptance list. Rules 1 and 6, and the rewording in 8,
+are things the agent does. Read the rule before changing the part of the generator that
 carries it.
 
 ## 1. Test it in the claude.ai phone viewer before sending the link
@@ -140,6 +143,45 @@ not run: banner, items readable, buttons off.
 
 ---
 
+## 8. The labeller understands each question before the real items
+
+**Do.**
+
+- Write each question in plain words, and give every answer option one `example`: a real item of
+  the study, in a line, that takes that answer. The page shows them on its first screen ("How to
+  answer"). The build refuses an option without one.
+- Write 3 or more practice items whose answers are known (`practice`, each with `expected` for every
+  question and a one-line `why`). The page shows them before the real items; after each answer it
+  shows the expected answer and the reason. The real items open once all are answered.
+- Make the answers tell apart at a glance: give each option an `icon` (an emoji), never a coloured
+  side stripe on a button (it reads as decoration; the look test refuses one), and put code, configs and identifiers in
+  `backticks` or the item's `code` field, so they render as code. During the practice each button also
+  shows its example; on the real items the buttons stay short.
+- Code is shown so it can be judged, not counted: a `code` block marks leading and trailing
+  whitespace (a space as ·, a tab as →) and prints each line's length at its end.
+- One question per screen: an item card asks one question at a time, and the answered ones stay as a
+  line with "Change".
+- A practice miss does not lock the page, but it stops the agent: ask the labeller which question
+  was unclear, reword it (and its examples), raise `practice_round`, rebuild, and have the labeller
+  take the practice again before their real answers count. The page keeps only the FIRST answer to a
+  practice item, so the score is not a retry.
+- Keep a record in the study folder: the practice file `export` writes
+  (`<study>-practice-r<round>-<date>.json`: the round, who answered, what matched), and one line in the
+  study's README per round on what was reworded after it.
+
+**Why.** A labeller who misreads the question gives labels that agree with nothing, and the agreement
+statistic then blames the items. Pages that passed rules 1 to 7 (blind, phone-tested, saved on click)
+still left their labeller unable to say what was being asked. A pilot round before the real labelling
+is ordinary practice in labelling studies: one rule-mining study labelled 30 pilot samples, resolved
+the disagreements, aligned its criteria, and only then labelled the full set.
+
+**Check.** The page opens on "How to answer" with an example under every option, then "Practice: 0
+of N answered", and no real item. Answer the practice wrongly once: the expected answer and the
+reason appear, and the banner says to tell the agent. `export` writes the labels without any
+practice answer and the practice file beside them.
+
+---
+
 ## Who can answer
 
 A page with the `db` capability opens only for people with access to the artifact: the owner, or a

package/skills/build-label-page/references/closed-choice-generator.md

@@ -0,0 +1,88 @@
+# The closed-choice generator
+
+The page shape this generator builds: the labeller looks at one item and picks from a closed list of
+answers, one question at a time, with an optional reason box and code blocks. For any other shape,
+see "Which page you need" in `SKILL.md`: the agent writes that page itself.
+
+The generator is `scripts/label-page.ts` in this skill, compiled beside itself to `label-page.js`.
+It runs from the skill and is never copied into a study, so a fix to it reaches every page built
+after the fix. The commands are in `SKILL.md` (procedure steps 3 and 8); its third subcommand,
+`schema`, prints the declared schema of `items.json`. Its header holds the full `items.json`
+example; this file says how to fill it.
+
+It is two parts. The core, `label-page-core.ts`, has no page layout: the `items.json` schema and its
+checks, the leak guard, the neutral ids, the key with its `allowed` and `needs_why` lists, the check
+of the store's dump, and the export of labels and practice. The page, `label-page-html.ts`, fills the
+one closed-choice template (`page-template.html`, its style and browser script kept as exact bytes)
+with the core's payload. Another page shape would be another template on the same core.
+
+## What it does by construction
+
+Rules 2 to 5, 7 and the page half of 8 of [`checklist.md`](checklist.md): a filter instead of
+scrolling, complete-item counting, neutral ids in a seeded order, the leak guard (an item field
+outside the allow-list stops the build), save on click with a time, the store-unavailable banner,
+the "How to answer" screen, the practice before the real items, one question per screen. Its look:
+one dark palette, one column, answer buttons as full-width rows with space between them, an optional
+icon per answer and never a coloured side stripe, `backticks` and the `code` field shown as code,
+code blocks that show leading and trailing whitespace (a space as ·, a tab as →) and each line's
+length.
+
+## items.json
+
+- **`questions`**: each with a closed set of `options`. Every option has a `value`, a `label` and a
+  required `example`: a real case of the study, in a line, that takes that answer. `icon` (an
+  emoji) is optional. Two labels that read the same, or two equal values, stop the build.
+- **`practice`**: 3 or more real cases whose answer you are sure of, each with `expected` (one answer
+  per question) and a one-line `why`. Never an item that is also on the page to be labelled.
+- **`practice_round`**: raise it after rewording a question; the practice reopens and the answers of
+  the earlier round are kept but not counted.
+- **`items`**: `source_id` (the study's own, never shown), `group`, `body`, and optionally `title`,
+  `code`, `context`. Nothing else: the allow-list is fixed in the generator, and a field outside it
+  is taken out of the item, not added to the list to make a build pass.
+- **`seed`**: fixes the order. Rebuilding with the same seed keeps every id, and a new item gets the
+  next one; a different seed is refused once a key exists.
+- **`ask_why`** on an option: a prompt. That answer is incomplete, on the page and in the export,
+  until a reason is saved.
+
+## The approval page
+
+The page on which the author checks a drawn sample before the runs (the plan's `author_check`: "is
+this what you meet in practice?") is this generator's shape: one question whose answers are keep and
+drop, the drop carrying `ask_why`.
+
+```json
+{
+  "id": "keep",
+  "text": "Is this what you meet in practice?",
+  "short": "keep?",
+  "options": [
+    {
+      "value": "keep",
+      "label": "Keep",
+      "icon": "👍",
+      "example": "`Use 2-space indent.`"
+    },
+    {
+      "value": "drop",
+      "label": "Drop",
+      "icon": "🗑️",
+      "example": "`Be nice to reviewers.`",
+      "ask_why": "Why drop it? (not a rule, a process rule, a misreading…)"
+    }
+  ]
+}
+```
+
+The items are the drawn units verbatim. The practice is three units whose fate is obvious. Each
+dropped unit carries its reason in the labels file. The author's words, with the date, then go into
+`author_check` of the plan; a dropped unit is replaced by the next one in the draw.
+
+## The export
+
+`export` reads the store's three collections (`answers`, `groups`, `practice`) from `dump.json`, in
+the shape the generator's header gives. It refuses, naming the path, any name the key does not know
+or any document of the wrong type. An answer or familiarity value outside the offered options is not
+counted: the item is marked incomplete and printed (same rule as on the page); correct it in the dump
+and export again. It also prints every item that is missing an answer or a reason, and every group
+with no familiarity answer. The practice of the current round is scored into its own file and never
+into the labels.

package/skills/build-label-page/scripts/label-page-cli.js

@@ -0,0 +1,72 @@
+/**
+ * The command line of build-label-page's generator, as a function: one argv in, what it printed and
+ * its exit code out. `label-page.ts` runs it on the process's own argv.
+ */
+import { parseArgs } from "node:util";
+import { COMMANDS } from "./label-page-commands.js";
+import { CONFIG_SCHEMA, exportLabels, fail, Refusal, say, takePrinted, } from "./label-page-core.js";
+import { build } from "./label-page-html.js";
+const USAGE = "usage: label-page.js build <items.json> <out-dir> | export <dump.json> <key.json> <data-dir> [--date YYYY-MM-DD] | schema";
+const usage = () => {
+    throw new Refusal(2, USAGE);
+};
+const today = () => {
+    const d = new Date();
+    const two = (n) => String(n).padStart(2, "0");
+    return `${String(d.getFullYear())}-${two(d.getMonth() + 1)}-${two(d.getDate())}`;
+};
+/** How many paths each subcommand takes, and whether it takes `--date`. */
+const ARITY = {
+    [COMMANDS.schema]: { paths: 0, dated: false },
+    [COMMANDS.build]: { paths: 2, dated: false },
+    [COMMANDS.export]: { paths: 3, dated: true },
+};
+const parsedArgs = (argv) => {
+    try {
+        return parseArgs({
+            args: [...argv],
+            options: { date: { type: "string" } },
+            allowPositionals: true,
+        });
+    }
+    catch {
+        return usage();
+    }
+};
+function exportCommand(paths, date) {
+    const [dump = "", key = "", out = ""] = paths;
+    const day = date ?? today();
+    if (!/^\d{4}-\d{2}-\d{2}$/.test(day))
+        fail("--date must be YYYY-MM-DD");
+    exportLabels(dump, key, out, day);
+}
+function main(argv) {
+    const { values, positionals } = parsedArgs(argv);
+    const [cmd = "", ...paths] = positionals;
+    const arity = ARITY[cmd];
+    const fits = arity !== undefined &&
+        arity.paths === paths.length &&
+        (arity.dated || values.date === undefined);
+    if (!fits)
+        usage();
+    if (cmd === COMMANDS.schema)
+        say(JSON.stringify(CONFIG_SCHEMA, null, 2));
+    else if (cmd === COMMANDS.build)
+        build(String(paths[0]), String(paths[1]));
+    else
+        exportCommand(paths, values.date);
+}
+/** One command line: what it printed to stdout and to stderr, and its exit code. */
+export function cli(argv) {
+    takePrinted(); // nothing left over from an earlier run in the same process
+    try {
+        main(argv);
+        return { status: 0, stdout: takePrinted(), stderr: "" };
+    }
+    catch (e) {
+        // An unexpected error (a file that cannot be written) prints its message, not a stack.
+        const message = String(Reflect.get(Object(e), "message"));
+        const status = e instanceof Refusal ? e.exitCode : 1;
+        return { status, stdout: takePrinted(), stderr: `${message}\n` };
+    }
+}

package/skills/build-label-page/scripts/label-page-cli.ts

@@ -0,0 +1,85 @@
+/**
+ * The command line of build-label-page's generator, as a function: one argv in, what it printed and
+ * its exit code out. `label-page.ts` runs it on the process's own argv.
+ */
+import { parseArgs } from "node:util";
+import { COMMANDS } from "./label-page-commands.ts";
+import {
+  CONFIG_SCHEMA,
+  exportLabels,
+  fail,
+  Refusal,
+  say,
+  takePrinted,
+} from "./label-page-core.ts";
+import { build } from "./label-page-html.ts";
+
+const USAGE =
+  "usage: label-page.js build <items.json> <out-dir> | export <dump.json> <key.json> <data-dir> [--date YYYY-MM-DD] | schema";
+const usage = (): never => {
+  throw new Refusal(2, USAGE);
+};
+const today = (): string => {
+  const d = new Date();
+  const two = (n: number): string => String(n).padStart(2, "0");
+  return `${String(d.getFullYear())}-${two(d.getMonth() + 1)}-${two(d.getDate())}`;
+};
+/** How many paths each subcommand takes, and whether it takes `--date`. */
+const ARITY: Readonly<
+  Record<string, { readonly paths: number; readonly dated: boolean }>
+> = {
+  [COMMANDS.schema]: { paths: 0, dated: false },
+  [COMMANDS.build]: { paths: 2, dated: false },
+  [COMMANDS.export]: { paths: 3, dated: true },
+};
+const parsedArgs = (argv: readonly string[]) => {
+  try {
+    return parseArgs({
+      args: [...argv],
+      options: { date: { type: "string" } },
+      allowPositionals: true,
+    });
+  } catch {
+    return usage();
+  }
+};
+function exportCommand(
+  paths: readonly string[],
+  date: string | undefined,
+): void {
+  const [dump = "", key = "", out = ""] = paths;
+  const day = date ?? today();
+  if (!/^\d{4}-\d{2}-\d{2}$/.test(day)) fail("--date must be YYYY-MM-DD");
+  exportLabels(dump, key, out, day);
+}
+function main(argv: readonly string[]): void {
+  const { values, positionals } = parsedArgs(argv);
+  const [cmd = "", ...paths] = positionals;
+  const arity = ARITY[cmd];
+  const fits =
+    arity !== undefined &&
+    arity.paths === paths.length &&
+    (arity.dated || values.date === undefined);
+  if (!fits) usage();
+  if (cmd === COMMANDS.schema) say(JSON.stringify(CONFIG_SCHEMA, null, 2));
+  else if (cmd === COMMANDS.build) build(String(paths[0]), String(paths[1]));
+  else exportCommand(paths, values.date);
+}
+
+/** One command line: what it printed to stdout and to stderr, and its exit code. */
+export function cli(argv: readonly string[]): {
+  readonly status: number;
+  readonly stdout: string;
+  readonly stderr: string;
+} {
+  takePrinted(); // nothing left over from an earlier run in the same process
+  try {
+    main(argv);
+    return { status: 0, stdout: takePrinted(), stderr: "" };
+  } catch (e) {
+    // An unexpected error (a file that cannot be written) prints its message, not a stack.
+    const message = String(Reflect.get(Object(e), "message"));
+    const status = e instanceof Refusal ? e.exitCode : 1;
+    return { status, stdout: takePrinted(), stderr: `${message}\n` };
+  }
+}

package/skills/build-label-page/scripts/label-page-commands.js

@@ -0,0 +1,14 @@
+/**
+ * The subcommands of `label-page`, by name. One table, read twice: the dispatcher in `label-page.ts`
+ * compares against these, and `SKILL.md.spec.ts` interpolates them into every command it instructs
+ * (a name that is not here is a type error there). So the skill cannot instruct a subcommand the
+ * script does not have.
+ *
+ * A module of its own, with no side effects: the spec imports it at compile time, and importing
+ * `label-page.ts` itself would run the command line.
+ */
+export const COMMANDS = {
+    build: "build",
+    export: "export",
+    schema: "schema",
+};

package/skills/build-label-page/scripts/label-page-commands.ts

@@ -0,0 +1,14 @@
+/**
+ * The subcommands of `label-page`, by name. One table, read twice: the dispatcher in `label-page.ts`
+ * compares against these, and `SKILL.md.spec.ts` interpolates them into every command it instructs
+ * (a name that is not here is a type error there). So the skill cannot instruct a subcommand the
+ * script does not have.
+ *
+ * A module of its own, with no side effects: the spec imports it at compile time, and importing
+ * `label-page.ts` itself would run the command line.
+ */
+export const COMMANDS = {
+  build: "build",
+  export: "export",
+  schema: "schema",
+} as const;