eklavya 1.59.0 → 1.61.0

package/dist/assets/tutor/references/grading.md

@@ -1,15 +1,34 @@
 # Grading, and what to do with a blank
 
-Required reading before you grade. Two scales, one teaching sequence, and the
-one failure mode that makes the whole tool pointless.
+Required reading before you grade. Call `record_attempt` for **every** answer,
+blanks and skips included, with `question` verbatim (that text is what stops the
+same question coming back) and an `outcome`: `answered`, `dont_know`, `declined`
+or `invalid`. A question that was itself at fault is not a grade at all: see
+*When the question is at fault*.
 
-## Every answer gets recorded
+## What every response earns
 
-Call `record_attempt` for **every** answer, including blanks and skips — pass
-`question` verbatim, because that text is what stops the same question coming
-back later. Pass `outcome` as well: `answered`, `dont_know`, or `declined`.
+Find the row for who asked and what the learner did. The sections below say how
+to do each cell.
 
-Grade honestly on SM-2's 0–5:
+| Learner | Automatic question: checkpoint, sweep, while-waiting | Panel | Parallel tutor | Asked-for quiz or lesson |
+|---|---|---|---|---|
+| Right | record; verdict | the panel records and judges; do nothing | record; verdict | record; verdict |
+| Wrong | record; verdict, right answer, one line of why; no follow-up | as left | as left, then the handoff block | do not give the answer yet: one narrower question, then teach |
+| Blank ("I don't know") | `dont_know`; answer and one line of why | as left | as left | `dont_know`; the full four-step teaching |
+| Skip | `declined`, nothing said | the Skip button records it | `declined` | `declined` |
+| Question at fault | `invalid`; own it, give the answer, no replacement this turn | n/a: nothing to press | `invalid` | `invalid`; own it, then ask a fixed question now |
+| Silence | nothing | nothing | nothing, do not chase | wait |
+
+Then **straight back to the task** in every automatic case: no summary, no
+second question, no re-plan. A verdict is *right*, or *wrong plus the right
+answer and one line of why*, and it always comes first.
+
+## Grading honestly
+
+Grade on SM-2's 0–5. Before you pick a number, state to yourself what in their
+answer justifies it: a gate built on inflated grades teaches nothing, and being
+generous is the one failure that makes this tool pointless.
 
 | Grade | Means |
 |---|---|
@@ -20,18 +39,9 @@ Grade honestly on SM-2's 0–5:
 | 4 | correct and clean |
 | 5 | correct, and explained *why*, or caught a nuance you didn't ask for |
 
-Before you pick a number, state to yourself what in their answer justifies it. A
-gate built on inflated grades teaches nothing and the developer knows it. Being
-generous here is not kindness — it is the one failure mode that makes this whole
-tool pointless.
-
-## Multiple choice caps at 4, and the server enforces it
-
-Grade 5 means *correct, and explained why*, and picking an option cannot show
-that — one in four is a coin. `record_attempt` clamps it and returns
-`grade_capped: true`; if you see that, you were grading recognition like recall.
-
-Within the cap, still grade honestly:
+**Multiple choice caps at 4, and the server enforces it** (`grade_capped: true`
+means you graded recognition like recall). Picking one of four cannot show *why*.
+Within the cap:
 
 | Grade | Means |
 |---|---|
@@ -39,104 +49,91 @@ Within the cap, still grade honestly:
 | 3 | right option, but their "Other" text or follow-up showed it was a guess |
 | 2 | picked a distractor that is the shape of the idea |
 | 1 | picked a distractor built on a misconception |
-| 0 | "Other" with *I don't know* (`outcome: dont_know` — **teach it**), or a decline (`outcome: declined`) |
-
-**A defensible pick is a correct answer.** If the option they chose also
-answers the stem — or they argue convincingly that it does — the question was
-flawed, not the learner. Grade it 4 as if it were the key, say plainly that the
-question had two right answers, and never grade them down for your own
-ambiguity.
+| 0 | "Other" with *I don't know* (`dont_know`, teach it), or a decline (`declined`) |
 
-**If they want to explain, let them, and say so.** Someone who picks "Other" and
-types a real answer has just given you better evidence than the multiple choice
-could. Grade that as the free answer it is — **omit `format`**, and the cap does
-not apply.
+**A defensible pick is a correct answer.** If their option also answers the
+stem, or they argue convincingly that it does, the question was flawed, not the
+learner: grade it 4, say plainly it had two right answers, and never mark them
+down for your ambiguity. If they did not pick but said two are right, that is
+`invalid`.
 
-## Feedback
+**If they want to explain, let them.** Someone who picks "Other" and types a real
+answer has given better evidence than the choice could. Grade it as free recall:
+omit `format`, and the cap does not apply.
 
-**Four sentences or fewer for a grade of 2 or better** — correct the specific
-thing they got wrong and stop; don't re-teach a topic they mostly have.
-
-If they answer and get it wrong, do not immediately give the answer. Ask one
-narrower question that isolates the gap. If they miss that too, teach it as
-below.
-
-**Always say whether they were right first.** At an `[Eklavya checkpoint]`
-there is no narrower question — the checkpoint is one question — so a miss
-gets the right answer and one line of why, then the task resumes.
+Feedback for a grade of 2 or better is four sentences or fewer: correct the
+specific thing they got wrong and stop.
 
 ## When they say "I don't know"
 
-**This is the most important thing in this skill.** A blank is not a skip. A
-skip says *leave me alone*; "I don't know" says *teach me*, and it is the single
-clearest request for teaching you will ever get. Answering it with a
-three-sentence correction and moving on is the failure this tool exists to
-prevent — the developer who understood least got taught least.
-
-Both record as grade 0. What separates them is `outcome`, and what you do next.
-
-**They are not interchangeable, and the asymmetry is worth knowing.** A concept
-recorded as `declined` is never offered again — deliberately, because "leave me
-alone" is a choice. So labelling a blank as a decline removes that concept from
-the gate-retry path, which when enforced is the only route out of a blocked
-commit. If you explained it, it was a blank: `dont_know`. `record_attempt`
-returns `outcome_conflict` when it is given `declined` together with feedback,
-because a decline you dropped immediately has nothing to explain. It rejects
-`declined` or `dont_know` with a grade of 3 or more (`outcome_grade_conflict`)
-and records nothing: call again with the pair that is true.
-
-**Teach it. Properly, in this order:**
-
-1. **Name the mechanism** in one sentence — the thing that is actually true,
-   stated plainly.
-2. **Show the code.** Quote the two or three real lines from the diff that make
-   it true. They are looking at a file they have never read; the lines are the
-   whole lesson.
-3. **Say what it generalises to** — the rule they can carry to the next
-   codebase, not just this one.
+**The most important thing in this skill.** A blank is not a skip. A skip says
+*leave me alone*; "I don't know" says *teach me*, the clearest request for teaching you will ever get.
+Answering it with a three-sentence correction is the failure this tool exists to
+prevent. Both record grade 0; `outcome` separates them.
+
+They are not interchangeable. A `declined` concept is not offered again in this
+session's gate retry (a later session's project review may bring it back once it
+is due), so labelling a blank as a decline removes it from the only route out of
+a blocked commit. If you explained it, it was a blank: `dont_know`.
+`record_attempt` returns `outcome_conflict` for `declined` with feedback (a
+decline you dropped has nothing to explain), and rejects `declined` or
+`dont_know` with a grade of 3 or more (`outcome_grade_conflict`) without
+recording anything.
+
+**Who teaches, and how much.** The full sequence is for a quiz or lesson the
+developer asked for. For an automatic question the answer and one line of why is
+the whole inline reply, and when `record_attempt` returns `explain` the
+background page carries the long version: teach in one place, never both.
+
+1. **Name the mechanism** in one sentence, plainly.
+2. **Show the code, or a small example.** With a `context` from the plan, quote
+   the two or three lines from the diff that make it true, only lines you have
+   read. With `context: null` (`concept` focus, a review, a topic lesson) write a
+   three-to-six-line worked example and say it is one. Never present invented
+   lines as theirs.
+3. **Say what it generalises to**: the rule they carry to the next codebase.
 4. **One-line takeaway.** What to remember if they forget everything else.
 
-Six to ten sentences. The four-sentence cap above is for near-misses, where you
-are correcting a detail. Here there is no detail to correct: the topic *is* the
-gap.
-
-**Then record and move on.** `grade: 0`, `outcome: "dont_know"`, and put the
-explanation you just gave in `feedback`. Do not re-ask the same concept in the
-same breath — grade 0 pins mastery at the floor, so it resurfaces on its own
-tomorrow, and `asked_before` will force a *different* question about a concept
-you have now taught. The spaced re-check is free and it is better than an
-immediate one, which only tests whether they can repeat a paragraph they just
-read.
+Six to ten sentences. Then record: `grade: 0`, `outcome: "dont_know"`, your
+explanation in `feedback`. Do not re-ask in the same breath: grade 0 pins mastery
+at the floor, so it resurfaces tomorrow, and `asked_before` forces a different
+question. Never offer to stop because they are blanking: two blanks mean the
+pitch was too high, the planner already lowers `tier_to_ask`, so ask at the tier
+the plan gives and wait to be told to stop. Never dump the remaining answers as a
+list.
 
-**Never offer to stop because they are blanking.** Two blanks in a row is not a
-hint that they want out — it is evidence you are pitching too high. Drop a tier
-and keep going. Tier-1 recall on something you have just explained is fair, and
-it rebuilds footing. If they want to stop, they will say so; wait to be told.
+## When the question is at fault
 
-**Never dump the remaining answers as a list.** If the quiz ends early, it ends.
-A wall of four explanations at the door is not teaching, it is a receipt.
+"This question does not have enough context", or "two of these are right" with
+no pick made, is a verdict on the question, not a decline and not a blank. The
+miss is yours and must cost them nothing:
 
-## When they say the question lacks context
-
-"This question does not have enough context" is a verdict on the question, not
-a decline. The question assumed something they never saw, so the miss is
-yours. Never drop it untaught:
-
-1. **Own it** in a few words — the question leaned on something you had not
-   shown them.
+1. **Own it** in a few words.
 2. **Give the answer and one line of why**, defining the name or situation the
    stem skipped.
-3. **Record it as a blank**: `grade: 0`, `outcome: "dont_know"`, with that
-   answer in `feedback`. It is not `declined` — a decline is never offered
-   again, and they did not ask to be left alone.
-
-The next question about it, now or later, starts with the situation sentence
-the first one was missing (`writing-mcq.md`, *Write for a cold reader*).
+3. **Record `invalid`**: `outcome: "invalid"`, `grade: 0` (ignored), what was
+   wrong in `feedback`. Nothing is graded: no mastery, level, accuracy, review
+   date or gate changes, no `attempt_id` comes back, and an enforced gate does not
+   count it, so the concept is still asked. The stem is marked spent.
+4. **Replace it** where the pacing allows (the table above). The replacement
+   opens with the situation sentence the first one lacked (`writing-mcq.md`,
+   *Write for a cold reader*).
+
+A learner who really blanked on a clear question is `dont_know`: only the
+question's own defect makes it `invalid`.
+
+## Skip, stop and opting out
+
+- **Skip this question**: `declined`, no teaching, no second ask.
+- **Stop for now** ("enough"): stop asking this session and record nothing for
+  questions never shown. Nothing remembers it past the session, so for a lasting
+  stop tell them `eklavya config set quiz.enabled false` (the dashboard Settings
+  page does the same).
+- **Opt out of a topic**: there is no per-topic switch; do not promise one.
 
 ## `already_taught`
 
-When it is true on a plan item, they blanked on this before and you explained
-it. Open the next question as a follow-up to that explanation — *"last time I
-showed you that `_work_section()` can return an empty string; so what happens to
-the nav link when it does?"* — not as a first encounter. Building on a lesson is
-what makes it stick; asking cold throws it away.
+When true on a plan item, they blanked on this before and you explained it. Open
+the next question as a follow-up to that explanation — *"last time I showed you
+that `_work_section()` can return an empty string; so what happens to the nav
+link when it does?"* — not as a first encounter.

package/dist/assets/tutor/references/panel.md

@@ -18,7 +18,7 @@ Make one call to `present_question` instead:
 - `options`: four, in display order, each `{label, description, grade}`. The
   `label` is the option itself, a complete claim (what `writing-mcq.md` calls the
   option), never `A` or `1`: the panel numbers them, and a bare marker is rejected.
-  The `description` is the one clause from part 4. Mark the right one
+  The `description` is the neutral clause from part 4: what the option claims, with no verdict and no "tempting because". Mark the right one
   `correct: true`, **in the `answer_position` slot** (the call is rejected with
   `wrong_answer_position` otherwise, and nothing is stored).
 - Each option's `grade` is what a learner who picks it has shown, on the scale in
@@ -32,7 +32,10 @@ Make one call to `present_question` instead:
   `` `shouldRepaint` ``) and nothing else: no headings, lists or emphasis. Option `label`s are drawn as
   plain text, so write them in words and leave the backticks out of them.
 - `explanation`: one line saying why the right option is right. The panel shows
-  it after the answer, so it is the whole of your feedback.
+  it after the answer, so it is the whole of your feedback. A call whose options
+  are lopsided in visible length, or whose descriptions explain themselves, is
+  rejected once with `conspicuous_options` and nothing stored: rewrite and call
+  again with the same stem.
 
 It returns at once. Do not wait for the answer, do not call `record_attempt`
 (the panel records it, once), do not give a verdict, and go straight back to

package/dist/assets/tutor/references/writing-mcq.md

@@ -95,18 +95,25 @@ Before asking, read each distractor as if it were the key: if a senior reviewer
 could argue for it, narrow the stem ("why *here*", "what does *this line*
 prevent") until it cannot, or replace it with a different row.
 
-**4. One clause of `description` per option.** This is where a near-miss earns
-its place — the sentence that makes the wrong answer tempting.
-
-The parity rule from step 2 holds here too, and this is where it usually
-breaks: the correct option gets the careful sentence that explains the
-mechanism, and each distractor gets a few throwaway words. A learner reads that
-as the answer without reading the claim. Give every description the same
-weight — one clause of similar length, none standing out as the longest or the
-shortest — and spend the effort on the distractors: each one says why it is
-tempting, as specifically as the correct one says why it holds. Overcorrecting
-is a tell too: if the right answer's note is never the longest, a learner
-learns to rule the longest one out.
+**4. One neutral clause of `description` per option.** The learner reads it
+before answering, so it says what the option *does or claims*, never whether it
+is right, why it is tempting or which misconception it is. "Tempting because…",
+"a common mistake" and "this is why it works" are feedback, and feedback comes
+after the answer.
+
+Judge parity on what is on screen: the label and its description together. Give
+every option the same grammar, the same number of claims and the same kind of
+detail. Do not keep the mechanism, the example or the caveat for the correct
+one. Bring the short ones up with real content, never filler; if a distractor
+cannot reach the length honestly, the stem is too thin to have a near miss. The
+correct option should not be the longest, and it should not be systematically
+the shortest either: that is a tell in the other direction. The server and the
+`ask-label` hook send back an option that outruns the next by a few words, or
+whose description explains itself, once; rewrite it and ask again.
+
+The reasoning behind each option goes in `option_notes` when you record the
+attempt (the card path) and in the panel's `explanation` (the panel path), both
+shown only after the answer.
 
 **5. Placement.** The plan gives each question an `answer_position`, 1 to 4. Put
 the correct option in that slot. Left to your own judgement you will put the
@@ -157,9 +164,8 @@ telling you which part to rebuild.
 - All four options are within a few words of the same length and use the same
   grammar. A visibly longer or more careful option reads as the correct one, and
   gets picked without engaging — the same leak as always answering first.
-- The four descriptions are within a few words of each other too, and the
-  correct option's is no more careful than the rest. Each distractor's says why
-  it is tempting, not just that it is wrong.
+- Each option, label plus description, is within a few words of the others, and
+  no description says why an option is right, wrong or tempting.
 - Each of the three wrong options came from a different row of the table above.
 - Only the correct option answers the stem; no expert could argue for another.
 - The correct option sits at `answer_position`.
@@ -194,9 +200,10 @@ concept has unmet prerequisites:
 
 - Ask about the prerequisite instead, if it is in the plan — the plan already
   orders foundations first.
-- Otherwise drop the question a tier and make it mechanism-level. "Why this
-  rather than the alternative" is not answerable by someone who does not yet
-  have the alternative.
+- Otherwise ask at `tier_to_ask` (the planner owns the tier) and write the
+  question so it needs no alternative: about what this thing does, not why it
+  beats another. "Why this rather than the alternative" is not answerable by
+  someone who does not yet have the alternative.
 - Say the dependency out loud in your feedback. Knowing *what to learn next* is
   half of what the graph is for.
 
@@ -204,8 +211,9 @@ concept has unmet prerequisites:
 
 `record_attempt` with `format: "mcq"`, `options` as the labels you offered,
 `answer` as the one they picked, `correct` as the right option's label
-verbatim, `option_notes` as the `description` you wrote under each option (same
-order as `options`), and `question` as the **stem only**.
+verbatim, `option_notes` as what you want shown under each option after the answer (same
+order as `options`; the reasoning each option earns, which the neutral
+`description` left out), and `question` as the **stem only**.
 
 `correct` and `option_notes` are what let the learner correct a missed answer
 from its explainer page: the dashboard grades the new pick against `correct`

package/dist/eval/conversation-score.js

@@ -0,0 +1,98 @@
+/**
+ * The deterministic half of the conversation eval.
+ *
+ * `question-checks.ts` scores one question. The rules that decide whether a
+ * session goes well live elsewhere: what a blank earns, what a decline earns,
+ * whether a checkpoint stays one question, whether a parallel tutor hands the
+ * explainer everything it needs. A scenario freezes one moment (the plan, the
+ * question asked, what the learner said) and the model replies with the tool
+ * calls and message it would produce next; this module judges that reply
+ * against rules written as data.
+ *
+ * Pure like `question-checks.ts`: no model, no files, no clock.
+ */
+/** `mcp__plugin_eklavya_eklavya__record_attempt` and `record_attempt` are one tool. */
+export function toolName(raw) {
+    return raw.replace(/^mcp__.*?__/, '');
+}
+const ASKING = new Set(['AskUserQuestion', 'present_question']);
+/** Four lettered lines is a question in a renderer that has no tool for it. */
+function lettered(message) {
+    return (message.match(/^\s*\(?[A-D][).]\s+\S/gm) ?? []).length >= 4;
+}
+export function questionCount(turn) {
+    const calls = turn.tool_calls.filter((c) => ASKING.has(toolName(c.tool))).length;
+    return calls + (lettered(turn.message) ? 1 : 0);
+}
+function sentences(text) {
+    return text.split(/(?<=[.!?])\s+/).filter((s) => /\w/.test(s)).length;
+}
+function dig(obj, path) {
+    return path.split('.').reduce((o, k) => (o && typeof o === 'object' ? o[k] : undefined), obj);
+}
+function matches(value, m) {
+    if (typeof m !== 'object')
+        return value === m;
+    if (m.absent)
+        return value === undefined || value === null || value === '';
+    if (m.oneOf)
+        return m.oneOf.includes(value);
+    if (m.regex !== undefined)
+        return typeof value === 'string' && new RegExp(m.regex, 'i').test(value);
+    if (m.lengthMin !== undefined || m.lengthMax !== undefined) {
+        const n = typeof value === 'string' ? value.trim().length : Array.isArray(value) ? value.length : -1;
+        return n >= (m.lengthMin ?? 0) && n <= (m.lengthMax ?? Infinity);
+    }
+    return typeof value === 'number' && value >= (m.min ?? -Infinity) && value <= (m.max ?? Infinity);
+}
+export function checkTurn(turn, rules) {
+    return rules.map((rule) => {
+        if (rule.kind === 'call') {
+            const where = Object.entries(rule.where ?? {});
+            const hits = turn.tool_calls.filter((c) => toolName(c.tool) === rule.tool && where.every(([k, m]) => matches(dig(c.args, k), m))).length;
+            // A rule that only sets `max: 0` is a prohibition; it must not also demand a call.
+            const min = rule.min ?? (rule.max === 0 ? 0 : 1);
+            const ok = hits >= min && hits <= (rule.max ?? Infinity);
+            return { id: rule.id, ok, detail: `${hits} matching ${rule.tool} call(s), wanted ${min}-${rule.max ?? '∞'}` };
+        }
+        if (rule.kind === 'questions') {
+            const n = questionCount(turn);
+            return { id: rule.id, ok: n <= rule.max, detail: `${n} question(s) asked, limit ${rule.max}` };
+        }
+        const problems = [];
+        const text = turn.message.trim();
+        if (rule.maxSentences !== undefined && sentences(text) > rule.maxSentences)
+            problems.push(`${sentences(text)} sentences, limit ${rule.maxSentences}`);
+        if (rule.minChars !== undefined && text.length < rule.minChars)
+            problems.push(`${text.length} characters, wanted ${rule.minChars}+`);
+        if (rule.match !== undefined && !new RegExp(rule.match, 'is').test(text))
+            problems.push(`does not match /${rule.match}/`);
+        if (rule.notMatch !== undefined && new RegExp(rule.notMatch, 'is').test(text))
+            problems.push(`matches /${rule.notMatch}/`);
+        if (rule.notEndsWithQuestion && text.endsWith('?'))
+            problems.push('ends on a question');
+        return { id: rule.id, ok: problems.length === 0, detail: problems.length ? problems.join('; ') : 'message ok' };
+    });
+}
+/** Per-rule and per-scenario pass rates; an unparsed reply counts against every rule. */
+export function summarise(results) {
+    const byScenario = {};
+    for (const r of results) {
+        const row = (byScenario[r.scenario] ??= { trials: 0, clean: 0, unparsed: 0, failed: {} });
+        row.trials++;
+        if (r.unparsed) {
+            row.unparsed++;
+            continue;
+        }
+        const bad = r.verdicts.filter((v) => !v.ok);
+        if (bad.length === 0)
+            row.clean++;
+        for (const v of bad)
+            row.failed[v.id] = (row.failed[v.id] ?? 0) + 1;
+    }
+    const trials = results.length;
+    const clean = Object.values(byScenario).reduce((n, r) => n + r.clean, 0);
+    const unparsed = Object.values(byScenario).reduce((n, r) => n + r.unparsed, 0);
+    return { trials, clean, unparsed, byScenario };
+}
+//# sourceMappingURL=conversation-score.js.map

package/dist/eval/conversation-score.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"conversation-score.js","sourceRoot":"","sources":["../../src/eval/conversation-score.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;GAYG;AAgCH,uFAAuF;AACvF,MAAM,UAAU,QAAQ,CAAC,GAAW;IAClC,OAAO,GAAG,CAAC,OAAO,CAAC,aAAa,EAAE,EAAE,CAAC,CAAC;AACxC,CAAC;AAED,MAAM,MAAM,GAAG,IAAI,GAAG,CAAC,CAAC,iBAAiB,EAAE,kBAAkB,CAAC,CAAC,CAAC;AAEhE,+EAA+E;AAC/E,SAAS,QAAQ,CAAC,OAAe;IAC/B,OAAO,CAAC,OAAO,CAAC,KAAK,CAAC,yBAAyB,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,IAAI,CAAC,CAAC;AACtE,CAAC;AAED,MAAM,UAAU,aAAa,CAAC,IAAe;IAC3C,MAAM,KAAK,GAAG,IAAI,CAAC,UAAU,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,MAAM,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC;IACjF,OAAO,KAAK,GAAG,CAAC,QAAQ,CAAC,IAAI,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;AAClD,CAAC;AAED,SAAS,SAAS,CAAC,IAAY;IAC7B,OAAO,IAAI,CAAC,KAAK,CAAC,eAAe,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC;AACxE,CAAC;AAED,SAAS,GAAG,CAAC,GAAY,EAAE,IAAY;IACrC,OAAO,IAAI,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,MAAM,CAAU,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,OAAO,CAAC,KAAK,QAAQ,CAAC,CAAC,CAAE,CAA6B,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC,EAAE,GAAG,CAAC,CAAC;AACtI,CAAC;AAED,SAAS,OAAO,CAAC,KAAc,EAAE,CAAQ;IACvC,IAAI,OAAO,CAAC,KAAK,QAAQ;QAAE,OAAO,KAAK,KAAK,CAAC,CAAC;IAC9C,IAAI,CAAC,CAAC,MAAM;QAAE,OAAO,KAAK,KAAK,SAAS,IAAI,KAAK,KAAK,IAAI,IAAI,KAAK,KAAK,EAAE,CAAC;IAC3E,IAAI,CAAC,CAAC,KAAK;QAAE,OAAO,CAAC,CAAC,KAAK,CAAC,QAAQ,CAAC,KAAK,CAAC,CAAC;IAC5C,IAAI,CAAC,CAAC,KAAK,KAAK,SAAS;QAAE,OAAO,OAAO,KAAK,KAAK,QAAQ,IAAI,IAAI,MAAM,CAAC,CAAC,CAAC,KAAK,EAAE,GAAG,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAC;IACpG,IAAI,CAAC,CAAC,SAAS,KAAK,SAAS,IAAI,CAAC,CAAC,SAAS,KAAK,SAAS,EAAE,CAAC;QAC3D,MAAM,CAAC,GAAG,OAAO,KAAK,KAAK,QAAQ,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,EAAE,CAAC,MAAM,CAAC,CAAC,CAAC,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;QACrG,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,SAAS,IAAI,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,SAAS,IAAI,QAAQ,CAAC,CAAC;IACnE,CAAC;IACD,OAAO,OAAO,KAAK,KAAK,QAAQ,IAAI,KAAK,IAAI,CAAC,CAAC,CAAC,GAAG,IAAI,CAAC,QAAQ,CAAC,IAAI,KAAK,IAAI,CAAC,CAAC,CAAC,GAAG,IAAI,QAAQ,CAAC,CAAC;AACpG,CAAC;AAED,MAAM,UAAU,SAAS,CAAC,IAAe,EAAE,KAAa;IACtD,OAAO,KAAK,CAAC,GAAG,CAAC,CAAC,IAAI,EAAW,EAAE;QACjC,IAAI,IAAI,CAAC,IAAI,KAAK,MAAM,EAAE,CAAC;YACzB,MAAM,KAAK,GAAG,MAAM,CAAC,OAAO,CAAC,IAAI,CAAC,KAAK,IAAI,EAAE,CAAC,CAAC;YAC/C,MAAM,IAAI,GAAG,IAAI,CAAC,UAAU,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,IAAI,CAAC,IAAI,IAAI,KAAK,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,EAAE,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC;YACzI,mFAAmF;YACnF,MAAM,GAAG,GAAG,IAAI,CAAC,GAAG,IAAI,CAAC,IAAI,CAAC,GAAG,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;YACjD,MAAM,EAAE,GAAG,IAAI,IAAI,GAAG,IAAI,IAAI,IAAI,CAAC,IAAI,CAAC,GAAG,IAAI,QAAQ,CAAC,CAAC;YACzD,OAAO,EAAE,EAAE,EAAE,IAAI,CAAC,EAAE,EAAE,EAAE,EAAE,MAAM,EAAE,GAAG,IAAI,aAAa,IAAI,CAAC,IAAI,oBAAoB,GAAG,IAAI,IAAI,CAAC,GAAG,IAAI,GAAG,EAAE,EAAE,CAAC;QAChH,CAAC;QACD,IAAI,IAAI,CAAC,IAAI,KAAK,WAAW,EAAE,CAAC;YAC9B,MAAM,CAAC,GAAG,aAAa,CAAC,IAAI,CAAC,CAAC;YAC9B,OAAO,EAAE,EAAE,EAAE,IAAI,CAAC,EAAE,EAAE,EAAE,EAAE,CAAC,IAAI,IAAI,CAAC,GAAG,EAAE,MAAM,EAAE,GAAG,CAAC,6BAA6B,IAAI,CAAC,GAAG,EAAE,EAAE,CAAC;QACjG,CAAC;QACD,MAAM,QAAQ,GAAa,EAAE,CAAC;QAC9B,MAAM,IAAI,GAAG,IAAI,CAAC,OAAO,CAAC,IAAI,EAAE,CAAC;QACjC,IAAI,IAAI,CAAC,YAAY,KAAK,SAAS,IAAI,SAAS,CAAC,IAAI,CAAC,GAAG,IAAI,CAAC,YAAY;YAAE,QAAQ,CAAC,IAAI,CAAC,GAAG,SAAS,CAAC,IAAI,CAAC,qBAAqB,IAAI,CAAC,YAAY,EAAE,CAAC,CAAC;QACtJ,IAAI,IAAI,CAAC,QAAQ,KAAK,SAAS,IAAI,IAAI,CAAC,MAAM,GAAG,IAAI,CAAC,QAAQ;YAAE,QAAQ,CAAC,IAAI,CAAC,GAAG,IAAI,CAAC,MAAM,uBAAuB,IAAI,CAAC,QAAQ,GAAG,CAAC,CAAC;QACrI,IAAI,IAAI,CAAC,KAAK,KAAK,SAAS,IAAI,CAAC,IAAI,MAAM,CAAC,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC;YAAE,QAAQ,CAAC,IAAI,CAAC,mBAAmB,IAAI,CAAC,KAAK,GAAG,CAAC,CAAC;QAC1H,IAAI,IAAI,CAAC,QAAQ,KAAK,SAAS,IAAI,IAAI,MAAM,CAAC,IAAI,CAAC,QAAQ,EAAE,IAAI,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC;YAAE,QAAQ,CAAC,IAAI,CAAC,YAAY,IAAI,CAAC,QAAQ,GAAG,CAAC,CAAC;QAC3H,IAAI,IAAI,CAAC,mBAAmB,IAAI,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC;YAAE,QAAQ,CAAC,IAAI,CAAC,oBAAoB,CAAC,CAAC;QACxF,OAAO,EAAE,EAAE,EAAE,IAAI,CAAC,EAAE,EAAE,EAAE,EAAE,QAAQ,CAAC,MAAM,KAAK,CAAC,EAAE,MAAM,EAAE,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC,QAAQ,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,YAAY,EAAE,CAAC;IAClH,CAAC,CAAC,CAAC;AACL,CAAC;AAUD,yFAAyF;AACzF,MAAM,UAAU,SAAS,CAAC,OAAyB;IACjD,MAAM,UAAU,GAAwG,EAAE,CAAC;IAC3H,KAAK,MAAM,CAAC,IAAI,OAAO,EAAE,CAAC;QACxB,MAAM,GAAG,GAAG,CAAC,UAAU,CAAC,CAAC,CAAC,QAAQ,CAAC,KAAK,EAAE,MAAM,EAAE,CAAC,EAAE,KAAK,EAAE,CAAC,EAAE,QAAQ,EAAE,CAAC,EAAE,MAAM,EAAE,EAAE,EAAE,CAAC,CAAC;QAC1F,GAAG,CAAC,MAAM,EAAE,CAAC;QACb,IAAI,CAAC,CAAC,QAAQ,EAAE,CAAC;YACf,GAAG,CAAC,QAAQ,EAAE,CAAC;YACf,SAAS;QACX,CAAC;QACD,MAAM,GAAG,GAAG,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC;QAC5C,IAAI,GAAG,CAAC,MAAM,KAAK,CAAC;YAAE,GAAG,CAAC,KAAK,EAAE,CAAC;QAClC,KAAK,MAAM,CAAC,IAAI,GAAG;YAAE,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,GAAG,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC;IACtE,CAAC;IACD,MAAM,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC;IAC9B,MAAM,KAAK,GAAG,MAAM,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC;IACzE,MAAM,QAAQ,GAAG,MAAM,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,CAAC,CAAC,CAAC;IAC/E,OAAO,EAAE,MAAM,EAAE,KAAK,EAAE,QAAQ,EAAE,UAAU,EAAE,CAAC;AACjD,CAAC"}

package/dist/eval/question-checks.js

@@ -222,6 +222,37 @@ export function checkQuestion(q) {
     push('no_settings_line', !q.stem.trimStart().startsWith('['), 'stem opens with the question');
     return checks;
 }
+/**
+ * Phrasing that explains an option before the learner has answered. A
+ * description that says why a distractor is "tempting" or names the
+ * misconception is feedback, and feedback belongs after the answer.
+ */
+const PRE_ANSWER_RATIONALE = /\b(?:tempting|common (?:mistake|misconception)|misconception|is the (?:right|correct) answer|(?:is|are) (?:wrong|incorrect) because|looks (?:right|correct) but)\b/i;
+/**
+ * Why a question should not be shown as written, or null when it can be.
+ *
+ * Judged on what is on screen -- label plus description -- and on no knowledge
+ * of the key, so it works where the key is not known (the `AskUserQuestion`
+ * card) as well as where it is (the panel). An option that outruns the next by
+ * `LENGTH_MARGIN_WORDS` is conspicuous whichever one is correct: if it is the
+ * key it leaks, and if it is not, the learner is being steered off the key.
+ * Only the outlier is named, so the rewrite is aimed at one option.
+ */
+export function visibleOptionProblem(options) {
+    for (const [i, o] of options.entries()) {
+        const phrase = PRE_ANSWER_RATIONALE.exec(o.description ?? '');
+        if (phrase) {
+            return `option ${i + 1}'s description explains the option before the answer (${phrase[0]}). Keep descriptions neutral and put the reasoning in the explanation shown after the answer`;
+        }
+    }
+    const lengths = options.map((o) => words(`${o.label} ${o.description ?? ''}`).length);
+    const longest = Math.max(...lengths, 0);
+    const runnerUp = Math.max(...lengths.filter((_, i) => i !== lengths.indexOf(longest)), 0);
+    if (longest > PARITY_MIN_WORDS && longest - runnerUp >= LENGTH_MARGIN_WORDS) {
+        return `option ${lengths.indexOf(longest) + 1} is ${longest} visible words (label plus description) against ${runnerUp} for the next longest, so it stands out. Shorten it or give the others the same number of claims, without padding`;
+    }
+    return null;
+}
 export function scoreAll(questions) {
     const scored = questions.map((question) => {
         const checks = checkQuestion(question);

package/dist/eval/question-checks.js.map

@@ -1 +1 @@
-{"version":3,"file":"question-checks.js","sourceRoot":"","sources":["../../src/eval/question-checks.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;GAiBG;AA+BH,sEAAsE;AACtE,SAAS,KAAK,CAAC,IAAY;IACzB,OAAO,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,eAAe,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC;AAClE,CAAC;AAED;;;;;;;;;GASG;AACH,SAAS,UAAU,CAAC,IAAY;IAC9B,OAAO,KAAK,CAAC,IAAI,CAAC,WAAW,EAAE,CAAC,OAAO,CAAC,uBAAuB,EAAE,GAAG,CAAC,CAAC,CAAC;AACzE,CAAC;AAED;;;;;;;GAOG;AACH,MAAM,UAAU,gBAAgB,CAAC,CAAS,EAAE,CAAS;IACnD,MAAM,CAAC,GAAG,UAAU,CAAC,CAAC,CAAC,CAAC;IACxB,MAAM,CAAC,GAAG,UAAU,CAAC,CAAC,CAAC,CAAC;IACxB,IAAI,CAAC,CAAC,MAAM,KAAK,CAAC,IAAI,CAAC,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,CAAC,CAAC;IAE/C,wEAAwE;IACxE,+CAA+C;IAC/C,IAAI,IAAI,GAAG,CAAC,CAAC;IACb,MAAM,IAAI,GAAG,IAAI,KAAK,CAAS,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IACrD,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACnC,IAAI,QAAQ,GAAG,CAAC,CAAC;QACjB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;YACnC,MAAM,KAAK,GAAG,IAAI,CAAC,CAAC,CAAE,CAAC;YACvB,MAAM,GAAG,GAAG,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,QAAQ,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;YACrD,QAAQ,GAAG,KAAK,CAAC;YACjB,IAAI,CAAC,CAAC,CAAC,GAAG,GAAG,CAAC;YACd,IAAI,GAAG,GAAG,IAAI;gBAAE,IAAI,GAAG,GAAG,CAAC;QAC7B,CAAC;IACH,CAAC;IACD,OAAO,IAAI,CAAC;AACd,CAAC;AAED;;;GAGG;AACH,MAAM,CAAC,MAAM,eAAe,GAAG,EAAE,CAAC;AAElC;;;;;GAKG;AACH,MAAM,CAAC,MAAM,gBAAgB,GAAG,EAAE,CAAC;AAEnC;;;;;;GAMG;AACH,MAAM,UAAU,SAAS,CAAC,IAAY;IACpC,MAAM,SAAS,GAAG,IAAI;SACnB,IAAI,EAAE;SACN,KAAK,CAAC,eAAe,CAAC;SACtB,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;IAC/B,IAAI,SAAS,CAAC,MAAM,IAAI,CAAC;QAAE,OAAO,EAAE,KAAK,EAAE,EAAE,EAAE,QAAQ,EAAE,IAAI,CAAC,IAAI,EAAE,EAAE,CAAC;IACvE,OAAO,EAAE,KAAK,EAAE,SAAS,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,QAAQ,EAAE,SAAS,CAAC,SAAS,CAAC,MAAM,GAAG,CAAC,CAAE,EAAE,CAAC;AACjG,CAAC;AAED,4EAA4E;AAC5E,MAAM,CAAC,MAAM,gBAAgB,GAAG,CAAC,CAAC;AAElC;;;;GAIG;AACH,MAAM,CAAC,MAAM,kBAAkB,GAAG,GAAG,CAAC;AAEtC;;;;GAIG;AACH,MAAM,CAAC,MAAM,gBAAgB,GAAG,CAAC,CAAC;AAElC;;;;GAIG;AACH,MAAM,CAAC,MAAM,mBAAmB,GAAG,CAAC,CAAC;AAErC;;;;;;;;GAQG;AACH,MAAM,YAAY,GAAG;IACnB,4DAA4D;IAC5D,6HAA6H;IAC7H,kFAAkF;IAClF,gEAAgE;IAChE,wCAAwC;IACxC,mCAAmC;IACnC,wCAAwC;CACzC,CAAC;AAEF;;;;;;GAMG;AACH,MAAM,aAAa,GAAG,qCAAqC,CAAC;AAE5D,kEAAkE;AAClE,SAAS,cAAc,CAAC,CAAoB;IAC1C,OAAO,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,YAAY,CAAC,IAAI,CAAC,CAAC,YAAY,CAAC,MAAM,KAAK,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,IAAI,CAAC;AAC7G,CAAC;AAED,yDAAyD;AACzD,MAAM,UAAU,eAAe,CAAC,KAAe,EAAE,CAAS;IACxD,MAAM,OAAO,GAAG,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC;IAClD,MAAM,IAAI,GAAG,OAAO,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;IAC7B,OAAO,IAAI,KAAK,IAAI,CAAC,GAAG,CAAC,GAAG,OAAO,EAAE,CAAC,CAAC,IAAI,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,KAAK,IAAI,CAAC,CAAC,MAAM,KAAK,CAAC,CAAC;AAC5F,CAAC;AAED,SAAS,eAAe,CAAC,IAAY;IACnC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,aAAa,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,IAAI,CAAC,CAAC;AACvD,CAAC;AAED,MAAM,UAAU,aAAa,CAAC,CAAoB;IAChD,MAAM,MAAM,GAAY,EAAE,CAAC;IAC3B,MAAM,IAAI,GAAG,CAAC,EAAU,EAAE,EAAW,EAAE,MAAc,EAAE,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,EAAE,EAAE,EAAE,EAAE,MAAM,EAAE,CAAC,CAAC;IAE1F,MAAM,MAAM,GAAG,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;IAC1E,MAAM,MAAM,GAAG,IAAI,GAAG,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC;IAC3D,IAAI,CACF,cAAc,EACd,MAAM,CAAC,MAAM,KAAK,CAAC,IAAI,MAAM,CAAC,IAAI,KAAK,CAAC,EACxC,GAAG,MAAM,CAAC,MAAM,yBAAyB,MAAM,CAAC,IAAI,WAAW,CAChE,CAAC;IAEF,wEAAwE;IACxE,0EAA0E;IAC1E,6EAA6E;IAC7E,oCAAoC;IACpC,MAAM,MAAM,GAAG,CAAC,CAAC,OAAO,KAAK,CAAC,CAAC,eAAe,CAAC;IAC/C,IAAI,CACF,oBAAoB,EACpB,MAAM,EACN,MAAM;QACJ,CAAC,CAAC,qBAAqB,CAAC,CAAC,eAAe,YAAY;QACpD,CAAC,CAAC,aAAa,CAAC,CAAC,eAAe,gBAAgB,CAAC,CAAC,OAAO,EAAE,CAC9D,CAAC;IAEF,MAAM,EAAE,KAAK,EAAE,QAAQ,EAAE,GAAG,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;IAC9C,MAAM,UAAU,GAAG,KAAK,CAAC,KAAK,CAAC,CAAC,MAAM,CAAC;IACvC,MAAM,aAAa,GAAG,KAAK,CAAC,QAAQ,CAAC,CAAC,MAAM,CAAC;IAC7C,IAAI,CACF,aAAa,EACb,UAAU,IAAI,gBAAgB,IAAI,aAAa,IAAI,eAAe,EAClE,SAAS,UAAU,iBAAiB,gBAAgB,eAAe,aAAa,iBAAiB,eAAe,GAAG,CACpH,CAAC;IAEF,6EAA6E;IAC7E,yEAAyE;IACzE,uEAAuE;IACvE,6EAA6E;IAC7E,sBAAsB;IACtB,MAAM,OAAO,GAAG,MAAM,CAAC,SAAS,CAAC,CAAC,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,OAAO,IAAI,CAAC,IAAI,CAAC,CAAC,OAAO,IAAI,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC;IAC/F,IAAI,CACF,kBAAkB,EAClB,OAAO,EACP,OAAO,CAAC,CAAC,CAAC,aAAa,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,aAAa,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,OAAO,CAAC,eAAe,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,CAC7G,CAAC;IACF,MAAM,OAAO,GAAG,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,OAAO,GAAG,CAAC,CAAE,CAAC,CAAC,CAAC,EAAE,CAAC;IACzD,MAAM,GAAG,GAAG,gBAAgB,CAAC,CAAC,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC;IAC9C,IAAI,CACF,oBAAoB,EACpB,GAAG,GAAG,gBAAgB,EACtB,+CAA+C,GAAG,UAAU,CAC7D,CAAC;IAEF,MAAM,OAAO,GAAG,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC;IACtD,MAAM,OAAO,GAAG,IAAI,CAAC,GAAG,CAAC,GAAG,OAAO,EAAE,CAAC,CAAC,CAAC;IACxC,MAAM,QAAQ,GAAG,IAAI,CAAC,GAAG,CAAC,GAAG,OAAO,EAAE,QAAQ,CAAC,CAAC;IAChD,MAAM,KAAK,GAAG,QAAQ,GAAG,CAAC,CAAC,CAAC,CAAC,OAAO,GAAG,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC;IAC3D,MAAM,sBAAsB,GAAG,OAAO,IAAI,gBAAgB,CAAC;IAC3D,IAAI,CACF,eAAe,EACf,sBAAsB,IAAI,KAAK,IAAI,kBAAkB,EACrD,sBAAsB;QACpB,CAAC,CAAC,wBAAwB,OAAO,2CAA2C;QAC5E,CAAC,CAAC,sBAAsB,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,WAAW,kBAAkB,GAAG,CAC3E,CAAC;IAEF,6EAA6E;IAC7E,8EAA8E;IAC9E,4DAA4D;IAC5D,EAAE;IACF,2EAA2E;IAC3E,uEAAuE;IACvE,8EAA8E;IAC9E,8EAA8E;IAC9E,8EAA8E;IAC9E,+DAA+D;IAC/D,8DAA8D;IAC9D,MAAM,aAAa,GAAG,KAAK,CAAC,OAAO,CAAC,CAAC,MAAM,CAAC;IAC5C,MAAM,MAAM,GAAG,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,OAAO,GAAG,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC;IAC3F,MAAM,QAAQ,GAAG,IAAI,CAAC,GAAG,CAAC,GAAG,MAAM,EAAE,CAAC,CAAC,CAAC;IACxC,MAAM,MAAM,GAAG,aAAa,GAAG,QAAQ,CAAC;IACxC,IAAI,CACF,yBAAyB,EACzB,MAAM,GAAG,mBAAmB,EAC5B,qBAAqB,aAAa,iCAAiC,QAAQ,YAAY,MAAM,WAAW,mBAAmB,GAAG,CAC/H,CAAC;IAEF,4EAA4E;IAC5E,oEAAoE;IACpE,6EAA6E;IAC7E,iFAAiF;IACjF,MAAM,YAAY,GAAG,cAAc,CAAC,CAAC,CAAC,CAAC;IACvC,IAAI,YAAY,IAAI,OAAO,EAAE,CAAC;QAC5B,MAAM,IAAI,GAAG,KAAK,CAAC,YAAY,CAAC,CAAC,CAAC,OAAO,GAAG,CAAC,CAAE,CAAC,CAAC,MAAM,CAAC;QACxD,MAAM,IAAI,GAAG,IAAI,CAAC,GAAG,CACnB,GAAG,YAAY,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,OAAO,GAAG,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,EACjF,CAAC,CACF,CAAC;QACF,IAAI,CACF,6BAA6B,EAC7B,IAAI,GAAG,IAAI,GAAG,mBAAmB,EACjC,0BAA0B,IAAI,iCAAiC,IAAI,YAAY,IAAI,GAAG,IAAI,WAAW,mBAAmB,GAAG,CAC5H,CAAC;IACJ,CAAC;IAED,MAAM,OAAO,GAAG,YAAY,CAAC,IAAI,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;IAC3D,IAAI,CAAC,eAAe,EAAE,CAAC,OAAO,EAAE,OAAO,CAAC,CAAC,CAAC,oBAAoB,OAAO,EAAE,CAAC,CAAC,CAAC,6BAA6B,CAAC,CAAC;IAEzG,MAAM,QAAQ,GAAG,eAAe,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;IACzC,IAAI,CACF,sBAAsB,EACtB,CAAC,QAAQ,EACT,QAAQ,CAAC,CAAC,CAAC,oCAAoC,CAAC,CAAC,CAAC,4CAA4C,CAC/F,CAAC;IAEF,yEAAyE;IACzE,2EAA2E;IAC3E,+CAA+C;IAC/C,IAAI,CAAC,kBAAkB,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,SAAS,EAAE,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE,8BAA8B,CAAC,CAAC;IAE9F,OAAO,MAAM,CAAC;AAChB,CAAC;AAsCD,MAAM,UAAU,QAAQ,CAAC,SAA8B;IACrD,MAAM,MAAM,GAAG,SAAS,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,EAAE;QACxC,MAAM,MAAM,GAAG,aAAa,CAAC,QAAQ,CAAC,CAAC;QACvC,OAAO,EAAE,QAAQ,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC;IACjE,CAAC,CAAC,CAAC;IAEH,MAAM,OAAO,GAAuB,EAAE,CAAC;IACvC,KAAK,MAAM,EAAE,MAAM,EAAE,IAAI,MAAM,EAAE,CAAC;QAChC,KAAK,MAAM,CAAC,IAAI,MAAM,EAAE,CAAC;YACvB,MAAM,GAAG,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC,KAAK,EAAE,MAAM,EAAE,CAAC,EAAE,MAAM,EAAE,CAAC,EAAE,CAAC,CAAC;YACzD,IAAI,CAAC,CAAC,EAAE;gBAAE,GAAG,CAAC,MAAM,EAAE,CAAC;;gBAClB,GAAG,CAAC,MAAM,EAAE,CAAC;QACpB,CAAC;IACH,CAAC;IAED,MAAM,KAAK,GAAqC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,CAAC;IAC7D,IAAI,cAAc,GAAG,CAAC,CAAC;IACvB,IAAI,SAAS,GAAG,CAAC,CAAC;IAClB,IAAI,yBAAyB,GAAG,CAAC,CAAC;IAClC,KAAK,MAAM,EAAE,QAAQ,EAAE,IAAI,MAAM,EAAE,CAAC;QAClC,MAAM,CAAC,GAAG,QAAQ,CAAC,OAAO,GAAG,CAAC,CAAC;QAC/B,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC;YAAE,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;QAC5D,mEAAmE;QACnE,qEAAqE;QACrE,8CAA8C;QAC9C,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;YAC7C,IAAI,eAAe,CAAC,QAAQ,CAAC,OAAO,EAAE,CAAC,CAAC;gBAAE,cAAc,EAAE,CAAC;YAC3D,MAAM,YAAY,GAAG,cAAc,CAAC,QAAQ,CAAC,CAAC;YAC9C,IAAI,YAAY,EAAE,CAAC;gBACjB,SAAS,EAAE,CAAC;gBACZ,IAAI,eAAe,CAAC,YAAY,EAAE,CAAC,CAAC;oBAAE,yBAAyB,EAAE,CAAC;YACpE,CAAC;QACH,CAAC;IACH,CAAC;IAED,OAAO;QACL,MAAM;QACN,OAAO,EAAE;YACP,SAAS,EAAE,MAAM,CAAC,MAAM;YACxB,KAAK,EAAE,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,MAAM;YAC5C,OAAO;YACP,KAAK;YACL,cAAc;YACd,SAAS;YACT,yBAAyB;SAC1B;KACF,CAAC;AACJ,CAAC"}
+{"version":3,"file":"question-checks.js","sourceRoot":"","sources":["../../src/eval/question-checks.ts"],"names":[],"mappings":"AAAA;;;;;;;;;;;;;;;;;GAiBG;AA+BH,sEAAsE;AACtE,SAAS,KAAK,CAAC,IAAY;IACzB,OAAO,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,eAAe,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC;AAClE,CAAC;AAED;;;;;;;;;GASG;AACH,SAAS,UAAU,CAAC,IAAY;IAC9B,OAAO,KAAK,CAAC,IAAI,CAAC,WAAW,EAAE,CAAC,OAAO,CAAC,uBAAuB,EAAE,GAAG,CAAC,CAAC,CAAC;AACzE,CAAC;AAED;;;;;;;GAOG;AACH,MAAM,UAAU,gBAAgB,CAAC,CAAS,EAAE,CAAS;IACnD,MAAM,CAAC,GAAG,UAAU,CAAC,CAAC,CAAC,CAAC;IACxB,MAAM,CAAC,GAAG,UAAU,CAAC,CAAC,CAAC,CAAC;IACxB,IAAI,CAAC,CAAC,MAAM,KAAK,CAAC,IAAI,CAAC,CAAC,MAAM,KAAK,CAAC;QAAE,OAAO,CAAC,CAAC;IAE/C,wEAAwE;IACxE,+CAA+C;IAC/C,IAAI,IAAI,GAAG,CAAC,CAAC;IACb,MAAM,IAAI,GAAG,IAAI,KAAK,CAAS,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IACrD,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;QACnC,IAAI,QAAQ,GAAG,CAAC,CAAC;QACjB,KAAK,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC,MAAM,EAAE,CAAC,EAAE,EAAE,CAAC;YACnC,MAAM,KAAK,GAAG,IAAI,CAAC,CAAC,CAAE,CAAC;YACvB,MAAM,GAAG,GAAG,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,QAAQ,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;YACrD,QAAQ,GAAG,KAAK,CAAC;YACjB,IAAI,CAAC,CAAC,CAAC,GAAG,GAAG,CAAC;YACd,IAAI,GAAG,GAAG,IAAI;gBAAE,IAAI,GAAG,GAAG,CAAC;QAC7B,CAAC;IACH,CAAC;IACD,OAAO,IAAI,CAAC;AACd,CAAC;AAED;;;GAGG;AACH,MAAM,CAAC,MAAM,eAAe,GAAG,EAAE,CAAC;AAElC;;;;;GAKG;AACH,MAAM,CAAC,MAAM,gBAAgB,GAAG,EAAE,CAAC;AAEnC;;;;;;GAMG;AACH,MAAM,UAAU,SAAS,CAAC,IAAY;IACpC,MAAM,SAAS,GAAG,IAAI;SACnB,IAAI,EAAE;SACN,KAAK,CAAC,eAAe,CAAC;SACtB,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;IAC/B,IAAI,SAAS,CAAC,MAAM,IAAI,CAAC;QAAE,OAAO,EAAE,KAAK,EAAE,EAAE,EAAE,QAAQ,EAAE,IAAI,CAAC,IAAI,EAAE,EAAE,CAAC;IACvE,OAAO,EAAE,KAAK,EAAE,SAAS,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,EAAE,QAAQ,EAAE,SAAS,CAAC,SAAS,CAAC,MAAM,GAAG,CAAC,CAAE,EAAE,CAAC;AACjG,CAAC;AAED,4EAA4E;AAC5E,MAAM,CAAC,MAAM,gBAAgB,GAAG,CAAC,CAAC;AAElC;;;;GAIG;AACH,MAAM,CAAC,MAAM,kBAAkB,GAAG,GAAG,CAAC;AAEtC;;;;GAIG;AACH,MAAM,CAAC,MAAM,gBAAgB,GAAG,CAAC,CAAC;AAElC;;;;GAIG;AACH,MAAM,CAAC,MAAM,mBAAmB,GAAG,CAAC,CAAC;AAErC;;;;;;;;GAQG;AACH,MAAM,YAAY,GAAG;IACnB,4DAA4D;IAC5D,6HAA6H;IAC7H,kFAAkF;IAClF,gEAAgE;IAChE,wCAAwC;IACxC,mCAAmC;IACnC,wCAAwC;CACzC,CAAC;AAEF;;;;;;GAMG;AACH,MAAM,aAAa,GAAG,qCAAqC,CAAC;AAE5D,kEAAkE;AAClE,SAAS,cAAc,CAAC,CAAoB;IAC1C,OAAO,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,YAAY,CAAC,IAAI,CAAC,CAAC,YAAY,CAAC,MAAM,KAAK,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,IAAI,CAAC;AAC7G,CAAC;AAED,yDAAyD;AACzD,MAAM,UAAU,eAAe,CAAC,KAAe,EAAE,CAAS;IACxD,MAAM,OAAO,GAAG,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC;IAClD,MAAM,IAAI,GAAG,OAAO,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;IAC7B,OAAO,IAAI,KAAK,IAAI,CAAC,GAAG,CAAC,GAAG,OAAO,EAAE,CAAC,CAAC,IAAI,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,KAAK,IAAI,CAAC,CAAC,MAAM,KAAK,CAAC,CAAC;AAC5F,CAAC;AAED,SAAS,eAAe,CAAC,IAAY;IACnC,OAAO,CAAC,IAAI,CAAC,KAAK,CAAC,aAAa,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,IAAI,CAAC,CAAC;AACvD,CAAC;AAED,MAAM,UAAU,aAAa,CAAC,CAAoB;IAChD,MAAM,MAAM,GAAY,EAAE,CAAC;IAC3B,MAAM,IAAI,GAAG,CAAC,EAAU,EAAE,EAAW,EAAE,MAAc,EAAE,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,EAAE,EAAE,EAAE,EAAE,MAAM,EAAE,CAAC,CAAC;IAE1F,MAAM,MAAM,GAAG,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;IAC1E,MAAM,MAAM,GAAG,IAAI,GAAG,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,WAAW,EAAE,CAAC,CAAC,CAAC;IAC3D,IAAI,CACF,cAAc,EACd,MAAM,CAAC,MAAM,KAAK,CAAC,IAAI,MAAM,CAAC,IAAI,KAAK,CAAC,EACxC,GAAG,MAAM,CAAC,MAAM,yBAAyB,MAAM,CAAC,IAAI,WAAW,CAChE,CAAC;IAEF,wEAAwE;IACxE,0EAA0E;IAC1E,6EAA6E;IAC7E,oCAAoC;IACpC,MAAM,MAAM,GAAG,CAAC,CAAC,OAAO,KAAK,CAAC,CAAC,eAAe,CAAC;IAC/C,IAAI,CACF,oBAAoB,EACpB,MAAM,EACN,MAAM;QACJ,CAAC,CAAC,qBAAqB,CAAC,CAAC,eAAe,YAAY;QACpD,CAAC,CAAC,aAAa,CAAC,CAAC,eAAe,gBAAgB,CAAC,CAAC,OAAO,EAAE,CAC9D,CAAC;IAEF,MAAM,EAAE,KAAK,EAAE,QAAQ,EAAE,GAAG,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;IAC9C,MAAM,UAAU,GAAG,KAAK,CAAC,KAAK,CAAC,CAAC,MAAM,CAAC;IACvC,MAAM,aAAa,GAAG,KAAK,CAAC,QAAQ,CAAC,CAAC,MAAM,CAAC;IAC7C,IAAI,CACF,aAAa,EACb,UAAU,IAAI,gBAAgB,IAAI,aAAa,IAAI,eAAe,EAClE,SAAS,UAAU,iBAAiB,gBAAgB,eAAe,aAAa,iBAAiB,eAAe,GAAG,CACpH,CAAC;IAEF,6EAA6E;IAC7E,yEAAyE;IACzE,uEAAuE;IACvE,6EAA6E;IAC7E,sBAAsB;IACtB,MAAM,OAAO,GAAG,MAAM,CAAC,SAAS,CAAC,CAAC,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,OAAO,IAAI,CAAC,IAAI,CAAC,CAAC,OAAO,IAAI,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC;IAC/F,IAAI,CACF,kBAAkB,EAClB,OAAO,EACP,OAAO,CAAC,CAAC,CAAC,aAAa,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,aAAa,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,OAAO,CAAC,eAAe,CAAC,CAAC,OAAO,CAAC,MAAM,EAAE,CAC7G,CAAC;IACF,MAAM,OAAO,GAAG,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,OAAO,GAAG,CAAC,CAAE,CAAC,CAAC,CAAC,EAAE,CAAC;IACzD,MAAM,GAAG,GAAG,gBAAgB,CAAC,CAAC,CAAC,IAAI,EAAE,OAAO,CAAC,CAAC;IAC9C,IAAI,CACF,oBAAoB,EACpB,GAAG,GAAG,gBAAgB,EACtB,+CAA+C,GAAG,UAAU,CAC7D,CAAC;IAEF,MAAM,OAAO,GAAG,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC;IACtD,MAAM,OAAO,GAAG,IAAI,CAAC,GAAG,CAAC,GAAG,OAAO,EAAE,CAAC,CAAC,CAAC;IACxC,MAAM,QAAQ,GAAG,IAAI,CAAC,GAAG,CAAC,GAAG,OAAO,EAAE,QAAQ,CAAC,CAAC;IAChD,MAAM,KAAK,GAAG,QAAQ,GAAG,CAAC,CAAC,CAAC,CAAC,OAAO,GAAG,QAAQ,CAAC,CAAC,CAAC,QAAQ,CAAC;IAC3D,MAAM,sBAAsB,GAAG,OAAO,IAAI,gBAAgB,CAAC;IAC3D,IAAI,CACF,eAAe,EACf,sBAAsB,IAAI,KAAK,IAAI,kBAAkB,EACrD,sBAAsB;QACpB,CAAC,CAAC,wBAAwB,OAAO,2CAA2C;QAC5E,CAAC,CAAC,sBAAsB,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,WAAW,kBAAkB,GAAG,CAC3E,CAAC;IAEF,6EAA6E;IAC7E,8EAA8E;IAC9E,4DAA4D;IAC5D,EAAE;IACF,2EAA2E;IAC3E,uEAAuE;IACvE,8EAA8E;IAC9E,8EAA8E;IAC9E,8EAA8E;IAC9E,+DAA+D;IAC/D,8DAA8D;IAC9D,MAAM,aAAa,GAAG,KAAK,CAAC,OAAO,CAAC,CAAC,MAAM,CAAC;IAC5C,MAAM,MAAM,GAAG,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,OAAO,GAAG,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC;IAC3F,MAAM,QAAQ,GAAG,IAAI,CAAC,GAAG,CAAC,GAAG,MAAM,EAAE,CAAC,CAAC,CAAC;IACxC,MAAM,MAAM,GAAG,aAAa,GAAG,QAAQ,CAAC;IACxC,IAAI,CACF,yBAAyB,EACzB,MAAM,GAAG,mBAAmB,EAC5B,qBAAqB,aAAa,iCAAiC,QAAQ,YAAY,MAAM,WAAW,mBAAmB,GAAG,CAC/H,CAAC;IAEF,4EAA4E;IAC5E,oEAAoE;IACpE,6EAA6E;IAC7E,iFAAiF;IACjF,MAAM,YAAY,GAAG,cAAc,CAAC,CAAC,CAAC,CAAC;IACvC,IAAI,YAAY,IAAI,OAAO,EAAE,CAAC;QAC5B,MAAM,IAAI,GAAG,KAAK,CAAC,YAAY,CAAC,CAAC,CAAC,OAAO,GAAG,CAAC,CAAE,CAAC,CAAC,MAAM,CAAC;QACxD,MAAM,IAAI,GAAG,IAAI,CAAC,GAAG,CACnB,GAAG,YAAY,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC,OAAO,GAAG,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,MAAM,CAAC,EACjF,CAAC,CACF,CAAC;QACF,IAAI,CACF,6BAA6B,EAC7B,IAAI,GAAG,IAAI,GAAG,mBAAmB,EACjC,0BAA0B,IAAI,iCAAiC,IAAI,YAAY,IAAI,GAAG,IAAI,WAAW,mBAAmB,GAAG,CAC5H,CAAC;IACJ,CAAC;IAED,MAAM,OAAO,GAAG,YAAY,CAAC,IAAI,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;IAC3D,IAAI,CAAC,eAAe,EAAE,CAAC,OAAO,EAAE,OAAO,CAAC,CAAC,CAAC,oBAAoB,OAAO,EAAE,CAAC,CAAC,CAAC,6BAA6B,CAAC,CAAC;IAEzG,MAAM,QAAQ,GAAG,eAAe,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;IACzC,IAAI,CACF,sBAAsB,EACtB,CAAC,QAAQ,EACT,QAAQ,CAAC,CAAC,CAAC,oCAAoC,CAAC,CAAC,CAAC,4CAA4C,CAC/F,CAAC;IAEF,yEAAyE;IACzE,2EAA2E;IAC3E,+CAA+C;IAC/C,IAAI,CAAC,kBAAkB,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,SAAS,EAAE,CAAC,UAAU,CAAC,GAAG,CAAC,EAAE,8BAA8B,CAAC,CAAC;IAE9F,OAAO,MAAM,CAAC;AAChB,CAAC;AAQD;;;;GAIG;AACH,MAAM,oBAAoB,GACxB,qKAAqK,CAAC;AAExK;;;;;;;;;GASG;AACH,MAAM,UAAU,oBAAoB,CAAC,OAAwB;IAC3D,KAAK,MAAM,CAAC,CAAC,EAAE,CAAC,CAAC,IAAI,OAAO,CAAC,OAAO,EAAE,EAAE,CAAC;QACvC,MAAM,MAAM,GAAG,oBAAoB,CAAC,IAAI,CAAC,CAAC,CAAC,WAAW,IAAI,EAAE,CAAC,CAAC;QAC9D,IAAI,MAAM,EAAE,CAAC;YACX,OAAO,UAAU,CAAC,GAAG,CAAC,yDAAyD,MAAM,CAAC,CAAC,CAAC,8FAA8F,CAAC;QACzL,CAAC;IACH,CAAC;IACD,MAAM,OAAO,GAAG,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,KAAK,IAAI,CAAC,CAAC,WAAW,IAAI,EAAE,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC;IACtF,MAAM,OAAO,GAAG,IAAI,CAAC,GAAG,CAAC,GAAG,OAAO,EAAE,CAAC,CAAC,CAAC;IACxC,MAAM,QAAQ,GAAG,IAAI,CAAC,GAAG,CAAC,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,KAAK,OAAO,CAAC,OAAO,CAAC,OAAO,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC;IAC1F,IAAI,OAAO,GAAG,gBAAgB,IAAI,OAAO,GAAG,QAAQ,IAAI,mBAAmB,EAAE,CAAC;QAC5E,OAAO,UAAU,OAAO,CAAC,OAAO,CAAC,OAAO,CAAC,GAAG,CAAC,OAAO,OAAO,mDAAmD,QAAQ,mHAAmH,CAAC;IAC5O,CAAC;IACD,OAAO,IAAI,CAAC;AACd,CAAC;AAsCD,MAAM,UAAU,QAAQ,CAAC,SAA8B;IACrD,MAAM,MAAM,GAAG,SAAS,CAAC,GAAG,CAAC,CAAC,QAAQ,EAAE,EAAE;QACxC,MAAM,MAAM,GAAG,aAAa,CAAC,QAAQ,CAAC,CAAC;QACvC,OAAO,EAAE,QAAQ,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC;IACjE,CAAC,CAAC,CAAC;IAEH,MAAM,OAAO,GAAuB,EAAE,CAAC;IACvC,KAAK,MAAM,EAAE,MAAM,EAAE,IAAI,MAAM,EAAE,CAAC;QAChC,KAAK,MAAM,CAAC,IAAI,MAAM,EAAE,CAAC;YACvB,MAAM,GAAG,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,CAAC,KAAK,EAAE,MAAM,EAAE,CAAC,EAAE,MAAM,EAAE,CAAC,EAAE,CAAC,CAAC;YACzD,IAAI,CAAC,CAAC,EAAE;gBAAE,GAAG,CAAC,MAAM,EAAE,CAAC;;gBAClB,GAAG,CAAC,MAAM,EAAE,CAAC;QACpB,CAAC;IACH,CAAC;IAED,MAAM,KAAK,GAAqC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC,EAAE,CAAC,CAAC,CAAC;IAC7D,IAAI,cAAc,GAAG,CAAC,CAAC;IACvB,IAAI,SAAS,GAAG,CAAC,CAAC;IAClB,IAAI,yBAAyB,GAAG,CAAC,CAAC;IAClC,KAAK,MAAM,EAAE,QAAQ,EAAE,IAAI,MAAM,EAAE,CAAC;QAClC,MAAM,CAAC,GAAG,QAAQ,CAAC,OAAO,GAAG,CAAC,CAAC;QAC/B,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC;YAAE,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;QAC5D,mEAAmE;QACnE,qEAAqE;QACrE,8CAA8C;QAC9C,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC;YAC7C,IAAI,eAAe,CAAC,QAAQ,CAAC,OAAO,EAAE,CAAC,CAAC;gBAAE,cAAc,EAAE,CAAC;YAC3D,MAAM,YAAY,GAAG,cAAc,CAAC,QAAQ,CAAC,CAAC;YAC9C,IAAI,YAAY,EAAE,CAAC;gBACjB,SAAS,EAAE,CAAC;gBACZ,IAAI,eAAe,CAAC,YAAY,EAAE,CAAC,CAAC;oBAAE,yBAAyB,EAAE,CAAC;YACpE,CAAC;QACH,CAAC;IACH,CAAC;IAED,OAAO;QACL,MAAM;QACN,OAAO,EAAE;YACP,SAAS,EAAE,MAAM,CAAC,MAAM;YACxB,KAAK,EAAE,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,MAAM;YAC5C,OAAO;YACP,KAAK;YACL,cAAc;YACd,SAAS;YACT,yBAAyB;SAC1B;KACF,CAAC;AACJ,CAAC"}

package/dist/hooks/ask-label.js

@@ -1,6 +1,7 @@
 /**
  * PreToolUse(`AskUserQuestion`): an Eklavya question on a card host must carry
- * its `[Eklavya]` line.
+ * its `[Eklavya]` line, and on any host its options must not give the answer
+ * away (`visibleOptionProblem`, sent back once per stem).
  *
  * The plan's `ask_attribution` asks for it, and a model can read past an
  * instruction. This is the check behind the instruction: a question headed
@@ -12,27 +13,49 @@
  * Fails open like every hook, and stays on the lightest imports.
  */
 import { run } from './lib.js';
+import { createHash } from 'node:crypto';
+import fs from 'node:fs';
+import os from 'node:os';
+import path from 'node:path';
 import { needsInlineAttribution } from '../surface.js';
+import { visibleOptionProblem } from '../eval/question-checks.js';
 const PREFIX = /^\s*\[eklavya\]/i;
+function deny(reason) {
+    process.stdout.write(`${JSON.stringify({ hookSpecificOutput: { hookEventName: 'PreToolUse', permissionDecision: 'deny', permissionDecisionReason: reason } })}\n`);
+}
+/**
+ * True the first time a stem is seen, false after: the rewrite is asked for
+ * once, and a question that comes back with the same stem is shown as written.
+ * A marker file in the temp directory is the whole state; losing it only means
+ * one more request for a rewrite.
+ */
+function firstSight(sessionId, stem) {
+    const key = createHash('sha1').update(`${String(sessionId)}\n${stem}`).digest('hex').slice(0, 16);
+    const marker = path.join(os.tmpdir(), `eklavya-ask-${key}`);
+    if (fs.existsSync(marker))
+        return false;
+    fs.writeFileSync(marker, '');
+    return true;
+}
 await run(async (input) => {
-    if (input.tool_name !== 'AskUserQuestion' || input.agent_id || !needsInlineAttribution())
+    if (input.tool_name !== 'AskUserQuestion' || input.agent_id)
         return 0;
     const questions = input.tool_input?.questions;
     if (!Array.isArray(questions))
         return 0;
-    const unsigned = questions.some((q) => typeof q?.header === 'string' &&
-        q.header.trim().toLowerCase() === 'eklavya' &&
-        typeof q.question === 'string' &&
-        !PREFIX.test(q.question));
-    if (!unsigned)
+    const ours = questions.filter((q) => typeof q?.header === 'string' && q.header.trim().toLowerCase() === 'eklavya' && typeof q.question === 'string');
+    if (needsInlineAttribution() && ours.some((q) => !PREFIX.test(q.question))) {
+        deny('This host draws no header chip, so an Eklavya question must open with "[Eklavya]" on its own line, then the stem. Ask the same question again with that line added; change nothing else.');
         return 0;
-    process.stdout.write(`${JSON.stringify({
-        hookSpecificOutput: {
-            hookEventName: 'PreToolUse',
-            permissionDecision: 'deny',
-            permissionDecisionReason: 'This host draws no header chip, so an Eklavya question must open with "[Eklavya]" on its own line, then the stem. Ask the same question again with that line added; change nothing else.',
-        },
-    })}\n`);
+    }
+    for (const q of ours) {
+        const options = Array.isArray(q.options) ? q.options.filter((o) => typeof o?.label === 'string') : [];
+        const problem = visibleOptionProblem(options);
+        if (problem && firstSight(input.session_id, q.question)) {
+            deny(`Rewrite the options and ask again with the same question: ${problem}.`);
+            return 0;
+        }
+    }
     return 0;
 });
 //# sourceMappingURL=ask-label.js.map