@codio-ai/opencode-authoring-agent 0.1.0

package/src/skills/reference-assessment-types/SKILL.md

@@ -0,0 +1,309 @@
+---
+name: reference-assessment-types
+description: Payload field reference for multiple-choice, code-output-compare (Standard Code Test), fill-in-the-blanks, parsons-puzzle, and llm-based-auto-rubric assessments. Use when building any assessment payload.
+---
+
+# Assessment types
+
+Five Codio assessment types are supported by the `assessment-author` subagent. All five share the same orchestration shape:
+
+1. The agent produces a **payload** (only the creative parts — name, instructions, answer content, test cases, etc.).
+2. The orchestrator pipes the payload to the `create_assessment` tool (handled by the `create_assessment` tool), which stamps it into the canonical template, generates the taskId, writes `<workspace>/.guides/assessments/<taskId>.json`, and (for code tests) writes the starter file. The tool prints the full taskId.
+3. The orchestrator splices an embed line into the parent page's markdown:
+
+   ```
+   {<short label> |assessment}(<taskId>)
+   ```
+
+TaskId convention is uniform: `<type>-<10-digit-numeric>` (no leading zero). The tool generates this — never hand-author taskIds.
+
+---
+
+## Tool invocation
+
+Assessment files are created by the `create_assessment` tool. Pass the payload as structured arguments. The tool exits non-zero on payload errors (missing required fields, malformed JSON, missing `.guides/` directory, MCQ without a correct answer, FITB without `<<<blank>>>` markers).
+
+---
+
+## Type: `multiple-choice`
+
+Single-correct multiple-choice with four options (one correct, three plausible distractors). MCQ pedagogy lives in [`mcq-quality.md`](mcq-quality.md) — read it before authoring.
+
+### Payload
+
+```json
+{
+  "name": "Question display name",
+  "instructions": "Question text shown to the student.",
+  "answers": [
+    { "answer": "Correct answer text",  "correct": true  },
+    { "answer": "Distractor 1" },
+    { "answer": "Distractor 2" },
+    { "answer": "Distractor 3" }
+  ],
+  "guidance": "One sentence why correct, plus one sentence per distractor on the misconception.",
+  "lo": "SWBAT ...",
+  "blooms": "2",
+  "tags": []
+}
+```
+
+The tool fills: `_id` UUIDs per answer, `taskId`, `metadata.tags` (prepends `Assessment Type: Multiple Choice`), all schema defaults (`isRandomized: true`, `points: 20`, etc.).
+
+### Optional payload fields
+
+- `multiple_response` — `true` if more than one answer is correct. Default `false`.
+- `randomize` — answer-order shuffle. Default `true`.
+- `points` — default `20`.
+- `max_attempts` — default `1`.
+
+---
+
+## Type: `code-output-compare` (Standard Code Test)
+
+Runs student code with one or more test cases (stdin / argv / expected stdout). Requires a workspace file the student edits in the left panel of a 2-panel layout.
+
+**Layout requirement:** the parent page must be `2-panels-tree` (or another 2-panel layout) with the starter file opened in panel 0. The orchestrator switches the page layout if it isn't already correct, then calls this tool.
+
+### Payload
+
+```json
+{
+  "name": "Short test name",
+  "instructions": "Submit your code when ready",
+  "command": "python3 code/path/to/student-file.py",
+  "guidance": "Markdown-formatted explanation, may include a worked solution in a fenced block.",
+  "lo": "SWBAT ...",
+  "blooms": "3",
+
+  "starter_file_path": "code/path/to/student-file.py",
+  "starter_file_content": "# DO NOT ALTER THIS CODE\n<scaffolding>\n\n# Write your code here\n",
+
+  "test_cases": [
+    { "arguments": "1,2,3*4,5,6",  "input": "", "output": "21", "feedback": "" },
+    { "arguments": "10,20*30,40", "input": "", "output": "100", "feedback": "" }
+  ],
+
+  "tags": [
+    { "name": "Programming Language", "value": "Python" },
+    { "name": "Category",             "value": "Inheritance" },
+    { "name": "Content",              "value": "Extending the parent class" }
+  ]
+}
+```
+
+Tool fills: `taskId`, `metadata.files` and `metadata.opened` from `starter_file_path`, `Assessment Type: Standard Code Test` tag, and option defaults (see below). Writes `starter_file_content` to `<workspace>/<starter_file_path>` **only if the file does not already exist** — so re-running on an existing exercise won't overwrite student work.
+
+### Authoring the starter file
+
+The starter file is the scaffolding students extend. It typically has three parts:
+
+1. A `# DO NOT ALTER THIS CODE` block with provided helpers, parsing logic, or parent classes.
+2. A blank section with a `# Write your code here` marker.
+3. (Optional) Print/output statements that produce the test-case outputs.
+
+Make starter and test cases coherent: the test arguments must reach the student's code through the existing scaffolding, and the expected outputs must be reachable from a correct student solution.
+
+### Test-case design
+
+- **3 cases minimum, 5 typical.** First case is "happy path", subsequent cases probe edges (empty input, large numbers, boundary conditions).
+- `arguments` is the program's argv (single string; the student's code is responsible for parsing it).
+- `input` is stdin (used when the program reads from stdin).
+- `output` is the **exact stdout** expected. The default options (`ignoreWhitespaces`, `ignoreNewline`, `ignoreCase`, `matchSubstring` all `true`) make matching forgiving but not arbitrary.
+- `feedback` is shown only if `show_feedback: true` on that case — usually leave empty.
+
+### Optional payload fields
+
+- `partial_points` — default `true`.
+- `points` — default `20`.
+- `one_time_test` — default `true`. Set `false` to allow re-submission.
+- `pre_execute_command` — runs before `command`. Use for setup (e.g. installing a package). Default empty.
+- `options` — overrides the default matching options:
+  ```json
+  "options": {
+    "ignoreCase": true, "ignoreNewline": true, "ignoreWhitespaces": true,
+    "matchSubstring": true, "timeout": 30
+  }
+  ```
+- `show_expected_answer`, `show_guidance_after_response`, `show_name` — default `true`, `true`, `false`.
+
+---
+
+## Type: `fill-in-the-blanks`
+
+Cloze-style questions. Students fill blanks (free text or pick from a distractor pool).
+
+### Payload
+
+```json
+{
+  "name": "Internal name",
+  "instructions": "**Fill in the blanks below.**",
+  "text": "* A <<<child>>> class inherits attributes from a parent class\n* The <<<super()>>> function calls the parent's method",
+  "distractors": "sibling\nself\nbase",
+  "guidance": "Explains the correct answer for each blank.",
+  "lo": "SWBAT ...",
+  "blooms": "",
+  "tags": []
+}
+```
+
+The `text` field uses `<<<answer>>>` to mark each blank. The tool parses these into the canonical `tokens.blank` array (the answers in order) and `tokens.text` array (alternating string segments and `0` placeholders) automatically — never hand-build the tokens object.
+
+`distractors` is a newline-separated list of additional wrong-answer options shown in the distractor pool (Codio mixes them with the correct answers when rendering). One per line.
+
+### Pedagogy
+
+- **One concept per blank.** A blank should test a single fact or term, not a long phrase.
+- **Cap at 6 blanks** per question. More than that and students lose track.
+- **Distractors should be plausible siblings** of the correct answers — same grammatical category, same domain, common misconceptions.
+- **Don't make blanks trivially copy-pasted from earlier sentences** in the same `text`.
+
+### Optional payload fields
+
+- `points` — default `20`.
+- `max_attempts` — default `1`.
+- `partial_points` — default `false`. Set `true` to score per-blank.
+- `show_name`, `show_values` — default `true`, `true`.
+
+---
+
+## Embedding
+
+After the tool returns the `<taskId>`, the orchestrator inserts one line into the target page's markdown:
+
+```
+{<short label> |assessment}(<taskId>)
+```
+
+When inserting more than one assessment into a page, group them under a single `## Check your understanding` heading (preceded by a blank line). Order them in increasing Bloom's level when the page covers a progression.
+
+The orchestrator does the embedding — `assessment-author` only writes JSON.
+
+---
+
+## Type: `parsons-puzzle`
+
+Students drag-and-drop code blocks into the correct order. Supports indentation for Python. The `options` field is a structured object at input time; the tool serializes it as a JSON-encoded string in the output JSON (Codio expects a string there).
+
+### Payload
+
+```json
+{
+  "name": "Python Loop Structure",
+  "instructions": "Arrange the code blocks to produce the correct output.",
+  "initial": "for i in range(2, 11):\n    if i % 2 == 0:\n        print(i)\nfor i in range(2, 10):  #distractor\nif i % 2 == 1:  #distractor",
+  "options": {
+    "max_wrong_lines": 10,
+    "python3": true,
+    "lang": "en"
+  },
+  "grader": "1",
+  "guidance": "The correct solution uses range(2, 11) to include 10...",
+  "lo": "SWBAT correctly structure a for loop in Python",
+  "blooms": "3",
+  "points": 20
+}
+```
+
+The tool fills: `taskId`, `metadata.tags` (prepends `Assessment Type: Parsons Puzzle`), all `options` defaults, and serializes `options` to a JSON string. `metadata.files` and `metadata.opened` are always empty arrays for Parsons puzzles.
+
+### `options` fields (all optional, defaults shown)
+
+| Field | Default | Description |
+|---|---|---|
+| `sortableId` | `"sortable"` | DOM id of the sortable answer area |
+| `max_wrong_lines` | `10` | Maximum distractor lines shown |
+| `exec_limit` | `2500` | Execution time limit (ms) |
+| `can_indent` | `true` | Whether students can indent blocks |
+| `x_indent` | `50` | Pixel width of one indent level |
+| `lang` | `"en"` | UI language |
+| `show_feedback` | `true` | Whether to show correctness feedback |
+| `python3` | `true` | Python 3 mode |
+| `trashId` | `"sortableTrash"` | DOM id of the trash/unused-blocks area |
+
+### Optional payload fields
+
+- `grader` — default `"1"`.
+- `points` — default `20`.
+- `max_attempts` — default `0` (unlimited).
+- `use_maximum_score` — default `false`.
+- `show_name` — default `true`.
+- `show_guidance_after_attempts` — default `2`.
+
+### Distractor design (Parsons)
+
+- Place the correct solution lines first in `initial`, then append distractors each ending in `  #distractor`.
+- Aim for **2–4 distractors** per puzzle — enough to require careful thinking without overwhelming.
+- Choose distractors that reflect **real, common mistakes**: off-by-one in range bounds, wrong relational operator (`==` vs `!=`), wrong variable name, wrong indentation level.
+- For Python puzzles, the indentation of each block in `initial` matters — the tool passes it verbatim to Codio's renderer, which uses it to determine the block's starting indent level.
+
+---
+
+## Type: `llm-based-auto-rubric`
+
+An open-ended question graded by an LLM using a rubric. The grader receives the student's submission alongside the rubric items and a reference solution. All solution files go under `.guides/secure/` — never `.guides/content/` (which is student-visible).
+
+### Payload
+
+```json
+{
+  "name": "Test",
+  "instructions": "Write a Python program that prints all even numbers from 2 to 10.",
+  "files": ["loops.py"],
+  "rubrics": [
+    {
+      "title": "Loop Structure and Syntax",
+      "description": "Evaluates whether the student correctly implements loop structures with proper syntax including correct indentation, colons, and loop variable initialization.",
+      "points": 1
+    },
+    {
+      "title": "Loop Logic and Control Flow",
+      "description": "Assesses the student's understanding of loop control flow including proper use of loop conditions, iteration variables, and logical flow.",
+      "points": 1
+    }
+  ],
+  "rubrics_requirements": "rubrics to test loops application",
+  "solutions": [
+    {
+      "path": ".guides/secure/solution.py",
+      "content": "for i in range(2, 11):\n    if i % 2 == 0:\n        print(i)\n"
+    }
+  ],
+  "guidance": "The correct solution uses range(2, 11) inclusive...",
+  "lo": "SWBAT use for loops with conditional logic",
+  "blooms": "3",
+  "points": 5
+}
+```
+
+The tool fills: `taskId`, `metadata.tags` (always empty — no auto-tag for this type), `metadata.files` (the `files` array), `metadata.opened` (one `{type: "file", panelNumber: 0, content: path}` per file), auto-generated string `id` values for each rubric item (`"1"`, `"2"`, ...). The tool also writes each solution to its `path` relative to the workspace, using the same no-overwrite atomic pattern as code-output-compare starter files.
+
+### `metadata.opened` key
+
+Note: auto-rubric uses `panelNumber` (not `panel`) in the opened entries.
+
+### Optional payload fields
+
+- `files` — paths opened in the student's panel. Default `[]`.
+- `rubrics_requirements` — free-text requirements context for the LLM grader. Default `""`.
+- `solutions` — reference solutions written to `.guides/secure/`. Default `[]`.
+- `points` — default `5`.
+- `partial_points` — default `true`.
+- `max_attempts` — default `0` (unlimited).
+- `show_name` — default `false`.
+- `show_guidance_after_attempts` — default `2`.
+
+### Rubric item authoring
+
+- Write **3–5 rubric items** per question, each scoped to one observable dimension.
+- Good dimensions: syntax correctness, algorithmic logic, boundary/edge case handling, output format, code style/readability.
+- `title` — short label (4–8 words). The LLM grader sees this.
+- `description` — concrete grading criteria written for the LLM (not the student). Be specific: "checks whether the loop uses `range(2, 11)` or equivalent to include 10" beats "correct loop bounds."
+- Each rubric item's `points` typically equals `total_points / rubric_count` (integer division). All items summing to `points` is the convention.
+
+### Solution path discipline
+
+- All solutions: `.guides/secure/<filename>` — hidden from students.
+- Never: `.guides/content/<anything>` — student-visible.
+- The tool writes the file only if it does not already exist (safe to re-run).

package/src/skills/reference-authoring-notes/SKILL.md

@@ -0,0 +1,297 @@
+---
+name: reference-authoring-notes
+description: Codio guide JSON schemas, markdown conventions, button syntax, callout block types, folder layout. Use when authoring or validating page JSON or markdown.
+---
+
+# Codio authoring notes
+
+## Purpose
+Use this reference when generating Codio guide assignments to produce output that matches Codio's import format exactly.
+
+---
+
+## Folder structure
+
+All guide files live under `.guides/`:
+
+```
+.guides/
+├── content/        # index.json (root config) + page/section/chapter files and folders
+├── assessments/    # one JSON file per assessment
+├── img/            # image assets (referenced as .guides/img/filename.png)
+└── secure/         # files hidden from students
+```
+
+Workspace files (source code, data files, etc.) live outside `.guides/` in normal project directories.
+
+---
+
+## Root configuration: `.guides/content/index.json`
+
+```json
+{
+  "title": "Assignment Title",
+  "theme": "light",
+  "order": ["page-stem-xxxx", "chapter-folder-xxxx"],
+  "useSubmitButtons": true,
+  "useMarkAsComplete": false,
+  "hideMenu": false,
+  "allowGuideClose": false,
+  "protectLayout": false,
+  "suppressPageNumbering": true,
+  "collapsedOnStart": false,
+  "hideSectionsToggle": false,
+  "hideBackToDashboard": false
+}
+```
+
+The `order` array lists stems (for pages) or folder names (for sections/chapters) in reading sequence.
+
+---
+
+## Page files
+
+Each page is a pair of files with the same stem, e.g. `intro-to-loops-a1b2`:
+
+- `.guides/content/intro-to-loops-a1b2.json` — metadata
+- `.guides/content/intro-to-loops-a1b2.md` — markdown content
+
+### Page JSON
+
+```json
+{
+  "id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
+  "title": "Page Title",
+  "type": "page",
+  "contentType": "markdown",
+  "layout": "1-panel",
+  "files": [],
+  "teacherOnly": false,
+  "learningObjectives": ""
+}
+```
+
+**Layout options:** `1-panel`, `2-panels-tree`, `3-cell`, `3-columns-guides-left`, `4-cell`
+
+Use `2-panels-tree` when the page interacts with workspace files; `1-panel` for conceptual/text-only pages.
+
+### Files array
+
+Controls which files and terminals open when the student lands on the page:
+
+```json
+"files": [
+  { "path": "solution.py", "panel": 0, "action": "open" }
+]
+```
+
+Special path values: `#tabs` targets the open editor group; `#terminal:` opens a terminal. Panel indices: 0–3.
+
+---
+
+## Sections and chapters
+
+Sections and chapters are optional containers. Each lives in its own subfolder with an `index.json`:
+
+**`.guides/content/chapter-intro-xxxx/index.json`**
+
+```json
+{
+  "id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
+  "title": "Chapter Title",
+  "type": "chapter",
+  "order": ["page-stem-xxxx", "page-stem-yyyy"],
+  "files": [],
+  "layout": "1-panel",
+  "teacherOnly": false,
+  "closeTerminalSession": false,
+  "learningObjectives": ""
+}
+```
+
+Use `"type": "section"` for a lower-level grouping or `"type": "chapter"` for a top-level grouping. Containers do not need their own markdown unless they display content directly.
+
+---
+
+## Multiple-choice assessment JSON
+
+**File location:** `.guides/assessments/multiple-choice-<10-digit-number>.json`
+
+```json
+{
+  "type": "multiple-choice",
+  "taskId": "multiple-choice-1234567890",
+  "source": {
+    "name": "Question display name",
+    "showName": true,
+    "instructions": "Question text shown to the student.",
+    "multipleResponse": false,
+    "isRandomized": true,
+    "answers": [
+      { "_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "correct": true,  "answer": "Correct answer text" },
+      { "_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "correct": false, "answer": "Distractor 1" },
+      { "_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "correct": false, "answer": "Distractor 2" },
+      { "_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx", "correct": false, "answer": "Distractor 3" }
+    ],
+    "metadata": {
+      "tags": [{ "name": "Assessment Type", "value": "Multiple Choice" }],
+      "files": [],
+      "opened": []
+    },
+    "bloomsObjectiveLevel": "",
+    "learningObjectives": "",
+    "guidance": "",
+    "showGuidanceAfterResponseOption": { "type": "Attempts", "passedFrom": 1 },
+    "maxAttemptsCount": 1,
+    "showExpectedAnswerOption": { "type": "Always" },
+    "points": 20,
+    "incorrectPoints": 0,
+    "arePartialPointsAllowed": false,
+    "useMaximumScore": false
+  }
+}
+```
+
+**Embed the assessment in a page's markdown:**
+
+```
+{Question Label |assessment}(multiple-choice-1234567890)
+```
+
+### Field reference
+
+| Field | Options |
+|---|---|
+| `multipleResponse` | `false` = single correct answer, `true` = multiple correct |
+| `isRandomized` | `true` to shuffle answer order (default) |
+| `showGuidanceAfterResponseOption.type` | `"Attempts"`, `"Score"`, `"Always"`, `"Never"`, `"WhenGradesReleased"` |
+| `showExpectedAnswerOption.type` | `"Always"`, `"Never"`, `"WhenGradesReleased"` |
+| `maxAttemptsCount` | `0` = unlimited, `1`, `9`, or any integer |
+| `points` | `0`, `20`, `50`, or custom integer |
+| `bloomsObjectiveLevel` | `""`, `"1"`, `"2"` |
+
+---
+
+## Custom buttons (process launch only — no section buttons)
+
+Embed directly in page markdown. Three variants:
+
+```
+{Button Label}(command arg1 arg2)
+{Button Label | terminal}(command)
+{Button Label | background}(command)
+```
+
+- **Default:** runs the command and shows output inline within the guide panel.
+- **`| terminal`:** opens a terminal window — required when the program reads standard input.
+- **`| background`:** launches the command asynchronously without blocking the guide.
+
+Examples:
+
+```
+{Run Code}(python3 solution.py)
+{Compile & Run | terminal}(javac Main.java && java Main)
+{Run Tests}(bash run_tests.sh)
+```
+
+Do not add section buttons. Only add process-launch buttons when the assignment includes runnable code or commands that materially support the learning activity.
+
+---
+
+## Markdown styles
+
+### Headings
+
+```
+# Heading 1
+## Heading 2
+### Heading 3
+```
+
+### Emphasis
+
+```
+**bold**   *italic*
+```
+
+### Lists
+
+```
+- Bullet item
+- Another item
+  - Nested (2-space indent)
+
+1. Numbered item
+1. Auto-increments
+```
+
+### Code
+
+Fenced block with language label:
+
+    ```python
+    def greet(name):
+        return f"Hello, {name}"
+    ```
+
+Suppress copy button:
+
+    ```python `-hide-clipboard`
+    # students cannot copy this block
+    ```
+
+Inline: `` `variable_name` ``
+
+### Math
+
+Block (LaTeX):
+
+    ```math
+    F = G \frac{m_1 m_2}{r^2}
+    ```
+
+Inline: `$a \ne 0$`
+
+Multi-line: `$$...$$`
+
+### Callout blocks
+
+```
+|||info
+# Title
+Content here
+
+|||
+
+|||important
+Content here
+
+|||
+
+|||warning
+Content here
+
+|||
+```
+
+Supported types: `important`, `info`, `warning`, `topic`, `definition`, `challenge`, `guidance`, `debugging`.
+
+### Collapsible content
+
+```html
+<details>
+<summary>Click to expand</summary>
+
+Hidden content here.
+
+</details>
+```
+
+---
+
+## File output rules
+
+- All guide files are written directly into the live Codio workspace under `.guides/content/` and `.guides/assessments/`.
+- Filenames, folder names, and stems must be consistent across `index.json` order arrays and paired files.
+- When revising an existing assignment, preserve structure unless the requested change requires reorganisation.
+- This tooling itself lives in `.guides/secure/` and never appears in student-visible output.

package/src/skills/reference-mcq-quality/SKILL.md

@@ -0,0 +1,56 @@
+---
+name: reference-mcq-quality
+description: MCQ pedagogy and distractor quality rules. Use when authoring multiple-choice questions or reviewing them.
+---
+
+# MCQ pedagogy and distractor quality
+
+This reference is loaded by `assessment-author` (when authoring multiple-choice questions) and `validator`. It defines what makes a Codio MCQ a *good* MCQ — not just well-formatted JSON.
+
+## Question stem
+
+- Tests understanding, not trivial wording recall. Avoid questions that can be answered by string-matching the most recent paragraph of the lesson.
+- One clearly stated problem per question. No nested clauses, no multi-part questions joined by "and".
+- Use positive phrasing. If you must use negation, capitalise the negation word ("Which is **NOT** an example of…") so students don't miss it.
+- Match the cognitive level to the learning objective: recall ≠ apply ≠ analyse. If the LO is "apply X", the stem should require applying X, not naming X.
+
+## Correct answer
+
+- Exactly one correct answer (unless the user explicitly requested `multipleResponse: true`).
+- Unambiguous — a domain expert reading the question with no other context picks the same answer every time.
+- Not the longest option by a noticeable margin (a common test-taking heuristic exploits this).
+
+## Distractors
+
+- All distractors must be **plausibly wrong** — they reflect a real misconception a learner might hold, not a random nonsense option.
+- Aim for one distractor per common misconception or error pattern relevant to the topic. If you can't think of three real misconceptions, the question may not be worth asking.
+- Distractors should be roughly the same length and grammatical form as the correct answer.
+- Avoid:
+  - "All of the above" / "None of the above" unless there's a strong pedagogical reason.
+  - Joke options or absurd distractors.
+  - Distractors that are technically also correct under some interpretation.
+  - Distractors that overlap (e.g. "between 1 and 10" and "between 5 and 8" — students who know the answer is 7 can't choose).
+
+## Guidance text
+
+- Always populate `guidance` when the topic supports it.
+- Format: one sentence on why the correct answer is right, followed by one short sentence per distractor explaining the specific misconception it represents.
+- Don't restate the question. Don't lecture. Treat it as a hint a tutor would whisper after the student commits an answer.
+
+## Set-level rules (across all MCQs in an assignment)
+
+- **Answer-position balance.** Authored correct-answer positions should be roughly uniform across A/B/C/D. With `isRandomized: true` Codio shuffles at runtime, but clustering at authoring time is still a code smell — usually a sign the author wrote correct answers first and distractors second.
+- **Difficulty consistency.** Don't mix trivial recall with deep application questions in one assessment unless the user explicitly asked for a difficulty progression.
+- **No content-leak across questions.** A later question's stem shouldn't give away an earlier question's answer.
+- **No duplicates.** Two MCQs that test the same fact in slightly different words is a sign the assignment over-assesses one LO.
+
+## Common failure modes (validator should flag these)
+
+| Symptom | Why it's a problem |
+|---|---|
+| All distractors are obviously wrong | Question becomes a recall task with no thought required |
+| Distractors of very different lengths | Correct answer becomes guessable by length |
+| Two options say essentially the same thing | Confusing; one of them is wrong by definition |
+| Question stem contains the answer (e.g. "What is the *primary* reason X happens, primarily?") | Trivially solvable |
+| Negation buried in the middle of the stem | Students misread; failure becomes about reading, not knowledge |
+| Same correct-answer position 4+ times in a row | Authoring laziness; student pattern-matching |