@jutge.org/toolkit 4.4.17 → 4.4.20

package/docs/quiz-anatomy.md

@@ -4,17 +4,13 @@ This document describes the anatomy of a **quiz problem** in [Jutge.org](https:/
 
 ## Terminology
 
-A **quiz** is a problem whose handler is set to `quiz`. It is made of a **quiz root** and a list of **questions**. The quiz root is defined in `quiz.yml` and holds the quiz title, statement, whether questions are shuffled, and the list of questions with their scores. Each **question** is defined in a separate YAML file (e.g. `single-choice.yml`) and has a **type**: SingleChoice, MultipleChoice, FillIn, Ordering, Matching, or OpenQuestion.
+A **quiz** is a problem whose handler is set to `quiz`. It is made of a **quiz root** and a list of **questions**. The quiz root is defined in `quiz.yml` and holds the quiz title, author, statement, whether questions are shuffled, and the list of questions with their scores. Each **question** is defined in a separate YAML file (e.g. `single-choice.yml`) and has a `type`: SingleChoice, MultipleChoice, FillIn, Ordering, Matching, or OpenQuestion.
 
 Quiz content can be **localized**: the same quiz can have different `quiz.yml` and question files per language (e.g. under `en/` and `ca/`). The toolkit runs or lints the quiz for a given directory, so you typically run it from a language-specific subdirectory. Each language should live inside its own folder.
 
-**Variable substitution** allows question text and options to depend on values generated at run time. If a question file is named `example.yml`, the toolkit looks for `example.py` in the same directory. When present, it runs the Python script with a random **seed** and collects the script’s global variables. Those variables can be referenced in the question YAML with `$name` or `${name}`. This makes it possible to have different numbers, strings, or options for each run while keeping the same correct answer logic (e.g. “What is $a + $b?” with `a` and `b` random).
+**Variable substitution** allows question text and options to depend on values generated at run time. If a question file is named `example.yml`, the toolkit looks for `example.py` in the same directory. When present, it runs the Python script with a random `seed` and collects the script’s global variables. Those variables can be referenced in the question YAML with `$name` or `${name}`. This makes it possible to have different numbers, strings, or options for each run while keeping the same correct answer logic (e.g. “What is $a + $b?” with `a` and `b` random).
 
-**Scoring**: Each question has a **score** between 0–100, and the total of all question scores listed in `quiz.yml` must add up to 100. Users earn points for each question based on question type and that question’s **partial_answer** setting. The **partial_answer** option is set per question in the question YAML (not in `quiz.yml`):
-
-- If `partial_answer` is set to `false` (default), users get full points for that question only when their answer is completely correct; any mistake gives zero points for that question.
-
-- If `partial_answer` is set to `true`, users can receive partial points for that question when the answer is partially correct (e.g. proportional to how many parts are right), and the response may still be marked as "correct" if at least one part is right.
+**Scoring**: Each question has a `score` between 0–100, and the total of all question scores listed in `quiz.yml` must add up to 100. Users earn points for each question.
 
 ## Quiz structure
 
@@ -43,25 +39,26 @@ You run or lint the quiz from the directory that contains `quiz.yml` (e.g. `jtk
 
 `yml` files are YAML (YAML Ain't Markup Language) files. YAML is a human-readable data-serialization language; see [YAML documentation](https://yaml.org/) for more details. Also, see [YAML multiline info](https://yaml-multiline.info/) for more details on how to write multiline strings in YAML.
 
-Many items are written in Markdown. See [Markdown documentation](https://www.markdownguide.org/) for more details. In addition, you can use a small subset of LaTeX for mathematical expressions but these have to be enclosed between `·` signs, not standard `$` signs.
-
 ## The `quiz.yml` file
 
 The file `quiz.yml` defines the quiz root.
 
-- **title**: Title of the quiz.
-- **statement**: Short description or instructions shown for the quiz (Markdown).
-- **questions**: List of question entries. Each entry has:
-    - **title**: Title of the question (e.g. for display in a table of contents).
-    - **file**: Base name of the question file, without the `.yml` extension (e.g. `question` for `question.yml`).
-    - **score**: Integer from 0 to 100. The sum of all `score` values in the list must be 100.
-- **shuffle** (optional): Whether to shuffle the order of questions when running the quiz. Defaults to `false`.
+- `title`: Title of the quiz.
+- `author`: Author of the quiz.
+- `statement`: Short description or instructions shown for the quiz (Markdown).
+- `questions`: List of question entries. Each entry has:
+    - `title`: Title of the question (e.g. for display in a table of contents).
+    - `file`: Base name of the question file, without the `.yml` extension (e.g. `question` for `question.yml`).
+    - `score`: Integer from 0 to 100. The sum of all `score` values in the list must be 100.
+- `shuffle` (optional): Whether to shuffle the order of questions when running the quiz. Defaults to `false`.
 
 ### Example
 
 ```yaml
 title: Demo quiz
 
+author: Jordi Petit
+
 statement: This quiz showcases the possibilities of the quiz problems at Jutge.org.
 
 shuffle: true
@@ -90,16 +87,24 @@ questions:
 
 ## Question types
 
-Each question is stored in a YAML file whose name matches the `file` field in `quiz.yml` (e.g. `question.yml`). The file must contain a **type** field that identifies the kind of question. All question types support an optional **hide_score** (default `false`) and an optional **partial_answer** (default `false`); see [Scoring](#terminology) above. Variable substitution applies to text fields and options when a corresponding `.py` file exists.
+Each question is stored in a YAML file whose name matches the `file` field in `quiz.yml` (e.g. `question.yml`). The file must contain a `type` field that identifies the kind of question. Variable substitution applies to text fields and options when a corresponding `.py` file exists. All question types support an optional `hide_score` (default `false`) and an optional `partial_answer` (default `false`).
+
+The `partial_answer` option is set per question in the question YAML:
+
+- If `partial_answer` is set to `false` (default), users get full points for that question only when their answer is completely correct; any mistake gives zero points for that question.
+
+- If `partial_answer` is set to `true`, users can receive partial points for that question when the answer is partially correct (e.g. proportional to how many parts are right), and the response may still be marked as "correct" if at least one part is right.
+
+The `hide_score` option is set per question in the question YAML. If set to `true`, the question score is not shown to the user.
 
 ### SingleChoice
 
-One correct option among several. Exactly one choice must have `correct: true`. Choices can be shuffled (optional **shuffle**, default `true`). Each choice can have an optional **hint**. Duplicate choice text is not allowed.
+One correct option among several. Exactly one choice must have `correct: true`. Choices can be shuffled (optional `shuffle`, default `true`). Each choice can have an optional `hint`. Duplicate choice text is not allowed.
 
-- **text**: Question text (supports `$var` and `${var}`).
-- **choices**: List of `{ text, correct?, hint? }`. One and only one choice must have `correct: true`.
-- **shuffle** (optional): Whether to shuffle choices. Defaults to `true`.
-- **partial_answer** (optional): Whether to award partial credit for this question. Defaults to `false`.
+- `text`: Question text (supports `$var` and `${var}`).
+- `choices`: List of `{ text, correct?, hint? }`. One and only one choice must have `correct: true`.
+- `shuffle` (optional): Whether to shuffle choices. Defaults to `true`.
+- `partial_answer` (optional): Whether to award partial credit for this question. Defaults to `false`.
 
 Example:
 
@@ -121,12 +126,12 @@ Variables `a`, `b`, `s1`, `s2`, `s3` would be produced by a `single-choice.py` s
 
 ### MultipleChoice
 
-Zero or more correct options. Multiple choices can have `correct: true`. Choices can be shuffled (optional **shuffle**, default `true`).
+Zero or more correct options. Multiple choices can have `correct: true`. Choices can be shuffled (optional `shuffle`, default `true`).
 
-- **text**: Question text (supports variables).
-- **choices**: List of `{ text, correct?, hint? }`.
-- **shuffle** (optional): Whether to shuffle choices. Defaults to `true`.
-- **partial_answer** (optional): Whether to award partial credit for this question. Defaults to `false`.
+- `text`: Question text (supports variables).
+- `choices`: List of `{ text, correct?, hint? }`.
+- `shuffle` (optional): Whether to shuffle choices. Defaults to `true`.
+- `partial_answer` (optional): Whether to award partial credit for this question. Defaults to `false`.
 
 Example:
 
@@ -146,19 +151,19 @@ choices:
 
 ### FillIn
 
-One or more blanks in a text or code block. Each blank is identified by a placeholder (e.g. `S1`, `XXXX`) and has a correct answer and optional options (dropdown). If **options** are given, the correct answer must be one of them.
+One or more blanks in a text or code block. Each blank is identified by a placeholder (e.g. `S1`, `XXXX`) and has a correct answer and optional options (dropdown). If `options` are given, the correct answer must be one of them.
 
-- **text**: Question or instructions (supports variables).
-- **context**: Text containing placeholders (e.g. `S1`, `S2`, `XXXX`). Placeholders are the keys in **items**.
-- **items**: Map from placeholder name to an item object:
-    - **correct**: Correct answer (string).
-    - **options** (optional): List of strings; if present, the blank is shown as a dropdown and **correct** must be in this list.
-    - **maxlength** (optional): Max length for the answer. Defaults to 100.
-    - **placeholder** (optional): Placeholder text in the input (e.g. `"?"`).
-    - **ignorecase** (optional): Whether to ignore case when checking. Defaults to `true`.
-    - **trim** (optional): Whether to trim spaces. Defaults to `true`.
-    - **partial_answer** (optional): Whether this blank contributes to partial credit for the question. Defaults to `false`.
-- **partial_answer** (optional, at question level): Whether to award partial credit for this question. Defaults to `false`.
+- `text`: Question or instructions (supports variables).
+- `context`: Text containing placeholders (e.g. `S1`, `S2`, `XXXX`). Placeholders are the keys in `items`.
+- `items`: Map from placeholder name to an item object:
+    - `correct`: Correct answer (string).
+    - `options` (optional): List of strings; if present, the blank is shown as a dropdown and `correct` must be in this list.
+    - `maxlength` (optional): Max length for the answer. Defaults to 100.
+    - `placeholder` (optional): Placeholder text in the input (e.g. `"?"`).
+    - `ignorecase` (optional): Whether to ignore case when checking. Defaults to `true`.
+    - `trim` (optional): Whether to trim spaces. Defaults to `true`.
+    - `partial_answer` (optional): Whether this blank contributes to partial credit for the question. Defaults to `false`.
+- `partial_answer` (optional, at question level): Whether to award partial credit for this question. Defaults to `false`.
 
 Example with dropdowns:
 
@@ -217,13 +222,13 @@ items:
 
 ### Ordering
 
-User must order a list of items (e.g. chronological order). Items can be shown in shuffled order (optional **shuffle**, default `true`).
+User must order a list of items (e.g. chronological order). Items can be shown in shuffled order (optional `shuffle`, default `true`).
 
-- **text**: Question text (supports variables).
-- **label**: Label for the list (e.g. “Programming language”).
-- **items**: List of strings in the **correct** order.
-- **shuffle** (optional): Whether to show items in random order. Defaults to `true`.
-- **partial_answer** (optional): Whether to award partial credit for this question. Defaults to `false`.
+- `text`: Question text (supports variables).
+- `label`: Label for the list (e.g. “Programming language”).
+- `items`: List of strings in the correct order.
+- `shuffle` (optional): Whether to show items in random order. Defaults to `true`.
+- `partial_answer` (optional): Whether to award partial credit for this question. Defaults to `false`.
 
 Example:
 
@@ -245,14 +250,14 @@ items:
 
 ### Matching
 
-Two columns: user matches each left item with one right item. Left and right lists can be shuffled (optional **shuffle**, default `true`).
+Two columns: user matches each left item with one right item. Left and right lists can be shuffled (optional `shuffle`, default `true`).
 
-- **text**: Question text (supports variables).
-- **labels**: Two strings, e.g. `["Countries", "Capitals"]`.
-- **left**: List of strings (e.g. countries).
-- **right**: List of strings (e.g. capitals), in the same order as **left** (left[i] matches right[i]).
-- **shuffle** (optional): Whether to shuffle left and right columns. Defaults to `true`.
-- **partial_answer** (optional): Whether to award partial credit for this question. Defaults to `false`.
+- `text`: Question text (supports variables).
+- `labels`: Two strings, e.g. `["Countries", "Capitals"]`.
+- `left`: List of strings (e.g. countries).
+- `right`: List of strings (e.g. capitals), in the same order as `left` (left[i] matches right[i]).
+- `shuffle` (optional): Whether to shuffle left and right columns. Defaults to `true`.
+- `partial_answer` (optional): Whether to award partial credit for this question. Defaults to `false`.
 
 Example:
 
@@ -284,9 +289,9 @@ right:
 
 Free-text answer with no automatic correction. Useful for open-ended or reflective answers.
 
-- **text**: Question text (supports variables).
-- **placeholder** (optional): Placeholder for the text area. Defaults to `""`. Supports variables.
-- **partial_answer** (optional): Whether to award partial credit for this question. Defaults to `false`.
+- `text`: Question text (supports variables).
+- `placeholder` (optional): Placeholder for the text area. Defaults to `""`. Supports variables.
+- `partial_answer` (optional): Whether to award partial credit for this question. Defaults to `false`.
 
 Example:
 
@@ -300,11 +305,30 @@ placeholder: 'My name is **$name** and I want to pass this course.'
 
 The variable `name` can be set by an optional `open-ended.py` (or the same base name as the question file) in the same directory.
 
+## Markdown usage
+
+Markdown is supported in most fields. See [Markdown documentation](https://www.markdownguide.org/) for more details.
+
+For quizzes, you may use:
+
+- Lists (e.g. `- Item 1`, `- Item 2`, `- Item 3`)
+- Italic text (e.g. `*italic text*`)
+- Bold text (e.g. `**bold text**`)
+- Underline text (e.g. `__underline text__`)
+- Code (with backticks, e.g. `` `code` ``)
+- Code blocks (with triple backticks and a programming language specifier)
+
+Avoid sections, links, and tables. HTML is not supported.
+
+In order to include images, you can use the `![Image description](FIG_DIR/image.png)` syntax. The `FIG_DIR` variable will be replaced by the URL where the images are stored. PNG and SVG images are supported.
+
+In addition, you can use a small subset of LaTeX for mathematical expressions but these have to be enclosed between `·` signs, not standard `$` signs. Take into account that math in quizzes is fragile, as it conflicts with Markdown and other features.
+
 ## Variable substitution (`.py` files)
 
 If a question file is named `example.yml`, the toolkit looks for `example.py` in the same directory. When present:
 
-1. The Python script is run with a given **seed** (passed as an argument by the toolkit) so that the run is reproducible.
+1. The Python script is run with a given `seed` (passed as an argument by the toolkit) so that the run is reproducible.
 2. The script’s global variables (that are JSON-serializable and whose names do not start with `__`) are collected.
 3. In the question YAML, any string field that supports substitution can use `$name` or `${name}` to be replaced by the value of `name`.
 
@@ -337,11 +361,11 @@ handler: quiz
 
 Other handler options (e.g. `std`, `graphic`) are for non-quiz problems. See [Problem anatomy — handler.yml](problem-anatomy.md#the-handleryml-file) for the full list of handler and option descriptions.
 
-## Running and linting quizzes
+## Linting, running and playing quizzes
 
 From the toolkit CLI:
 
-- **Lint** a quiz (validate `quiz.yml` and all referenced question YAML files):
+- `jtk quiz lint` — lint a quiz (validate `quiz.yml` and all referenced question YAML files):
 
     ```bash
     jtk quiz lint -d <directory>
@@ -349,12 +373,20 @@ From the toolkit CLI:
 
     Use the directory that contains `quiz.yml` (e.g. the `en` subdirectory).
 
-- **Run** a quiz (build questions, apply variables, output JSON or YAML):
+- `jtk quiz run` — run a quiz (build questions, apply variables, output JSON or YAML):
+
     ```bash
     jtk quiz run -d <directory> [-s <seed>] [-f json|yaml]
     ```
+
     If no seed is provided, a random one is used. Running the quiz applies variable substitution and, if `shuffle` is true, shuffles question order (and, per question, choices or ordering/matching items when their `shuffle` is true).
 
+- `jtk quiz play` — play a quiz in the terminal:
+    ```bash
+    jtk quiz play -d <directory> [-i <input>] [-o <output>] [-s <seed>]
+    ```
+    If no seed is provided, a random one is used. Playing the quiz applies variable substitution and, if `shuffle` is true, shuffles question order (and, per question, choices or ordering/matching items when their `shuffle` is true).
+
 ## Quick checklist
 
 - Use a problem folder with `handler: quiz` in `handler.yml`.
@@ -362,6 +394,6 @@ From the toolkit CLI:
 - Ensure every `file` in `quiz.yml` has a corresponding `file.yml` in the same directory.
 - Ensure question scores in `quiz.yml` sum to 100.
 - For SingleChoice, set exactly one choice with `correct: true`.
-- For FillIn with **options**, ensure **correct** is in the **options** list.
-- For Matching, ensure **left** and **right** have the same length and are in matching order.
+- For FillIn with `options`, ensure `correct` is in the `options` list.
+- For Matching, ensure `left` and `right` have the same length and are in matching order.
 - Use variable names in `$name` / `${name}` that are produced by the optional `file.py` script; the script is run with the toolkit’s seed for reproducibility.

package/package.json

@@ -1,7 +1,7 @@
 {
     "name": "@jutge.org/toolkit",
     "description": "Toolkit to prepare problems for Jutge.org",
-    "version": "4.4.17",
+    "version": "4.4.20",
     "homepage": "https://jutge.org",
     "author": {
         "name": "Jutge.org",
@@ -72,6 +72,7 @@
         "chalk": "^5.6.2",
         "chokidar": "^5.0.0",
         "cli-highlight": "^2.1.11",
+        "cli-table3": "^0.6.5",
         "commander": "^14.0.3",
         "dayjs": "^1.11.19",
         "env-paths": "^4.0.0",

package/toolkit/quiz.ts

@@ -1,5 +1,10 @@
 import { Command } from '@commander-js/extra-typings'
 import { runQuiz, lintQuiz } from '../lib/quiz'
+import {
+    loadQuizTestInput,
+    playQuiz,
+    writeQuizTestOutput,
+} from '../lib/play-quiz'
 import tui from '../lib/tui'
 import { findRealDirectories } from '../lib/helpers'
 import { random } from 'radash'
@@ -30,12 +35,11 @@ quizCmd
     .command('run')
     .summary('Run a quiz problem')
     .description(
-        `Run a quiz problem. This command is work-in-progress and may not work as expected yet.
+        `Run a quiz problem.
 
 This command will run the quiz problem and print the resulting object to stdout.
 A random seed will be generated if not provided.
-`,
-    )
+`)
 
     .option('-d, --directory <directory>', 'problem directory', '.')
     .option('-s, --seed <seed>', 'random seed')
@@ -50,3 +54,26 @@ A random seed will be generated if not provided.
             tui.yaml(object)
         }
     })
+
+
+quizCmd
+    .command('play')
+    .summary('Play a quiz problem')
+    .description(
+        `Play an interactive quiz test. Questions are shown in sequence; you can review and change answers before submitting.
+Input is a JSON file from \`quiz run\`; if --input is omitted, a run is generated with a random seed from the given directory.
+If --output is given, the results (your answers, correct answers, and score per question) are written to that file.`)
+
+    .option('-i, --input <input>', 'input JSON file from quiz run')
+    .option('-o, --output <output>', 'output file for results')
+    .option('-d, --directory <directory>', 'problem directory (used when --input is not provided)', '.')
+    .option('-s, --seed <seed>', 'random seed (used when --input is not provided)')
+
+    .action(async ({ input, output, directory, seed }) => {
+        const seedValue = seed !== undefined ? parseInt(seed, 10) : undefined
+        const quizInput = await loadQuizTestInput(input, directory, seedValue)
+        const results = await playQuiz(quizInput)
+        if (output !== undefined) {
+            await writeQuizTestOutput(output, results)
+        }
+    })