@doenet/doenetml-headless 0.0.0-stage → 0.7.27-dev.867

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,3 +1,258 @@
1
- # Temporary Holding Version
1
+ # @doenet/doenetml-headless
2
2
 
3
- This version is a temporary placeholder for this package. An operational version to replace this has been submitted for review and is awaiting a staged release.
3
+ Run DoenetML without a browser, in plain Node: grade a saved state of a
4
+ document, check a document for problems, and set a document's inputs and
5
+ points from a script or a test.
6
+
7
+ ```bash
8
+ npm install @doenet/doenetml-headless math-expressions
9
+ ```
10
+
11
+ `math-expressions` is a peer dependency, the math engine the core computes
12
+ with. Node 20 or later.
13
+
14
+ ## What it is for, and what it is not
15
+
16
+ **`gradeState` reports the credit the document assigns to a stored state in a
17
+ given variant. It is not evidence of what the student did.** The saved state
18
+ comes from the student's browser, and everything in it can be changed there:
19
+ the input values, the random selections that made the variant, and any other
20
+ state the credit depends on. Nothing run in a student's browser can be made
21
+ tamper-proof, and this package does not make it so. What recomputing on a
22
+ server does buy is that the server no longer takes the browser's word for the
23
+ credit itself.
24
+
25
+ So, in a result:
26
+
27
+ - `credit` is the answer's `creditAchievedIfSubmit`, recomputed when the state
28
+ is loaded, from the inputs as the state holds them. It ignores attempt
29
+ limits (an answer whose attempts are used up is still graded) and it is not
30
+ reduced by `creditByAttempt`.
31
+ - `stored.creditAchieved` and `stored.numSubmissions` are what the state
32
+ itself says. They are restored exactly as stored, so they are the browser's
33
+ claim, shown beside the recomputed credit for comparison and nothing more.
34
+ - `document.credit` is the document's own `creditAchievedIfSubmit`. The
35
+ document weighs its parts itself, by their weights, and a section with
36
+ `sectionWideCheckWork` counts as the one part it is; read this, do not add
37
+ up the answers. It is `NaN` (`null` in JSON) when the document weighs
38
+ nothing (no answers, or every weight 0), and also when a `<problem>` or
39
+ other aggregating section in it has no answers, whatever the rest earns
40
+ (each divides by the weight of what it holds, with no guard for none).
41
+
42
+ `answers` are the document's scored parts in the order a reader meets them (a
43
+ repeat's iteration by iteration, a select's in the order selected): each
44
+ `<answer>`, and each component scored as a whole, such as a `<pretzel>`
45
+ (`componentType: "pretzel"`, with no `inputs`). A copy of an answer is not
46
+ reported beside it, an answer inside an unopened `<solution>` does not exist
47
+ yet, and the `<answer>` that gives each of a pretzel's problems' answer is not
48
+ among them, since the pretzel turns it into text.
49
+
50
+ **Snapshot the state at the moment of submission.** `creditAchievedIfSubmit`
51
+ reflects the inputs as they were when the state was saved, not as they were
52
+ when the student last submitted. A state saved after the student changed an
53
+ input grades work they never submitted. The viewer reports a state in the same
54
+ update as each submission, marked `onSubmission: true` inside the payload; a
55
+ host that stores those reports as the snapshots of its checks, and treats the
56
+ other reports as the running state only, grades what was submitted. A graded
57
+ result carries `savedState.onSubmission` so a host can tell which it has.
58
+
59
+ **There is no grading from events.** The `response` in a `submitted` event is
60
+ what the author chose to show an instructor, and need not determine the
61
+ credit. In
62
+
63
+ ```xml
64
+ <mathInput name="a" forAnswer="$ans" />
65
+ <mathInput name="b" />
66
+ <answer name="ans"><award><when>$a = $b</when></award></answer>
67
+ ```
68
+
69
+ `b` decides the credit but is not part of the response, so no mapping from
70
+ responses back to inputs can be right in general. Grade from the saved state.
71
+
72
+ ## Grading a saved state
73
+
74
+ ```js
75
+ import { gradeState } from "@doenet/doenetml-headless";
76
+
77
+ const result = await gradeState(doenetML, variantIndex, savedState);
78
+ // result.answers: [{ stableKey, credit, stored: { creditAchieved, numSubmissions }, inputs, ... }]
79
+ // result.document: { credit, stored: { creditAchieved } }
80
+ // result.diagnostics, result.variantIndex, result.numVariants, result.savedState
81
+ ```
82
+
83
+ `savedState` is the payload the viewer reports to its host, stored as it came,
84
+ as an object or as its JSON text: the `state` of `reportScoreAndState` (the
85
+ `SPLICE.reportScoreAndState` message's `state`, for the iframe). It carries
86
+ `data_format_version`, `cid`, `coreInfo`, `coreState`, `attemptNumber` and
87
+ `onSubmission`, among others.
88
+
89
+ Each grade runs a fresh core: the JavaScript core applies a saved state only
90
+ when it is created.
91
+
92
+ `gradeState` refuses, with a `HeadlessError` whose `code` says why:
93
+
94
+ | `code` | when |
95
+ | ------------------- | ----------------------------------------------------------------------------------------- |
96
+ | `invalidState` | the state is not such a payload |
97
+ | `dataFormatVersion` | its `data_format_version` is not the one the core reads; the error names both |
98
+ | `documentMismatch` | it was written for a different document source (its `cid`, a hash of the source, differs) |
99
+ | `variantMismatch` | it was written for a different variant than `variantIndex` |
100
+ | `attemptMismatch` | with the `attemptNumber` option, it was written for a different attempt |
101
+
102
+ `data_format_version` marks the format of the saved state. It changes only
103
+ when an older state would no longer be read correctly, and states are never
104
+ converted between formats, so a state in another format is refused rather
105
+ than misapplied. It is not a runtime version: many releases share one format
106
+ version, so a match does not mean the same release wrote the state. Whether to
107
+ grade each state with the release that wrote it is the host's decision.
108
+
109
+ Options: `flags` (viewer flags, as `DoenetViewer` takes them; none of them
110
+ changes the credit, and `allowLoadState` is not consulted, since the state is
111
+ always applied), `attemptNumber` (a state that records no attempt is not
112
+ checked), and `coreFactory` (see [Cores](#cores)).
113
+
114
+ ## Checking a document
115
+
116
+ ```js
117
+ import { check } from "@doenet/doenetml-headless";
118
+
119
+ const result = await check(doenetML, { variantIndex: 1 });
120
+ // result.diagnostics: every diagnostic, of every type (error, warning, info, accessibility)
121
+ // result.answers: [{ stableKey, componentType, position, inputs }]
122
+ // result.answersWithoutStableKey, result.numVariants
123
+ ```
124
+
125
+ ## Stable keys
126
+
127
+ A component's stable key is a path from the document root that reaches it and
128
+ nothing else, each step resolving uniquely from the one before it: the same
129
+ condition under which the resolver gives a component a root name. It is
130
+ written as a reference without the `$`, such as `ans`, `p1.same` or
131
+ `r[2].inRepeat` (the shortest such path is reported).
132
+
133
+ Having a name is not enough. Two `<answer name="same">` in different
134
+ paragraphs have no stable key until each paragraph has a name of its own, and
135
+ an answer with no name has none. Inside a saved state components are keyed by
136
+ an index that stays the same only while the document does not change, so it
137
+ is not a key a file of expected answers can use; `componentIdx` in a result is
138
+ that index, for correlating within one run, never to store.
139
+
140
+ ## Setting values
141
+
142
+ For tests and content checks: give inputs the raw values a student enters,
143
+ move points as a student drags them, submit, and read the credit.
144
+
145
+ ```js
146
+ import {
147
+ createHeadlessCore,
148
+ setValue,
149
+ submit,
150
+ } from "@doenet/doenetml-headless";
151
+
152
+ const core = await createHeadlessCore({ doenetML, variantIndex: 1 });
153
+ try {
154
+ await setValue(core, "x", "2x+1"); // by path
155
+ await setValue(core, { answer: "ans", position: 1 }, "3"); // an answer's own input
156
+ await submit(core, "ans");
157
+ const answers = await core.getAnswers();
158
+ const state = await core.serializeState(); // the state a viewer would report
159
+ } finally {
160
+ await core.terminate();
161
+ }
162
+ ```
163
+
164
+ The target is either a path from the root (`"x"`, `"$r[3].g.mi"`) or an
165
+ answer's path and the position of one of the answer's own inputs, numbered
166
+ from 1. The second form is the only way to reach an input that cannot be
167
+ named, such as the one `<answer name="ans">x</answer>` makes for itself, or an
168
+ unnamed `<mathInput/>` written inside the answer. Either way the target needs a
169
+ stable key, and an input reached through an answer needs the answer to have
170
+ one. A target without one is refused (`noStableKey`), never guessed; a path
171
+ that names nothing, an `answer` that names something other than an
172
+ `<answer>`, or a position the answer has no input at is refused as `notFound`. A string that is not written as
173
+ a path at all (names joined by `.`, literal indices) throws a plain `Error`.
174
+
175
+ | component | value |
176
+ | ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
177
+ | `<mathInput>` | the LaTeX typed, a string |
178
+ | `<textInput>` | the text, a string |
179
+ | `<booleanInput>` | `true` or `false` |
180
+ | `<choiceInput>` | the positions of the selected choices, numbered from 1 in the order the reader sees them (shuffled, if the choices are), as an array or one number; one position unless it has `selectMultiple`, each at most once, none of a disabled choice; several are selected in increasing position, as checkboxes select them, or, for an `inline` input, in the order given, as its dropdown keeps the order they were picked |
181
+ | `<fractionInput>` | `{ numerator, denominator }`, each the LaTeX typed |
182
+ | `<matrixInput>` | the LaTeX of every cell, as an array of rows of the matrix's current size |
183
+ | `<point>` | its coordinates, `[x, y]` or `[x, y, z]` as the point has, moved as a drag moves it |
184
+
185
+ Each value is turned into the actions the component's renderer sends. Any other
186
+ type is refused (`unsupportedTarget`), as are a disabled or fixed input, a disabled choice, or a point that
187
+ cannot be moved (`targetDisabled`, since it would ignore the value) and a value
188
+ of the wrong shape (`invalidValue`). A value the document does not take is
189
+ refused as `targetDisabled` too: an input bound to a fixed component
190
+ (`<mathInput bindValueTo="$m"/>` with `<math name="m" fixed>`) keeps its value,
191
+ and a point that extends one with a fixed location stays where it is. A point
192
+ a constraint or attractor moves somewhere else has taken the value, as a drag
193
+ does, and so has an input whose value the document changes into something else
194
+ (a bound `<math simplify>` simplifying what was typed). A value the target
195
+ already has is not refused for coming from a fixed source, since nothing would
196
+ change; a target that is itself disabled or fixed is refused all the same.
197
+
198
+ Two cases are not caught:
199
+
200
+ - A value the document turns into exactly the value the input already has is
201
+ refused, though it was taken: `x+x` typed into an input showing `2x` and
202
+ bound to a `<math simplify>`, or `2.50` into a `<textInput>` bound to a
203
+ `<number>` that is 2.5. Reading the value back cannot tell this from a fixed
204
+ source.
205
+ - A point whose coordinates come from a fixed point (`<point>$A</point>` or
206
+ `coords="$A"` with `A` fixed) stays where it is, and `setValue` returns
207
+ normally. The document reports nothing for it, and a point is not read
208
+ back, since one that a constraint or attractor keeps in place has taken the
209
+ value.
210
+
211
+ `submit(core, path)` submits an `<answer>` or a `<pretzel>`, or, for any other
212
+ component (a section with `sectionWideCheckWork`, the document), all the
213
+ answers under it, as its check-work button does.
214
+
215
+ ## Command line
216
+
217
+ ```bash
218
+ doenet-headless check <file> [--variant <n>] [--json]
219
+ doenet-headless grade <file> --variant <n> --state <state.json> [--attempt <n>] [--json]
220
+ ```
221
+
222
+ Human-readable by default, JSON with `--json`. Standard output carries only
223
+ that report: anything the core logs while it runs (with `console.log`,
224
+ `console.info`, `console.debug` or `console.warn`) goes to standard error, as
225
+ do refusals and errors. Exit codes: 0 when the
226
+ document has no errors; 1 when it has an error among its diagnostics (the
227
+ report is still printed); 2 for a usage or file problem or any other failure;
228
+ 3 when `grade` refuses the saved state.
229
+
230
+ ## Cores
231
+
232
+ The package depends only on the `HeadlessCore` interface (in `@doenet/utils`):
233
+ create a core from a document, a variant, flags and an optional saved state;
234
+ dispatch an action; resolve a path; read a component's children and
235
+ properties; read the answers, the document credit and the diagnostics; read
236
+ the variant count; serialize the state; optionally load a state into a running
237
+ core (`loadState`, when `capabilities.loadState` is true); terminate. A
238
+ `HeadlessCoreFactory` makes cores and says which `data_format_version` they
239
+ read, so a state is checked before a core is made from it.
240
+
241
+ The default is the current JavaScript core, over the resolver WASM that ships
242
+ in this package beside its code (`lib_doenetml_worker_bg.wasm`, read from disk
243
+ when the first core is made). A host that keeps that file elsewhere names it
244
+ with the `DOENET_HEADLESS_RESOLVER_WASM` environment variable. Another core
245
+ plugs in by passing its factory as the `coreFactory` option of `createHeadlessCore`,
246
+ `gradeState` or `check`; making it the default is a change to one file,
247
+ `src/defaultCore.ts`.
248
+
249
+ The JavaScript core computes in English, even for a document that declares
250
+ another language with `lang` or that a viewer shows with a non-English
251
+ `documentLocale`, where the viewer computes the words the core writes into the
252
+ content (a boolean's text, for one) in that language; an answer that compares a
253
+ response with such words is graded here against the English ones. It has no
254
+ host to fetch external DoenetML from (a `copy="doenet:..."` reference is reported as an error, and
255
+ the document is graded without the content it names, where a viewer whose host
256
+ fetches it shows that content and its answers), and
257
+ runs with the viewer's default flags except that it sends no events (as the
258
+ viewer does by default, it keeps no local state and does not auto-submit).
package/cli.d.ts ADDED
@@ -0,0 +1,25 @@
1
+ export declare const EXIT: {
2
+ readonly ok: 0;
3
+ readonly documentErrors: 1;
4
+ readonly usage: 2;
5
+ readonly refused: 3;
6
+ };
7
+
8
+ declare type Output = {
9
+ out: (text: string) => void;
10
+ err: (text: string) => void;
11
+ };
12
+
13
+ /**
14
+ * Run the CLI on `argv` (without the node and script paths) and return the
15
+ * exit code. Writes through `output`, so a test can run it in-process.
16
+ *
17
+ * Standard output carries only the command's own report: while it runs,
18
+ * whatever the core writes with `console.log`, `console.info`,
19
+ * `console.debug` or `console.warn` goes to `output.err` instead, and the
20
+ * console is restored afterwards. `console.error` is left as it is (it
21
+ * already writes to standard error).
22
+ */
23
+ export declare function runCli(argv: string[], output?: Output): Promise<number>;
24
+
25
+ export { }
package/cli.js ADDED
@@ -0,0 +1,199 @@
1
+ #!/usr/bin/env node
2
+ import { realpathSync } from "node:fs";
3
+ import { readFile } from "node:fs/promises";
4
+ import { pathToFileURL } from "node:url";
5
+ import { format, parseArgs } from "node:util";
6
+ import { c as check, g as gradeState, H as HeadlessError } from "./gradeState-DgaD_akY.js";
7
+ const EXIT = {
8
+ ok: 0,
9
+ documentErrors: 1,
10
+ usage: 2,
11
+ refused: 3
12
+ };
13
+ const USAGE = `Usage:
14
+ doenet-headless check <file> [--variant <n>] [--json]
15
+ doenet-headless grade <file> --variant <n> --state <state.json> [--attempt <n>] [--json]
16
+
17
+ check reports every diagnostic, the answers with their stable keys (and
18
+ which have none), and the number of variants.
19
+ grade loads a saved state (the viewer's reportScoreAndState payload, as
20
+ stored) and reports the credit the document assigns to it.
21
+
22
+ Exit codes: 0 no errors; 1 the document has errors; 2 usage, file or other
23
+ failure; 3 the saved state was refused.`;
24
+ const STDOUT_CONSOLE_METHODS = ["log", "info", "debug", "warn"];
25
+ async function runCli(argv, output = {
26
+ out: (text) => process.stdout.write(text + "\n"),
27
+ err: (text) => process.stderr.write(text + "\n")
28
+ }) {
29
+ const saved = STDOUT_CONSOLE_METHODS.map((name) => console[name]);
30
+ for (const name of STDOUT_CONSOLE_METHODS) {
31
+ console[name] = (...args) => output.err(format(...args));
32
+ }
33
+ try {
34
+ return await run(argv, output);
35
+ } finally {
36
+ STDOUT_CONSOLE_METHODS.forEach((name, i) => {
37
+ console[name] = saved[i];
38
+ });
39
+ }
40
+ }
41
+ async function run(argv, output) {
42
+ let parsed;
43
+ try {
44
+ parsed = parseArgs({
45
+ args: argv,
46
+ allowPositionals: true,
47
+ options: {
48
+ variant: { type: "string" },
49
+ state: { type: "string" },
50
+ attempt: { type: "string" },
51
+ json: { type: "boolean", default: false },
52
+ help: { type: "boolean", short: "h", default: false }
53
+ }
54
+ });
55
+ } catch (e) {
56
+ output.err(`${e instanceof Error ? e.message : String(e)}
57
+
58
+ ${USAGE}`);
59
+ return EXIT.usage;
60
+ }
61
+ const { values, positionals } = parsed;
62
+ if (values.help) {
63
+ output.out(USAGE);
64
+ return EXIT.ok;
65
+ }
66
+ const [command, file, ...extra] = positionals;
67
+ if (command !== "check" && command !== "grade" || file === void 0 || extra.length > 0) {
68
+ output.err(USAGE);
69
+ return EXIT.usage;
70
+ }
71
+ const variant = parsePositiveInteger(values.variant, "--variant", output);
72
+ const attempt = parsePositiveInteger(values.attempt, "--attempt", output);
73
+ if (variant === null || attempt === null) {
74
+ return EXIT.usage;
75
+ }
76
+ const doenetML = await readText(file, output);
77
+ if (doenetML === null) {
78
+ return EXIT.usage;
79
+ }
80
+ if (command === "check") {
81
+ const result2 = await check(doenetML, { variantIndex: variant });
82
+ output.out(
83
+ values.json ? JSON.stringify(result2, null, 2) : formatCheck(file, result2)
84
+ );
85
+ return hasErrors(result2.diagnostics) ? EXIT.documentErrors : EXIT.ok;
86
+ }
87
+ if (variant === void 0 || values.state === void 0) {
88
+ output.err(`grade needs --variant and --state.
89
+
90
+ ${USAGE}`);
91
+ return EXIT.usage;
92
+ }
93
+ const stateText = await readText(values.state, output);
94
+ if (stateText === null) {
95
+ return EXIT.usage;
96
+ }
97
+ let result;
98
+ try {
99
+ result = await gradeState(doenetML, variant, stateText, {
100
+ attemptNumber: attempt
101
+ });
102
+ } catch (e) {
103
+ if (e instanceof HeadlessError) {
104
+ output.err(
105
+ values.json ? JSON.stringify({ refused: e.code, message: e.message }) : `Refused (${e.code}): ${e.message}`
106
+ );
107
+ return EXIT.refused;
108
+ }
109
+ throw e;
110
+ }
111
+ output.out(
112
+ values.json ? JSON.stringify(result, null, 2) : formatGrade(file, result)
113
+ );
114
+ return hasErrors(result.diagnostics) ? EXIT.documentErrors : EXIT.ok;
115
+ }
116
+ function parsePositiveInteger(text, flag, output) {
117
+ if (text === void 0) {
118
+ return void 0;
119
+ }
120
+ const value = Number(text);
121
+ if (!/^\d+$/.test(text) || value < 1) {
122
+ output.err(`${flag} takes a whole number from 1, not "${text}".`);
123
+ return null;
124
+ }
125
+ return value;
126
+ }
127
+ async function readText(file, output) {
128
+ try {
129
+ return await readFile(file, "utf8");
130
+ } catch (e) {
131
+ output.err(
132
+ `Cannot read ${file}: ${e instanceof Error ? e.message : String(e)}`
133
+ );
134
+ return null;
135
+ }
136
+ }
137
+ function hasErrors(diagnostics) {
138
+ return diagnostics.some((d) => d.type === "error");
139
+ }
140
+ function plural(n, word) {
141
+ return `${n} ${word}${n === 1 ? "" : "s"}`;
142
+ }
143
+ function at(position) {
144
+ return position ? `line ${position.start.line}, column ${position.start.column}` : "position unknown";
145
+ }
146
+ function formatDiagnostics(diagnostics) {
147
+ const counts = { error: 0, warning: 0, info: 0, accessibility: 0 };
148
+ for (const d of diagnostics) {
149
+ counts[d.type]++;
150
+ }
151
+ return [
152
+ `Diagnostics: ${plural(counts.error, "error")}, ${plural(counts.warning, "warning")}, ${plural(counts.info, "info")}, ${counts.accessibility} accessibility`,
153
+ ...diagnostics.map(
154
+ (d) => ` ${d.type}${d.type === "accessibility" ? ` (level ${d.level})` : ""}, ${at(d.position)}: ${d.message}`
155
+ )
156
+ ];
157
+ }
158
+ function formatCheck(file, result) {
159
+ return [
160
+ `${file}: variant ${result.variantIndex} of ${result.numVariants}, ${plural(result.answers.length, "answer")}, ${result.answersWithoutStableKey.length} without a stable key`,
161
+ ...result.answers.length > 0 ? ["Answers:"] : [],
162
+ ...result.answers.map(
163
+ (a) => ` ${a.stableKey ?? `(no stable key, ${at(a.position)})`}: ${a.componentType === "answer" ? a.inputs.map((i) => i.componentType).join(", ") || "no inputs of its own" : `a <${a.componentType}>, scored as a whole`}`
164
+ ),
165
+ ...formatDiagnostics(result.diagnostics)
166
+ ].join("\n");
167
+ }
168
+ function formatGrade(file, result) {
169
+ return [
170
+ `${file}: variant ${result.variantIndex} of ${result.numVariants}, state ${result.savedState.onSubmission ? "saved at a submission" : "not saved at a submission"}`,
171
+ `Document credit: ${result.document.credit} (stored: ${result.document.stored.creditAchieved})`,
172
+ ...result.answers.length > 0 ? [
173
+ "Answers (credit recomputed; stored values are the browser's claim):"
174
+ ] : [],
175
+ ...result.answers.map(
176
+ (a) => ` ${a.stableKey ?? `(no stable key, ${at(a.position)})`}: credit ${a.credit} (stored: ${a.stored.creditAchieved}, ${plural(a.stored.numSubmissions, "submission")})`
177
+ ),
178
+ ...formatDiagnostics(result.diagnostics)
179
+ ].join("\n");
180
+ }
181
+ if (process.argv[1] && import.meta.url === pathToFileURL(realpathSync(process.argv[1])).href) {
182
+ runCli(process.argv.slice(2)).then(
183
+ (code) => {
184
+ process.exitCode = code;
185
+ },
186
+ (e) => {
187
+ process.stderr.write(
188
+ `${e instanceof Error ? e.stack ?? e.message : String(e)}
189
+ `
190
+ );
191
+ process.exitCode = EXIT.usage;
192
+ }
193
+ );
194
+ }
195
+ export {
196
+ EXIT,
197
+ runCli
198
+ };
199
+ //# sourceMappingURL=cli.js.map
package/cli.js.map ADDED
@@ -0,0 +1 @@
1
+ {"version":3,"file":"cli.js","sources":["../src/cli.ts"],"sourcesContent":["#!/usr/bin/env node\n/**\n * `doenet-headless`: check a DoenetML document, or grade a saved state of one,\n * from the command line.\n *\n * doenet-headless check <file> [--variant <n>] [--json]\n * doenet-headless grade <file> --variant <n> --state <state.json> [--attempt <n>] [--json]\n *\n * Exit codes: 0 when the document has no errors; 1 when it has an error among\n * its diagnostics (the report is still printed); 2 for a usage or file\n * problem, or any other failure; 3 when `grade` refuses the saved state.\n */\nimport { realpathSync } from \"node:fs\";\nimport { readFile } from \"node:fs/promises\";\nimport { pathToFileURL } from \"node:url\";\nimport { format, parseArgs } from \"node:util\";\nimport type { DiagnosticRecord } from \"@doenet/utils\";\nimport { check, type CheckResult } from \"./check\";\nimport { HeadlessError } from \"./errors\";\nimport { gradeState, type GradeStateResult } from \"./gradeState\";\n\nexport const EXIT = {\n ok: 0,\n documentErrors: 1,\n usage: 2,\n refused: 3,\n} as const;\n\nconst USAGE = `Usage:\n doenet-headless check <file> [--variant <n>] [--json]\n doenet-headless grade <file> --variant <n> --state <state.json> [--attempt <n>] [--json]\n\ncheck reports every diagnostic, the answers with their stable keys (and\n which have none), and the number of variants.\ngrade loads a saved state (the viewer's reportScoreAndState payload, as\n stored) and reports the credit the document assigns to it.\n\nExit codes: 0 no errors; 1 the document has errors; 2 usage, file or other\nfailure; 3 the saved state was refused.`;\n\ntype Output = { out: (text: string) => void; err: (text: string) => void };\n\n/**\n * The console methods the CLI redirects while it runs. `log`, `info` and\n * `debug` write to standard output, and the core logs through them\n * (`console.log` on a text-format `<math>` whose content does not parse, for\n * one), so a line of that would come before the report and break `--json`.\n * `warn` already writes to standard error; it is redirected too, so that a\n * caller's `output.err` receives the core's warnings as well.\n */\nconst STDOUT_CONSOLE_METHODS = [\"log\", \"info\", \"debug\", \"warn\"] as const;\n\n/**\n * Run the CLI on `argv` (without the node and script paths) and return the\n * exit code. Writes through `output`, so a test can run it in-process.\n *\n * Standard output carries only the command's own report: while it runs,\n * whatever the core writes with `console.log`, `console.info`,\n * `console.debug` or `console.warn` goes to `output.err` instead, and the\n * console is restored afterwards. `console.error` is left as it is (it\n * already writes to standard error).\n */\nexport async function runCli(\n argv: string[],\n output: Output = {\n out: (text) => process.stdout.write(text + \"\\n\"),\n err: (text) => process.stderr.write(text + \"\\n\"),\n },\n): Promise<number> {\n const saved = STDOUT_CONSOLE_METHODS.map((name) => console[name]);\n for (const name of STDOUT_CONSOLE_METHODS) {\n console[name] = (...args: unknown[]) => output.err(format(...args));\n }\n try {\n return await run(argv, output);\n } finally {\n STDOUT_CONSOLE_METHODS.forEach((name, i) => {\n console[name] = saved[i];\n });\n }\n}\n\nasync function run(argv: string[], output: Output): Promise<number> {\n let parsed;\n try {\n parsed = parseArgs({\n args: argv,\n allowPositionals: true,\n options: {\n variant: { type: \"string\" },\n state: { type: \"string\" },\n attempt: { type: \"string\" },\n json: { type: \"boolean\", default: false },\n help: { type: \"boolean\", short: \"h\", default: false },\n },\n });\n } catch (e) {\n output.err(`${e instanceof Error ? e.message : String(e)}\\n\\n${USAGE}`);\n return EXIT.usage;\n }\n const { values, positionals } = parsed;\n if (values.help) {\n output.out(USAGE);\n return EXIT.ok;\n }\n const [command, file, ...extra] = positionals;\n if (\n (command !== \"check\" && command !== \"grade\") ||\n file === undefined ||\n extra.length > 0\n ) {\n output.err(USAGE);\n return EXIT.usage;\n }\n\n const variant = parsePositiveInteger(values.variant, \"--variant\", output);\n const attempt = parsePositiveInteger(values.attempt, \"--attempt\", output);\n if (variant === null || attempt === null) {\n return EXIT.usage;\n }\n\n const doenetML = await readText(file, output);\n if (doenetML === null) {\n return EXIT.usage;\n }\n\n if (command === \"check\") {\n const result = await check(doenetML, { variantIndex: variant });\n output.out(\n values.json\n ? JSON.stringify(result, null, 2)\n : formatCheck(file, result),\n );\n return hasErrors(result.diagnostics) ? EXIT.documentErrors : EXIT.ok;\n }\n\n if (variant === undefined || values.state === undefined) {\n output.err(`grade needs --variant and --state.\\n\\n${USAGE}`);\n return EXIT.usage;\n }\n const stateText = await readText(values.state, output);\n if (stateText === null) {\n return EXIT.usage;\n }\n let result: GradeStateResult;\n try {\n result = await gradeState(doenetML, variant, stateText, {\n attemptNumber: attempt,\n });\n } catch (e) {\n if (e instanceof HeadlessError) {\n output.err(\n values.json\n ? JSON.stringify({ refused: e.code, message: e.message })\n : `Refused (${e.code}): ${e.message}`,\n );\n return EXIT.refused;\n }\n throw e;\n }\n output.out(\n values.json\n ? JSON.stringify(result, null, 2)\n : formatGrade(file, result),\n );\n return hasErrors(result.diagnostics) ? EXIT.documentErrors : EXIT.ok;\n}\n\nfunction parsePositiveInteger(\n text: string | undefined,\n flag: string,\n output: Output,\n): number | undefined | null {\n if (text === undefined) {\n return undefined;\n }\n const value = Number(text);\n if (!/^\\d+$/.test(text) || value < 1) {\n output.err(`${flag} takes a whole number from 1, not \"${text}\".`);\n return null;\n }\n return value;\n}\n\nasync function readText(file: string, output: Output): Promise<string | null> {\n try {\n return await readFile(file, \"utf8\");\n } catch (e) {\n output.err(\n `Cannot read ${file}: ${e instanceof Error ? e.message : String(e)}`,\n );\n return null;\n }\n}\n\nfunction hasErrors(diagnostics: DiagnosticRecord[]) {\n return diagnostics.some((d) => d.type === \"error\");\n}\n\nfunction plural(n: number, word: string) {\n return `${n} ${word}${n === 1 ? \"\" : \"s\"}`;\n}\n\nfunction at(position: { start: { line: number; column: number } } | undefined) {\n return position\n ? `line ${position.start.line}, column ${position.start.column}`\n : \"position unknown\";\n}\n\nfunction formatDiagnostics(diagnostics: DiagnosticRecord[]): string[] {\n const counts = { error: 0, warning: 0, info: 0, accessibility: 0 };\n for (const d of diagnostics) {\n counts[d.type]++;\n }\n return [\n `Diagnostics: ${plural(counts.error, \"error\")}, ${plural(counts.warning, \"warning\")}, ${plural(counts.info, \"info\")}, ${counts.accessibility} accessibility`,\n ...diagnostics.map(\n (d) =>\n ` ${d.type}${d.type === \"accessibility\" ? ` (level ${d.level})` : \"\"}, ${at(d.position)}: ${d.message}`,\n ),\n ];\n}\n\nfunction formatCheck(file: string, result: CheckResult): string {\n return [\n `${file}: variant ${result.variantIndex} of ${result.numVariants}, ${plural(result.answers.length, \"answer\")}, ${result.answersWithoutStableKey.length} without a stable key`,\n ...(result.answers.length > 0 ? [\"Answers:\"] : []),\n ...result.answers.map(\n (a) =>\n ` ${a.stableKey ?? `(no stable key, ${at(a.position)})`}: ${a.componentType === \"answer\" ? a.inputs.map((i) => i.componentType).join(\", \") || \"no inputs of its own\" : `a <${a.componentType}>, scored as a whole`}`,\n ),\n ...formatDiagnostics(result.diagnostics),\n ].join(\"\\n\");\n}\n\nfunction formatGrade(file: string, result: GradeStateResult): string {\n return [\n `${file}: variant ${result.variantIndex} of ${result.numVariants}, state ${result.savedState.onSubmission ? \"saved at a submission\" : \"not saved at a submission\"}`,\n `Document credit: ${result.document.credit} (stored: ${result.document.stored.creditAchieved})`,\n ...(result.answers.length > 0\n ? [\n \"Answers (credit recomputed; stored values are the browser's claim):\",\n ]\n : []),\n ...result.answers.map(\n (a) =>\n ` ${a.stableKey ?? `(no stable key, ${at(a.position)})`}: credit ${a.credit} (stored: ${a.stored.creditAchieved}, ${plural(a.stored.numSubmissions, \"submission\")})`,\n ),\n ...formatDiagnostics(result.diagnostics),\n ].join(\"\\n\");\n}\n\n// Run when this module is the program, not when a test imports it.\nif (\n process.argv[1] &&\n import.meta.url === pathToFileURL(realpathSync(process.argv[1])).href\n) {\n runCli(process.argv.slice(2)).then(\n (code) => {\n process.exitCode = code;\n },\n (e) => {\n process.stderr.write(\n `${e instanceof Error ? (e.stack ?? e.message) : String(e)}\\n`,\n );\n process.exitCode = EXIT.usage;\n },\n );\n}\n"],"names":["result"],"mappings":";;;;;;AAqBO,MAAM,OAAO;AAAA,EAChB,IAAI;AAAA,EACJ,gBAAgB;AAAA,EAChB,OAAO;AAAA,EACP,SAAS;AACb;AAEA,MAAM,QAAQ;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAAA;AAsBd,MAAM,yBAAyB,CAAC,OAAO,QAAQ,SAAS,MAAM;AAY9D,eAAsB,OAClB,MACA,SAAiB;AAAA,EACb,KAAK,CAAC,SAAS,QAAQ,OAAO,MAAM,OAAO,IAAI;AAAA,EAC/C,KAAK,CAAC,SAAS,QAAQ,OAAO,MAAM,OAAO,IAAI;AACnD,GACe;AACf,QAAM,QAAQ,uBAAuB,IAAI,CAAC,SAAS,QAAQ,IAAI,CAAC;AAChE,aAAW,QAAQ,wBAAwB;AACvC,YAAQ,IAAI,IAAI,IAAI,SAAoB,OAAO,IAAI,OAAO,GAAG,IAAI,CAAC;AAAA,EACtE;AACA,MAAI;AACA,WAAO,MAAM,IAAI,MAAM,MAAM;AAAA,EACjC,UAAA;AACI,2BAAuB,QAAQ,CAAC,MAAM,MAAM;AACxC,cAAQ,IAAI,IAAI,MAAM,CAAC;AAAA,IAC3B,CAAC;AAAA,EACL;AACJ;AAEA,eAAe,IAAI,MAAgB,QAAiC;AAChE,MAAI;AACJ,MAAI;AACA,aAAS,UAAU;AAAA,MACf,MAAM;AAAA,MACN,kBAAkB;AAAA,MAClB,SAAS;AAAA,QACL,SAAS,EAAE,MAAM,SAAA;AAAA,QACjB,OAAO,EAAE,MAAM,SAAA;AAAA,QACf,SAAS,EAAE,MAAM,SAAA;AAAA,QACjB,MAAM,EAAE,MAAM,WAAW,SAAS,MAAA;AAAA,QAClC,MAAM,EAAE,MAAM,WAAW,OAAO,KAAK,SAAS,MAAA;AAAA,MAAM;AAAA,IACxD,CACH;AAAA,EACL,SAAS,GAAG;AACR,WAAO,IAAI,GAAG,aAAa,QAAQ,EAAE,UAAU,OAAO,CAAC,CAAC;AAAA;AAAA,EAAO,KAAK,EAAE;AACtE,WAAO,KAAK;AAAA,EAChB;AACA,QAAM,EAAE,QAAQ,YAAA,IAAgB;AAChC,MAAI,OAAO,MAAM;AACb,WAAO,IAAI,KAAK;AAChB,WAAO,KAAK;AAAA,EAChB;AACA,QAAM,CAAC,SAAS,MAAM,GAAG,KAAK,IAAI;AAClC,MACK,YAAY,WAAW,YAAY,WACpC,SAAS,UACT,MAAM,SAAS,GACjB;AACE,WAAO,IAAI,KAAK;AAChB,WAAO,KAAK;AAAA,EAChB;AAEA,QAAM,UAAU,qBAAqB,OAAO,SAAS,aAAa,MAAM;AACxE,QAAM,UAAU,qBAAqB,OAAO,SAAS,aAAa,MAAM;AACxE,MAAI,YAAY,QAAQ,YAAY,MAAM;AACtC,WAAO,KAAK;AAAA,EAChB;AAEA,QAAM,WAAW,MAAM,SAAS,MAAM,MAAM;AAC5C,MAAI,aAAa,MAAM;AACnB,WAAO,KAAK;AAAA,EAChB;AAEA,MAAI,YAAY,SAAS;AACrB,UAAMA,UAAS,MAAM,MAAM,UAAU,EAAE,cAAc,SAAS;AAC9D,WAAO;AAAA,MACH,OAAO,OACD,KAAK,UAAUA,SAAQ,MAAM,CAAC,IAC9B,YAAY,MAAMA,OAAM;AAAA,IAAA;AAElC,WAAO,UAAUA,QAAO,WAAW,IAAI,KAAK,iBAAiB,KAAK;AAAA,EACtE;AAEA,MAAI,YAAY,UAAa,OAAO,UAAU,QAAW;AACrD,WAAO,IAAI;AAAA;AAAA,EAAyC,KAAK,EAAE;AAC3D,WAAO,KAAK;AAAA,EAChB;AACA,QAAM,YAAY,MAAM,SAAS,OAAO,OAAO,MAAM;AACrD,MAAI,cAAc,MAAM;AACpB,WAAO,KAAK;AAAA,EAChB;AACA,MAAI;AACJ,MAAI;AACA,aAAS,MAAM,WAAW,UAAU,SAAS,WAAW;AAAA,MACpD,eAAe;AAAA,IAAA,CAClB;AAAA,EACL,SAAS,GAAG;AACR,QAAI,aAAa,eAAe;AAC5B,aAAO;AAAA,QACH,OAAO,OACD,KAAK,UAAU,EAAE,SAAS,EAAE,MAAM,SAAS,EAAE,QAAA,CAAS,IACtD,YAAY,EAAE,IAAI,MAAM,EAAE,OAAO;AAAA,MAAA;AAE3C,aAAO,KAAK;AAAA,IAChB;AACA,UAAM;AAAA,EACV;AACA,SAAO;AAAA,IACH,OAAO,OACD,KAAK,UAAU,QAAQ,MAAM,CAAC,IAC9B,YAAY,MAAM,MAAM;AAAA,EAAA;AAElC,SAAO,UAAU,OAAO,WAAW,IAAI,KAAK,iBAAiB,KAAK;AACtE;AAEA,SAAS,qBACL,MACA,MACA,QACyB;AACzB,MAAI,SAAS,QAAW;AACpB,WAAO;AAAA,EACX;AACA,QAAM,QAAQ,OAAO,IAAI;AACzB,MAAI,CAAC,QAAQ,KAAK,IAAI,KAAK,QAAQ,GAAG;AAClC,WAAO,IAAI,GAAG,IAAI,sCAAsC,IAAI,IAAI;AAChE,WAAO;AAAA,EACX;AACA,SAAO;AACX;AAEA,eAAe,SAAS,MAAc,QAAwC;AAC1E,MAAI;AACA,WAAO,MAAM,SAAS,MAAM,MAAM;AAAA,EACtC,SAAS,GAAG;AACR,WAAO;AAAA,MACH,eAAe,IAAI,KAAK,aAAa,QAAQ,EAAE,UAAU,OAAO,CAAC,CAAC;AAAA,IAAA;AAEtE,WAAO;AAAA,EACX;AACJ;AAEA,SAAS,UAAU,aAAiC;AAChD,SAAO,YAAY,KAAK,CAAC,MAAM,EAAE,SAAS,OAAO;AACrD;AAEA,SAAS,OAAO,GAAW,MAAc;AACrC,SAAO,GAAG,CAAC,IAAI,IAAI,GAAG,MAAM,IAAI,KAAK,GAAG;AAC5C;AAEA,SAAS,GAAG,UAAmE;AAC3E,SAAO,WACD,QAAQ,SAAS,MAAM,IAAI,YAAY,SAAS,MAAM,MAAM,KAC5D;AACV;AAEA,SAAS,kBAAkB,aAA2C;AAClE,QAAM,SAAS,EAAE,OAAO,GAAG,SAAS,GAAG,MAAM,GAAG,eAAe,EAAA;AAC/D,aAAW,KAAK,aAAa;AACzB,WAAO,EAAE,IAAI;AAAA,EACjB;AACA,SAAO;AAAA,IACH,gBAAgB,OAAO,OAAO,OAAO,OAAO,CAAC,KAAK,OAAO,OAAO,SAAS,SAAS,CAAC,KAAK,OAAO,OAAO,MAAM,MAAM,CAAC,KAAK,OAAO,aAAa;AAAA,IAC5I,GAAG,YAAY;AAAA,MACX,CAAC,MACG,KAAK,EAAE,IAAI,GAAG,EAAE,SAAS,kBAAkB,WAAW,EAAE,KAAK,MAAM,EAAE,KAAK,GAAG,EAAE,QAAQ,CAAC,KAAK,EAAE,OAAO;AAAA,IAAA;AAAA,EAC9G;AAER;AAEA,SAAS,YAAY,MAAc,QAA6B;AAC5D,SAAO;AAAA,IACH,GAAG,IAAI,aAAa,OAAO,YAAY,OAAO,OAAO,WAAW,KAAK,OAAO,OAAO,QAAQ,QAAQ,QAAQ,CAAC,KAAK,OAAO,wBAAwB,MAAM;AAAA,IACtJ,GAAI,OAAO,QAAQ,SAAS,IAAI,CAAC,UAAU,IAAI,CAAA;AAAA,IAC/C,GAAG,OAAO,QAAQ;AAAA,MACd,CAAC,MACG,KAAK,EAAE,aAAa,mBAAmB,GAAG,EAAE,QAAQ,CAAC,GAAG,KAAK,EAAE,kBAAkB,WAAW,EAAE,OAAO,IAAI,CAAC,MAAM,EAAE,aAAa,EAAE,KAAK,IAAI,KAAK,yBAAyB,MAAM,EAAE,aAAa,sBAAsB;AAAA,IAAA;AAAA,IAE3N,GAAG,kBAAkB,OAAO,WAAW;AAAA,EAAA,EACzC,KAAK,IAAI;AACf;AAEA,SAAS,YAAY,MAAc,QAAkC;AACjE,SAAO;AAAA,IACH,GAAG,IAAI,aAAa,OAAO,YAAY,OAAO,OAAO,WAAW,WAAW,OAAO,WAAW,eAAe,0BAA0B,2BAA2B;AAAA,IACjK,oBAAoB,OAAO,SAAS,MAAM,aAAa,OAAO,SAAS,OAAO,cAAc;AAAA,IAC5F,GAAI,OAAO,QAAQ,SAAS,IACtB;AAAA,MACI;AAAA,IAAA,IAEJ,CAAA;AAAA,IACN,GAAG,OAAO,QAAQ;AAAA,MACd,CAAC,MACG,KAAK,EAAE,aAAa,mBAAmB,GAAG,EAAE,QAAQ,CAAC,GAAG,YAAY,EAAE,MAAM,aAAa,EAAE,OAAO,cAAc,KAAK,OAAO,EAAE,OAAO,gBAAgB,YAAY,CAAC;AAAA,IAAA;AAAA,IAE1K,GAAG,kBAAkB,OAAO,WAAW;AAAA,EAAA,EACzC,KAAK,IAAI;AACf;AAGA,IACI,QAAQ,KAAK,CAAC,KACd,YAAY,QAAQ,cAAc,aAAa,QAAQ,KAAK,CAAC,CAAC,CAAC,EAAE,MACnE;AACE,SAAO,QAAQ,KAAK,MAAM,CAAC,CAAC,EAAE;AAAA,IAC1B,CAAC,SAAS;AACN,cAAQ,WAAW;AAAA,IACvB;AAAA,IACA,CAAC,MAAM;AACH,cAAQ,OAAO;AAAA,QACX,GAAG,aAAa,QAAS,EAAE,SAAS,EAAE,UAAW,OAAO,CAAC,CAAC;AAAA;AAAA,MAAA;AAE9D,cAAQ,WAAW,KAAK;AAAA,IAC5B;AAAA,EAAA;AAER;"}