@intellectif/lk-core 1.3.0 → 1.5.0

package/CHANGELOG.md

@@ -1,5 +1,64 @@
 # @intellectif/lk-core
 
+## 1.5.0
+
+### Minor Changes
+
+- [#112](https://github.com/intellectif/learning-kit/pull/112) [`3fc1049`](https://github.com/intellectif/learning-kit/commit/3fc1049d67e14f6f4a12c235f19952a988472978) Thanks [@diego28e](https://github.com/diego28e)! - **Assisted grading**: a model grades a written response for the record, and in an exam no model's grade counts until it has been measured against your teachers.
+
+  **Action required:** none. Nothing grades until you call it. Four union members are new: `DeferredReason` gains `awaiting_human_review`, `AiRefusal` gains `misses-criteria` and `out-of-scale`, and `AiCheckCase['feature']` gains `grading`. A `switch` over one of them without a `default` branch needs them.
+
+  - **Levels on a rubric:** a criterion can carry `levels`, lowest first, from 2 to 10, each a `label` and an optional `description`. A level scores `index / (levels − 1)`.
+    - `validateActivity` does not check them in 1.x, so a field of that name you already use stays valid.
+    - The critic warns `wr_levels_unreadable` when it cannot read them.
+    - `redact()` passes readable levels to the learner, `label` and `description` only.
+  - **The request and its check:**
+    - `aiGradingRequest` builds what a model is told: the response verbatim, the rubric with its levels, and a group's material.
+    - `checkAiGrading` accepts a grade whole or not at all. It needs every criterion once (`misses-criteria`), each on its own scale (`out-of-scale`), with corrections that quote the response (`misquotes-answer`).
+  - **The grade:** `gradeFromJudgement` computes the total as `gradeFromRubric` does, with the item's weights and pass line. It records the model and the prompt revision on `grader`.
+  - **Calibration:** `calibrationReport` measures one grader configuration (a model, a prompt revision, a rubric and a target) against people's grades of the same responses. It reports:
+
+    - exact and adjacent agreement, quadratic kappa and a confusion matrix, on levels;
+    - Pearson's r, mean absolute error and the standardised mean difference, everywhere;
+    - calibration error and the Brier score, where grades carry level probabilities;
+    - the degradation from two people's agreement.
+
+    It ships no thresholds.
+
+  - **Routing:** `routeGrade` decides whether a grade counts or goes to a person, and gives every reason, in a fixed order:
+
+    - no matching calibration, in an exam;
+    - a report below your thresholds;
+    - a model unsure of a level, or one that asked for review;
+    - low confidence;
+    - the audit sample, chosen by `inAuditSample`.
+
+    `outcomeFromRoutedGrade` holds a grade for a person as `deferred` / `awaiting_human_review`, with the grade kept on `pendingGrade`, so `composeAssessmentScore` keeps the attempt provisional.
+
+  - **New optional fields:**
+    - `levels` on a rubric criterion, its redacted form included;
+    - `levelProbabilities` on `CriterionScore`;
+    - `routing` on `GradeRecord`;
+    - `pendingGrade` on a deferred `ItemOutcome`.
+  - **`@intellectif/lk-core/ai-check`** has five grading cases, for a `grading` port.
+  - **The grade-stability corpus** gains 123 vectors for the new functions. The 635 before them are unchanged.
+
+  Guide: https://github.com/intellectif/learning-kit/blob/main/docs/grading.md
+
+## 1.4.0
+
+### Minor Changes
+
+- [#107](https://github.com/intellectif/learning-kit/pull/107) [`8bf7d6e`](https://github.com/intellectif/learning-kit/commit/8bf7d6edb76ff45ce32e425ef83663692127acc9) Thanks [@diego28e](https://github.com/diego28e)! - **AI help reads a question's group**: hints, explanations and feedback on writing now carry the passage, or the shared instructions, a grouped question is read beside.
+
+  **Action required:** none. Requests gain an optional `material`, which your prompt uses once you tell it to. A port that sends the whole request to its model sends the material too: up to 12,000 code points more per call on a question in a group.
+
+  - **lk-core:** `aiExplanationRequest`, `aiHintRequest` and `aiWritingFeedbackRequest` take an optional `group` (an `ItemGroup`, a redacted one, or your own `{ title, stimulus }`) and add `material`: the kind, the title, the text, the media type and its alt text, the attribution and the language. The text is cut past `AI_MATERIAL_MAX_LENGTH` (12,000 code points), at a word boundary, and marked `truncated`. The author's transcript and media addresses never go, so a whole group and a redacted one give the same material. The checks are unchanged: a hint that quotes the passage where it prints the answer is refused like any other.
+  - **lk-react:** inside `<ActivitySequence>` and `<InteractiveVideo>`, the SDK's own components, and a question you draw with `useAiHints`, `useAiExplanation` or `useAiWritingFeedback`, send their group's material by themselves. Outside a pager, pass `group` to the hooks. lk-react 23.2.0 peers on lk-core `^1.4.0`, so upgrade the two together. Coaching on a reading is unchanged.
+  - **`@intellectif/lk-core/ai-check`** has four more cases in item groups: a reading question explained, and hinted at, from a passage that prints its answer; a word-formation blank under its group's instructions; and a summary of the passage.
+
+  Guide: https://github.com/intellectif/learning-kit/blob/main/docs/ai.md#a-question-in-an-item-group-140
+
 ## 1.3.0
 
 ### Minor Changes

package/README.md

@@ -25,6 +25,7 @@ npm install @intellectif/lk-core
 - **Dictation marking as data** — a dictation is graded by the character-level similarity of the typed text to the transcript, with one `ScoringDetail` per word carrying its own `score`. `alignDictation(data, text)` returns the same comparison the grade was made of (normalised strings, word pairings, which accepted transcript won), `diffDictationChars(reference, attempt)` the character-level edits for a marked display, and `dictationReferenceWords(data)` the ids and words a stored result can be checked against — so a review screen or an analytics job never re-implements the arithmetic. `score()` and `evaluate()` take an optional `{ rounding }` to compare the pass line as displayed.
 - **`evaluate(data, response)` → `ItemOutcome`** — the resilient result: `scored`, `deferred` (graded later by an AI or a human), `graded` (the grade came back), or `unscorable`. "Not graded yet" is expressible in the type system and is never conflated with a zero.
 - **Deferred grading, both directions** — `GradeRecord`, `gradeFromRubric()` (the weighted total as a pure function of the grader's judgements — never the model's arithmetic; declare `maxScore` per criterion if your grader works out of 100 or out of 9), `outcomeFromGrade()` (which refuses a record whose numbers cannot be a grade instead of mirroring them), `hasGrade()`.
+- **Assisted grading, behind a calibration gate** — `aiGradingRequest` / `checkAiGrading` hold a model's grade of a written response to every criterion of the rubric, each on its own scale (levels the author writes, or a score); `gradeFromJudgement` computes the total. `calibrationReport` measures a model, prompt, rubric and target against your teachers' grades, and `routeGrade` sends a grade to a person, for reasons it names: in an exam, until a matching calibration is supplied. No thresholds ship: they are yours.
 - **Assessment composition** — `composeAssessmentScore()` for weighted, sectioned totals, with a `provisional` status so an unmarked essay never deflates a grade — and a grade out of contract (`NaN`, 85 "out of 1") never finalises one — and `RoundingPolicy` / `roundGrade` / `gte` / `classifyBand` keeping grade rounding and band classification as the two distinct operations they are.
 - **Item groups** — `Stimulus` + `ItemGroup` (one passage or recording serving several questions), `flattenSequence()` for seeded, shuffle-atomic presentation order with stable slot identity. Slot ids are positional by default (`"3"`, `"1.0"`); author a `slotKey` on an entry and its id survives an insert, so a stored response cannot slide onto a different question.
 - **Delivery policies** — `DeliveryPolicy` says what a school lets a learner see around a question: feedback, solutions, hints and AI help. `resolveDeliveryPolicy` spells one out (an unreadable setting restricts, never allows), `validateDeliveryPolicy` checks one before it is stored, and `combineDeliveryPolicies` keeps a setting on only where every policy leaves it on.
@@ -161,6 +162,7 @@ Zod is the single runtime dependency, pinned to one exact version and private: n
 - [Delivery policies](https://github.com/intellectif/learning-kit/blob/main/docs/delivery.md) — `DeliveryPolicy`, and recording it with an attempt.
 - [Scoring policies](https://github.com/intellectif/learning-kit/blob/main/docs/scoring.md) — `ItemScoringPolicy`: tries, which counts, what hints cost, and scoring them on a server.
 - [AI help for learners](https://github.com/intellectif/learning-kit/blob/main/docs/ai.md) — the AI contract, the checks, and the `ai-check` kit for your prompts.
+- [Grading with a model](https://github.com/intellectif/learning-kit/blob/main/docs/grading.md) — a model's grade for the record: levels, calibration against your teachers, and where a grade goes.
 - [Project README](https://github.com/intellectif/learning-kit#readme) — the full picture, including the React renderers.
 - [Contributing](https://github.com/intellectif/learning-kit/blob/main/CONTRIBUTING.md) — adding a new activity type.
 

package/dist/{ai-B222dtHH.d.ts → ai-BYryFkF9.d.cts}

@@ -134,6 +134,57 @@ interface CriterionScore {
   weight?: number;
   /** Not applicable to this submission (e.g. interaction on a monologue task). */
   notApplicable?: boolean;
+  /**
+   * How likely each of the criterion's levels was, lowest first, when the
+   * grader said: a distribution, whose likeliest level is `band`. Kept on the
+   * record as the evidence a routing rule read (`routeGrade`) and a
+   * calibration measures (`calibrationReport`).
+   */
+  levelProbabilities?: number[];
+}
+/**
+ * What one calibration certifies: a model, a prompt revision, a rubric and a
+ * target language. A change to any of them is a different grader, measured
+ * afresh. Built by `graderConfiguration`.
+ */
+interface GraderConfiguration {
+  /** The model's identifier, from the grade's `grader.model`. */
+  model: string;
+  /** The prompt revision, from the grade's `grader.promptHash`. */
+  promptHash: string;
+  /** `contentHash` of the item's rubric, levels and descriptors included. */
+  rubricHash: string;
+  /** The item's `languageTarget`, or `null` when it has none. */
+  languageTarget: string | null;
+}
+/**
+ * Why `routeGrade` sent a grade to a person, in this order:
+ *
+ * - `no-calibration` — in `exam`, no calibration report for this grader's exact
+ *   configuration was supplied: a model's grade counts on a paper of record
+ *   only once it has been measured;
+ * - `calibration-below-policy` — the report misses one of the policy's
+ *   thresholds, for a criterion or for the total;
+ * - `unsure` — a criterion's likeliest level is less likely than the policy's
+ *   `minLevelProbability`;
+ * - `model-asked` — the grader set `requiresHumanReview`;
+ * - `low-confidence` — the grader's own `confidence` is `low`, and the policy
+ *   routes on it;
+ * - `audit-sample` — the response is in the audit sample: a person marks it
+ *   too, without the model's grade, to keep the calibration current.
+ */
+type GradeRoutingReason = 'no-calibration' | 'calibration-below-policy' | 'unsure' | 'model-asked' | 'low-confidence' | 'audit-sample';
+/** Where a grade went, and why: the record an appeal reads. */
+interface GradeRouting {
+  /** `model` when its grade counts; `person` when a person must mark it first. */
+  to: 'model' | 'person';
+  /** Every reason it went to a person, in {@link GradeRoutingReason}'s order; empty when it did not. */
+  reasons: GradeRoutingReason[];
+  /** The calibration report the decision read, by its hash and size, when one matched. */
+  calibration?: {
+    reportHash: string;
+    pairs: number;
+  };
 }
 /** A suggested correction anchored in the learner's own text. */
 interface InlineCorrection {
@@ -189,6 +240,8 @@ interface GradeRecord {
   usage?: GraderUsage;
   /** ISO 8601 timestamp of when the grade was produced. */
   gradedAt?: string;
+  /** Where `routeGrade` sent this grade, and why, when it was routed. */
+  routing?: GradeRouting;
 }
 //#endregion
 //#region src/types/xapi.d.ts
@@ -770,6 +823,17 @@ interface FillInTheBlanksData {
   /** What an author allows AI to do for this item: see {@link ActivityAiPermissions}. */
   ai?: ActivityAiPermissions;
 }
+/**
+ * One level of a rubric criterion: what a response at this level does. Levels
+ * run from the lowest to the highest, and a grader judging against them names
+ * one.
+ */
+interface WrittenResponseRubricLevel {
+  /** A short name, unique within its criterion: "Developing", "B1". */
+  label: string;
+  /** What a response at this level does, for the grader and the learner. */
+  description?: string;
+}
 /** A single criterion within a written-response grading rubric. */
 interface WrittenResponseRubricCriterion {
   /** Criterion name (e.g. "Task achievement", "Grammar range"). */
@@ -778,6 +842,20 @@ interface WrittenResponseRubricCriterion {
   description?: string;
   /** Non-negative weight of this criterion in the overall grade. */
   weight: number;
+  /**
+   * The criterion's levels, lowest first: 2 to 10, each with a label unique
+   * within the criterion once case and spacing are ignored. A grader judging
+   * against levels names one, and the criterion scores the level's place on
+   * the scale, `index / (levels.length - 1)`: 0 for the lowest, 1 for the
+   * highest.
+   *
+   * Read by the grading functions (`aiGradingRequest`, `gradeFromJudgement`,
+   * `calibrationReport`), which report levels they cannot use. In 1.x
+   * `validateActivity` does not check it: your own fields pass through it, and
+   * a field of yours named `levels` stays valid. The item critic warns about
+   * levels the grading functions cannot read.
+   */
+  levels?: WrittenResponseRubricLevel[];
 }
 /** Grading rubric attached to a written-response activity. */
 interface WrittenResponseRubric {
@@ -1232,8 +1310,12 @@ interface DeferredScoringPartial {
  *   a finite number, a `maxScore` of 0 or less, or a score outside 0 to
  *   `maxScore`). The record is kept on `rejectedGrade`. A real grade is still
  *   owed, so re-grade the slot: waiting will not bring one.
+ * - `awaiting_human_review` — a model graded the response, and a routing rule
+ *   (`routeGrade`) sent it to a person: the grade does not count until a
+ *   person marks it. The model's grade is kept on `pendingGrade`, for the
+ *   person and for the next calibration.
  */
-type DeferredReason = 'requires_async_grading' | 'no_response_recorded' | 'grade_rejected';
+type DeferredReason = 'requires_async_grading' | 'no_response_recorded' | 'grade_rejected' | 'awaiting_human_review';
 /**
  * The outcome of evaluating a learner response against an activity — the
  * union `evaluate()` returns. Unlike {@link ScoringResult}, it can express
@@ -1266,6 +1348,13 @@ type ItemOutcome = {
    * is mirrored onto the outcome, and nothing reads it as a grade.
    */
   rejectedGrade?: GradeRecord;
+  /**
+   * The model's grade a routing rule held for a person, when `reason` is
+   * `awaiting_human_review`, with the decision on its `routing`. Kept for
+   * the person and for the next calibration; nothing on it is mirrored
+   * onto the outcome, and nothing reads it as a grade.
+   */
+  pendingGrade?: GradeRecord;
 } | {
   status: 'graded';
   /**
@@ -1491,6 +1580,220 @@ interface DraftContext {
   newId: () => string;
 }
 //#endregion
+//#region src/types/item-group.d.ts
+/** What a stimulus primarily is. Drives validation and layout, never scoring. */
+type StimulusKind = 'text' | 'audio' | 'video' | 'image' | 'mixed';
+/**
+ * Shared material that several items refer to: a reading passage, a
+ * recording, a chart. Modelled as CONTENT — it lives inside the item group
+ * that carries it — so it works in a lesson quiz, a sequence and an exam
+ * section alike, and needs no blueprint to exist.
+ *
+ * Learner-visible by definition: the point of a stimulus is that the learner
+ * reads or hears it. The one exception is `transcript`, an author asset that
+ * `redactItemGroup` removes.
+ */
+interface Stimulus {
+  id: string;
+  kind: StimulusKind;
+  /** Optional heading (a passage title). */
+  title?: string;
+  /**
+   * Plain-text body: the passage itself, or short instructions above media.
+   * Required for `text` and `mixed`.
+   */
+  body?: string;
+  /**
+   * Optional sanitised rich-HTML sidecar of `body`. Requires `body`, which is
+   * the accessible fallback rendered when no sanitiser is supplied.
+   */
+  bodyHtml?: string;
+  /** The recording, video or image. Required for `audio`, `video`, `image` and `mixed`. */
+  media?: ActivityMedia;
+  /**
+   * Author-only transcript of `media` — for item generation and grading,
+   * never shown to the learner. Removed by `redactItemGroup`.
+   */
+  transcript?: string;
+  /** BCP 47 language tag of the material, when it differs from the items'. */
+  locale?: string;
+  /** Source credit, shown to the learner after the material. */
+  attribution?: string;
+}
+/**
+ * A quiz inside an interactive video: one moment of the video, and the
+ * questions shown there. The video pauses when playback reaches `at`, and the
+ * questions are shown one at a time, in `itemIds` order.
+ */
+interface TimelineCue {
+  /** Unique within the timeline. Names the quiz in events, the contents panel and drafts. */
+  id: string;
+  /**
+   * Seconds from the start of the video: finite and 0 or more. The SDK never
+   * reads the video, so it cannot check this against the duration: a quiz at
+   * or past the end opens when the video ends.
+   */
+  at: number;
+  /**
+   * `id`s of items in this group, in the order the quiz shows them. Non-empty.
+   * Every item of the group belongs to exactly one quiz.
+   */
+  itemIds: string[];
+  /** Optional heading shown above the questions. At most 120 characters. */
+  title?: string;
+  /**
+   * Every question must be answered (practice) or submitted (exam) before
+   * playback may pass `at`. Default false.
+   */
+  required?: boolean;
+}
+/** A titled section of the video. */
+interface TimelineChapter {
+  /** Seconds from the start: finite, 0 or more, strictly increasing across the list. */
+  at: number;
+  /** Non-empty after trimming; at most 120 characters. */
+  title: string;
+}
+/**
+ * The arrangement that turns an item group with a video stimulus into an
+ * interactive video: when each quiz opens, the video's chapters, and how far
+ * ahead a learner may seek.
+ *
+ * It lives on the container rather than inside `stimulus` because it describes
+ * how THESE questions are arranged over the material, not the material itself.
+ */
+interface MediaTimeline {
+  cues: TimelineCue[];
+  chapters?: TimelineChapter[];
+  /**
+   * `free` (default): seek anywhere. `no-skip-ahead`: rewind freely, never past
+   * the furthest point reached. Either way, playback never passes a `required`
+   * quiz that is not finished.
+   */
+  navigation?: 'free' | 'no-skip-ahead';
+}
+/**
+ * Where a learner stands in an interactive video, stored by the host so the
+ * video can resume. Kept apart from `AttemptState`, which holds answers by slot
+ * and drops fields it does not know. Read it back through
+ * `readMediaProgress`, never directly.
+ */
+interface MediaProgress {
+  progressVersion: '1.0';
+  /** Seconds: where playback last stood. */
+  at: number;
+  /**
+   * Seconds: the furthest point reached. Decides how far `no-skip-ahead` lets
+   * the learner seek, and how much of the transcript it shows.
+   */
+  furthest: number;
+}
+/**
+ * One stimulus serving several items. Two rules, both taken from observed
+ * failures: a group is SHUFFLE-ATOMIC (shuffling a sequence moves the group
+ * as one block — interleaving two passages' questions is the defect), and its
+ * stimulus is presented persistently alongside every item, never behind a
+ * toggle the learner must reopen per question.
+ *
+ * Generic over the item type so one shape describes a group of full activity
+ * data (authoring, server) and a group of `redact()` projections (client)
+ * without a cast at every boundary.
+ */
+interface ItemGroup<TItem = ActivityData> {
+  schemaVersion: '1.0';
+  type: 'item-group';
+  id: string;
+  title?: string;
+  /**
+   * Stable identity for this entry's slots, independent of where it sits in
+   * the array. See {@link SequenceSlot.slotId}: without one, inserting a
+   * question above this group re-maps every slot id beneath it, and stored
+   * grades quietly start naming different questions.
+   */
+  slotKey?: string;
+  stimulus: Stimulus;
+  /**
+   * The items, in authored order. Non-empty; ids unique within the group; no
+   * nested groups.
+   *
+   * An item may declare its own `slotKey`, for the same reason an entry can:
+   * it pins the item's identity within the group (`"reading.q1"` rather than
+   * `"reading.0"`), so inserting a question into a published group does not
+   * re-map the ones after it.
+   */
+  items: (TItem & {
+    slotKey?: string;
+  })[];
+  /**
+   * `none` (default) keeps authored order; `within-group` shuffles the items
+   * among themselves under the sequence seed. Either way the group stays one
+   * contiguous block.
+   */
+  shuffle?: 'none' | 'within-group';
+  /**
+   * Makes the group an interactive video: the stimulus must be a video, every
+   * item must be one of `INTERACTIVE_VIDEO_ITEM_TYPES`, and every item belongs
+   * to exactly one quiz. Items are then presented in quiz order, never
+   * shuffled. An older lk-core that does not know this field presents the same
+   * items as an ordinary video testlet, with the same slots and grades.
+   */
+  timeline?: MediaTimeline;
+}
+/**
+ * What a sequence is made of: loose activities and item groups, in authored
+ * order.
+ *
+ * A plain activity may carry `slotKey` here even though no activity SCHEMA
+ * declares one, because a slot key describes an item's PLACE in a paper, not
+ * its content — the same question keeps its own id in every paper it appears
+ * in. Putting it on the entry rather than on the activity is what lets it be
+ * authored without every activity type having to know about assessment
+ * assembly. See {@link SequenceSlot.slotId}.
+ */
+type SequenceEntry<TItem = ActivityData> = (TItem & {
+  slotKey?: string;
+}) | ItemGroup<TItem>;
+/** The group a presented slot belongs to. */
+interface SequenceSlotGroup {
+  id: string;
+  title?: string;
+  stimulus: Stimulus;
+  /** 0-based position of this slot within the group's PRESENTED order. */
+  position: number;
+  /** Number of slots in the group. */
+  size: number;
+  /** The group's timeline, when the group is an interactive video. */
+  timeline?: MediaTimeline;
+  /** The quiz this slot's item sits in, when the group is an interactive video. */
+  cue?: TimelineCue;
+}
+/** One presented position in a flattened sequence. */
+interface SequenceSlot<TItem = ActivityData> {
+  /**
+   * Identity of the slot within the sequence definition — unique, and stable
+   * under shuffling. Feed it to `composeAssessmentScore` as `slotId`.
+   *
+   * By default it is derived from the AUTHORED position (`"2"` for the third
+   * top-level entry; `"2.1"` for the second item of that entry when it is a
+   * group), so the same activity can appear in two entries and still be two
+   * slots.
+   *
+   * **A positional id is only valid against one version of the entries
+   * array.** Insert a question at the top of a published paper and every id
+   * below it shifts: rows stored as `"3"` now name what used to be entry 2,
+   * and a re-grade or a review render pairs each response with the wrong
+   * question — silently, because the ids still look valid. For any content
+   * whose slot ids you persist, give the entries an explicit `slotKey`; it is
+   * used verbatim here and survives insertion, deletion and re-ordering.
+   */
+  slotId: string;
+  /** 0-based PRESENTED position, after shuffling. */
+  index: number;
+  activity: TItem;
+  /** Present when the slot comes from an item group. */
+  group?: SequenceSlotGroup;
+}
+//#endregion
 //#region src/types/ai.d.ts
 /**
  * What an author allows AI to do for one item. Absent, or a feature left out,
@@ -1629,6 +1932,46 @@ interface AiDictationFacts extends AiFactsBase {
  * learner saw it, what they answered, and — part by part — the SDK's verdict.
  */
 type AiItemFacts = AiMultipleChoiceFacts | AiFillInTheBlanksFacts | AiGapSelectFacts | AiDictationFacts;
+/**
+ * The item group a question sits in, as a renderer holds it: an `ItemGroup`,
+ * a redacted one, a sequence slot's `group`, or a host's own object with the
+ * same two fields. Structural, and read defensively, like
+ * {@link AiActivityInput}: a group whose material cannot be read gives none.
+ */
+interface AiGroupInput {
+  readonly title?: unknown;
+  readonly stimulus?: unknown;
+}
+/**
+ * What a model is told about the material of the group a question sits in:
+ * what the learner reads and sees beside the question — a passage, or the
+ * instructions a run of questions shares. Never a media address, and never
+ * the author-only transcript: the same facts whether the group is whole or
+ * redacted.
+ */
+interface AiMaterialFacts {
+  /** What the material primarily is. */
+  kind: StimulusKind;
+  /** The material's heading, or the group's title when it has none. */
+  title?: string;
+  /**
+   * The text beside the questions, as plain text: the passage, or the
+   * instructions above a recording or image. Cut past `AI_MATERIAL_MAX_LENGTH`
+   * code points, at a word boundary where there is one.
+   */
+  text?: string;
+  /** Present when `text` was cut: the model is given only its start. */
+  truncated?: true;
+  /** The recording, video or image, without its address. An image's `alt` says what it shows. */
+  media?: {
+    type: ActivityMedia['type'];
+    alt?: string;
+  };
+  /** The source credit the learner sees after the material. */
+  attribution?: string;
+  /** The language of the material, when the author gave one. */
+  locale?: string;
+}
 /**
  * Asks for an explanation of a graded answer. Sent after the SDK has graded
  * it, so `grade` is always present, and an explanation that contradicts
@@ -1638,6 +1981,8 @@ interface AiExplanationRequest {
   feature: 'explanation';
   facts: AiItemFacts;
   grade: AiGrade;
+  /** The material of the group the question sits in; absent outside a group. */
+  material?: AiMaterialFacts;
   /** The language to answer in: the learner's interface language, when known. */
   learnerLocale?: string;
 }
@@ -1653,6 +1998,8 @@ interface AiHintRequest {
   hintNumber: number;
   /** The hints already shown for this item, oldest first. */
   previousHints: string[];
+  /** The material of the group the question sits in; absent outside a group. */
+  material?: AiMaterialFacts;
   learnerLocale?: string;
 }
 /** What produced an AI text, for the record a host keeps. */
@@ -1693,8 +2040,15 @@ interface AiTextResult {
  * a word the engine did not mark, or on a sound it did not report.
  * `contradicts-item` is a critique of an item that points at a field the item
  * does not have, or quotes words that field does not contain.
- */
-type AiRefusal = 'malformed' | 'empty' | 'too-long' | 'contradicts-grade' | 'reveals-answer' | 'misquotes-answer' | 'contradicts-marks' | 'contradicts-item';
+ * `misses-criteria` is a grade that leaves a rubric criterion out, renames
+ * one, judges one twice, or calls one not applicable: judging fewer criteria
+ * is judging a different task, and the total would be renormalised over the
+ * rest. `out-of-scale` is a judgement off its criterion's scale: a level the
+ * criterion does not have, a score outside 0 to 1, or level probabilities that
+ * are not a distribution over its levels, or whose likeliest level is not the
+ * one chosen.
+ */
+type AiRefusal = 'malformed' | 'empty' | 'too-long' | 'contradicts-grade' | 'reveals-answer' | 'misquotes-answer' | 'contradicts-marks' | 'contradicts-item' | 'misses-criteria' | 'out-of-scale';
 /** One criterion of a written response's rubric, as the author wrote it. */
 interface AiRubricCriterionFact {
   name: string;
@@ -1735,6 +2089,11 @@ interface AiWritingFeedbackRequest {
   draftNumber: number;
   /** The text of the feedback already shown on earlier drafts, oldest first. */
   previousFeedback: string[];
+  /**
+   * The material of the group the task sits in — the text a summary is of —
+   * absent outside a group.
+   */
+  material?: AiMaterialFacts;
   /** The language to write in: the learner's interface language, when known. */
   learnerLocale?: string;
 }
@@ -1800,6 +2159,104 @@ interface AiWritingFeedback {
   provenance?: AiProvenance;
   usage?: GraderUsage;
 }
+/** One criterion of the rubric a model grades against, as the author wrote it. */
+interface AiGradingCriterionFact {
+  name: string;
+  description?: string;
+  weight: number;
+  /**
+   * The levels to choose from, lowest first, as the author wrote them; `null`
+   * when the criterion is judged on a score from 0 to 1.
+   */
+  levels: {
+    label: string;
+    description?: string;
+  }[] | null;
+}
+/**
+ * What a model is told about a submitted written response it grades: the task
+ * as the learner saw it, the response verbatim, and the rubric with its levels.
+ */
+interface AiGradingFacts {
+  activityType: 'written-response';
+  activityId: string;
+  title: string;
+  /** The language the item is written in, when the author gave one. */
+  locale?: string;
+  prompt: string;
+  /** The response, exactly as submitted. Every correction quotes it. */
+  text: string;
+  /** Recomputed with `countWords`, as the grader counts it. */
+  wordCount: number;
+  minWords: number;
+  maxWords: number;
+  withinWordBounds: boolean;
+  /** Every criterion of the item's rubric, in its order. */
+  rubric: AiGradingCriterionFact[];
+  /** The target language and level, when the author gave one (e.g. `"en-B1"`). */
+  languageTarget?: string;
+}
+/**
+ * Asks a model to grade a submitted written response. Built on the server,
+ * from the whole item: grading needs the rubric, never the learner's view.
+ */
+interface AiGradingRequest {
+  feature: 'grading';
+  facts: AiGradingFacts;
+  /** The material of the group the task sits in; absent outside a group. */
+  material?: AiMaterialFacts;
+  /** The language to write comments and feedback in, when known. */
+  learnerLocale?: string;
+}
+/**
+ * Why `aiGradingRequest` built no request:
+ *
+ * - `not-written-response` — the item is another type;
+ * - `redacted` — a `redact()` projection: grading is done on the server, from
+ *   the whole item;
+ * - `no-rubric` — no rubric, or a criterion without a name and a weight: a
+ *   model has nothing to be held to;
+ * - `duplicate-criteria` — two criteria with one name once case and spacing are
+ *   ignored: a model cannot judge them apart (`path` names the later one);
+ * - `unreadable-levels` — a criterion's `levels` are there but cannot be used
+ *   (`path` names them; the item critic's `wr_levels_unreadable` says why);
+ * - `empty-response` — nothing was written, so there is nothing to judge.
+ */
+type AiGradingRequestProblem = 'not-written-response' | 'redacted' | 'no-rubric' | 'duplicate-criteria' | 'unreadable-levels' | 'empty-response';
+/** A model's judgement of one criterion, as `checkAiGrading` accepted it. */
+interface AiGradedCriterion {
+  /** The criterion's name, as the author wrote it. */
+  name: string;
+  /** The level chosen, as the author labelled it, on a criterion with levels. */
+  level?: string;
+  /**
+   * From 0 to 1: the level's place on the scale, `index / (levels - 1)`, or the
+   * model's score on a criterion without levels.
+   */
+  score: number;
+  /** How likely each level is, lowest first, when the model said: a distribution. */
+  probabilities?: number[];
+  comment?: string;
+}
+/**
+ * A model's grade of a written response as `checkAiGrading` accepted it: every
+ * criterion of the rubric judged, on its own scale. The total is not here —
+ * `gradeFromJudgement` computes it.
+ */
+interface AiGradingJudgement {
+  /** Every criterion of the rubric, once, in the rubric's order. */
+  criteria: AiGradedCriterion[];
+  /** Narrative feedback addressed to the learner. */
+  feedback: string | null;
+  /** Corrections, each anchored in the response, as on feedback on writing. */
+  corrections: InlineCorrection[];
+  /** How the model reached its verdict; for the record, never shown. */
+  rationale?: string;
+  /** The model asked for a person to mark this response. */
+  requiresHumanReview: boolean;
+  provenance?: AiProvenance;
+  usage?: GraderUsage;
+}
 /** One sound of a word, as the engine reported it. */
 interface AiSoundFact {
   /** The sound expected, in the assessment's phoneme alphabet. */
@@ -2150,5 +2607,5 @@ interface AiVideoDraft {
   findings: ItemFinding[];
 }
 //#endregion
-export { DraftComplete as $, WrittenResponseLearnerResponse as $t, AiGapSelectFacts as A, ItemOutcome as At, AiRefusal as B, ReadAloudData as Bt, AiDraftsCall as C, GapSelectBank as Ct, AiFeature as D, GapSelectLearnerResponse as Dt, AiExplanationRequest as E, GapSelectGap as Et, AiMultipleChoiceFacts as F, MultipleChoiceData as Ft, AiVerdict as G, RecordingBounds as Gt, AiSoundFact as H, ReadAloudDimensionWeight as Ht, AiOptionFact as I, MultipleChoiceLearnerResponse as It, AiWritingCorrection as J, ScoringOutcome as Jt, AiVideoDraft as K, RecordingRef as Kt, AiProvenance as L, MultipleChoiceOption as Lt, AiGrade as M, LearnerResponseMap as Mt, AiHintRequest as N, MediaPlaybackPolicy as Nt, AiFillInTheBlanksFacts as O, InteractionEvent as Ot, AiItemFacts as P, MediaTrack as Pt, AiWritingFeedbackResult as Q, WrittenResponseData as Qt, AiReadingFacts as R, MultipleChoiceOptionMedia as Rt, AiDrafts as S, matchText as Sn, FillInTheBlanksLearnerResponse as St, AiDraftsRun as T, GapSelectData as Tt, AiSupportedActivityType as U, ReadAloudLearnerResponse as Ut, AiRubricCriterionFact as V, ReadAloudDimension as Vt, AiTextResult as W, ReadAloudSlowMedia as Wt, AiWritingFeedback as X, ValidationError as Xt, AiWritingFacts as Y, ScoringResult as Yt, AiWritingFeedbackRequest as Z, ValidationResult as Zt, AiDraftCaption as _, GradingState as _n, DictationEquivalence as _t, AiCoachingRequest as a, XAPIContextActivities as an, ItemFinding as at, AiDraftSource as b, TextMatchResult as bn, DictationTolerance as bt, AiCriterionJudgement as c, XAPIResult as cn, ActivityDataMap as ct, AiCritiqueFinding as d, XAPIVerbObject as dn, ActivityResult as dt, WrittenResponseRubric as en, DraftContext as et, AiCritiqueFindingResult as f, CriterionScore as fn, ActivityType as ft, AiDictationFacts as g, GraderUsage as gn, DictationData as gt, AiCritiqueResult as h, GraderKind as hn, DeferredScoringPartial as ht, AiCoaching as i, XAPIContext as in, DraftValidationResult as it, AiGeneratedDraft as j, LearnerResponse as jt, AiGapFact as k, InteractionKind as kt, AiCritique as l, XAPIScore as ln, ActivityFeedback as lt, AiCritiqueRequest as m, Grader as mn, DeferredReason as mt, AiActivityInput as n, XAPIActor as nn, DraftNotComplete as nt, AiCoachingResult as o, XAPIError as on, ItemFindingSeverity as ot, AiCritiqueKind as p, GradeRecord as pn, BlankConfig as pt, AiWordFact as q, ScoringDetail as qt, AiBlankFact as r, XAPIConfig as rn, DraftSeverity as rt, AiCoachingWord as s, XAPIObject as sn, ActivityData as st, ActivityAiPermissions as t, WrittenResponseRubricCriterion as tn, DraftIssue as tt, AiCritiqueField as u, XAPIStatement as un, ActivityMedia as ut, AiDraftProblem as v, InlineCorrection as vn, DictationLearnerResponse as vt, AiDraftsRequest as w, GapSelectChoice as wt, AiDraftType as x, levenshteinDistance as xn, FillInTheBlanksData as xt, AiDraftSettings as y, TextMatchPolicy as yn, DictationSlowMedia as yt, AiReadingWordFact as z, NativeControlHint as zt };
-//# sourceMappingURL=ai-B222dtHH.d.ts.map
+export { AiVerdict as $, MultipleChoiceData as $t, AiGapSelectFacts as A, XAPIVerbObject as An, ActivityType as At, AiHintRequest as B, InlineCorrection as Bn, FillInTheBlanksLearnerResponse as Bt, AiDraftsCall as C, XAPIContext as Cn, ItemFinding as Ct, AiFeature as D, XAPIResult as Dn, ActivityFeedback as Dt, AiExplanationRequest as E, XAPIObject as En, ActivityDataMap as Et, AiGradingFacts as F, Grader as Fn, DictationEquivalence as Ft, AiProvenance as G, GapSelectLearnerResponse as Gt, AiMaterialFacts as H, TextMatchResult as Hn, GapSelectChoice as Ht, AiGradingJudgement as I, GraderConfiguration as In, DictationLearnerResponse as It, AiRefusal as J, ItemOutcome as Jt, AiReadingFacts as K, InteractionEvent as Kt, AiGradingRequest as L, GraderKind as Ln, DictationSlowMedia as Lt, AiGrade as M, GradeRecord as Mn, DeferredReason as Mt, AiGradedCriterion as N, GradeRouting as Nn, DeferredScoringPartial as Nt, AiFillInTheBlanksFacts as O, XAPIScore as On, ActivityMedia as Ot, AiGradingCriterionFact as P, GradeRoutingReason as Pn, DictationData as Pt, AiTextResult as Q, MediaTrack as Qt, AiGradingRequestProblem as R, GraderUsage as Rn, DictationTolerance as Rt, AiDrafts as S, XAPIConfig as Sn, DraftValidationResult as St, AiDraftsRun as T, XAPIError as Tn, ActivityData as Tt, AiMultipleChoiceFacts as U, levenshteinDistance as Un, GapSelectData as Ut, AiItemFacts as V, TextMatchPolicy as Vn, GapSelectBank as Vt, AiOptionFact as W, matchText as Wn, GapSelectGap as Wt, AiSoundFact as X, LearnerResponseMap as Xt, AiRubricCriterionFact as Y, LearnerResponse as Yt, AiSupportedActivityType as Z, MediaPlaybackPolicy as Zt, AiDraftCaption as _, WrittenResponseLearnerResponse as _n, DraftComplete as _t, AiCoachingRequest as a, ReadAloudDimension as an, AiWritingFeedbackRequest as at, AiDraftSource as b, WrittenResponseRubricLevel as bn, DraftNotComplete as bt, AiCriterionJudgement as c, ReadAloudSlowMedia as cn, MediaProgress as ct, AiCritiqueFinding as d, ScoringDetail as dn, SequenceSlot as dt, MultipleChoiceLearnerResponse as en, AiVideoDraft as et, AiCritiqueFindingResult as f, ScoringOutcome as fn, SequenceSlotGroup as ft, AiDictationFacts as g, WrittenResponseData as gn, TimelineCue as gt, AiCritiqueResult as h, ValidationResult as hn, TimelineChapter as ht, AiCoaching as i, ReadAloudData as in, AiWritingFeedback as it, AiGeneratedDraft as j, CriterionScore as jn, BlankConfig as jt, AiGapFact as k, XAPIStatement as kn, ActivityResult as kt, AiCritique as l, RecordingBounds as ln, MediaTimeline as lt, AiCritiqueRequest as m, ValidationError as mn, StimulusKind as mt, AiActivityInput as n, MultipleChoiceOptionMedia as nn, AiWritingCorrection as nt, AiCoachingResult as o, ReadAloudDimensionWeight as on, AiWritingFeedbackResult as ot, AiCritiqueKind as p, ScoringResult as pn, Stimulus as pt, AiReadingWordFact as q, InteractionKind as qt, AiBlankFact as r, NativeControlHint as rn, AiWritingFacts as rt, AiCoachingWord as s, ReadAloudLearnerResponse as sn, ItemGroup as st, ActivityAiPermissions as t, MultipleChoiceOption as tn, AiWordFact as tt, AiCritiqueField as u, RecordingRef as un, SequenceEntry as ut, AiDraftProblem as v, WrittenResponseRubric as vn, DraftContext as vt, AiDraftsRequest as w, XAPIContextActivities as wn, ItemFindingSeverity as wt, AiDraftType as x, XAPIActor as xn, DraftSeverity as xt, AiDraftSettings as y, WrittenResponseRubricCriterion as yn, DraftIssue as yt, AiGroupInput as z, GradingState as zn, FillInTheBlanksData as zt };
+//# sourceMappingURL=ai-BYryFkF9.d.cts.map