@intellectif/lk-core 1.0.0 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +15 -0
- package/README.md +1 -1
- package/dist/{activity-DgpBJeBc.d.cts → activity-MBOYLBjR.d.cts} +117 -7
- package/dist/{activity-DgpBJeBc.d.ts → activity-MBOYLBjR.d.ts} +117 -7
- package/dist/ai-check.cjs +137 -8
- package/dist/ai-check.cjs.map +1 -1
- package/dist/ai-check.d.cts +6 -3
- package/dist/ai-check.d.ts +6 -3
- package/dist/ai-check.js +132 -3
- package/dist/ai-check.js.map +1 -1
- package/dist/{chunk-W5KB3DNT.cjs → chunk-LDFQUZII.cjs} +230 -9
- package/dist/chunk-LDFQUZII.cjs.map +1 -0
- package/dist/{chunk-RCZCP5OE.js → chunk-NQHETFDA.js} +227 -6
- package/dist/chunk-NQHETFDA.js.map +1 -0
- package/dist/{index-BlOLlwOi.d.cts → index-DlCW0XaQ.d.cts} +1 -1
- package/dist/{index-BVNeSCWC.d.ts → index-DlcqkM_L.d.ts} +1 -1
- package/dist/index.cjs +10 -2
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +68 -11
- package/dist/index.d.ts +68 -11
- package/dist/index.js +9 -1
- package/dist/index.js.map +1 -1
- package/dist/schemas.d.cts +2 -2
- package/dist/schemas.d.ts +2 -2
- package/dist/scoring.d.cts +242 -242
- package/dist/scoring.d.ts +242 -242
- package/dist/xapi.d.cts +1 -1
- package/dist/xapi.d.ts +1 -1
- package/package.json +1 -1
- package/dist/chunk-RCZCP5OE.js.map +0 -1
- package/dist/chunk-W5KB3DNT.cjs.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,20 @@
|
|
|
1
1
|
# @intellectif/lk-core
|
|
2
2
|
|
|
3
|
+
## 1.1.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- [#86](https://github.com/intellectif/learning-kit/pull/86) [`2215060`](https://github.com/intellectif/learning-kit/commit/22150600b0d8f5f058214a0551566b58e154276d) Thanks [@diego28e](https://github.com/diego28e)! - **Coaching on a reading aloud**: "Coach me on this reading" under a read-aloud's marks, where a model may coach only the words the speech engine marked.
|
|
8
|
+
|
|
9
|
+
**Action required:** none unless your code switches over `AiFeature`, `AiRefusal` or `InteractionKind` without a `default` branch (new members `'pronunciation-coaching'`, `'contradicts-marks'` and `'ai-coaching-shown'`), reads `AiCheckCase.request` without narrowing on `feature`, or implements a complete `LkStrings` (six new keys). Nothing changes until you pass a `pronunciationCoaching` port. Install lk-core 1.1.0 with lk-react 23.1.0: lk-react calls the new lk-core functions, and its peer range moves to `^1.1.0`. See https://github.com/intellectif/learning-kit/blob/main/docs/upgrading.md
|
|
10
|
+
|
|
11
|
+
- **lk-core:** `aiCoachingRequest` builds what a model is given: the text, the engine's marks word by word — with each word's sounds and what each was heard as, when the marks come from the assessment — its dimension scores, and the grade. `checkAiCoaching` checks the reply before a learner sees it. Every word it coaches must be one the engine marked mispronounced or omitted, and every sound one the engine reported for that word; anything else refuses the whole reply as `contradicts-marks`. The words come back in reading order, each as the marks show it, and no number in the reply is read. An assessment the grader would not read (no speech, unscripted, another text or locale) gives no coaching.
|
|
12
|
+
- **lk-react:** a `pronunciationCoaching` port on `LearnerAi`. `<ReadAloud>` offers coaching under its marks in `practice` once a take is graded and in `review` wherever marks are shown, from a kept assessment or a stored grade; never in `exam`. `<PronunciationFeedback>` offers it too, given the item's `id` and `title` in `data`, and takes `ai`, `renderMode`, `delivery` and `onInteraction` for it. It follows the author's and the paper's `ai.explanations` and needs `feedback`, not `solutions`. One coaching per take. `useAiCoaching` gives marks you draw yourself the same rules.
|
|
13
|
+
- **Records:** `ai-coaching-shown` (how many words were coached, provenance and usage; never the text), and `ai-help-refused` with `feature: 'pronunciation-coaching'`.
|
|
14
|
+
- **ai-check:** a `pronunciationCoaching` port runs four reading cases: a mispronounced "th" and a word left out, a clean reading, the same slips from a stored grade without sounds, and coaching asked for in Spanish.
|
|
15
|
+
- Six new strings: `aiCoaching`, `aiCoachingLoading`, `aiCoachingHeading`, `aiCoachingUnavailable`, `aiCoachingWords`, `aiCoachingSound`.
|
|
16
|
+
- **Stability:** the promise now says what a minor may add to a type the SDK hands you — a union member, a `LkStrings` key, an optional property — and that a `default` branch keeps a switch correct across minors.
|
|
17
|
+
|
|
3
18
|
## 1.0.0
|
|
4
19
|
|
|
5
20
|
### Major Changes
|
package/README.md
CHANGED
|
@@ -28,7 +28,7 @@ npm install @intellectif/lk-core
|
|
|
28
28
|
- **Item groups** — `Stimulus` + `ItemGroup` (one passage or recording serving several questions), `flattenSequence()` for seeded, shuffle-atomic presentation order with stable slot identity. Slot ids are positional by default (`"3"`, `"1.0"`); author a `slotKey` on an entry and its id survives an insert, so a stored response cannot slide onto a different question.
|
|
29
29
|
- **Delivery policies** — `DeliveryPolicy` says what a school lets a learner see around a question: feedback, solutions, hints and AI help. `resolveDeliveryPolicy` spells one out (an unreadable setting restricts, never allows), `validateDeliveryPolicy` checks one before it is stored, and `combineDeliveryPolicies` keeps a setting on only where every policy leaves it on.
|
|
30
30
|
- **Scoring policies** — `ItemScoringPolicy` says how a paper's questions are scored when a learner can try again or ask for hints: `retries`, which try `counts`, and what a try and a hint cost. `scoreTries` does the arithmetic, `evaluate(…, { scoring })` and `evaluateTries` reach a question's outcome from its answers, and `validateItemScoringPolicy` / `resolveItemScoringPolicy` refuse a policy that cannot be applied rather than guess — every setting moves a grade. No policy changes nothing, and the arithmetic is pinned by the grade vectors.
|
|
31
|
-
- **AI help contracts** — the SDK calls no model; it builds what yours is given and checks what comes back. `aiExplanationRequest`, `aiHintRequest` and `
|
|
31
|
+
- **AI help contracts** — the SDK calls no model; it builds what yours is given and checks what comes back. `aiExplanationRequest`, `aiHintRequest`, `aiWritingFeedbackRequest` and `aiCoachingRequest` build the requests a component makes, so a server can rebuild them from its own copy of an item; `checkAiExplanation`, `checkAiHint`, `checkAiWritingFeedback` and `checkAiCoaching` refuse an explanation that contradicts the grade, a hint that contains the answer, writing feedback that corrects words the learner never wrote, and coaching on a reading that names a word the speech engine did not mark or a sound it did not report; `checkAiWritingFeedback` computes the rubric's indicative score itself.
|
|
32
32
|
- **Attempt plans** — `planAttempt(entries, { seed, shuffleEntries, points, delivery })` freezes the exact paper a learner sat, and the delivery policy it was sat under: slot order, per-slot points, a content hash of each item, and a `planHash` over the lot. `verifyAttemptPlan(plan, current)` reports the drift when that content is later edited (`changedSlotIds`, `changedStimulusSlotIds`, `changedPointsSlotIds`, `missingSlotIds`, …) — the thing you need during a remark or an appeal. `scoredItemsFromPlan(plan, outcomes)` turns collected outcomes into the `ScoredItem[]` composition wants, and defaults a slot with **no** outcome to `deferred` rather than silently shrinking the denominator.
|
|
33
33
|
- **Attempt state** — `serializeAttemptState(plan, progress)` / `restoreAttemptState(plan, snapshot)` round-trip an interrupted attempt (responses, submitted slots, position) and are **bound to the plan**: restoring answers onto a paper the learner never sat throws instead of succeeding quietly. `diffResponses(before, after)` reports what moved between two snapshots, including an answer the learner *cleared*.
|
|
34
34
|
- **Media playback policy** — `media.playback` on an audio recording: `maxPlays`, `seek`, `rate`, and advisory `nativeControlHints`. `resolvePlaybackPolicy()` derives the defaults once so the schema, the plan and the renderer cannot drift. Budgets are frozen into the plan by `planAttempt`, so editing `maxPlays` mid-window cannot change what a past learner was held to, and `planAttempt` refuses a paper where one recording is budgeted under two keys.
|
|
@@ -226,8 +226,8 @@ interface GradeRecord {
|
|
|
226
226
|
interface ActivityAiPermissions {
|
|
227
227
|
/**
|
|
228
228
|
* An explanation of the learner's graded answer — and, on a written
|
|
229
|
-
* response, feedback on a draft
|
|
230
|
-
* learner's own answer.
|
|
229
|
+
* response, feedback on a draft, and on a read-aloud, coaching on the
|
|
230
|
+
* marks: each is a model's words about the learner's own answer.
|
|
231
231
|
*/
|
|
232
232
|
explanations?: boolean;
|
|
233
233
|
/** Hints before the learner submits. */
|
|
@@ -248,7 +248,7 @@ interface AiActivityInput {
|
|
|
248
248
|
readonly ai?: unknown;
|
|
249
249
|
}
|
|
250
250
|
/** The kinds of help a learner can be given. */
|
|
251
|
-
type AiFeature = 'explanation' | 'hint' | 'writing-feedback';
|
|
251
|
+
type AiFeature = 'explanation' | 'hint' | 'writing-feedback' | 'pronunciation-coaching';
|
|
252
252
|
/** The activity types the SDK builds facts for. */
|
|
253
253
|
type AiSupportedActivityType = 'multiple-choice' | 'fill-in-the-blanks' | 'gap-select' | 'dictation';
|
|
254
254
|
/**
|
|
@@ -411,9 +411,10 @@ interface AiTextResult {
|
|
|
411
411
|
/**
|
|
412
412
|
* Why a result was not shown. `misquotes-answer` is writing feedback that
|
|
413
413
|
* corrects words the learner did not write: a correction whose quote is not in
|
|
414
|
-
* the draft, or not where it claims to be.
|
|
414
|
+
* the draft, or not where it claims to be. `contradicts-marks` is coaching on
|
|
415
|
+
* a word the engine did not mark, or on a sound it did not report.
|
|
415
416
|
*/
|
|
416
|
-
type AiRefusal = 'malformed' | 'empty' | 'too-long' | 'contradicts-grade' | 'reveals-answer' | 'misquotes-answer';
|
|
417
|
+
type AiRefusal = 'malformed' | 'empty' | 'too-long' | 'contradicts-grade' | 'reveals-answer' | 'misquotes-answer' | 'contradicts-marks';
|
|
417
418
|
/** One criterion of a written response's rubric, as the author wrote it. */
|
|
418
419
|
interface AiRubricCriterionFact {
|
|
419
420
|
name: string;
|
|
@@ -519,6 +520,110 @@ interface AiWritingFeedback {
|
|
|
519
520
|
provenance?: AiProvenance;
|
|
520
521
|
usage?: GraderUsage;
|
|
521
522
|
}
|
|
523
|
+
/** One sound of a word, as the engine reported it. */
|
|
524
|
+
interface AiSoundFact {
|
|
525
|
+
/** The sound expected, in the assessment's phoneme alphabet. */
|
|
526
|
+
symbol: string;
|
|
527
|
+
/** How close it came, 0..100, when the engine scored it. */
|
|
528
|
+
accuracy?: number;
|
|
529
|
+
/** What the engine heard instead, best first, each with its score 0..100. */
|
|
530
|
+
heardAs?: {
|
|
531
|
+
symbol: string;
|
|
532
|
+
score: number;
|
|
533
|
+
}[];
|
|
534
|
+
}
|
|
535
|
+
/** One word of a reading, as the engine marked it. */
|
|
536
|
+
interface AiReadingWordFact {
|
|
537
|
+
/** `w<n>` for a word of the text; absent for a word the learner added. */
|
|
538
|
+
itemId?: string;
|
|
539
|
+
/** The word of the text, normalised; `''` for a word the learner added. */
|
|
540
|
+
word: string;
|
|
541
|
+
/** What was heard, normalised; `''` for an omitted word. */
|
|
542
|
+
heard: string;
|
|
543
|
+
state: 'correct' | 'mispronounced' | 'omitted' | 'inserted';
|
|
544
|
+
/** The engine's accuracy for the word, 0..100, when it has one. */
|
|
545
|
+
accuracy?: number;
|
|
546
|
+
/** The word's sounds, from the full assessment only. */
|
|
547
|
+
sounds?: AiSoundFact[];
|
|
548
|
+
}
|
|
549
|
+
/**
|
|
550
|
+
* What a model is told about a read-aloud: the text, the engine's marks word
|
|
551
|
+
* by word, and its scores. The marks are the engine's; a model explains them
|
|
552
|
+
* and never makes its own.
|
|
553
|
+
*/
|
|
554
|
+
interface AiReadingFacts {
|
|
555
|
+
activityType: 'read-aloud';
|
|
556
|
+
activityId: string;
|
|
557
|
+
title: string;
|
|
558
|
+
/** The language the text is in. */
|
|
559
|
+
locale: string;
|
|
560
|
+
referenceText: string;
|
|
561
|
+
instructions?: string;
|
|
562
|
+
/** Who made the marks: a measurement engine, a model, or a person. */
|
|
563
|
+
assessor: 'auto' | 'ai' | 'human' | 'unknown';
|
|
564
|
+
/** The alphabet `sounds` are written in, when any are. */
|
|
565
|
+
phonemeAlphabet?: 'ipa' | 'sapi';
|
|
566
|
+
/** The engine's dimension scores, each 0..100, where it measured them. */
|
|
567
|
+
scores: {
|
|
568
|
+
accuracy?: number;
|
|
569
|
+
fluency?: number;
|
|
570
|
+
completeness?: number;
|
|
571
|
+
prosody?: number;
|
|
572
|
+
};
|
|
573
|
+
words: AiReadingWordFact[];
|
|
574
|
+
}
|
|
575
|
+
/**
|
|
576
|
+
* Asks for coaching on a graded reading: what to work on, word by word, and
|
|
577
|
+
* how. Never in an exam.
|
|
578
|
+
*/
|
|
579
|
+
interface AiCoachingRequest {
|
|
580
|
+
feature: 'pronunciation-coaching';
|
|
581
|
+
facts: AiReadingFacts;
|
|
582
|
+
/** The grade of the reading, when there is one. */
|
|
583
|
+
grade?: {
|
|
584
|
+
score: number;
|
|
585
|
+
maxScore: number;
|
|
586
|
+
passed: boolean;
|
|
587
|
+
};
|
|
588
|
+
/** The language to write in: the learner's interface language, when known. */
|
|
589
|
+
learnerLocale?: string;
|
|
590
|
+
}
|
|
591
|
+
/** One word a model coaches, as it returns it. */
|
|
592
|
+
interface AiCoachingWord {
|
|
593
|
+
/** The `itemId` of a word the engine marked mispronounced or omitted. */
|
|
594
|
+
itemId: string;
|
|
595
|
+
/** What to do about it, addressed to the learner. */
|
|
596
|
+
tip: string;
|
|
597
|
+
/**
|
|
598
|
+
* A sound the engine reported for this word — `expected` — and, if the
|
|
599
|
+
* model names one, what the engine heard instead.
|
|
600
|
+
*/
|
|
601
|
+
sound?: {
|
|
602
|
+
expected: string;
|
|
603
|
+
heard?: string;
|
|
604
|
+
};
|
|
605
|
+
}
|
|
606
|
+
/** What a host's `pronunciationCoaching` port returns. */
|
|
607
|
+
interface AiCoachingResult {
|
|
608
|
+
/** The coaching as a whole, addressed to the learner. Plain text, never HTML. */
|
|
609
|
+
text: string;
|
|
610
|
+
words?: AiCoachingWord[];
|
|
611
|
+
provenance?: AiProvenance;
|
|
612
|
+
usage?: GraderUsage;
|
|
613
|
+
}
|
|
614
|
+
/** Coaching, as the SDK accepted it: what a learner is shown. */
|
|
615
|
+
interface AiCoaching {
|
|
616
|
+
text: string;
|
|
617
|
+
/**
|
|
618
|
+
* The words to work on, in reading order, each with the word of the text as
|
|
619
|
+
* the SDK has it — never the model's spelling of it.
|
|
620
|
+
*/
|
|
621
|
+
words: (AiCoachingWord & {
|
|
622
|
+
word: string;
|
|
623
|
+
})[];
|
|
624
|
+
provenance?: AiProvenance;
|
|
625
|
+
usage?: GraderUsage;
|
|
626
|
+
}
|
|
522
627
|
|
|
523
628
|
/**
|
|
524
629
|
* An xAPI Activity object — the thing a statement is about.
|
|
@@ -1714,7 +1819,12 @@ type InteractionKind = 'option-selected' | 'option-deselected' | 'blank-filled'
|
|
|
1714
1819
|
* number of `corrections`, the `indicativeScore` when there was one, and the
|
|
1715
1820
|
* port's `provenance` and `usage` when it sent them). Never the text.
|
|
1716
1821
|
*/
|
|
1717
|
-
| 'ai-writing-feedback-shown'
|
|
1822
|
+
| 'ai-writing-feedback-shown'
|
|
1823
|
+
/**
|
|
1824
|
+
* Coaching on a read-aloud was shown (the number of `words` it coached, and
|
|
1825
|
+
* the port's `provenance` and `usage` when it sent them). Never the text.
|
|
1826
|
+
*/
|
|
1827
|
+
| 'ai-coaching-shown' | (string & {});
|
|
1718
1828
|
/** Fired by activity components on every discrete learner interaction. */
|
|
1719
1829
|
interface InteractionEvent {
|
|
1720
1830
|
/** The kind of interaction that occurred. */
|
|
@@ -1727,4 +1837,4 @@ interface InteractionEvent {
|
|
|
1727
1837
|
payload: Record<string, unknown>;
|
|
1728
1838
|
}
|
|
1729
1839
|
|
|
1730
|
-
export { type
|
|
1840
|
+
export { type GapSelectData as $, type ActivityAiPermissions as A, type AiReadingWordFact as B, type AiRefusal as C, type AiRubricCriterionFact as D, type AiSoundFact as E, type AiSupportedActivityType as F, type AiTextResult as G, type AiVerdict as H, type AiWordFact as I, type AiWritingCorrection as J, type AiWritingFacts as K, type AiWritingFeedback as L, type AiWritingFeedbackRequest as M, type AiWritingFeedbackResult as N, type BlankConfig as O, type CriterionScore as P, type DeferredReason as Q, type DeferredScoringPartial as R, type DictationData as S, type DictationEquivalence as T, type DictationLearnerResponse as U, type DictationSlowMedia as V, type DictationTolerance as W, type FillInTheBlanksData as X, type FillInTheBlanksLearnerResponse as Y, type GapSelectBank as Z, type GapSelectChoice as _, type ActivityData as a, type GapSelectGap as a0, type GapSelectLearnerResponse as a1, type GradeRecord as a2, type Grader as a3, type GraderKind as a4, type GraderUsage as a5, type GradingState as a6, type InlineCorrection as a7, type InteractionEvent as a8, type InteractionKind as a9, type WrittenResponseRubric as aA, type WrittenResponseRubricCriterion as aB, type XAPIActor as aC, type XAPIConfig as aD, type XAPIContext as aE, type XAPIContextActivities as aF, type XAPIError as aG, type XAPIObject as aH, type XAPIResult as aI, type XAPIScore as aJ, type XAPIStatement as aK, type XAPIVerbObject as aL, levenshteinDistance as aM, matchText as aN, type ItemOutcome as aa, type LearnerResponse as ab, type LearnerResponseMap as ac, type MediaPlaybackPolicy as ad, type MediaTrack as ae, type MultipleChoiceData as af, type MultipleChoiceLearnerResponse as ag, type MultipleChoiceOption as ah, type MultipleChoiceOptionMedia as ai, type NativeControlHint as aj, type ReadAloudData as ak, type ReadAloudDimension as al, type ReadAloudDimensionWeight as am, type ReadAloudLearnerResponse as an, type ReadAloudSlowMedia as ao, type RecordingBounds as ap, type RecordingRef as aq, type ScoringDetail as ar, type ScoringOutcome as as, type ScoringResult as at, type TextMatchPolicy as au, type TextMatchResult as av, type ValidationError as aw, type ValidationResult as ax, type WrittenResponseData as ay, type WrittenResponseLearnerResponse as az, type ActivityDataMap as b, type ActivityFeedback as c, type ActivityMedia as d, type ActivityResult as e, type ActivityType as f, type AiActivityInput as g, type AiBlankFact as h, type AiCoaching as i, type AiCoachingRequest as j, type AiCoachingResult as k, type AiCoachingWord as l, type AiCriterionJudgement as m, type AiDictationFacts as n, type AiExplanationRequest as o, type AiFeature as p, type AiFillInTheBlanksFacts as q, type AiGapFact as r, type AiGapSelectFacts as s, type AiGrade as t, type AiHintRequest as u, type AiItemFacts as v, type AiMultipleChoiceFacts as w, type AiOptionFact as x, type AiProvenance as y, type AiReadingFacts as z };
|
|
@@ -226,8 +226,8 @@ interface GradeRecord {
|
|
|
226
226
|
interface ActivityAiPermissions {
|
|
227
227
|
/**
|
|
228
228
|
* An explanation of the learner's graded answer — and, on a written
|
|
229
|
-
* response, feedback on a draft
|
|
230
|
-
* learner's own answer.
|
|
229
|
+
* response, feedback on a draft, and on a read-aloud, coaching on the
|
|
230
|
+
* marks: each is a model's words about the learner's own answer.
|
|
231
231
|
*/
|
|
232
232
|
explanations?: boolean;
|
|
233
233
|
/** Hints before the learner submits. */
|
|
@@ -248,7 +248,7 @@ interface AiActivityInput {
|
|
|
248
248
|
readonly ai?: unknown;
|
|
249
249
|
}
|
|
250
250
|
/** The kinds of help a learner can be given. */
|
|
251
|
-
type AiFeature = 'explanation' | 'hint' | 'writing-feedback';
|
|
251
|
+
type AiFeature = 'explanation' | 'hint' | 'writing-feedback' | 'pronunciation-coaching';
|
|
252
252
|
/** The activity types the SDK builds facts for. */
|
|
253
253
|
type AiSupportedActivityType = 'multiple-choice' | 'fill-in-the-blanks' | 'gap-select' | 'dictation';
|
|
254
254
|
/**
|
|
@@ -411,9 +411,10 @@ interface AiTextResult {
|
|
|
411
411
|
/**
|
|
412
412
|
* Why a result was not shown. `misquotes-answer` is writing feedback that
|
|
413
413
|
* corrects words the learner did not write: a correction whose quote is not in
|
|
414
|
-
* the draft, or not where it claims to be.
|
|
414
|
+
* the draft, or not where it claims to be. `contradicts-marks` is coaching on
|
|
415
|
+
* a word the engine did not mark, or on a sound it did not report.
|
|
415
416
|
*/
|
|
416
|
-
type AiRefusal = 'malformed' | 'empty' | 'too-long' | 'contradicts-grade' | 'reveals-answer' | 'misquotes-answer';
|
|
417
|
+
type AiRefusal = 'malformed' | 'empty' | 'too-long' | 'contradicts-grade' | 'reveals-answer' | 'misquotes-answer' | 'contradicts-marks';
|
|
417
418
|
/** One criterion of a written response's rubric, as the author wrote it. */
|
|
418
419
|
interface AiRubricCriterionFact {
|
|
419
420
|
name: string;
|
|
@@ -519,6 +520,110 @@ interface AiWritingFeedback {
|
|
|
519
520
|
provenance?: AiProvenance;
|
|
520
521
|
usage?: GraderUsage;
|
|
521
522
|
}
|
|
523
|
+
/** One sound of a word, as the engine reported it. */
|
|
524
|
+
interface AiSoundFact {
|
|
525
|
+
/** The sound expected, in the assessment's phoneme alphabet. */
|
|
526
|
+
symbol: string;
|
|
527
|
+
/** How close it came, 0..100, when the engine scored it. */
|
|
528
|
+
accuracy?: number;
|
|
529
|
+
/** What the engine heard instead, best first, each with its score 0..100. */
|
|
530
|
+
heardAs?: {
|
|
531
|
+
symbol: string;
|
|
532
|
+
score: number;
|
|
533
|
+
}[];
|
|
534
|
+
}
|
|
535
|
+
/** One word of a reading, as the engine marked it. */
|
|
536
|
+
interface AiReadingWordFact {
|
|
537
|
+
/** `w<n>` for a word of the text; absent for a word the learner added. */
|
|
538
|
+
itemId?: string;
|
|
539
|
+
/** The word of the text, normalised; `''` for a word the learner added. */
|
|
540
|
+
word: string;
|
|
541
|
+
/** What was heard, normalised; `''` for an omitted word. */
|
|
542
|
+
heard: string;
|
|
543
|
+
state: 'correct' | 'mispronounced' | 'omitted' | 'inserted';
|
|
544
|
+
/** The engine's accuracy for the word, 0..100, when it has one. */
|
|
545
|
+
accuracy?: number;
|
|
546
|
+
/** The word's sounds, from the full assessment only. */
|
|
547
|
+
sounds?: AiSoundFact[];
|
|
548
|
+
}
|
|
549
|
+
/**
|
|
550
|
+
* What a model is told about a read-aloud: the text, the engine's marks word
|
|
551
|
+
* by word, and its scores. The marks are the engine's; a model explains them
|
|
552
|
+
* and never makes its own.
|
|
553
|
+
*/
|
|
554
|
+
interface AiReadingFacts {
|
|
555
|
+
activityType: 'read-aloud';
|
|
556
|
+
activityId: string;
|
|
557
|
+
title: string;
|
|
558
|
+
/** The language the text is in. */
|
|
559
|
+
locale: string;
|
|
560
|
+
referenceText: string;
|
|
561
|
+
instructions?: string;
|
|
562
|
+
/** Who made the marks: a measurement engine, a model, or a person. */
|
|
563
|
+
assessor: 'auto' | 'ai' | 'human' | 'unknown';
|
|
564
|
+
/** The alphabet `sounds` are written in, when any are. */
|
|
565
|
+
phonemeAlphabet?: 'ipa' | 'sapi';
|
|
566
|
+
/** The engine's dimension scores, each 0..100, where it measured them. */
|
|
567
|
+
scores: {
|
|
568
|
+
accuracy?: number;
|
|
569
|
+
fluency?: number;
|
|
570
|
+
completeness?: number;
|
|
571
|
+
prosody?: number;
|
|
572
|
+
};
|
|
573
|
+
words: AiReadingWordFact[];
|
|
574
|
+
}
|
|
575
|
+
/**
|
|
576
|
+
* Asks for coaching on a graded reading: what to work on, word by word, and
|
|
577
|
+
* how. Never in an exam.
|
|
578
|
+
*/
|
|
579
|
+
interface AiCoachingRequest {
|
|
580
|
+
feature: 'pronunciation-coaching';
|
|
581
|
+
facts: AiReadingFacts;
|
|
582
|
+
/** The grade of the reading, when there is one. */
|
|
583
|
+
grade?: {
|
|
584
|
+
score: number;
|
|
585
|
+
maxScore: number;
|
|
586
|
+
passed: boolean;
|
|
587
|
+
};
|
|
588
|
+
/** The language to write in: the learner's interface language, when known. */
|
|
589
|
+
learnerLocale?: string;
|
|
590
|
+
}
|
|
591
|
+
/** One word a model coaches, as it returns it. */
|
|
592
|
+
interface AiCoachingWord {
|
|
593
|
+
/** The `itemId` of a word the engine marked mispronounced or omitted. */
|
|
594
|
+
itemId: string;
|
|
595
|
+
/** What to do about it, addressed to the learner. */
|
|
596
|
+
tip: string;
|
|
597
|
+
/**
|
|
598
|
+
* A sound the engine reported for this word — `expected` — and, if the
|
|
599
|
+
* model names one, what the engine heard instead.
|
|
600
|
+
*/
|
|
601
|
+
sound?: {
|
|
602
|
+
expected: string;
|
|
603
|
+
heard?: string;
|
|
604
|
+
};
|
|
605
|
+
}
|
|
606
|
+
/** What a host's `pronunciationCoaching` port returns. */
|
|
607
|
+
interface AiCoachingResult {
|
|
608
|
+
/** The coaching as a whole, addressed to the learner. Plain text, never HTML. */
|
|
609
|
+
text: string;
|
|
610
|
+
words?: AiCoachingWord[];
|
|
611
|
+
provenance?: AiProvenance;
|
|
612
|
+
usage?: GraderUsage;
|
|
613
|
+
}
|
|
614
|
+
/** Coaching, as the SDK accepted it: what a learner is shown. */
|
|
615
|
+
interface AiCoaching {
|
|
616
|
+
text: string;
|
|
617
|
+
/**
|
|
618
|
+
* The words to work on, in reading order, each with the word of the text as
|
|
619
|
+
* the SDK has it — never the model's spelling of it.
|
|
620
|
+
*/
|
|
621
|
+
words: (AiCoachingWord & {
|
|
622
|
+
word: string;
|
|
623
|
+
})[];
|
|
624
|
+
provenance?: AiProvenance;
|
|
625
|
+
usage?: GraderUsage;
|
|
626
|
+
}
|
|
522
627
|
|
|
523
628
|
/**
|
|
524
629
|
* An xAPI Activity object — the thing a statement is about.
|
|
@@ -1714,7 +1819,12 @@ type InteractionKind = 'option-selected' | 'option-deselected' | 'blank-filled'
|
|
|
1714
1819
|
* number of `corrections`, the `indicativeScore` when there was one, and the
|
|
1715
1820
|
* port's `provenance` and `usage` when it sent them). Never the text.
|
|
1716
1821
|
*/
|
|
1717
|
-
| 'ai-writing-feedback-shown'
|
|
1822
|
+
| 'ai-writing-feedback-shown'
|
|
1823
|
+
/**
|
|
1824
|
+
* Coaching on a read-aloud was shown (the number of `words` it coached, and
|
|
1825
|
+
* the port's `provenance` and `usage` when it sent them). Never the text.
|
|
1826
|
+
*/
|
|
1827
|
+
| 'ai-coaching-shown' | (string & {});
|
|
1718
1828
|
/** Fired by activity components on every discrete learner interaction. */
|
|
1719
1829
|
interface InteractionEvent {
|
|
1720
1830
|
/** The kind of interaction that occurred. */
|
|
@@ -1727,4 +1837,4 @@ interface InteractionEvent {
|
|
|
1727
1837
|
payload: Record<string, unknown>;
|
|
1728
1838
|
}
|
|
1729
1839
|
|
|
1730
|
-
export { type
|
|
1840
|
+
export { type GapSelectData as $, type ActivityAiPermissions as A, type AiReadingWordFact as B, type AiRefusal as C, type AiRubricCriterionFact as D, type AiSoundFact as E, type AiSupportedActivityType as F, type AiTextResult as G, type AiVerdict as H, type AiWordFact as I, type AiWritingCorrection as J, type AiWritingFacts as K, type AiWritingFeedback as L, type AiWritingFeedbackRequest as M, type AiWritingFeedbackResult as N, type BlankConfig as O, type CriterionScore as P, type DeferredReason as Q, type DeferredScoringPartial as R, type DictationData as S, type DictationEquivalence as T, type DictationLearnerResponse as U, type DictationSlowMedia as V, type DictationTolerance as W, type FillInTheBlanksData as X, type FillInTheBlanksLearnerResponse as Y, type GapSelectBank as Z, type GapSelectChoice as _, type ActivityData as a, type GapSelectGap as a0, type GapSelectLearnerResponse as a1, type GradeRecord as a2, type Grader as a3, type GraderKind as a4, type GraderUsage as a5, type GradingState as a6, type InlineCorrection as a7, type InteractionEvent as a8, type InteractionKind as a9, type WrittenResponseRubric as aA, type WrittenResponseRubricCriterion as aB, type XAPIActor as aC, type XAPIConfig as aD, type XAPIContext as aE, type XAPIContextActivities as aF, type XAPIError as aG, type XAPIObject as aH, type XAPIResult as aI, type XAPIScore as aJ, type XAPIStatement as aK, type XAPIVerbObject as aL, levenshteinDistance as aM, matchText as aN, type ItemOutcome as aa, type LearnerResponse as ab, type LearnerResponseMap as ac, type MediaPlaybackPolicy as ad, type MediaTrack as ae, type MultipleChoiceData as af, type MultipleChoiceLearnerResponse as ag, type MultipleChoiceOption as ah, type MultipleChoiceOptionMedia as ai, type NativeControlHint as aj, type ReadAloudData as ak, type ReadAloudDimension as al, type ReadAloudDimensionWeight as am, type ReadAloudLearnerResponse as an, type ReadAloudSlowMedia as ao, type RecordingBounds as ap, type RecordingRef as aq, type ScoringDetail as ar, type ScoringOutcome as as, type ScoringResult as at, type TextMatchPolicy as au, type TextMatchResult as av, type ValidationError as aw, type ValidationResult as ax, type WrittenResponseData as ay, type WrittenResponseLearnerResponse as az, type ActivityDataMap as b, type ActivityFeedback as c, type ActivityMedia as d, type ActivityResult as e, type ActivityType as f, type AiActivityInput as g, type AiBlankFact as h, type AiCoaching as i, type AiCoachingRequest as j, type AiCoachingResult as k, type AiCoachingWord as l, type AiCriterionJudgement as m, type AiDictationFacts as n, type AiExplanationRequest as o, type AiFeature as p, type AiFillInTheBlanksFacts as q, type AiGapFact as r, type AiGapSelectFacts as s, type AiGrade as t, type AiHintRequest as u, type AiItemFacts as v, type AiMultipleChoiceFacts as w, type AiOptionFact as x, type AiProvenance as y, type AiReadingFacts as z };
|
package/dist/ai-check.cjs
CHANGED
|
@@ -5,8 +5,12 @@
|
|
|
5
5
|
|
|
6
6
|
|
|
7
7
|
|
|
8
|
-
|
|
9
|
-
|
|
8
|
+
|
|
9
|
+
|
|
10
|
+
var _chunkLDFQUZIIcjs = require('./chunk-LDFQUZII.cjs');
|
|
11
|
+
|
|
12
|
+
|
|
13
|
+
var _chunkVSH7ECQLcjs = require('./chunk-VSH7ECQL.cjs');
|
|
10
14
|
require('./chunk-DSDXL3JO.cjs');
|
|
11
15
|
|
|
12
16
|
// src/ai-check/cases.ts
|
|
@@ -87,6 +91,68 @@ var weekend = {
|
|
|
87
91
|
]
|
|
88
92
|
}
|
|
89
93
|
};
|
|
94
|
+
var weather = {
|
|
95
|
+
schemaVersion: "1.0",
|
|
96
|
+
type: "read-aloud",
|
|
97
|
+
id: "ai-check-reading",
|
|
98
|
+
title: "Read the forecast aloud",
|
|
99
|
+
referenceText: "The weather is warm in the north.",
|
|
100
|
+
locale: "en-US",
|
|
101
|
+
recording: { maxSeconds: 20 },
|
|
102
|
+
scoring: {
|
|
103
|
+
dimensions: [
|
|
104
|
+
{ name: "accuracy", weight: 2 },
|
|
105
|
+
{ name: "fluency", weight: 1 }
|
|
106
|
+
]
|
|
107
|
+
}
|
|
108
|
+
};
|
|
109
|
+
var said = (text, over = {}) => ({
|
|
110
|
+
text,
|
|
111
|
+
error: "none",
|
|
112
|
+
accuracy: 92,
|
|
113
|
+
...over
|
|
114
|
+
});
|
|
115
|
+
function forecast(slips) {
|
|
116
|
+
return {
|
|
117
|
+
assessmentVersion: "1.0",
|
|
118
|
+
status: "assessed",
|
|
119
|
+
task: "scripted",
|
|
120
|
+
locale: "en-US",
|
|
121
|
+
referenceText: weather.referenceText,
|
|
122
|
+
recordingKey: "ai-check-take",
|
|
123
|
+
assessor: { kind: "auto", id: "pronunciation-engine" },
|
|
124
|
+
scale: 100,
|
|
125
|
+
scores: slips ? { accuracy: 68, fluency: 81, completeness: 86 } : { accuracy: 94, fluency: 90 },
|
|
126
|
+
miscue: "assessor",
|
|
127
|
+
phonemeAlphabet: "ipa",
|
|
128
|
+
words: [
|
|
129
|
+
said("The"),
|
|
130
|
+
slips ? said("weather", {
|
|
131
|
+
accuracy: 38,
|
|
132
|
+
error: "mispronunciation",
|
|
133
|
+
phonemes: [
|
|
134
|
+
{ symbol: "w", accuracy: 90 },
|
|
135
|
+
{ symbol: "\u025B", accuracy: 85 },
|
|
136
|
+
{
|
|
137
|
+
symbol: "\xF0",
|
|
138
|
+
accuracy: 12,
|
|
139
|
+
heardAs: [
|
|
140
|
+
{ symbol: "d", score: 64 },
|
|
141
|
+
{ symbol: "z", score: 21 },
|
|
142
|
+
{ symbol: "\xF0", score: 12 }
|
|
143
|
+
]
|
|
144
|
+
},
|
|
145
|
+
{ symbol: "\u025A", accuracy: 80 }
|
|
146
|
+
]
|
|
147
|
+
}) : said("weather"),
|
|
148
|
+
said("is"),
|
|
149
|
+
said("warm"),
|
|
150
|
+
said("in"),
|
|
151
|
+
said("the"),
|
|
152
|
+
slips ? { text: "north", error: "omission" } : said("north")
|
|
153
|
+
]
|
|
154
|
+
};
|
|
155
|
+
}
|
|
90
156
|
var chose = (ids) => ({
|
|
91
157
|
type: "multiple-choice",
|
|
92
158
|
selectedOptionIds: ids
|
|
@@ -106,21 +172,21 @@ var drafted = (text) => ({
|
|
|
106
172
|
wordCount: 0
|
|
107
173
|
});
|
|
108
174
|
function explanationCase(id, about, data, response) {
|
|
109
|
-
const request =
|
|
175
|
+
const request = _chunkLDFQUZIIcjs.aiExplanationRequest.call(void 0, { data, response });
|
|
110
176
|
if (request === null) {
|
|
111
177
|
throw new Error(`ai-check: no explanation request for "${id}"`);
|
|
112
178
|
}
|
|
113
179
|
return { id, feature: "explanation", about, request };
|
|
114
180
|
}
|
|
115
181
|
function hintCase(id, about, data, response, previousHints = []) {
|
|
116
|
-
const request =
|
|
182
|
+
const request = _chunkLDFQUZIIcjs.aiHintRequest.call(void 0, { data, response, previousHints });
|
|
117
183
|
if (request === null) {
|
|
118
184
|
throw new Error(`ai-check: no hint request for "${id}"`);
|
|
119
185
|
}
|
|
120
186
|
return { id, feature: "hint", about, request };
|
|
121
187
|
}
|
|
122
188
|
function writingCase(id, about, text, previousFeedback = [], learnerLocale) {
|
|
123
|
-
const request =
|
|
189
|
+
const request = _chunkLDFQUZIIcjs.aiWritingFeedbackRequest.call(void 0, {
|
|
124
190
|
data: weekend,
|
|
125
191
|
response: drafted(text),
|
|
126
192
|
previousFeedback,
|
|
@@ -131,6 +197,31 @@ function writingCase(id, about, text, previousFeedback = [], learnerLocale) {
|
|
|
131
197
|
}
|
|
132
198
|
return { id, feature: "writing-feedback", about, request };
|
|
133
199
|
}
|
|
200
|
+
function coachingCase(id, about, marks, learnerLocale) {
|
|
201
|
+
const assessment = _nullishCoalesce(marks.assessment, () => ( forecast(true)));
|
|
202
|
+
const grade = _chunkVSH7ECQLcjs.gradeReadAloud.call(void 0,
|
|
203
|
+
weather,
|
|
204
|
+
{ type: "read-aloud", recording: { key: "ai-check-take", mimeType: "audio/wav" } },
|
|
205
|
+
assessment,
|
|
206
|
+
{
|
|
207
|
+
measured: { durationMs: 4e3, voicedMs: 3e3 },
|
|
208
|
+
plausibility: { maxWordsPerSecond: 6, minVoicedMs: 500 }
|
|
209
|
+
}
|
|
210
|
+
);
|
|
211
|
+
if ("unscorable" in grade) {
|
|
212
|
+
throw new Error(`ai-check: the reading for "${id}" did not grade: ${grade.reason}`);
|
|
213
|
+
}
|
|
214
|
+
const request = _chunkLDFQUZIIcjs.aiCoachingRequest.call(void 0, {
|
|
215
|
+
data: weather,
|
|
216
|
+
...marks.fromGradeOnly === true ? {} : { assessment },
|
|
217
|
+
grade,
|
|
218
|
+
...learnerLocale !== void 0 ? { learnerLocale } : {}
|
|
219
|
+
});
|
|
220
|
+
if (request === null) {
|
|
221
|
+
throw new Error(`ai-check: no coaching request for "${id}"`);
|
|
222
|
+
}
|
|
223
|
+
return { id, feature: "pronunciation-coaching", about, request };
|
|
224
|
+
}
|
|
134
225
|
function aiCheckCases() {
|
|
135
226
|
return [
|
|
136
227
|
explanationCase("mc-correct", "Chose the right city", capital, chose(["madrid"])),
|
|
@@ -220,6 +311,23 @@ function aiCheckCases() {
|
|
|
220
311
|
"Last weekend I visit my grandmother. She cook a big lunch and we eat together in the garden.",
|
|
221
312
|
[],
|
|
222
313
|
"es"
|
|
314
|
+
),
|
|
315
|
+
coachingCase("coaching-slips", 'A reading with a mispronounced "th" and a word left out', {
|
|
316
|
+
assessment: forecast(true)
|
|
317
|
+
}),
|
|
318
|
+
coachingCase("coaching-clean", "A reading the engine marked correct throughout", {
|
|
319
|
+
assessment: forecast(false)
|
|
320
|
+
}),
|
|
321
|
+
coachingCase(
|
|
322
|
+
"coaching-from-grade",
|
|
323
|
+
"The same slips, from a stored grade: marks without sounds",
|
|
324
|
+
{ fromGradeOnly: true }
|
|
325
|
+
),
|
|
326
|
+
coachingCase(
|
|
327
|
+
"coaching-in-spanish",
|
|
328
|
+
"The same slips, with coaching asked for in Spanish",
|
|
329
|
+
{ assessment: forecast(true) },
|
|
330
|
+
"es"
|
|
223
331
|
)
|
|
224
332
|
];
|
|
225
333
|
}
|
|
@@ -231,7 +339,8 @@ var REFUSALS = [
|
|
|
231
339
|
"too-long",
|
|
232
340
|
"contradicts-grade",
|
|
233
341
|
"reveals-answer",
|
|
234
|
-
"misquotes-answer"
|
|
342
|
+
"misquotes-answer",
|
|
343
|
+
"contradicts-marks"
|
|
235
344
|
];
|
|
236
345
|
function portFor(ports, one) {
|
|
237
346
|
switch (one.request.feature) {
|
|
@@ -239,6 +348,8 @@ function portFor(ports, one) {
|
|
|
239
348
|
return ports.explain;
|
|
240
349
|
case "hint":
|
|
241
350
|
return ports.hint;
|
|
351
|
+
case "pronunciation-coaching":
|
|
352
|
+
return ports.pronunciationCoaching;
|
|
242
353
|
default:
|
|
243
354
|
return ports.writingFeedback;
|
|
244
355
|
}
|
|
@@ -262,8 +373,26 @@ async function runCase(ports, one) {
|
|
|
262
373
|
};
|
|
263
374
|
}
|
|
264
375
|
const ms = now() - started;
|
|
376
|
+
if (one.request.feature === "pronunciation-coaching") {
|
|
377
|
+
const checked2 = _chunkLDFQUZIIcjs.checkAiCoaching.call(void 0, raw, one.request);
|
|
378
|
+
if (checked2.ok) {
|
|
379
|
+
const { text, provenance, usage } = checked2.coaching;
|
|
380
|
+
return {
|
|
381
|
+
case: one,
|
|
382
|
+
ok: true,
|
|
383
|
+
result: {
|
|
384
|
+
text,
|
|
385
|
+
...provenance !== void 0 ? { provenance } : {},
|
|
386
|
+
...usage !== void 0 ? { usage } : {}
|
|
387
|
+
},
|
|
388
|
+
coaching: checked2.coaching,
|
|
389
|
+
ms
|
|
390
|
+
};
|
|
391
|
+
}
|
|
392
|
+
return refused(one, raw, checked2.refusal, ms);
|
|
393
|
+
}
|
|
265
394
|
if (one.request.feature === "writing-feedback") {
|
|
266
|
-
const checked2 =
|
|
395
|
+
const checked2 = _chunkLDFQUZIIcjs.checkAiWritingFeedback.call(void 0, raw, one.request);
|
|
267
396
|
if (checked2.ok) {
|
|
268
397
|
const { text, provenance, usage } = checked2.feedback;
|
|
269
398
|
return {
|
|
@@ -280,7 +409,7 @@ async function runCase(ports, one) {
|
|
|
280
409
|
}
|
|
281
410
|
return refused(one, raw, checked2.refusal, ms);
|
|
282
411
|
}
|
|
283
|
-
const checked = one.request.feature === "explanation" ?
|
|
412
|
+
const checked = one.request.feature === "explanation" ? _chunkLDFQUZIIcjs.checkAiExplanation.call(void 0, raw, one.request) : _chunkLDFQUZIIcjs.checkAiHint.call(void 0, raw, one.request);
|
|
284
413
|
if (checked.ok) {
|
|
285
414
|
return { case: one, ok: true, result: checked.result, ms };
|
|
286
415
|
}
|