@intellectif/lk-core 0.21.0 → 0.22.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +16 -0
- package/README.md +1 -0
- package/dist/{activity-xhGV-Aeh.d.cts → activity-CYc7SD3d.d.cts} +126 -7
- package/dist/{activity-xhGV-Aeh.d.ts → activity-CYc7SD3d.d.ts} +126 -7
- package/dist/ai-check.cjs +102 -14
- package/dist/ai-check.cjs.map +1 -1
- package/dist/ai-check.d.cts +9 -5
- package/dist/ai-check.d.ts +9 -5
- package/dist/ai-check.js +101 -13
- package/dist/ai-check.js.map +1 -1
- package/dist/{chunk-IHOEP4EB.js → chunk-5UCR4UNR.js} +300 -19
- package/dist/chunk-5UCR4UNR.js.map +1 -0
- package/dist/{chunk-NRTP7WCW.cjs → chunk-OPMY2MNS.cjs} +299 -18
- package/dist/chunk-OPMY2MNS.cjs.map +1 -0
- package/dist/{index-DnJ_PI4d.d.ts → index-0Z3-nnAP.d.ts} +4 -4
- package/dist/{index-BuD8Hc8o.d.cts → index-CxxiNSSH.d.cts} +4 -4
- package/dist/index.cjs +10 -2
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +65 -8
- package/dist/index.d.ts +65 -8
- package/dist/index.js +9 -1
- package/dist/index.js.map +1 -1
- package/dist/schemas.d.cts +2 -2
- package/dist/schemas.d.ts +2 -2
- package/dist/scoring.d.cts +2 -2
- package/dist/scoring.d.ts +2 -2
- package/dist/xapi.d.cts +1 -1
- package/dist/xapi.d.ts +1 -1
- package/package.json +1 -1
- package/dist/chunk-IHOEP4EB.js.map +0 -1
- package/dist/chunk-NRTP7WCW.cjs.map +0 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,21 @@
|
|
|
1
1
|
# @intellectif/lk-core
|
|
2
2
|
|
|
3
|
+
## 0.22.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- [#80](https://github.com/intellectif/learning-kit/pull/80) [`782b01b`](https://github.com/intellectif/learning-kit/commit/782b01ba697a90583b472ab1a7a2be102fd888c4) Thanks [@diego28e](https://github.com/diego28e)! - **Feedback on writing**: "Get feedback on my draft" on a written response in practice. The learner revises and asks again, and every correction must quote what they wrote.
|
|
8
|
+
|
|
9
|
+
**Action required:** none unless your code switches exhaustively over `AiFeature` or `AiRefusal` (new members `'writing-feedback'` and `'misquotes-answer'`), reads `AiCheckCase.request` without narrowing on `feature`, or implements a complete `LkStrings` (eight new keys). Nothing changes until you pass a `writingFeedback` port. Otherwise this is a major only because the peer range moves to `@intellectif/lk-core@^0.22.0`.
|
|
10
|
+
|
|
11
|
+
- **lk-core:** `aiWritingFeedbackRequest` builds what a model is given: the task, the draft verbatim, its word count as the grader counts it, the rubric, and the feedback on earlier drafts. `checkAiWritingFeedback` checks the reply before a learner sees it. It finds each correction's quote in the draft itself (typographic quotes and runs of whitespace folded, case kept) and anchors it there. A correction of words the draft does not contain refuses the whole reply as `misquotes-answer`. Criteria must be the rubric's; the author's weights are attached, and a model's weight is ignored. The indicative score is `gradeFromRubric`'s total, computed only when every criterion was judged. It is never a grade. Limits: 20 corrections, 500 characters a field.
|
|
12
|
+
- **lk-react:** a `writingFeedback` port and `maxWritingFeedback` (3 by default, at most 10) on `LearnerAi`. `<WrittenResponse>` shows the feedback with its corrections, rubric comments, the indicative score marked "Not a grade." and the AI notice, and says so once the draft has changed. It is never in `exam` or `review`, and it follows the author's and the paper's `ai.explanations`, `feedback` and `solutions`. `useAiWritingFeedback` gives a written response you draw yourself the same rules.
|
|
13
|
+
- **Records:** `ai-writing-feedback-shown` (the draft number, how many corrections, the indicative score, provenance and usage; never the text), and `ai-help-refused` with `feature: 'writing-feedback'`.
|
|
14
|
+
- **ai-check:** a `writingFeedback` port runs four writing cases: a draft with mistakes, one without, a revision, and feedback asked for in Spanish.
|
|
15
|
+
- Eight new strings: `aiWritingFeedback`, `aiWritingFeedbackLoading`, `aiWritingFeedbackHeading`, `aiWritingFeedbackUnavailable`, `aiNoMoreWritingFeedback`, `aiWritingFeedbackOutdated`, `aiCorrections`, `aiIndicativeScore`.
|
|
16
|
+
|
|
17
|
+
Guide: https://github.com/intellectif/learning-kit/blob/main/docs/ai.md#feedback-on-writing
|
|
18
|
+
|
|
3
19
|
## 0.21.0
|
|
4
20
|
|
|
5
21
|
### Minor Changes
|
package/README.md
CHANGED
|
@@ -28,6 +28,7 @@ npm install @intellectif/lk-core
|
|
|
28
28
|
- **Item groups** — `Stimulus` + `ItemGroup` (one passage or recording serving several questions), `flattenSequence()` for seeded, shuffle-atomic presentation order with stable slot identity. Slot ids are positional by default (`"3"`, `"1.0"`); author a `slotKey` on an entry and its id survives an insert, so a stored response cannot slide onto a different question.
|
|
29
29
|
- **Delivery policies** — `DeliveryPolicy` says what a school lets a learner see around a question: feedback, solutions, hints and AI help. `resolveDeliveryPolicy` spells one out (an unreadable setting restricts, never allows), `validateDeliveryPolicy` checks one before it is stored, and `combineDeliveryPolicies` keeps a setting on only where every policy leaves it on.
|
|
30
30
|
- **Scoring policies** — `ItemScoringPolicy` says how a paper's questions are scored when a learner can try again or ask for hints: `retries`, which try `counts`, and what a try and a hint cost. `scoreTries` does the arithmetic, `evaluate(…, { scoring })` and `evaluateTries` reach a question's outcome from its answers, and `validateItemScoringPolicy` / `resolveItemScoringPolicy` refuse a policy that cannot be applied rather than guess — every setting moves a grade. No policy changes nothing, and the arithmetic is pinned by the grade vectors.
|
|
31
|
+
- **AI help contracts** — the SDK calls no model; it builds what yours is given and checks what comes back. `aiExplanationRequest`, `aiHintRequest` and `aiWritingFeedbackRequest` build the requests a component makes, so a server can rebuild them from its own copy of an item; `checkAiExplanation`, `checkAiHint` and `checkAiWritingFeedback` refuse an explanation that contradicts the grade, a hint that contains the answer, and writing feedback that corrects words the learner never wrote, and `checkAiWritingFeedback` computes the rubric's indicative score itself.
|
|
31
32
|
- **Attempt plans** — `planAttempt(entries, { seed, shuffleEntries, points, delivery })` freezes the exact paper a learner sat, and the delivery policy it was sat under: slot order, per-slot points, a content hash of each item, and a `planHash` over the lot. `verifyAttemptPlan(plan, current)` reports the drift when that content is later edited (`changedSlotIds`, `changedStimulusSlotIds`, `changedPointsSlotIds`, `missingSlotIds`, …) — the thing you need during a remark or an appeal. `scoredItemsFromPlan(plan, outcomes)` turns collected outcomes into the `ScoredItem[]` composition wants, and defaults a slot with **no** outcome to `deferred` rather than silently shrinking the denominator.
|
|
32
33
|
- **Attempt state** — `serializeAttemptState(plan, progress)` / `restoreAttemptState(plan, snapshot)` round-trip an interrupted attempt (responses, submitted slots, position) and are **bound to the plan**: restoring answers onto a paper the learner never sat throws instead of succeeding quietly. `diffResponses(before, after)` reports what moved between two snapshots, including an answer the learner *cleared*.
|
|
33
34
|
- **Media playback policy** — `media.playback` on an audio recording: `maxPlays`, `seek`, `rate`, and advisory `nativeControlHints`. `resolvePlaybackPolicy()` derives the defaults once so the schema, the plan and the renderer cannot drift. Budgets are frozen into the plan by `planAttempt`, so editing `maxPlays` mid-window cannot change what a past learner was held to, and `planAttempt` refuses a paper where one recording is budgeted under two keys.
|
|
@@ -224,7 +224,11 @@ interface GradeRecord {
|
|
|
224
224
|
* one-word vocabulary item, say.
|
|
225
225
|
*/
|
|
226
226
|
interface ActivityAiPermissions {
|
|
227
|
-
/**
|
|
227
|
+
/**
|
|
228
|
+
* An explanation of the learner's graded answer — and, on a written
|
|
229
|
+
* response, feedback on a draft: both are a model's words about the
|
|
230
|
+
* learner's own answer.
|
|
231
|
+
*/
|
|
228
232
|
explanations?: boolean;
|
|
229
233
|
/** Hints before the learner submits. */
|
|
230
234
|
hints?: boolean;
|
|
@@ -243,8 +247,8 @@ interface AiActivityInput {
|
|
|
243
247
|
readonly redacted?: unknown;
|
|
244
248
|
readonly ai?: unknown;
|
|
245
249
|
}
|
|
246
|
-
/** The
|
|
247
|
-
type AiFeature = 'explanation' | 'hint';
|
|
250
|
+
/** The kinds of help a learner can be given. */
|
|
251
|
+
type AiFeature = 'explanation' | 'hint' | 'writing-feedback';
|
|
248
252
|
/** The activity types the SDK builds facts for. */
|
|
249
253
|
type AiSupportedActivityType = 'multiple-choice' | 'fill-in-the-blanks' | 'gap-select' | 'dictation';
|
|
250
254
|
/**
|
|
@@ -404,8 +408,117 @@ interface AiTextResult {
|
|
|
404
408
|
*/
|
|
405
409
|
usage?: GraderUsage;
|
|
406
410
|
}
|
|
407
|
-
/**
|
|
408
|
-
|
|
411
|
+
/**
|
|
412
|
+
* Why a result was not shown. `misquotes-answer` is writing feedback that
|
|
413
|
+
* corrects words the learner did not write: a correction whose quote is not in
|
|
414
|
+
* the draft, or not where it claims to be.
|
|
415
|
+
*/
|
|
416
|
+
type AiRefusal = 'malformed' | 'empty' | 'too-long' | 'contradicts-grade' | 'reveals-answer' | 'misquotes-answer';
|
|
417
|
+
/** One criterion of a written response's rubric, as the author wrote it. */
|
|
418
|
+
interface AiRubricCriterionFact {
|
|
419
|
+
name: string;
|
|
420
|
+
description?: string;
|
|
421
|
+
weight: number;
|
|
422
|
+
}
|
|
423
|
+
/**
|
|
424
|
+
* What a model is told about a draft of a written response: the task as the
|
|
425
|
+
* learner saw it, the draft verbatim, and the rubric it will be graded on.
|
|
426
|
+
*/
|
|
427
|
+
interface AiWritingFacts {
|
|
428
|
+
activityType: 'written-response';
|
|
429
|
+
activityId: string;
|
|
430
|
+
title: string;
|
|
431
|
+
/** The language the item is written in, when the author gave one. */
|
|
432
|
+
locale?: string;
|
|
433
|
+
prompt: string;
|
|
434
|
+
/** The draft, exactly as the learner wrote it. Every correction quotes it. */
|
|
435
|
+
text: string;
|
|
436
|
+
/** Recomputed with `countWords`, as the grader will count it. */
|
|
437
|
+
wordCount: number;
|
|
438
|
+
minWords: number;
|
|
439
|
+
maxWords: number;
|
|
440
|
+
withinWordBounds: boolean;
|
|
441
|
+
/** The rubric the answer will be graded on; `null` when the item has none. */
|
|
442
|
+
rubric: AiRubricCriterionFact[] | null;
|
|
443
|
+
/** The target language and level, when the author gave one (e.g. `"en-A2"`). */
|
|
444
|
+
languageTarget?: string;
|
|
445
|
+
}
|
|
446
|
+
/**
|
|
447
|
+
* Asks for feedback on a draft of a written response, before the learner
|
|
448
|
+
* submits it. Practice only.
|
|
449
|
+
*/
|
|
450
|
+
interface AiWritingFeedbackRequest {
|
|
451
|
+
feature: 'writing-feedback';
|
|
452
|
+
facts: AiWritingFacts;
|
|
453
|
+
/** 1-based: which draft this is. */
|
|
454
|
+
draftNumber: number;
|
|
455
|
+
/** The text of the feedback already shown on earlier drafts, oldest first. */
|
|
456
|
+
previousFeedback: string[];
|
|
457
|
+
/** The language to write in: the learner's interface language, when known. */
|
|
458
|
+
learnerLocale?: string;
|
|
459
|
+
}
|
|
460
|
+
/**
|
|
461
|
+
* One correction a model proposes. `original` must be words the learner
|
|
462
|
+
* wrote; the SDK finds them in the draft itself, so a model need not count
|
|
463
|
+
* characters — leave `range` out unless you are sure of it, because a range
|
|
464
|
+
* that does not hold the quote is refused.
|
|
465
|
+
*/
|
|
466
|
+
interface AiWritingCorrection {
|
|
467
|
+
original: string;
|
|
468
|
+
corrected: string;
|
|
469
|
+
explanation?: string;
|
|
470
|
+
/** An error-type tag, e.g. `"tense"`, `"article"`. */
|
|
471
|
+
category?: string;
|
|
472
|
+
range?: {
|
|
473
|
+
start: number;
|
|
474
|
+
end: number;
|
|
475
|
+
};
|
|
476
|
+
}
|
|
477
|
+
/**
|
|
478
|
+
* A model's judgement of one rubric criterion. The SDK attaches the author's
|
|
479
|
+
* weight; a weight sent here is ignored.
|
|
480
|
+
*/
|
|
481
|
+
type AiCriterionJudgement = Omit<CriterionScore, 'weight'> & {
|
|
482
|
+
weight?: number;
|
|
483
|
+
};
|
|
484
|
+
/**
|
|
485
|
+
* What a host's `writingFeedback` port returns: the overall feedback as plain
|
|
486
|
+
* text, and optionally corrections and a judgement per rubric criterion.
|
|
487
|
+
*/
|
|
488
|
+
interface AiWritingFeedbackResult {
|
|
489
|
+
/** The overall feedback, addressed to the learner. Plain text, never HTML. */
|
|
490
|
+
text: string;
|
|
491
|
+
corrections?: AiWritingCorrection[];
|
|
492
|
+
criteria?: AiCriterionJudgement[];
|
|
493
|
+
provenance?: AiProvenance;
|
|
494
|
+
usage?: GraderUsage;
|
|
495
|
+
}
|
|
496
|
+
/**
|
|
497
|
+
* Feedback on a draft, as the SDK accepted it: what a learner is shown.
|
|
498
|
+
*/
|
|
499
|
+
interface AiWritingFeedback {
|
|
500
|
+
text: string;
|
|
501
|
+
/**
|
|
502
|
+
* Every correction, anchored in the draft: `range` is where the SDK found
|
|
503
|
+
* the quote, and `original` is the draft's own text at that range.
|
|
504
|
+
*/
|
|
505
|
+
corrections: InlineCorrection[];
|
|
506
|
+
/** The model's judgements, with the rubric's weights. */
|
|
507
|
+
criteria: CriterionScore[];
|
|
508
|
+
/**
|
|
509
|
+
* The rubric's weighted total of the judgements, from 0 to 1, computed by
|
|
510
|
+
* the SDK as `gradeFromRubric` computes a grade — when every criterion of
|
|
511
|
+
* the rubric was judged; `null` otherwise, and on an item with no rubric.
|
|
512
|
+
*
|
|
513
|
+
* **An indication, never a grade:** it is a model's reading of a draft the
|
|
514
|
+
* learner has not submitted, and nothing in the SDK scores an answer with
|
|
515
|
+
* it. `ai-writing-feedback-shown` reports it, so a record can say what the
|
|
516
|
+
* learner was shown.
|
|
517
|
+
*/
|
|
518
|
+
indicativeScore: number | null;
|
|
519
|
+
provenance?: AiProvenance;
|
|
520
|
+
usage?: GraderUsage;
|
|
521
|
+
}
|
|
409
522
|
|
|
410
523
|
/**
|
|
411
524
|
* An xAPI Activity object — the thing a statement is about.
|
|
@@ -1599,7 +1712,13 @@ type InteractionKind = 'option-selected' | 'option-deselected' | 'blank-filled'
|
|
|
1599
1712
|
* watches its own model in production, where the development console warning
|
|
1600
1713
|
* it accompanies is silent.
|
|
1601
1714
|
*/
|
|
1602
|
-
| 'ai-help-refused'
|
|
1715
|
+
| 'ai-help-refused'
|
|
1716
|
+
/**
|
|
1717
|
+
* Feedback on a draft of a written response was shown (`draftNumber`, the
|
|
1718
|
+
* number of `corrections`, the `indicativeScore` when there was one, and the
|
|
1719
|
+
* port's `provenance` and `usage` when it sent them). Never the text.
|
|
1720
|
+
*/
|
|
1721
|
+
| 'ai-writing-feedback-shown' | (string & {});
|
|
1603
1722
|
/** Fired by activity components on every discrete learner interaction. */
|
|
1604
1723
|
interface InteractionEvent {
|
|
1605
1724
|
/** The kind of interaction that occurred. */
|
|
@@ -1612,4 +1731,4 @@ interface InteractionEvent {
|
|
|
1612
1731
|
payload: Record<string, unknown>;
|
|
1613
1732
|
}
|
|
1614
1733
|
|
|
1615
|
-
export { type
|
|
1734
|
+
export { type GradingState as $, type ActivityAiPermissions as A, type AiWordFact as B, type AiWritingCorrection as C, type AiWritingFacts as D, type AiWritingFeedback as E, type AiWritingFeedbackRequest as F, type AiWritingFeedbackResult as G, type BlankConfig as H, type CriterionScore as I, type DeferredReason as J, type DeferredScoringPartial as K, type DictationData as L, type DictationEquivalence as M, type DictationLearnerResponse as N, type DictationSlowMedia as O, type DictationTolerance as P, type FillInTheBlanksData as Q, type FillInTheBlanksLearnerResponse as R, type GapSelectBank as S, type GapSelectChoice as T, type GapSelectData as U, type GapSelectGap as V, type GapSelectLearnerResponse as W, type GradeRecord as X, type Grader as Y, type GraderKind as Z, type GraderUsage as _, type ActivityData as a, type InlineCorrection as a0, type InteractionEvent as a1, type InteractionKind as a2, type ItemOutcome as a3, type LearnerResponse as a4, type LearnerResponseMap as a5, type MediaPlaybackPolicy as a6, type MediaTrack as a7, type MultipleChoiceData as a8, type MultipleChoiceLearnerResponse as a9, type XAPIObject as aA, type XAPIResult as aB, type XAPIScore as aC, type XAPIStatement as aD, type XAPIVerbObject as aE, levenshteinDistance as aF, matchText as aG, type MultipleChoiceOption as aa, type MultipleChoiceOptionMedia as ab, type NativeControlHint as ac, type ReadAloudData as ad, type ReadAloudDimension as ae, type ReadAloudDimensionWeight as af, type ReadAloudLearnerResponse as ag, type ReadAloudSlowMedia as ah, type RecordingBounds as ai, type RecordingRef as aj, type ScoringDetail as ak, type ScoringOutcome as al, type ScoringResult as am, type TextMatchPolicy as an, type TextMatchResult as ao, type ValidationError as ap, type ValidationResult as aq, type WrittenResponseData as ar, type WrittenResponseLearnerResponse as as, type WrittenResponseRubric as at, type WrittenResponseRubricCriterion as au, type XAPIActor as av, type XAPIConfig as aw, type XAPIContext as ax, type XAPIContextActivities as ay, type XAPIError as az, type ActivityDataMap as b, type ActivityFeedback as c, type ActivityMedia as d, type ActivityResult as e, type ActivityType as f, type AiActivityInput as g, type AiBlankFact as h, type AiCriterionJudgement as i, type AiDictationFacts as j, type AiExplanationRequest as k, type AiFeature as l, type AiFillInTheBlanksFacts as m, type AiGapFact as n, type AiGapSelectFacts as o, type AiGrade as p, type AiHintRequest as q, type AiItemFacts as r, type AiMultipleChoiceFacts as s, type AiOptionFact as t, type AiProvenance as u, type AiRefusal as v, type AiRubricCriterionFact as w, type AiSupportedActivityType as x, type AiTextResult as y, type AiVerdict as z };
|
|
@@ -224,7 +224,11 @@ interface GradeRecord {
|
|
|
224
224
|
* one-word vocabulary item, say.
|
|
225
225
|
*/
|
|
226
226
|
interface ActivityAiPermissions {
|
|
227
|
-
/**
|
|
227
|
+
/**
|
|
228
|
+
* An explanation of the learner's graded answer — and, on a written
|
|
229
|
+
* response, feedback on a draft: both are a model's words about the
|
|
230
|
+
* learner's own answer.
|
|
231
|
+
*/
|
|
228
232
|
explanations?: boolean;
|
|
229
233
|
/** Hints before the learner submits. */
|
|
230
234
|
hints?: boolean;
|
|
@@ -243,8 +247,8 @@ interface AiActivityInput {
|
|
|
243
247
|
readonly redacted?: unknown;
|
|
244
248
|
readonly ai?: unknown;
|
|
245
249
|
}
|
|
246
|
-
/** The
|
|
247
|
-
type AiFeature = 'explanation' | 'hint';
|
|
250
|
+
/** The kinds of help a learner can be given. */
|
|
251
|
+
type AiFeature = 'explanation' | 'hint' | 'writing-feedback';
|
|
248
252
|
/** The activity types the SDK builds facts for. */
|
|
249
253
|
type AiSupportedActivityType = 'multiple-choice' | 'fill-in-the-blanks' | 'gap-select' | 'dictation';
|
|
250
254
|
/**
|
|
@@ -404,8 +408,117 @@ interface AiTextResult {
|
|
|
404
408
|
*/
|
|
405
409
|
usage?: GraderUsage;
|
|
406
410
|
}
|
|
407
|
-
/**
|
|
408
|
-
|
|
411
|
+
/**
|
|
412
|
+
* Why a result was not shown. `misquotes-answer` is writing feedback that
|
|
413
|
+
* corrects words the learner did not write: a correction whose quote is not in
|
|
414
|
+
* the draft, or not where it claims to be.
|
|
415
|
+
*/
|
|
416
|
+
type AiRefusal = 'malformed' | 'empty' | 'too-long' | 'contradicts-grade' | 'reveals-answer' | 'misquotes-answer';
|
|
417
|
+
/** One criterion of a written response's rubric, as the author wrote it. */
|
|
418
|
+
interface AiRubricCriterionFact {
|
|
419
|
+
name: string;
|
|
420
|
+
description?: string;
|
|
421
|
+
weight: number;
|
|
422
|
+
}
|
|
423
|
+
/**
|
|
424
|
+
* What a model is told about a draft of a written response: the task as the
|
|
425
|
+
* learner saw it, the draft verbatim, and the rubric it will be graded on.
|
|
426
|
+
*/
|
|
427
|
+
interface AiWritingFacts {
|
|
428
|
+
activityType: 'written-response';
|
|
429
|
+
activityId: string;
|
|
430
|
+
title: string;
|
|
431
|
+
/** The language the item is written in, when the author gave one. */
|
|
432
|
+
locale?: string;
|
|
433
|
+
prompt: string;
|
|
434
|
+
/** The draft, exactly as the learner wrote it. Every correction quotes it. */
|
|
435
|
+
text: string;
|
|
436
|
+
/** Recomputed with `countWords`, as the grader will count it. */
|
|
437
|
+
wordCount: number;
|
|
438
|
+
minWords: number;
|
|
439
|
+
maxWords: number;
|
|
440
|
+
withinWordBounds: boolean;
|
|
441
|
+
/** The rubric the answer will be graded on; `null` when the item has none. */
|
|
442
|
+
rubric: AiRubricCriterionFact[] | null;
|
|
443
|
+
/** The target language and level, when the author gave one (e.g. `"en-A2"`). */
|
|
444
|
+
languageTarget?: string;
|
|
445
|
+
}
|
|
446
|
+
/**
|
|
447
|
+
* Asks for feedback on a draft of a written response, before the learner
|
|
448
|
+
* submits it. Practice only.
|
|
449
|
+
*/
|
|
450
|
+
interface AiWritingFeedbackRequest {
|
|
451
|
+
feature: 'writing-feedback';
|
|
452
|
+
facts: AiWritingFacts;
|
|
453
|
+
/** 1-based: which draft this is. */
|
|
454
|
+
draftNumber: number;
|
|
455
|
+
/** The text of the feedback already shown on earlier drafts, oldest first. */
|
|
456
|
+
previousFeedback: string[];
|
|
457
|
+
/** The language to write in: the learner's interface language, when known. */
|
|
458
|
+
learnerLocale?: string;
|
|
459
|
+
}
|
|
460
|
+
/**
|
|
461
|
+
* One correction a model proposes. `original` must be words the learner
|
|
462
|
+
* wrote; the SDK finds them in the draft itself, so a model need not count
|
|
463
|
+
* characters — leave `range` out unless you are sure of it, because a range
|
|
464
|
+
* that does not hold the quote is refused.
|
|
465
|
+
*/
|
|
466
|
+
interface AiWritingCorrection {
|
|
467
|
+
original: string;
|
|
468
|
+
corrected: string;
|
|
469
|
+
explanation?: string;
|
|
470
|
+
/** An error-type tag, e.g. `"tense"`, `"article"`. */
|
|
471
|
+
category?: string;
|
|
472
|
+
range?: {
|
|
473
|
+
start: number;
|
|
474
|
+
end: number;
|
|
475
|
+
};
|
|
476
|
+
}
|
|
477
|
+
/**
|
|
478
|
+
* A model's judgement of one rubric criterion. The SDK attaches the author's
|
|
479
|
+
* weight; a weight sent here is ignored.
|
|
480
|
+
*/
|
|
481
|
+
type AiCriterionJudgement = Omit<CriterionScore, 'weight'> & {
|
|
482
|
+
weight?: number;
|
|
483
|
+
};
|
|
484
|
+
/**
|
|
485
|
+
* What a host's `writingFeedback` port returns: the overall feedback as plain
|
|
486
|
+
* text, and optionally corrections and a judgement per rubric criterion.
|
|
487
|
+
*/
|
|
488
|
+
interface AiWritingFeedbackResult {
|
|
489
|
+
/** The overall feedback, addressed to the learner. Plain text, never HTML. */
|
|
490
|
+
text: string;
|
|
491
|
+
corrections?: AiWritingCorrection[];
|
|
492
|
+
criteria?: AiCriterionJudgement[];
|
|
493
|
+
provenance?: AiProvenance;
|
|
494
|
+
usage?: GraderUsage;
|
|
495
|
+
}
|
|
496
|
+
/**
|
|
497
|
+
* Feedback on a draft, as the SDK accepted it: what a learner is shown.
|
|
498
|
+
*/
|
|
499
|
+
interface AiWritingFeedback {
|
|
500
|
+
text: string;
|
|
501
|
+
/**
|
|
502
|
+
* Every correction, anchored in the draft: `range` is where the SDK found
|
|
503
|
+
* the quote, and `original` is the draft's own text at that range.
|
|
504
|
+
*/
|
|
505
|
+
corrections: InlineCorrection[];
|
|
506
|
+
/** The model's judgements, with the rubric's weights. */
|
|
507
|
+
criteria: CriterionScore[];
|
|
508
|
+
/**
|
|
509
|
+
* The rubric's weighted total of the judgements, from 0 to 1, computed by
|
|
510
|
+
* the SDK as `gradeFromRubric` computes a grade — when every criterion of
|
|
511
|
+
* the rubric was judged; `null` otherwise, and on an item with no rubric.
|
|
512
|
+
*
|
|
513
|
+
* **An indication, never a grade:** it is a model's reading of a draft the
|
|
514
|
+
* learner has not submitted, and nothing in the SDK scores an answer with
|
|
515
|
+
* it. `ai-writing-feedback-shown` reports it, so a record can say what the
|
|
516
|
+
* learner was shown.
|
|
517
|
+
*/
|
|
518
|
+
indicativeScore: number | null;
|
|
519
|
+
provenance?: AiProvenance;
|
|
520
|
+
usage?: GraderUsage;
|
|
521
|
+
}
|
|
409
522
|
|
|
410
523
|
/**
|
|
411
524
|
* An xAPI Activity object — the thing a statement is about.
|
|
@@ -1599,7 +1712,13 @@ type InteractionKind = 'option-selected' | 'option-deselected' | 'blank-filled'
|
|
|
1599
1712
|
* watches its own model in production, where the development console warning
|
|
1600
1713
|
* it accompanies is silent.
|
|
1601
1714
|
*/
|
|
1602
|
-
| 'ai-help-refused'
|
|
1715
|
+
| 'ai-help-refused'
|
|
1716
|
+
/**
|
|
1717
|
+
* Feedback on a draft of a written response was shown (`draftNumber`, the
|
|
1718
|
+
* number of `corrections`, the `indicativeScore` when there was one, and the
|
|
1719
|
+
* port's `provenance` and `usage` when it sent them). Never the text.
|
|
1720
|
+
*/
|
|
1721
|
+
| 'ai-writing-feedback-shown' | (string & {});
|
|
1603
1722
|
/** Fired by activity components on every discrete learner interaction. */
|
|
1604
1723
|
interface InteractionEvent {
|
|
1605
1724
|
/** The kind of interaction that occurred. */
|
|
@@ -1612,4 +1731,4 @@ interface InteractionEvent {
|
|
|
1612
1731
|
payload: Record<string, unknown>;
|
|
1613
1732
|
}
|
|
1614
1733
|
|
|
1615
|
-
export { type
|
|
1734
|
+
export { type GradingState as $, type ActivityAiPermissions as A, type AiWordFact as B, type AiWritingCorrection as C, type AiWritingFacts as D, type AiWritingFeedback as E, type AiWritingFeedbackRequest as F, type AiWritingFeedbackResult as G, type BlankConfig as H, type CriterionScore as I, type DeferredReason as J, type DeferredScoringPartial as K, type DictationData as L, type DictationEquivalence as M, type DictationLearnerResponse as N, type DictationSlowMedia as O, type DictationTolerance as P, type FillInTheBlanksData as Q, type FillInTheBlanksLearnerResponse as R, type GapSelectBank as S, type GapSelectChoice as T, type GapSelectData as U, type GapSelectGap as V, type GapSelectLearnerResponse as W, type GradeRecord as X, type Grader as Y, type GraderKind as Z, type GraderUsage as _, type ActivityData as a, type InlineCorrection as a0, type InteractionEvent as a1, type InteractionKind as a2, type ItemOutcome as a3, type LearnerResponse as a4, type LearnerResponseMap as a5, type MediaPlaybackPolicy as a6, type MediaTrack as a7, type MultipleChoiceData as a8, type MultipleChoiceLearnerResponse as a9, type XAPIObject as aA, type XAPIResult as aB, type XAPIScore as aC, type XAPIStatement as aD, type XAPIVerbObject as aE, levenshteinDistance as aF, matchText as aG, type MultipleChoiceOption as aa, type MultipleChoiceOptionMedia as ab, type NativeControlHint as ac, type ReadAloudData as ad, type ReadAloudDimension as ae, type ReadAloudDimensionWeight as af, type ReadAloudLearnerResponse as ag, type ReadAloudSlowMedia as ah, type RecordingBounds as ai, type RecordingRef as aj, type ScoringDetail as ak, type ScoringOutcome as al, type ScoringResult as am, type TextMatchPolicy as an, type TextMatchResult as ao, type ValidationError as ap, type ValidationResult as aq, type WrittenResponseData as ar, type WrittenResponseLearnerResponse as as, type WrittenResponseRubric as at, type WrittenResponseRubricCriterion as au, type XAPIActor as av, type XAPIConfig as aw, type XAPIContext as ax, type XAPIContextActivities as ay, type XAPIError as az, type ActivityDataMap as b, type ActivityFeedback as c, type ActivityMedia as d, type ActivityResult as e, type ActivityType as f, type AiActivityInput as g, type AiBlankFact as h, type AiCriterionJudgement as i, type AiDictationFacts as j, type AiExplanationRequest as k, type AiFeature as l, type AiFillInTheBlanksFacts as m, type AiGapFact as n, type AiGapSelectFacts as o, type AiGrade as p, type AiHintRequest as q, type AiItemFacts as r, type AiMultipleChoiceFacts as s, type AiOptionFact as t, type AiProvenance as u, type AiRefusal as v, type AiRubricCriterionFact as w, type AiSupportedActivityType as x, type AiTextResult as y, type AiVerdict as z };
|
package/dist/ai-check.cjs
CHANGED
|
@@ -3,7 +3,9 @@
|
|
|
3
3
|
|
|
4
4
|
|
|
5
5
|
|
|
6
|
-
|
|
6
|
+
|
|
7
|
+
|
|
8
|
+
var _chunkOPMY2MNScjs = require('./chunk-OPMY2MNS.cjs');
|
|
7
9
|
require('./chunk-WQVZPIN7.cjs');
|
|
8
10
|
require('./chunk-D4TOPSHV.cjs');
|
|
9
11
|
|
|
@@ -68,6 +70,23 @@ var listening = {
|
|
|
68
70
|
title: "Listen and type",
|
|
69
71
|
transcript: "The train leaves at half past nine."
|
|
70
72
|
};
|
|
73
|
+
var weekend = {
|
|
74
|
+
schemaVersion: "1.0",
|
|
75
|
+
type: "written-response",
|
|
76
|
+
id: "ai-check-writing",
|
|
77
|
+
title: "Your weekend",
|
|
78
|
+
prompt: "Write a few sentences about what you did last weekend.",
|
|
79
|
+
minWords: 15,
|
|
80
|
+
maxWords: 120,
|
|
81
|
+
languageTarget: "en-A2",
|
|
82
|
+
rubric: {
|
|
83
|
+
criteria: [
|
|
84
|
+
{ name: "Grammar", description: "Past tenses, articles, agreement", weight: 2 },
|
|
85
|
+
{ name: "Vocabulary", description: "Everyday words, used correctly", weight: 1 },
|
|
86
|
+
{ name: "Task", description: "Says what the learner did, in order", weight: 1 }
|
|
87
|
+
]
|
|
88
|
+
}
|
|
89
|
+
};
|
|
71
90
|
var chose = (ids) => ({
|
|
72
91
|
type: "multiple-choice",
|
|
73
92
|
selectedOptionIds: ids
|
|
@@ -81,20 +100,37 @@ var picked = (choices) => ({
|
|
|
81
100
|
selections: choices
|
|
82
101
|
});
|
|
83
102
|
var wrote = (text) => ({ type: "dictation", text });
|
|
103
|
+
var drafted = (text) => ({
|
|
104
|
+
type: "written-response",
|
|
105
|
+
text,
|
|
106
|
+
wordCount: 0
|
|
107
|
+
});
|
|
84
108
|
function explanationCase(id, about, data, response) {
|
|
85
|
-
const request =
|
|
109
|
+
const request = _chunkOPMY2MNScjs.aiExplanationRequest.call(void 0, { data, response });
|
|
86
110
|
if (request === null) {
|
|
87
111
|
throw new Error(`ai-check: no explanation request for "${id}"`);
|
|
88
112
|
}
|
|
89
113
|
return { id, feature: "explanation", about, request };
|
|
90
114
|
}
|
|
91
115
|
function hintCase(id, about, data, response, previousHints = []) {
|
|
92
|
-
const request =
|
|
116
|
+
const request = _chunkOPMY2MNScjs.aiHintRequest.call(void 0, { data, response, previousHints });
|
|
93
117
|
if (request === null) {
|
|
94
118
|
throw new Error(`ai-check: no hint request for "${id}"`);
|
|
95
119
|
}
|
|
96
120
|
return { id, feature: "hint", about, request };
|
|
97
121
|
}
|
|
122
|
+
function writingCase(id, about, text, previousFeedback = [], learnerLocale) {
|
|
123
|
+
const request = _chunkOPMY2MNScjs.aiWritingFeedbackRequest.call(void 0, {
|
|
124
|
+
data: weekend,
|
|
125
|
+
response: drafted(text),
|
|
126
|
+
previousFeedback,
|
|
127
|
+
...learnerLocale !== void 0 ? { learnerLocale } : {}
|
|
128
|
+
});
|
|
129
|
+
if (request === null) {
|
|
130
|
+
throw new Error(`ai-check: no writing feedback request for "${id}"`);
|
|
131
|
+
}
|
|
132
|
+
return { id, feature: "writing-feedback", about, request };
|
|
133
|
+
}
|
|
98
134
|
function aiCheckCases() {
|
|
99
135
|
return [
|
|
100
136
|
explanationCase("mc-correct", "Chose the right city", capital, chose(["madrid"])),
|
|
@@ -161,6 +197,29 @@ function aiCheckCases() {
|
|
|
161
197
|
landscape,
|
|
162
198
|
picked({ g1: "campo" }),
|
|
163
199
|
["Piensa en la forma del terreno."]
|
|
200
|
+
),
|
|
201
|
+
writingCase(
|
|
202
|
+
"writing-mistakes",
|
|
203
|
+
"A first draft with tense, article and agreement mistakes",
|
|
204
|
+
"Last weekend I go to the market and buyed a apple. My friends was there too, so we don't go home early. Then I go to the cinema."
|
|
205
|
+
),
|
|
206
|
+
writingCase(
|
|
207
|
+
"writing-clean",
|
|
208
|
+
"A first draft with no mistakes to correct",
|
|
209
|
+
"Last weekend I went to the market and bought an apple. My friends were there too, so we stayed until the evening. Then we went to the cinema."
|
|
210
|
+
),
|
|
211
|
+
writingCase(
|
|
212
|
+
"writing-revised",
|
|
213
|
+
"A second draft that fixed some mistakes",
|
|
214
|
+
"Last weekend I went to the market and bought a apple. My friends was there too, so we didn't go home early.",
|
|
215
|
+
['Check your past tenses: "go" and "buyed" should be past forms.']
|
|
216
|
+
),
|
|
217
|
+
writingCase(
|
|
218
|
+
"writing-feedback-in-spanish",
|
|
219
|
+
"A first draft, with feedback asked for in Spanish",
|
|
220
|
+
"Last weekend I visit my grandmother. She cook a big lunch and we eat together in the garden.",
|
|
221
|
+
[],
|
|
222
|
+
"es"
|
|
164
223
|
)
|
|
165
224
|
];
|
|
166
225
|
}
|
|
@@ -171,13 +230,23 @@ var REFUSALS = [
|
|
|
171
230
|
"empty",
|
|
172
231
|
"too-long",
|
|
173
232
|
"contradicts-grade",
|
|
174
|
-
"reveals-answer"
|
|
233
|
+
"reveals-answer",
|
|
234
|
+
"misquotes-answer"
|
|
175
235
|
];
|
|
236
|
+
function portFor(ports, one) {
|
|
237
|
+
switch (one.request.feature) {
|
|
238
|
+
case "explanation":
|
|
239
|
+
return ports.explain;
|
|
240
|
+
case "hint":
|
|
241
|
+
return ports.hint;
|
|
242
|
+
default:
|
|
243
|
+
return ports.writingFeedback;
|
|
244
|
+
}
|
|
245
|
+
}
|
|
176
246
|
var now = () => Date.now();
|
|
177
247
|
async function runCase(ports, one) {
|
|
178
248
|
const started = now();
|
|
179
|
-
const
|
|
180
|
-
const port = explaining ? ports.explain : ports.hint;
|
|
249
|
+
const port = portFor(ports, one);
|
|
181
250
|
if (port === void 0) {
|
|
182
251
|
throw new Error(`ai-check: no port for "${one.id}"`);
|
|
183
252
|
}
|
|
@@ -193,15 +262,36 @@ async function runCase(ports, one) {
|
|
|
193
262
|
};
|
|
194
263
|
}
|
|
195
264
|
const ms = now() - started;
|
|
196
|
-
|
|
265
|
+
if (one.request.feature === "writing-feedback") {
|
|
266
|
+
const checked2 = _chunkOPMY2MNScjs.checkAiWritingFeedback.call(void 0, raw, one.request);
|
|
267
|
+
if (checked2.ok) {
|
|
268
|
+
const { text, provenance, usage } = checked2.feedback;
|
|
269
|
+
return {
|
|
270
|
+
case: one,
|
|
271
|
+
ok: true,
|
|
272
|
+
result: {
|
|
273
|
+
text,
|
|
274
|
+
...provenance !== void 0 ? { provenance } : {},
|
|
275
|
+
...usage !== void 0 ? { usage } : {}
|
|
276
|
+
},
|
|
277
|
+
feedback: checked2.feedback,
|
|
278
|
+
ms
|
|
279
|
+
};
|
|
280
|
+
}
|
|
281
|
+
return refused(one, raw, checked2.refusal, ms);
|
|
282
|
+
}
|
|
283
|
+
const checked = one.request.feature === "explanation" ? _chunkOPMY2MNScjs.checkAiExplanation.call(void 0, raw, one.request) : _chunkOPMY2MNScjs.checkAiHint.call(void 0, raw, one.request);
|
|
197
284
|
if (checked.ok) {
|
|
198
285
|
return { case: one, ok: true, result: checked.result, ms };
|
|
199
286
|
}
|
|
287
|
+
return refused(one, raw, checked.refusal, ms);
|
|
288
|
+
}
|
|
289
|
+
function refused(one, raw, refusal, ms) {
|
|
200
290
|
const text = typeof _optionalChain([raw, 'optionalAccess', _ => _.text]) === "string" ? raw : void 0;
|
|
201
291
|
return {
|
|
202
292
|
case: one,
|
|
203
293
|
ok: false,
|
|
204
|
-
refusal
|
|
294
|
+
refusal,
|
|
205
295
|
...text !== void 0 ? { result: text } : {},
|
|
206
296
|
ms
|
|
207
297
|
};
|
|
@@ -210,9 +300,7 @@ async function runAiCheck(ports, options = {}) {
|
|
|
210
300
|
const all = _nullishCoalesce(options.cases, () => ( aiCheckCases()));
|
|
211
301
|
const asked = Math.trunc(_nullishCoalesce(options.concurrency, () => ( 1)));
|
|
212
302
|
const lanes = Number.isFinite(asked) && asked >= 1 ? Math.min(asked, 16) : 1;
|
|
213
|
-
const runnable = all.filter(
|
|
214
|
-
(one) => one.request.feature === "explanation" ? ports.explain !== void 0 : ports.hint !== void 0
|
|
215
|
-
);
|
|
303
|
+
const runnable = all.filter((one) => portFor(ports, one) !== void 0);
|
|
216
304
|
const results = new Array(runnable.length);
|
|
217
305
|
let next = 0;
|
|
218
306
|
const lane = async () => {
|
|
@@ -229,13 +317,13 @@ async function runAiCheck(ports, options = {}) {
|
|
|
229
317
|
await Promise.all(Array.from({ length: Math.min(lanes, runnable.length) }, lane));
|
|
230
318
|
const byRefusal = Object.fromEntries(REFUSALS.map((reason) => [reason, 0]));
|
|
231
319
|
let shown = 0;
|
|
232
|
-
let
|
|
320
|
+
let refused2 = 0;
|
|
233
321
|
let errors = 0;
|
|
234
322
|
for (const result of results) {
|
|
235
323
|
if (result.ok) {
|
|
236
324
|
shown += 1;
|
|
237
325
|
} else if (result.refusal !== void 0) {
|
|
238
|
-
|
|
326
|
+
refused2 += 1;
|
|
239
327
|
byRefusal[result.refusal] += 1;
|
|
240
328
|
} else {
|
|
241
329
|
errors += 1;
|
|
@@ -244,7 +332,7 @@ async function runAiCheck(ports, options = {}) {
|
|
|
244
332
|
return {
|
|
245
333
|
total: results.length,
|
|
246
334
|
shown,
|
|
247
|
-
refused,
|
|
335
|
+
refused: refused2,
|
|
248
336
|
errors,
|
|
249
337
|
skipped: all.length - runnable.length,
|
|
250
338
|
byRefusal,
|