@happyvertical/smrt-education 0.57.45 → 0.57.47

package/AGENTS.md

@@ -47,6 +47,11 @@ render ids and exact-revision references use [QuestionAuthoringService](agents/q
 Keep keys out of learner bundles; a missing key stays null. `attachGeneratedPrintable`
 stores the rendered PDFs; the guardian copy is always an `answer-key`, never `printable`.
 
+Structured questions from scans and imported course files use
+[QuestionExtractionService](agents/question-extraction.md) and the `extract-questions`
+ingestion handler. Anchors come from provider region catalogs only; match by worksheet
+id and layout, never text; ambiguous matches stay unattached; source-keyed replays.
+
 Source import: [host boundary and transaction contract](agents/source-import.md).
 `SourceImportService` requires host opt-in, matching active actor/tenant and
 explicit licensing authorization; never add a system/global write bypass.

package/README.md

@@ -98,6 +98,10 @@ can `getLearner`, `startLesson`, and `listProgress` for their own work.
 Guardian access requires a same-tenant directed `ProfileRelationship` plus the
 host's permission decision. Sibling membership does not grant access.
 
+`putBackLesson` returns started work to `not_started` and `setEnrollmentStatus`
+starts, pauses, resumes or drops an enrollment along a fixed transition table;
+both append immutable history (actor, and for status the host policy basis)
+and never touch submissions, grades or other progress.
 Repeated starts preserve the existing progress, including attempts and scores.
 A new curriculum/year creates a distinct enrollment and fresh progress history.
 An unscored lesson has `bestScore: null`, distinct from zero. The service does
@@ -114,7 +118,9 @@ integration on both SQLite and PostgreSQL.
 
 `SubmissionService` captures tenant-owned uploads, reviews or attests work with
 optional guardian scores, signs authorized asset claims, and splits private
-PDF packets by page range. It reuses learner ownership and transaction checks.
+PDF packets by page range. `attestOwnLesson` lets a learner mark their own
+Reading or Video lesson done as a self-reported, ungraded attestation that a
+guardian then confirms. It reuses learner ownership and transaction checks.
 See the [submission integration contract](agents/submission-service.md) for
 inputs, retry rules, signing requirements and external-storage compensation.
 

package/agents/learner-service.md

@@ -71,7 +71,36 @@ Same-tenant membership alone never grants sibling access.
 - `startLesson` allows the learner or authorized guardian to create their own
   enrollment's lesson progress in `in_progress`. It derives learner identity
   from the validated enrollment. Repeated starts return existing state without
-  resetting counters, scores, completion, or timestamps.
+  resetting counters, scores, completion, or timestamps. A `not_started` row
+  (a put-back lesson) moves to `in_progress` again with its counters, score and
+  first `startedAt` kept.
+- `putBackLesson({ enrollmentId, lessonId })` takes `in_progress` work back to
+  `not_started` for the learner or an authorized guardian under
+  `education.progress.put-back` (the host decides; own is not blocked). The
+  progress row keeps its one (enrollment, lesson) identity, so each change
+  appends an immutable `LessonProgressTransition` (from, to, kind, actor,
+  time) in the same transaction; restarting appends a `start` row when the
+  table exists. Counters, score, submissions, grades, workbook and evidence
+  rows are never touched. It returns `{ changed, progress }`: no progress or
+  already `not_started` is an idempotent no-op. `submitted`, `needs_review`,
+  `complete`, `skipped` (stored or workbook-projected) refuse with
+  `LessonPutBackError` (`.status`); completion is undone only through guardian
+  paths. A host without `lesson_progress_transitions` gets
+  `LearnerHistoryUnavailableError` from put-back; other operations still run.
+- `setEnrollmentStatus({ enrollmentId, status, policyBasis })` moves an
+  enrollment along `ENROLLMENT_STATUS_TRANSITIONS`: planned→active,
+  active→paused|dropped, paused→active|dropped. `completed` is reached only by
+  the completion path; completed and dropped are final. The authorizer gets
+  `education.enrollment.set-status` and decides whether a learner may act on
+  their own enrollment. `policyBasis` (1-200 characters, no control characters,
+  `requirePolicyBasis`) names the host setting relied on; it and the actor are
+  appended to an immutable `EnrollmentStatusChange` with the change, as
+  `pinEnrollment` records its basis. The first activation sets `startedAt`.
+  The current status is an idempotent `{ changed: false }`; any other move
+  throws `EnrollmentStatusTransitionError` (`.from`, `.to`). Progress,
+  submissions and history never cascade. Bad input throws
+  `LearnerValidationError`; a host without `enrollment_status_changes` gets
+  `LearnerHistoryUnavailableError`.
 - `listProgress` reads only the validated learner/enrollment and lessons still
   visible in that enrollment's course. Inconsistent persisted links fail
   closed or are omitted from the list.
@@ -91,7 +120,7 @@ Same-tenant membership alone never grants sibling access.
 Service results are data-only views, with no persistence or database methods.
 Extra request fields are never model patches: a learner cannot set score,
 attempt count, or completion status through this API. Curriculum activation,
-completion transitions and dashboard aggregation are not supported by this
+enrollment completion and dashboard aggregation are not supported by this
 base service. SubmissionService extends its owning transaction/auth boundary for
 capture and guardian grading; see [submission contract](submission-service.md).
 Direct model writes require trusted authority.

package/agents/question-extraction.md

@@ -0,0 +1,108 @@
+# Question extraction from scans and imported courses
+
+`QuestionExtractionService` (SMRT #4517, extends `QuestionAuthoringService`,
+consumed by Hearth #80 and #94) turns ingestion proposals into structured,
+versioned question records. Risk is high: new schema, a public contract and
+idempotent writes. It reuses the course authorizer: preserve and apply need
+`education.course.author`; `readExtraction`, `findExtraction` and
+`resolveWorksheet` need `education.course.read-draft`. Operations:
+`preserve-question-extraction`, `apply-question-extraction`,
+`read-question-extraction`, `read-worksheet`.
+
+## Flow
+
+1. Receive the scan or course file through ingestion and extract it (OCR or
+   vision, with `observeAnswerRegions` for image region catalogs).
+2. Offer `educationQuestionExtractionHandler({ extractions, request })` (from
+   `@happyvertical/smrt-education/ingestion`) in the execution catalog and
+   generate proposals. Its closed `QUESTION_EXTRACTION_ARGS_SCHEMA` is the
+   extracted question set. Keep `requireReview: true` on every policy layer.
+   Size the policy `maxBytes` and proposal output limits for whole worksheets:
+   up to 200 questions, 400 student-work entries and 1000 pages.
+3. `preserveQuestionExtractionAnalysis({ ingestion, extractions, request,
+   itemId, expectedWorksheetVersionId?, worksheets? })` reads the completed
+   proposal, validates it against retained evidence, matches it and records a
+   `QuestionExtractionRecord`. It returns `QuestionExtractionView` for
+   extraction review. Not ready, no proposal or competing proposals throw
+   `QuestionExtractionPendingError` (`reason`); invalid output throws
+   `QuestionExtractionValidationError`. No question records are written.
+4. The reviewer previews and decides through ingestion. The handler apply
+   calls `applyExtraction` on the ingestion transaction and returns the
+   per-question outcomes (`ExtractedQuestionResult[]`, also on `view.applied`).
+
+Hosts without an ingestion proposal (for example a trusted bundle importer)
+call `preserveExtraction` directly with their own grounding and extractor
+identity, then `applyExtraction`.
+
+## Extraction contract
+
+- Every retained source, and each page the provider reported, has a `pages`
+  entry with a role (`exercise`, `example`, `instructions`, `cover`,
+  `answer_key`, `mixed`, `unknown`) and readability. A missing page is invalid,
+  so unreadable or unsupported content is marked, never dropped. Items that
+  cannot be read keep their key and numbering with `status` `unreadable` or
+  `unsupported` and a null prompt; other gaps go in `omissions`.
+- What ingestion itself did not extract stays visible. A retained original with
+  no result, and pages that ingestion omitted or failed, are kept in the
+  grounding: the extractor may cite them, and any it does not cover are added as
+  `unreadable` pages with a `not_extracted` omission. Truncation and
+  `source_incomplete` warnings are carried to `extraction.sourceWarnings` and
+  the handler preview. More than 1000 pages is refused, never trimmed.
+- Questions hold printed text only. Learner handwriting goes in `studentWork`
+  (null `observed` when illegible) and never becomes question content.
+- Anchors cite an original `evidenceId`, a physical page the provider
+  reported (null when unknown) and a `regionId` from that evidence's
+  `observedRegions` catalog. The rect comes from the catalog only; an unknown
+  region is invalid. Coordinates are never accepted from the extractor.
+- `origin: 'source' | 'inferred'` marks response type, keys, rubrics and points.
+  On apply, printed facts get `extracted` provenance and inferred facts get
+  `inferred`. An empty key is invalid; a missing key stays null.
+- Diagram and media references are kept on the link with their anchor. They
+  are not question `media`, which must name a tenant asset.
+
+## Matching
+
+`matchExtractedQuestions(extraction, worksheets, { expectedWorksheetVersionId })`
+is pure. A match needs a worksheet identity (the `wsv1-` id generated printables
+print in their footer, or a host-supplied expectation), plus the printed
+position (`1`, `1.`, `Q1`) of an item on that worksheet. Every claimant of a
+position counts, including one that fails a layout check. Section label, a
+source-printed response type, and choice and subpart counts must agree. Prompt
+text is never used. A printed id that disagrees with the expectation, any layout
+conflict, or two items claiming one position is `ambiguous` and stays
+unattached. Examples are never matched. `resolveWorksheet` finds a worksheet by
+id through `generated_printables`; host-supplied worksheets must be this
+tenant's revisions. Student work gets a `questionRevisionId` only when its
+question matched. Recording attempts and grades stays with the workbook and
+response services.
+
+## Idempotency
+
+`sourceIdentity` is a digest of the source kind and the sorted retained evidence
+hashes, never text. Preserve is idempotent per (tenant, intake item, analysis
+attempt). A different payload for the same attempt throws
+`CourseRequestKeyConflictError`. The payload includes the grounding, the
+extractor identity and the expected and supplied worksheets. Apply writes questions through `createQuestion`
+with request key `qx:<sourceIdentity>:<questionKey>`, so re-ingesting the same
+bytes replays the same questions (`replayed`). Identical prompts under different
+keys stay separate records. If the same source is extracted again with different
+content, the outcome is `source_changed`: nothing is overwritten. A new scan has
+new bytes, so it is a new source, and it links to known questions only by
+matching. A repeated apply replays the stored `ExtractedQuestionLink` rows.
+
+Outcomes: `created`, `replayed`, `matched` (these carry a revision),
+`needs_review` (ambiguous), `source_changed`, `invalid_content` (fails question
+validation), `unreadable`, `unsupported`, `not_an_exercise`.
+
+## Models (closed, required tenant, no REST/CLI/MCP/AI, append-only)
+
+`QuestionExtractionRecord` (`question_extractions`: unique tenant/item/attempt,
+indexed source identity, extractor identity, validated extraction and match JSON,
+proposal digest) and `ExtractedQuestionLink` (`extracted_question_links`: unique
+tenant/extraction/key; outcome, question and revision ids, anchors and media).
+Both ship through normal manifest migrations. The originals stay in ingestion
+and are never copied. Imported material keeps its course licence; extraction
+does not change sharing.
+
+Not in this release: zip/bundle manifests (SMRT #4578), automatic new
+revisions on `source_changed`, and media assets for extracted diagrams.

package/agents/submission-service.md

@@ -51,6 +51,21 @@ completion decision; grading alone leaves it in progress, including through a
 plain SubmissionService instance. Existing explicit completion on the current
 attempt is retained. The owning workbook join determines this policy, never a
 caller flag.
+`attestOwnLesson({ enrollmentId, lessonId, policyBasis, notes? })` is the
+learner's own "I read it" / "I watched it" for `SELF_ATTESTABLE_LESSON_KINDS`
+(Reading, Video) under `education.learner.attest-own`. Any other actor is
+refused before the host authorizer; the host decides from its own setting
+(passed as `policyBasis`, 1-200 characters) whether the learner may. It writes
+an ungraded `Attestation` and an insert-only `LessonSelfAttestation` (learner
+actor, basis), moves progress to `submitted` with one attempt, and creates no
+Grade: it is not a grade and never completes the lesson. A repeat returns the
+same attestation. Other kinds throw `OwnAttestationLessonKindError`;
+submitted, review, complete, skipped or already-attested progress throws
+`OwnAttestationConflictError`. Submission views carry `selfReported: true` for
+it; a guardian confirms with `attestLesson` (which grades that attestation) or
+`reviewSubmission`, and the guardian grade then distinguishes confirmed from
+self-reported evidence. A host without `lesson_self_attestations` gets
+`LearnerHistoryUnavailableError`.
 Omitted/null score is unscored; zero is a real score. The latest guardian grade
 wins over auto/AI/instructor proposals, and its score replaces progress's score.
 Review history is retained. ExternalCompletion is available for trusted server