@happyvertical/smrt-education 0.57.50 → 0.57.52

package/AGENTS.md

@@ -52,7 +52,7 @@ Structured questions from scans and imported course files use
 ingestion handler. Anchors come from provider region catalogs only; match by worksheet
 id and layout, never text; ambiguous matches stay unattached; source-keyed replays.
 
-Course bundles (zip with a closed manifest, or a file set) use
+Course bundles (zip with a closed manifest, or a file set; Word/PowerPoint members render through an injected `DocumentConverter`, fail closed) use
 [CourseBundleImportService](agents/course-bundle-import.md). Parse only with the
 bounded reader; refuse, never trim, hostile input; unsupported members stay
 visible; proposals only, nothing auto-published.

package/README.md

@@ -109,7 +109,12 @@ host setting and optional evidence submission; guardian `attestLesson` is
 unchanged. `setEnrollmentPace` sets or clears an enrollment's planned lessons
 per week (1 to 14) under the host's `education.enrollment.set-pace` decision with
 an immutable change row, and `nextLessons({ count })` returns a week's set of
-unfinished lessons (default one).
+unfinished lessons (default one). `setInterestTheme` stores a learner's interest
+theme (a short plain label and an optional stable key such as `minecraft`; the
+host owns its starter list) under `education.learner.set-interest-theme`: a
+guardian may set any of their learners' themes, a learner only their own
+(the host decides), nobody another learner's. Each change appends an immutable
+row; `getLearner` and the tutor context read it back.
 `TutorContextService.compose` builds a bounded, learner-safe `TutorContext`
 (lesson text, learner-visible questions, own progress, released feedback; for the
 learner audience only `answerKeyAvailable`, never keys, rubrics, notes or storage
@@ -123,6 +128,18 @@ or file set under hard size, ratio, path and entry bounds, keeps unsupported and
 duplicate members as append-only evidence, hands accepted members to ingestion and
 proposes sections and lessons through `CourseGenerationReviewService`; see
 [bundle import](agents/course-bundle-import.md).
+`requestGeneration` also accepts a structured `brief` (subject, category, area,
+grade, objective, a `course` or one-`lesson` target and a per-request interest
+theme that beats the learner's saved theme for that request only); the theme used
+and whether it was an override show on the guardian's review, and `release`
+accepts a `policyBasis` so a host can honour its own approve-requested-lessons
+setting without bypassing the review gate; see
+[generation and review](agents/course-generation-review.md).
+`CourseAuthoringService.appendLesson` adds one lesson (materials and question
+records included) to a released course as a new, optionally released revision:
+append-only, idempotent by request key, refused on archived, unreleased or
+foreign courses; enrollments stay on their pinned release until the host
+upgrades them; see [course authoring](agents/course-authoring-service.md).
 Repeated starts preserve the existing progress, including attempts and scores.
 A new curriculum/year creates a distinct enrollment and fresh progress history.
 An unscored lesson has `bestScore: null`, distinct from zero. The service does

package/agents/course-authoring-service.md

@@ -78,6 +78,45 @@ lesson identity; `Lesson.externalRef` and `ReleasedLesson` carry the stable
 lesson identity for progress/reporting. Learners on different releases never
 share mutable rows. Personal-use releases stay `draft` status (the existing rule).
 
+## Append a lesson to a released course
+
+`appendLesson({ courseId, expectedHeadRevisionId, requestKey, sectionKey,
+sectionTitle?, lesson, release?, policyBasis? })` adds exactly one lesson (a
+`CourseLessonInput` with a required `lessonKey`, its `materials` and `questions`
+exactly as `saveDraft` takes them) to a course that already has a release, as one
+new revision, and releases that revision in the same transaction (`release:
+false` only appends). Returns `CourseAppendLessonView` (`revisionId`, `version`,
+`lessonKey`, `release | null`). Host mapping: action `education.course.author`,
+operation `append-lesson` (also `generate` for generated provenance), and, when
+releasing, `education.course.release` with the usual `policyBasis` and delegation
+rules. Hosts that grant `author` but not `release` use `release: false`.
+
+- **Append-only.** The new revision is the head plus the lesson at the end of
+  `sectionKey` (a new section needs `sectionTitle`). No lesson, revision or release
+  is changed; the release is a new Course row with the next release number.
+- **Refused.** Unknown or foreign-tenant course (`CourseNotFoundError`), discarded,
+  archived (`CourseArchivedError`), no release yet (`CourseAppendRefusedError`
+  `not-released`), head with changes that were never released (`unreleased-changes`:
+  nothing unreviewed rides along; release or discard it first), `lessonKey` already
+  in the course (`duplicate-lesson`), stale `expectedHeadRevisionId`
+  (`CourseStaleHeadError`).
+- **Review gate.** Releasing goes through the injected `releaseReviewGate` unchanged.
+  The default review-backed gate needs a current review of the new revision, which
+  cannot exist before it does, so with that gate use `release: false`, `recordReview`,
+  then `release`. A host whose gate honours its own approval (the guardian already
+  approved the requested lesson) can release in the one call. A gate that is not
+  current rolls the append back entirely.
+- **Idempotent.** The same `requestKey` and request replays the original result, also
+  after the head moved (a no-op: no revision, release or lesson rows); a changed lesson
+  or section or release intent (`release` true versus false) under the same key is
+  `CourseRequestKeyConflictError`; publish a `release: false` append later with
+  `release` and its own key.
+- **Enrollment pins.** Existing enrollments keep their pinned release and so never see
+  the new lesson implicitly. The host advances them with `upgradeEnrollment` on the new
+  release (explicit, recorded), or an enrollment/course that opted in with
+  `setAutoUpgrade` is moved by the release exactly as for any release (unstarted
+  lessons follow, started lessons stay on their row). Nothing else moves a pin.
+
 ## Enrollment pin and upgrades
 
 `Enrollment` is untouched (its `courseId` stays immutable): enroll through

package/agents/course-bundle-import.md

@@ -38,7 +38,8 @@ authorizer: `importBundle` needs `education.course.author`, `readImport`
 4. `readImport({ ...actor, importId })` returns the recorded evidence.
 
 Progress is an observer callback (`onProgress({ stage, completed, total })`,
-stages `unpack`, `classify`, `extract`, `propose`; each starts at `completed: 0`).
+stages `unpack`, `classify`, `render`, `extract`, `propose`; each starts at `completed: 0`;
+`render` is emitted only when the bundle has Word/PowerPoint members).
 Ingestion has no public job-progress seam, so none is invented. A throwing
 listener never affects the import.
 
@@ -55,6 +56,9 @@ listener never affects the import.
 | `maxManifestBytes` | 256 KiB |
 | `maxLessons` (the proposal schema's cap) | 50 |
 | `maxLessonBodyChars` | 100 000 |
+| `maxOfficeMembers` (Word/PowerPoint members rendered per bundle) | 50 |
+| `maxRenderedBytes` (PDF accepted back from the converter) | 50 MiB |
+| `conversionTimeoutMs` (one Office member) | 60 000 |
 | archive nesting | 1: a zip inside a zip is never expanded |
 
 A lesson whose combined text members exceed `maxLessonBodyChars` refuses the
@@ -80,7 +84,7 @@ too long or deep). An **honestly** oversize member (declared over
 
 ## Members and statuses
 
-By content (never by extension alone): `pdf` (`%PDF-` in the first 1 KiB),
+By content (never by extension alone): `docx`/`pptx` (see below), `pdf` (`%PDF-` in the first 1 KiB),
 `image` (PNG, JPEG, WebP), `text` (`.txt`/`.md`, strict UTF-8, no control
 characters, body-sized), `manifest` (root `manifest.json`), else `unsupported`.
 Status: `accepted` (handed to extraction), `duplicate` (same sha256 as an earlier
@@ -91,6 +95,74 @@ extracted again), `unsupported` (`unsupported_type`, `nested_archive`,
 `/Encrypt` token anywhere in its bytes (trailer, xref stream): a false positive
 on literal text inside an uncompressed stream is refused, never read.
 
+## Word and PowerPoint members (SMRT #4615)
+
+A `.docx`/`.pptx` member is an OOXML package only when the name and the content
+agree: a zip whose entries include `[Content_Types].xml` and `word/document.xml`
+(docx) or `ppt/presentation.xml` (pptx). Before the bytes go anywhere, the
+package itself passes the same bounded zip reader (ratio, overlap, paths, CRC,
+100 MiB inflated); a failure is `refused / unsafe_office`, never a refusal of the
+whole bundle. Other zips stay `unsupported / nested_archive`.
+
+Two separate things happen to an accepted Office member, both after
+authorization and before the evidence is written:
+
+- **Pages**: a `DocumentConverter` (`document-converter.ts`) renders the original
+  to PDF. The importer enforces the byte and time limits itself (input
+  `maxEntryBytes`, output `maxRenderedBytes`, `conversionTimeoutMs` with an abort
+  signal and a hard race, so an adapter that ignores the signal still times out),
+  and rejects output that is not a PDF or is encrypted. The converter is told a
+  neutral file name (`document.docx`), never the bundle path. Identical documents
+  are rendered once.
+- **Text**: an injectable `OfficeTextReader` (best effort; a failure or a `null`
+  costs the member nothing) returns whole-document text and, for decks, text and
+  speaker notes per slide. Office text is handed to the host on the accepted
+  member (`officeText`) and is **never** inlined into a proposed lesson body:
+  teacher guides and notes carry answers, and audience belongs to manifest v2.
+
+`receiveMember` gets the original `bytes` (store them as the private original
+asset: "Open in PowerPoint" for decks) and `rendered` (`{ bytes, sha256, mediaType:
+'application/pdf', converterId }`, store as a private asset beside the original:
+the pages for "Present"/reading). The member row records `renderedSha256` and
+`renderedSizeBytes`; bytes are still not retained here.
+
+**Fail closed.** Any converter problem records that member `unsupported` with a
+reason and writes the rest of the bundle normally: `converter_unavailable` (none
+configured), `converter_failed` (unreachable or 5xx), `converter_timeout`,
+`converter_too_large`, `converter_rejected` (HTTP 4xx, not a PDF, encrypted),
+`office_limit` (beyond `maxOfficeMembers`, in natural path order). A member that
+failed is not a lesson in the proposal, and identical copies of a failed document
+fail the same way rather than becoming duplicates of nothing. Outcomes are part
+of the append-only evidence, so a replay does not retry a recorded failure (to
+retry, change the bundle). A replay does re-render recorded-accepted members so the
+host receives the bytes again; if it can no longer render one it throws
+`DocumentConversionError` rather than silently dropping an accepted member.
+
+### Host configuration
+
+```ts
+const service = new CourseBundleImportService(db, authorizer, {
+  converter: new GotenbergDocumentConverter({
+    baseUrl: process.env.GOTENBERG_URL, // for example http://gotenberg:3000
+    timeoutMs: 60_000,
+    maxBytes: 50 * 1024 * 1024,
+  }),
+  officeTextReader, // optional, see below
+});
+```
+
+`GotenbergDocumentConverter` posts to `<baseUrl>/forms/libreoffice/convert`, talks
+to nothing else, refuses redirects and URLs with credentials, and caps the response
+as it streams. Run Gotenberg as an internal-only service with **no outbound
+network**, its own size/time limits and no shared volume with the app; this
+package assumes untrusted files never run next to the application. Tests use
+`FakeDocumentConverter` (records calls; `failWith`, `failWhen`, `delayMs`, `output`).
+
+`OfficeTextReader` is deliberately not wired to the SDK here, so this package
+does not depend on an unreleased `@happyvertical/documents`. Once the Office
+processors ship (sdk#1485, `extractOffice(bytes, 'docx' | 'pptx')`), implement
+`read` with it; the snippet is in the `OfficeTextReader` doc comment.
+
 ## Manifest (`manifest.json`, version 1, closed)
 
 ```json
@@ -139,6 +211,12 @@ Normal manifest migrations create both tables.
 
 ## Not in this release
 
+- Manifest version 2 (material roles with audience, lesson pages, course-level
+  members, external video references) waits for the phase 1 role vocabulary
+  (smrt#4614). The page-role vocabulary and the closed v1 schema are unchanged;
+  see the hand-off note on smrt#4615.
+- Re-rendering a member whose first import recorded a converter failure (outcomes
+  are append-only per bundle identity).
 - Ingestion's receipt contract is unchanged: there is no `application/zip` part
   type or `parentEvidenceId` in ingestion. The education side unpacks and the
   host receives members one by one, so archive-to-member lineage lives in

package/agents/course-generation-review.md

@@ -40,6 +40,59 @@ a revision on `expectedHeadRevisionId` and records the decision; supplied conten
 is replaced only that way. Never write generated content to a released revision or
 assign it to a learner.
 
+## Structured requests and one-lesson generation
+
+`requestGeneration` also takes an optional `brief` (`CourseGenerationBriefInput`)
+next to the free `instruction`: `target` (`'course'`, the default and today's
+behaviour, or `'lesson'`), `subject`, `category`, `area`, `grade` (whole 0-12),
+`objective`, `learningObjectiveId`, `learnerId` and a per-request `theme`
+(`{ label, key? }`). Text is bounded and sent to the provider as data. A request
+without `brief` behaves exactly as before and `brief` reads back `null`.
+
+- **Objective.** A `lesson` target needs an objective (`objective`, else `area`);
+  without one `CourseGenerationBriefError` is thrown before anything is written.
+  The objective is stored on the job's `CourseGenerationBriefRecord`, sent in the
+  generation brief as fixed, and copied verbatim onto the proposed lesson. The
+  theme changes story and examples only: output that states a different
+  objective fails the job (`CourseGenerationOutputError`).
+- **Theme.** The per-request `theme` is an override that wins for that request
+  only. Without one the learner's saved `interestTheme` (read-only, from
+  `brief.learnerId`) applies; with neither there is no theme. The brief records
+  `theme` and `themeSource` (`'override' | 'saved' | 'none'`) at request time, and
+  `CourseGenerationView.brief` / `CourseProposalView.brief` show it to the
+  guardian reviewing. Nothing in generation writes to the learner, so the saved
+  preference is never changed, on success or failure.
+- **Authority.** The host authorizer is asked for `education.course.generate`
+  with `learnerId` in the context so it can enforce its own "may this learner ask
+  for a lesson" policy; an unknown or foreign-tenant learner is `CourseNotFoundError`.
+  The package does not check the requester's relationship to `brief.learnerId` (the
+  brief echoes that learner's saved theme): the host authorizer MUST deny `generate`
+  when the actor is not that learner or a guardian with an effective relationship, as
+  it does for `assign`.
+  Replaying a request key with the same brief returns the same job; a changed
+  brief is `CourseRequestKeyConflictError`.
+- **Output.** For `lesson` the generator is given `request.brief` and
+  `COURSE_LESSON_PROPOSAL_ARGS_SCHEMA` (via `createProposalCourseGenerator`):
+  exactly one new lesson with `title`, `body` (the themed explanation) and
+  `questions` (worksheet; every question needs an `answerKey`), and no course-level
+  fields. The worksheet becomes immutable question records, linked to
+  `learningObjectiveId`, in the same transaction as the draft revision; answer keys
+  stay out of learner views. Review, `decideProposal` and release are unchanged.
+- **Approving requested lessons.** The package has no per-learner setting. To honour
+  the host's own "approve requested lessons" setting without bypassing authority,
+  the host calls `recordReview` then `release` under the guardian delegation as for
+  any course, passing `policyBasis` (for example `'auto-approve-requested-lessons'`)
+  to `release`, or returning `policyBasis` from its authorizer decision (the two
+  must agree). The basis is recorded on `CourseRelease.policyBasis` and
+  `CourseReleaseView.policyBasis`; it grants nothing and the review gate, the
+  `education.course.release` grant and the delegation check still apply. The
+  explicit alternative, a guardian approving each lesson, is the existing
+  `recordReview` plus `release` by the guardian.
+
+Additive schema: table `course_generation_briefs` and column
+`course_releases.policy_basis`. On a host without the table, plain requests still
+work and structured ones fail closed with `schema-unavailable`.
+
 ## Review
 
 `recordReview` (action `education.course.review`) binds to an exact revision and a

package/agents/learner-service.md

@@ -131,6 +131,26 @@ Same-tenant membership alone never grants sibling access.
   without `enrollment_pace_changes` gets `LearnerHistoryUnavailableError`.
   Apply the additive migration (new column plus table) before deploying writes;
   reads of an un-migrated table report no pace.
+- `setInterestTheme({ theme: { label, key? } | null, policyBasis })` stores the
+  learner's interest theme under `education.learner.set-interest-theme`. The
+  label is a short plain string (1-60 characters after whitespace is collapsed;
+  control characters, `@`, `://` and `www.` are refused so it cannot carry
+  contact details) and the optional key is lowercase `[a-z0-9-]` up to 40
+  characters. The package ships no starter list: the host owns its keys
+  (for Hearth `minecraft`, `roman`, `hockey`, `baseball`, `football`, `soccer`,
+  `cars`) and family-added themes are free labels without a key. A guardian with
+  an effective relationship may set any of their learners' themes; the learner
+  may set their own when the host authorizer answers `own: true` as allowed;
+  another learner's theme needs the actor's own guardian relationship, so a
+  learner, sibling or stranger is refused by the same actor/tenant checks as
+  every other operation. It is a skin for stories and examples only; it never
+  changes an objective. Columns `learners.interest_theme_label` and
+  `interest_theme_key` (additive; empty = none) hold the value and each change
+  appends a `LearnerInterestThemeChange` (actor, `policyBasis`, previous and new
+  value). An unchanged value is `{ changed: false }`; a bad value is
+  `InterestThemeError`; a host without `learner_interest_theme_changes` gets
+  `LearnerHistoryUnavailableError`. `getLearner` returns `interestTheme`
+  (`{ label, key } | null`). Apply the additive migration before deploying.
 - `nextLessons({ enrollmentId?, count? })`: `count` is an integer from 1 to
   14 (`MAX_NEXT_LESSONS_COUNT`, default 1, else `LearnerValidationError`) and
   returns up to that many unfinished (not complete or skipped) lessons per

package/agents/tutor-context.md

@@ -15,7 +15,8 @@ enrollment's effective set; a focus question must belong to that lesson
 (otherwise `LearnerAuthorizationError`). The guardian audience is refused for
 the learner's own actor whatever the host answers.
 
-Result: learner (name, level), enrollment (status, pace; no notes), course,
+Result: learner (name, level, `interestTheme`: the learner-safe label and key or
+null, read-only; never the free `settings`), enrollment (status, pace; no notes), course,
 section and lesson titles, learner-facing lesson body (withheld for
 personal-use-copy), objectives, own progress status, the learner-visible
 question rendering (prompt, response type, choices, subpart prompts, points; no