@learnpack/learnpack 5.0.353 → 5.0.355

package/src/scripts/README.md

@@ -0,0 +1,244 @@
+# Descriptions scripts (`src/scripts/`)
+
+Ops scripts for the **step descriptions** of published courses. They are dev/ops
+entry points, not oclif commands: they never appear under `learnpack <something>`
+and are run with `node` against the compiled output.
+
+| Script                    | What it does                                                            | Calls the LLM |
+| ------------------------- | ----------------------------------------------------------------------- | ------------- |
+| `descriptionsS3Backfill`  | Generates descriptions for **published** courses (S3)                   | Yes           |
+| `descriptionsGcsBackfill` | Copies existing descriptions from S3 into the **draft** (GCS)           | No            |
+| `descriptionsSweep`       | Finishes publications that stalled mid-flight; runs on Heroku Scheduler | Yes           |
+
+All three delegate the per-course work to the same service,
+[`generateCourseDescriptions`](../utils/descriptions/generateCourseDescriptions.ts),
+which is also what the post-publish stage uses. They differ only in **which**
+courses they run it on and **what else** they do around it.
+
+## Build and run
+
+```bash
+npm run tsc
+node lib/scripts/<script>.js [flags]
+```
+
+They run against `lib/`, not `src/`, so recompile after editing. There is no
+`--help`: the flags are documented here and in each script's header.
+
+## Environment
+
+Copy from [`.env.example`](../../.env.example) into `learnpack-cli/.env`. These
+scripts read `process.env` directly — they do **not** load the `.env` themselves,
+so export it (`set -a; . ./.env; set +a`) or pass the variables inline.
+
+| Variable                                   | Needed by                                                                       |
+| ------------------------------------------ | ------------------------------------------------------------------------------- |
+| `RIGOBOT_SYSTEM_TOKEN`                     | S3 backfill, sweep (generation)                                                 |
+| `BREATHECODE_SYSTEM_TOKEN`                 | `--emit-events`, sweep (pending events)                                         |
+| `AWS_ACCESS_KEY_ID` / `_SECRET_ACCESS_KEY` | anything reading the published bucket                                           |
+| `AWS_REGION`                               | optional, default `us-east-1`                                                   |
+| `S3_PACKAGES_BUCKET`                       | optional in the backfills (default `learnpack-paquetes`), required by the sweep |
+| `CLOUDFRONT_DISTRIBUTION_ID`               | optional; without it manifests are re-projected but not invalidated             |
+| `GCP_CREDENTIALS_JSON`                     | GCS backfill, sweep, `--mirror-draft`                                           |
+| `GCP_BUCKET_NAME`                          | same; no default on purpose                                                     |
+
+Descriptions are always generated with the Rigobot **service** account, never
+with a user token: they are platform enrichment, not something a creator is
+billed for.
+
+---
+
+## Mode 1 — targeted run (a few slugs, on demand)
+
+The everyday case: a course was published before descriptions existed, or one
+needs remediation. Three flags make the S3 backfill reproduce per course
+everything a publication does, in one process:
+
+```bash
+node lib/scripts/descriptionsS3Backfill.js --slug my-course \
+  --reproject-manifest --mirror-draft --emit-events --concurrency 2
+```
+
+| Flag                   | Without it                                                                              |
+| ---------------------- | --------------------------------------------------------------------------------------- |
+| `--reproject-manifest` | `package-manifest.json` keeps the old descriptions until a later pass                   |
+| `--mirror-draft`       | the draft has no fingerprints, so the **next publication regenerates the whole course** |
+| `--emit-events`        | breathecode is never told the manifest changed                                          |
+
+All three are off by default, so the catalogue-wide run below is unaffected.
+
+Run it once per slug. Do a `--dry-run` first if you want to see the scope —
+it reports what it would generate and mirror without writing anything.
+
+**During high-traffic hours**, lower `--concurrency` (default 5 completions in
+flight per course). A single completion takes ~15s.
+
+Notes:
+
+- `--emit-events` **requires** `--reproject-manifest`. The event reports a
+  manifest update, so emitting it without projecting would state something
+  untrue. It is required rather than implied on purpose: writing to S3 and the
+  CDN as a side effect of another flag is the kind of surprise a backfill should
+  not have.
+- `--emit-events` is rejected together with `--dry-run`: there is no dry run of
+  an event breathecode already received.
+- `--mirror-draft` is independent of the other two — it touches the draft, not
+  the manifest.
+- Prefer `--slug` over `--limit 4` when you care about _which_ courses are
+  touched: `--limit` counts **processed** courses (already-settled ones are
+  skipped without counting) and walks the catalogue by recency.
+
+### What each course does, in order
+
+1. Generate the missing descriptions and write them into the published
+   `initialSyllabus.json` (S3).
+2. `--reproject-manifest`: rebuild `package-manifest.json` from that syllabus
+   (the same `processPackage` the manifest backfill runs) and invalidate its
+   CloudFront path.
+3. `--mirror-draft`: copy the descriptions back into the GCS draft, guarded by
+   the README content hash — anything the teacher edited since publishing is
+   counted as `missed` and waits for the next publication.
+4. `--emit-events`: send `package_manifest_updated` to breathecode.
+
+A failure in step 3 does not change the event status: the published package and
+its manifest are already correct, only the draft lags behind. A failure in step 2
+**does** — see below.
+
+---
+
+## Mode 2 — catalogue backfill (fases 0a / 0b)
+
+The one-off sweep over the ~400 published courses and ~800 drafts. Here the
+defaults are the right ones: manifests are cheaper to project in a single later
+pass than course by course, and the draft is seeded by copy rather than by
+re-generating.
+
+```bash
+# Fase 0a — descriptions for the published catalogue, 50 courses per run.
+# Re-run until it stops reporting processed courses.
+node lib/scripts/descriptionsS3Backfill.js --limit 50
+
+# Then project every manifest in one pass (cheaper than per course).
+npm run backfill:package-manifest
+
+# Fase 0b — seed the drafts by copying from S3. Never calls the LLM.
+node lib/scripts/descriptionsGcsBackfill.js --limit 200
+```
+
+Order matters: 0b reads the **published** syllabus, so 0a must have written it
+first. Whatever 0b cannot reuse (drafts edited since their last publication) is
+left for the next publication of that course, or for a single-slug 0a run.
+
+No events are emitted in this mode. If breathecode needs to be told about the
+catalogue, that is a separate decision — adding `--emit-events` to a 400-course
+run would deliver 400 events.
+
+---
+
+## Flag reference
+
+### `descriptionsS3Backfill`
+
+| Flag                   | Default              | Description                                                       |
+| ---------------------- | -------------------- | ----------------------------------------------------------------- |
+| `--slug <slug>`        | —                    | single course (remediation / on-demand)                           |
+| `--limit <n>`          | 50                   | courses **processed** per run, ordered by syllabus recency        |
+| `--s3-bucket <name>`   | `learnpack-paquetes` | published bucket                                                  |
+| `--gcs-bucket <name>`  | `GCP_BUCKET_NAME`    | draft bucket, for `--mirror-draft`                                |
+| `--region <region>`    | `us-east-1`          | AWS region                                                        |
+| `--concurrency <n>`    | 5                    | completions in flight per course                                  |
+| `--target-words <n>`   | 25                   | words per description                                             |
+| `--dry-run`            | off                  | preview; no writes, no completions                                |
+| `--force`              | off                  | regenerate courses already settled at the current prompt version  |
+| `--no-reconcile`       | off                  | skip the additive syllabus reconciliation                         |
+| `--reproject-manifest` | off                  | rebuild + invalidate `package-manifest.json` per course           |
+| `--mirror-draft`       | off                  | copy the descriptions into the GCS draft                          |
+| `--emit-events`        | off                  | send `package_manifest_updated` (requires `--reproject-manifest`) |
+
+### `descriptionsGcsBackfill`
+
+| Flag                  | Default              | Description                      |
+| --------------------- | -------------------- | -------------------------------- |
+| `--slug <slug>`       | —                    | single course                    |
+| `--limit <n>`         | 50                   | courses per run                  |
+| `--s3-bucket <name>`  | `learnpack-paquetes` | source (published) bucket        |
+| `--gcs-bucket <name>` | `GCP_BUCKET_NAME`    | target (draft) bucket            |
+| `--region <region>`   | `us-east-1`          | AWS region                       |
+| `--dry-run`           | off                  | preview without writing          |
+| `--no-reconcile`      | off                  | skip the syllabus reconciliation |
+
+### `descriptionsSweep`
+
+No flags — it is a scheduled job, configured by env:
+
+```bash
+node lib/scripts/descriptionsSweep.js
+```
+
+Two sources of work:
+
+1. **Publish journals** (`publish-journal/` in GCS), the primary one. Every
+   publication that stalled is resumed: missing descriptions generated, draft
+   mirrored, and the owed `package_manifest_updated` emitted. Journals are
+   dropped once complete and abandoned after a few attempts.
+2. **A deep scan** of published courses (`DESCRIPTIONS_SWEEP_FULL=1`), for
+   packages that never got a journal at all. Expensive — it reads every course —
+   and off by default. It re-projects manifests but does **not** emit events.
+
+Tuning: `DESCRIPTIONS_SWEEP_LIMIT` (courses per deep scan, default 25),
+`DESCRIPTION_STALE_AFTER_MS` (how long a publication may look stalled before it
+is resumed, default 15 min).
+
+---
+
+## The `package_manifest_updated` event
+
+Emitted by `--emit-events`, and by the sweep for the publications it finishes.
+Built in [`publishEvents.ts`](../utils/publishEvents.ts), sent to the breathecode
+telemetry endpoint with `BREATHECODE_SYSTEM_TOKEN`.
+
+Breathecode processes it **independently of `package_published`**, so a backfill
+event with no preceding publication is fine.
+
+**`publish_id` in a backfill** is `backfill-{runId}-{slug}`:
+
+- the `backfill-` prefix makes the origin readable in the webhook log and
+  discriminable in code (`isBackfillPublishId`); real publish ids are bare
+  uuids, so the two spaces cannot collide;
+- `{slug}` keeps it unique per event;
+- `{runId}` is one uuid per **invocation**, shared by every course it touches, so
+  "every event from the run I fired at 15:40" is a single substring query. It is
+  printed at start and in the closing summary.
+
+Note that running four separate `--slug` commands produces four run ids. Use
+`--limit` if you want one id for the batch — at the cost of not choosing the
+slugs.
+
+**Status is tied to the real outcome**, not to the script finishing:
+
+| Outcome                                  | `status`  |
+| ---------------------------------------- | --------- |
+| Descriptions written, manifest projected | `success` |
+| Generation failed outright               | `failed`  |
+| Manifest projection failed               | `failed`  |
+| Course unchanged (`skipped`)             | no event  |
+| Course threw                             | no event  |
+
+A failed manifest projection is not a failed run — the syllabus, which is the
+source of truth, is already saved — but this event reports the manifest, so it
+cannot claim success over one that was never rewritten. That is what
+`GenerateCourseDescriptionsResult.manifestProjected` exists for.
+
+The last row is deliberate: unlike a publication, a backfill never _announced_ an
+event, so it owes none. The course is simply re-run.
+
+---
+
+## Related
+
+- [`../utils/descriptions/`](../utils/descriptions/) — the shared service, the
+  mirror, the post-publish stage and the resume logic
+- [`../../scripts/README.md`](../../scripts/README.md) — the package-manifest S3
+  backfill (`npm run backfill:package-manifest`)
+- [`../utils/publishJournal.ts`](../utils/publishJournal.ts) — why the sweep can
+  know an event is owed

package/src/scripts/descriptionsS3Backfill.ts

@@ -13,31 +13,52 @@
  *     advance through the catalogue.
  *   - Or target a single course with --slug (remediation / on-demand).
  *
- * Writes descriptions into the published initialSyllabus.json. Manifest
- * projection is a separate pass here: run the package-manifest backfill
- * (runBatch) afterwards, which is cheaper than re-projecting course by course.
+ * Writes descriptions into the published initialSyllabus.json. By default the
+ * manifest projection is a separate pass: run the package-manifest backfill
+ * (runBatch) afterwards, which over the whole catalogue is cheaper than
+ * re-projecting course by course.
  *
- * Secrets/infra via env: RIGOBOT_SYSTEM_TOKEN (required), plus S3_PACKAGES_BUCKET
- * and AWS_REGION as fallbacks for --s3-bucket / --region.
+ * That default inverts for small, targeted runs, where three flags reproduce per
+ * course what a publication does, in one process instead of three:
+ *
+ *   --reproject-manifest  projects each course as it is described (the same
+ *                         `processPackage` the separate pass runs, plus a
+ *                         CloudFront invalidation);
+ *   --mirror-draft        copies the new descriptions back into the GCS draft,
+ *                         so the next publication does not regenerate them;
+ *   --emit-events         announces the result to breathecode.
+ *
+ * Secrets/infra via env: RIGOBOT_SYSTEM_TOKEN (required), BREATHECODE_SYSTEM_TOKEN
+ * (required by --emit-events), GCP_CREDENTIALS_JSON and GCP_BUCKET_NAME (required
+ * by --mirror-draft), plus S3_PACKAGES_BUCKET, AWS_REGION and
+ * CLOUDFRONT_DISTRIBUTION_ID as fallbacks/optionals.
  *
  * Per-run flags:
- *   --s3-bucket <name>   S3 bucket (or env S3_PACKAGES_BUCKET, default learnpack-paquetes)
- *   --region <region>    AWS region (default us-east-1, or env AWS_REGION)
- *   --slug <slug>        process a single course (remediation / on-demand)
- *   --limit <n>          courses per run (default 50)
- *   --target-words <n>   words per description (default 25)
- *   --concurrency <n>    completions in flight per course (default 5)
- *   --dry-run            preview without writing/generating
- *   --no-reconcile       skip the additive syllabus reconciliation
- *   --force              regenerate even courses already settled at this prompt version
+ *   --s3-bucket <name>    S3 bucket (or env S3_PACKAGES_BUCKET, default learnpack-paquetes)
+ *   --gcs-bucket <name>   GCS draft bucket for --mirror-draft (or env GCP_BUCKET_NAME)
+ *   --region <region>     AWS region (default us-east-1, or env AWS_REGION)
+ *   --slug <slug>         process a single course (remediation / on-demand)
+ *   --limit <n>           courses per run (default 50)
+ *   --target-words <n>    words per description (default 25)
+ *   --concurrency <n>     completions in flight per course (default 5)
+ *   --dry-run             preview without writing/generating
+ *   --no-reconcile        skip the additive syllabus reconciliation
+ *   --force               regenerate even courses already settled at this prompt version
+ *   --reproject-manifest  re-project package-manifest.json per course, and invalidate it
+ *   --mirror-draft        mirror the descriptions back into the GCS draft
+ *   --emit-events         send package_manifest_updated per course (implies --reproject-manifest)
  *
  * Examples:
  *   node lib/scripts/descriptionsS3Backfill.js --limit 50
  *   node lib/scripts/descriptionsS3Backfill.js --slug my-course
  *   node lib/scripts/descriptionsS3Backfill.js --dry-run --limit 10
+ *   node lib/scripts/descriptionsS3Backfill.js --slug my-course \
+ *     --reproject-manifest --mirror-draft --emit-events
  */
 import { parseArgs } from "node:util"
 import { ListObjectsV2Command, S3Client } from "@aws-sdk/client-s3"
+import { CloudFrontClient } from "@aws-sdk/client-cloudfront"
+import { Storage } from "@google-cloud/storage"
 import { DESCRIPTION_TARGET_WORD_COUNT } from "../utils/packageManifest"
 import { AwsClient } from "../utils/s3/packageManifestBackfill"
 import {
@@ -47,7 +68,16 @@ import {
 import {
   createS3DescriptionsStorage,
   fetchExercises,
+  S3DescriptionsStorageOptions,
 } from "../utils/descriptions/s3Storage"
+import {
+  BackfillEventOutcome,
+  emitBackfillManifestEvent,
+  newBackfillRunId,
+} from "../utils/descriptions/backfillEvents"
+import { createGcsDescriptionsStorage } from "../utils/descriptions/gcsStorage"
+import { mirrorDescriptionsToDraft } from "../utils/descriptions/mirrorDescriptions"
+import { requireGcsBucketName } from "../utils/gcsBucketName"
 import { isCourseSettled } from "../utils/descriptions/workList"
 
 const SYLLABUS_KEY_PATTERN = /^([^/]+)\/\.learn\/initialSyllabus\.json$/
@@ -86,10 +116,33 @@ async function listCoursesByRecency(
     .map(([slug]) => slug)
 }
 
+/**
+ * CDN invalidation for the re-projected manifests. Optional: a stale edge cache
+ * expires on its own, so a missing distribution id degrades the run instead of
+ * stopping it.
+ */
+function cloudFrontFor(
+  region: string
+): S3DescriptionsStorageOptions["cloudFront"] {
+  const distributionId = (process.env.CLOUDFRONT_DISTRIBUTION_ID || "").trim()
+  if (!distributionId) {
+    console.warn(
+      "[s3-backfill] CLOUDFRONT_DISTRIBUTION_ID is not set: manifests will be re-projected but not invalidated"
+    )
+    return undefined
+  }
+
+  return {
+    client: new CloudFrontClient({ region }) as unknown as AwsClient,
+    distributionId,
+  }
+}
+
 async function main(): Promise<void> {
   const { values } = parseArgs({
     options: {
       "s3-bucket": { type: "string" },
+      "gcs-bucket": { type: "string" },
       region: { type: "string" },
       slug: { type: "string" },
       limit: { type: "string" },
@@ -98,6 +151,9 @@ async function main(): Promise<void> {
       "dry-run": { type: "boolean", default: false },
       "no-reconcile": { type: "boolean", default: false },
       force: { type: "boolean", default: false },
+      "reproject-manifest": { type: "boolean", default: false },
+      "mirror-draft": { type: "boolean", default: false },
+      "emit-events": { type: "boolean", default: false },
     },
   })
 
@@ -116,12 +172,69 @@ async function main(): Promise<void> {
   const limit = Number.parseInt(values.limit || "50", 10) || 50
   const dryRun = values["dry-run"] ?? false
   const force = values.force ?? false
+  const emitEvents = values["emit-events"] ?? false
+  const reprojectManifest = values["reproject-manifest"] ?? false
+  const mirrorDraft = values["mirror-draft"] ?? false
+
+  // The event announces that the manifest changed, so emitting it without
+  // projecting the manifest would state something untrue. Required rather than
+  // implied: turning on a write to S3 and the CDN as a side effect of another
+  // flag is exactly the kind of surprise a backfill should not have.
+  if (emitEvents && !reprojectManifest) {
+    console.error(
+      "[s3-backfill] --emit-events requires --reproject-manifest: the event " +
+        "reports a manifest update that would not have happened"
+    )
+    process.exit(1)
+  }
+
+  const breathecodeToken = process.env.BREATHECODE_SYSTEM_TOKEN
+  if (emitEvents && !breathecodeToken) {
+    console.error(
+      "[s3-backfill] BREATHECODE_SYSTEM_TOKEN (env) is required by --emit-events"
+    )
+    process.exit(1)
+  }
+
+  if (emitEvents && dryRun) {
+    console.error(
+      "[s3-backfill] --emit-events cannot be combined with --dry-run: there is " +
+        "no dry run of an event that breathecode already received"
+    )
+    process.exit(1)
+  }
+
+  // Built up front, not on first use: a misconfigured draft bucket must fail at
+  // boot, not after the first course has already been billed to Rigobot.
+  let draftStorage
+  if (mirrorDraft) {
+    const credentialsEnv = process.env.GCP_CREDENTIALS_JSON
+    if (!credentialsEnv) {
+      console.error(
+        "[s3-backfill] GCP_CREDENTIALS_JSON (env) is required by --mirror-draft"
+      )
+      process.exit(1)
+    }
+
+    draftStorage = createGcsDescriptionsStorage(
+      new Storage({ credentials: JSON.parse(credentialsEnv) }).bucket(
+        values["gcs-bucket"] || requireGcsBucketName()
+      )
+    )
+  }
 
   const s3 = new S3Client({ region }) as unknown as AwsClient
-  // Manifests are projected in a separate pass after this backfill.
-  const storage = createS3DescriptionsStorage(s3, bucket, {
-    reprojectManifest: false,
-  })
+  const storage = createS3DescriptionsStorage(
+    s3,
+    bucket,
+    // Off by default: over the whole catalogue manifests are cheaper to project
+    // in a single later pass than course by course.
+    reprojectManifest ?
+      { cloudFront: cloudFrontFor(region) } :
+      { reprojectManifest: false }
+  )
+
+  const runId = newBackfillRunId()
 
   const singleSlug = values.slug
   const slugs = singleSlug ?
@@ -131,12 +244,22 @@ async function main(): Promise<void> {
   console.log(
     `[s3-backfill] Starting${
       singleSlug ? ` (slug=${singleSlug})` : ` (limit=${limit})`
-    }${dryRun ? " (dry-run)" : ""} over ${slugs.length} candidate course(s)`
+    }${dryRun ? " (dry-run)" : ""} over ${slugs.length} candidate course(s)${
+      reprojectManifest ? ", re-projecting manifests" : ""
+    }${mirrorDraft ? ", mirroring into the draft" : ""}${
+      emitEvents ? `, run ${runId}` : ""
+    }`
   )
 
   let processed = 0
   let generated = 0
   let failed = 0
+  let mirrored = 0
+  const events: Record<BackfillEventOutcome, number> = {
+    skipped: 0,
+    delivered: 0,
+    failed: 0,
+  }
 
   for (const slug of slugs) {
     if (!singleSlug && processed >= limit) {
@@ -185,9 +308,50 @@ async function main(): Promise<void> {
           result.failed > 0 ? `, ${result.failed} failed` : ""
         }${result.missing > 0 ? `, ${result.missing} unanswered` : ""}`
       )
+
+      if (draftStorage) {
+        // Before the event, so a course is fully settled by the time it is
+        // announced. A failure here does NOT change the event status: the
+        // published package and its manifest are already correct, and only the
+        // draft lags behind — the next publication regenerates what is missing.
+        try {
+          const published = await storage.readSyllabus(slug)
+          const mirror = await mirrorDescriptionsToDraft(
+            draftStorage,
+            slug,
+            published,
+            { dryRun }
+          )
+          mirrored += mirror.copied
+          console.log(
+            `[s3-backfill] "${slug}": ${mirror.copied} mirrored, ${mirror.missed} missed, ${mirror.fresh} already fresh`
+          )
+        } catch (error) {
+          console.error(
+            `[s3-backfill] Could not mirror "${slug}" into the draft:`,
+            (error as Error).message
+          )
+        }
+      }
+
+      if (emitEvents) {
+        const outcome = await emitBackfillManifestEvent({
+          courseSlug: slug,
+          runId,
+          result,
+          storage,
+          rigobotToken: token,
+          breathecodeToken: breathecodeToken as string,
+        })
+
+        events[outcome] += 1
+        console.log(`[s3-backfill] "${slug}": manifest event ${outcome}`)
+      }
     } catch (error) {
       processed += 1
       failed += 1
+      // No event here on purpose: unlike a publication, a backfill never
+      // announced one, so nothing is owed. The course is simply re-run.
       console.error(
         `[s3-backfill] Failed processing "${slug}":`,
         (error as Error).message
@@ -196,7 +360,11 @@ async function main(): Promise<void> {
   }
 
   console.log(
-    `[s3-backfill] Done. ${processed} course(s), ${generated} description(s) written, ${failed} failure(s).`
+    `[s3-backfill] Done. ${processed} course(s), ${generated} description(s) written, ${failed} failure(s).` +
+      (mirrorDraft ? ` ${mirrored} mirrored into the draft.` : "") +
+      (emitEvents ?
+        ` Events: ${events.delivered} delivered, ${events.failed} failed (run ${runId}).` :
+        "")
   )
 }
 

package/src/utils/awsCredentials.ts

@@ -0,0 +1,57 @@
+/**
+ * AWS credentials for the published bucket, required rather than implicit.
+ *
+ * Building an `S3Client` with no credentials succeeds: the SDK defers to its
+ * provider chain and only fails at the first request, with
+ * "Could not load credentials from any providers". Inside a background job that
+ * surfaces minutes later, detached from the cause. Validating up front turns it
+ * into an obvious misconfiguration.
+ *
+ * Note this deliberately rejects the empty-string fallback pattern
+ * (`process.env.X || ""`): passing empty credentials defeats the provider chain
+ * instead of failing, which is how the opaque error appeared in the first place.
+ */
+
+export type AwsCredentials = {
+  accessKeyId: string;
+  secretAccessKey: string;
+};
+
+export function requireAwsCredentials(): AwsCredentials {
+  const accessKeyId = (process.env.AWS_ACCESS_KEY_ID || "").trim()
+  const secretAccessKey = (process.env.AWS_SECRET_ACCESS_KEY || "").trim()
+
+  const missing: string[] = []
+  if (!accessKeyId) {
+    missing.push("AWS_ACCESS_KEY_ID")
+  }
+
+  if (!secretAccessKey) {
+    missing.push("AWS_SECRET_ACCESS_KEY")
+  }
+
+  if (missing.length > 0) {
+    throw new Error(
+      `${missing.join(" and ")} (env) ${
+        missing.length > 1 ? "are" : "is"
+      } required to reach the published package bucket`
+    )
+  }
+
+  return { accessKeyId, secretAccessKey }
+}
+
+export function requireS3PackagesBucket(): string {
+  const bucket = (process.env.S3_PACKAGES_BUCKET || "").trim()
+  if (!bucket) {
+    throw new Error(
+      "S3_PACKAGES_BUCKET (env) is required: it names the bucket holding published packages"
+    )
+  }
+
+  return bucket
+}
+
+export function awsRegion(): string {
+  return (process.env.AWS_REGION || "").trim() || "us-east-1"
+}