@heroiclands/package-build 22.0.2 → 22.1.0

package/CHANGELOG.md

@@ -1,5 +1,84 @@
 # @heroiclands/package-build
 
+## 22.1.0
+
+### Minor Changes
+
+- b0ec919: **The site build generates the Hugo configuration.** `content-build site`
+  writes the whole Hugo source tree under `build/hugo/` — a generated
+  `hugo.toml`, the content mount at `build/hugo/content/`, Hugo's cache —
+  beside the deployment root `build/site/`, and the repository carries no Hugo
+  configuration of its own. Every value in the generated file has one source:
+  `package.json` (`homepage`, `description`, `author`),
+  `package-build.config.yaml` (`packageBuild.manifest.title`, `site.assets`,
+  `site.list`, `site.notfound`), the installed `@heroiclands/hugo-theme`, and
+  the organisation's brand links and navigation. `package.json`'s `homepage`
+  is checked on every site build: it must be
+  `https://www.heroiclands.org/<contentPackage>/`.
+  
+  **`deps fetch` also fetches the navigation.** The header menu every site
+  renders comes from `https://www.heroiclands.org/nav.json`, cached under
+  `build/cache/navigation/` by `content-build deps fetch` and read from the
+  cache by the site build — a cold cache fails naming `deps fetch`, so a
+  package with no other dependency now runs `deps fetch` before `site` too.
+  
+  **To migrate a repository:**
+  
+  - Delete `site/` and its `.gitignore` entries (`site/content`, `site/public`,
+    `site/resources`), and any `packageBuild.clean.extra` entry naming them.
+  - Delete `site.out` from `package-build.config.yaml`; the location is fixed.
+  - Move the `[params.notfound]` block from the old `hugo.toml` to
+    `site.notfound` (`tagline`, `sitenoun`, optional `heroimage` and
+    `links[]` of `{title, url, text}`), `params.list.shortcodes` to
+    `site.list.shortcodes`, and `params.cdnBaseURL` to `site.assets` where it
+    was not already there. Anything else the file carried that the generator
+    does not write — `markup.goldmark.extensions.linkify`, say — goes under
+    `site.hugo`, which is deep-merged last; every key the generator writes is
+    refused there, naming its source.
+  - Point the npm scripts at the generated tree:
+    `hugo --source build/hugo --minify --gc --cleanDestinationDir` and
+    `hugo server --source build/hugo`.
+  - Make sure `package.json` declares `homepage`, `description` and `author`,
+    and that `@heroiclands/hugo-theme` is under `devDependencies`.
+- a5e83fd: **A manifest's `description` comes from `package.json`.**
+  `packageBuild.manifest.description` is refused, the same way `id` and
+  `version` are — delete it; the manifest carries `package.json`'s own
+  `description`.
+  
+  **`package.json`'s `homepage` and `author` are read and normalised.**
+  `homepage` names the address every package's site is served at — an
+  absolute URL ending `/<contentPackage>/` — and `author` is accepted in
+  either of npm's forms, a string or `{name, email, url}`, and normalised to
+  the object form.
+
+## 22.0.3
+
+### Patch Changes
+
+- fd58dc0: **A package may ship assets and compile nothing.** `packs: []` is now a package
+  saying it has no documents, rather than one that forgot to say which, and a
+  package with no packs needs no content tree — its index is its asset records.
+  An alternative-art module is the case: the same addresses another package
+  publishes, resolving to different files when it is installed.
+  
+  A package that _does_ declare packs is unchanged: a missing content tree is
+  still a misconfigured path and still fails, and an index that would state a
+  package has no content at all is still refused.
+- cfdf061: **Two pieces of per-repository tooling move here.** Every site-publishing
+  package held its own copy of both, and a copy per consumer is a copy free to
+  drift — which is what happened, invisibly, because nobody reads all of them at
+  once.
+  
+  - `packageBuild.assetTransform: svg-theme` names a shipped transform, so an
+    icon follows the reader's colour scheme without a module in the repository.
+    The value still takes a path, so a consumer with a transform of its own is
+    unaffected.
+  - `package-build site-root` writes a deployment's `_headers` and `_redirects`:
+    indexing suppressed on every host-assigned address, and both spellings of the
+    prefix root redirected to the landing with a lifetime on the 301. The package
+    name comes from `contentPackage`, which was the only thing the copies actually
+    varied.
+
 ## 22.0.2
 
 ### Patch Changes

package/CONTENT.md

@@ -234,8 +234,9 @@ publish:
   site: content
   address: { prefix: guide/ }
 
-site:
-  out: site/content
+packageBuild:
+  manifest:
+    title: The Toolkit
 
 pdf:
   title: The Toolkit
@@ -2260,22 +2261,22 @@ address anyone links to. A package publishes its content index regardless — th
 licensing constraint is against publishing _pages_, not against the artifact
 existing — but nothing may declare it as a dependency.
 
-The homepage's file is written at the root of `site.out` — the package's own
-site root, one level above the content mount, which is where
-`publish.address.prefix` puts everything else — under the name its address gives
-it, `homepage-root.md`. As with every other page, the front matter's `url`
+The homepage's file is written at the root of the content mount,
+`build/hugo/content/` — the package's own site root, one level above the
+content mount, which is where `publish.address.prefix` puts everything else —
+under the name its address gives it, `homepage-root.md`. As with every other page, the front matter's `url`
 decides where it publishes, and states it relative to the site root — `site.base`
 does not reach it.
 
 **What it does not do is decide addresses.** Those come from `publish.address`,
 the same setting the content index reads, so a page and its index record cannot
 disagree about where the page is. Everything under `site:` is _framing_ —
-where the tree is written, what a section is called, which extra trees are
-published beside the content:
+what a section is called, which extra trees are published beside the content,
+and the residue of the generated Hugo configuration that is this repository's
+own:
 
 ```yaml
 site:
-  out: kb/content # required; wiped on every run
   base: /sohl/ # default: /<contentPackage>/ — hrefs only, never a page's `url:`
   packages: [sohl, thalorna] # default: just contentPackage
   backfillSections: true
@@ -2294,11 +2295,16 @@ site:
       description: Folk, animals and the things that walk the world.
   readmeSections:
     dev-docs: { title: Developer Documentation, banner: banners/dev-docs.webp }
+  list: { shortcodes: true }
+  notfound:
+    tagline: Song of Heroic Lands has no page at
+    sitenoun: site
+    links:
+      - { title: Knowledgebase, url: kb/, text: Every section, with the full content catalog. }
 ```
 
 | Key                | What it decides                                                                                                                                                                                                                                       |
 | ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| `out`              | The Hugo content root. **Required** in both modes, and wiped on every run — see below.                                                                                                                                                                |
 | `base`             | Where the package is served: the prefix on every rendered `href`, and what a manifest `path` is measured against. It reaches no page's own `url:` — see [A page's URL is its address](#a-pages-url-is-its-address). Defaults to `/<contentPackage>/`. |
 | `packages`         | Which content packages this site renders. Defaults to its own.                                                                                                                                                                                        |
 | `sections`         | The Hugo sections this site declares, and what each says about itself — see below.                                                                                                                                                                    |
@@ -2308,6 +2314,17 @@ site:
 | `trees`            | Extra source trees published beside the content, preserving their source layout below a section.                                                                                                                                                      |
 | `pass`             | A named bundle of this repository's own body rewrites.                                                                                                                                                                                                |
 | `passOptions`      | That bundle's options.                                                                                                                                                                                                                                |
+| `assets`           | The host every package's imagery is served from; the generated `params.cdnBaseURL`.                                                                                                                                                                   |
+| `list`             | How a listing page renders; the generated `params.list`.                                                                                                                                                                                              |
+| `notfound`         | The wording of the "page not found" page; the generated `params.notfound`.                                                                                                                                                                            |
+| `hugo`             | A mapping deep-merged over the generated Hugo configuration, last. Every key the generator writes is refused here — see `docs/configuration.md`.                                                                                                      |
+
+Where the tree is written is not among them. `content-build site` writes the
+whole Hugo source tree under `build/hugo/` — the generated `hugo.toml`, the
+content mount at `build/hugo/content/`, Hugo's cache — and the consumer's
+script runs `hugo --source build/hugo` over it. The generated file's every
+value has a source the repository already states; `docs/configuration.md`
+lists them.
 
 ### What a section may declare
 
@@ -2409,15 +2426,14 @@ The **writers** name no keys: a section's `_index.md` is whatever the entry
 resolved to, `title` first. So extending the vocabulary is a change to the
 schema alone, and the two can no longer drift apart.
 
-### Why `out` is required
+### Why the output location is fixed
 
-The output tree is a build artifact and is **deleted on every run**, so that a
-page whose note was renamed cannot linger and keep publishing. An unset `out`
-resolves to the repository root, and the wipe then deletes the working tree.
-That is not hypothetical — it happened while this command was being written, on
-a configuration that simply had no `site` section yet. So `out` is refused when
-unset, and refused again when it resolves anywhere that is not strictly inside
-the repository root.
+The content mount is a build artifact and is **deleted on every run**, so that a
+page whose note was renamed cannot linger and keep publishing. A configurable
+location is a location that can be unset — and an unset one resolves to the
+repository root, where the wipe deletes the working tree. A fixed one under
+`build/` can point nowhere else, so `site.out` is refused by name and the wipe
+needs no guard.
 
 ### Consumer passes are named, not imported
 

package/README.md

@@ -95,7 +95,7 @@ packageBuild:
 
   clean:
     # Beyond the conventional build artifacts, which the library already knows.
-    extra: [site/content, site/public, site/resources]
+    extra: [coverage]
 
   lang:
     sources: lang/*.json

package/bin/content-build.mjs

@@ -79,6 +79,7 @@ import {
     unaddressableForeignPackages,
     formatUnaddressableFinding,
 } from "../engine/metadata-index.mjs";
+import { fetchNavigation, generateHugoConfig, writeHugoConfig } from "../engine/site-config.mjs";
 import { renderItemFieldReference } from "../engine/field-reference.mjs";
 import { lintContentTree } from "../engine/content-lint.mjs";
 import { lintContentCharset } from "../engine/content-charset.mjs";
@@ -1484,7 +1485,14 @@ function pdfCommand() {
  * compiled into packs. Everything a consumer would otherwise write for itself —
  * the walk, the address derivation, the address index, table expansion,
  * wikilink resolution, code-fence protection, the foreign-manifest merge and
- * the section-landing backfill — happens here, from configuration.
+ * the section-landing backfill — happens here, from configuration. So does the
+ * Hugo configuration: the whole source tree Hugo reads lands under
+ * `build/hugo/`, and the consumer's script runs Hugo over it.
+ *
+ * **The Hugo configuration is generated before anything is written.** Its
+ * sources — `package.json`'s `homepage`, the cached navigation, the installed
+ * theme — are each a way the build can fail, and failing before the output
+ * tree is cleared leaves the last good site in place to be looked at.
  *
  * **Each gate is reported and the run stops at the first that fires.** They are
  * ordered so the report names the cause rather than its symptoms: an unusable
@@ -1497,20 +1505,17 @@ function pdfCommand() {
 function siteCommand() {
     return {
         command: "site",
-        describe: "Build a Hugo content tree from the content tree",
-        builder: (yargs) => {
-            yargs.option("out", {
-                describe: "Write the mount here instead of the configured `site.out`.",
-                type: "string",
-            });
-        },
-        handler: async (argv) => {
+        describe: "Build the Hugo source tree from the content tree",
+        builder: () => {},
+        handler: async () => {
             try {
+                const config = loadPackConfig();
+                const hugo = generateHugoConfig(config);
                 const result = buildSite({
-                    sqlTables: await prepareTreeSqlTables(loadPackConfig().paths.content, {
-                        skipDirectories: loadPackConfig().skipDirectories,
+                    config,
+                    sqlTables: await prepareTreeSqlTables(config.paths.content, {
+                        skipDirectories: config.skipDirectories,
                     }),
-                    ...(argv.out ? { outRoot: argv.out } : {}),
                 });
                 const { gates } = result;
 
@@ -1623,6 +1628,8 @@ function siteCommand() {
                         `${s.tree ?? 0} tree page(s) + ${s.landings} ` +
                         `landing(s) to ${path.relative(process.cwd(), s.out)}`,
                 );
+                const { file } = writeHugoConfig(config, hugo);
+                log.info(`wrote ${path.relative(process.cwd(), file)}`);
             } catch (err) {
                 reportFailure(err);
                 process.exitCode = 1;
@@ -1782,16 +1789,19 @@ async function fetchFromLocalArtifact(config, argv) {
 
 /**
  * `deps fetch` — fill the caches this build resolves other packages through:
- * the **content index** of every declared dependency, and the **item
- * catalogue** of those additionally declaring `itemCatalog: true`.
- *
- * The two sets differ deliberately. Citing another package's *addresses* and
- * embedding its *items* are separate edges, and a package may have either
- * without the other — `harn-ensemble` cites no foreign address and embeds
- * 324,016 item references.
- *
- * Its own command rather than a step of `package compile`, so that a compile
- * never reaches the network. A build that downloads silently is not
+ * the **content index** of every declared dependency, the **item catalogue**
+ * of those additionally declaring `itemCatalog: true`, and the **site
+ * navigation** heroiclands.org publishes, which the site build writes its
+ * menu from.
+ *
+ * The two dependency sets differ deliberately. Citing another package's
+ * *addresses* and embedding its *items* are separate edges, and a package may
+ * have either without the other — `harn-ensemble` cites no foreign address and
+ * embeds 324,016 item references. The navigation is fetched for every package,
+ * because every package publishes a site.
+ *
+ * Its own command rather than a step of `package compile` or `site`, so that
+ * neither reaches the network. A build that downloads silently is not
  * reproducible, breaks offline, and hides a dependency's version change behind
  * a passing run.
  *
@@ -1838,6 +1848,14 @@ function depsCommand() {
                     await fetchFromLocalArtifact(config, argv);
                     return;
                 }
+                // First, because every package needs it and it depends on
+                // nothing a repository declares: a dependency whose release
+                // cannot be read stops the run after the navigation is cached,
+                // not before.
+                const navigation = await fetchNavigation(config);
+                log.info(
+                    `Fetched the site navigation to ${path.relative(process.cwd(), navigation)}.`,
+                );
                 const indexes = await fetchAllMetadata(config);
                 if (indexes) log.info(`Fetched ${indexes} dependency content index(es).`);
                 const count = await fetchAllCatalogs(config);

package/bin/package-build.mjs

@@ -78,6 +78,8 @@ import yargs from "yargs";
 import { hideBin } from "yargs/helpers";
 
 import { loadPackageBuildConfig } from "../config.mjs";
+import { writeSiteRoot } from "../engine/site-root.mjs";
+import { DEPLOY_ROOT } from "../engine/site-config.mjs";
 import { compilesFoundryDocuments } from "../content-config.mjs";
 import { loadPackConfig, packConfigPath } from "../engine/pack-config.mjs";
 import { cleanBuildArtifacts, stageAssets } from "../stage.mjs";
@@ -466,6 +468,37 @@ function manifestCommand() {
     };
 }
 
+/**
+ * `package-build site-root` — the deployment's root files.
+ *
+ * Hugo owns everything under the `/<package>/` prefix; this owns what sits
+ * beside it, which is the pair Cloudflare Pages reads from the uploaded
+ * directory and nowhere else.
+ *
+ * @returns {object} The yargs command.
+ */
+function siteRootCommand() {
+    return {
+        command: "site-root",
+        describe: "Write the deployment's _headers and _redirects",
+        builder: (y) =>
+            y.option("out", {
+                type: "string",
+                describe: `The directory that is deployed (default: ${DEPLOY_ROOT})`,
+            }),
+        handler: handler(async (argv) => {
+            const config = loadPackageBuildConfig();
+            const shared = loadPackConfig();
+            const out = path.resolve(config.rootDir, argv.out ?? DEPLOY_ROOT);
+
+            const { files } = writeSiteRoot({ pkg: shared.contentPackage, out });
+            for (const file of files) {
+                console.log(`✅ Wrote ${path.relative(config.rootDir, file)}.`);
+            }
+        }),
+    };
+}
+
 /**
  * Every file matching a glob, as `{ path, text }` with the path relative to the
  * repository root — the form both the rules and the findings want.
@@ -1214,6 +1247,7 @@ yargs(hideBin(process.argv))
     .command(cleanCommand())
     .command(assetsCommand())
     .command(manifestCommand())
+    .command(siteRootCommand())
     .command(schemaCommand())
     .command(langCommand())
     .command(labelsCommand())

package/config.mjs

@@ -46,7 +46,7 @@
  *     assetTransform: ./utils/svg-theme.mjs
  *     stageDir: build/stage
  *     clean:
- *         extra: [site/content, site/public]
+ *         extra: [coverage]
  *     lang:
  *         sources: lang/*.json
  *     deploy:
@@ -57,6 +57,7 @@
  */
 
 import path from "node:path";
+import { fileURLToPath } from "node:url";
 import { loadPackConfig, locateConfigError, packConfigPath } from "./engine/pack-config.mjs";
 
 /** Keys the reserved section may declare. */
@@ -90,6 +91,7 @@ const SECTION_KEYS = [
 export const DERIVED_MANIFEST_KEYS = Object.freeze({
     id: "`foundryPackage`, itself derived from package.json `name`",
     version: "package.json `version`",
+    description: "package.json `description`",
     url: "package.json `repository`",
     bugs: "package.json `repository`",
     manifest: "package.json `repository` and the release tag",
@@ -565,6 +567,89 @@ function normalizeExceptions(value, field, where) {
  *   collections, as collection → source directory.
  */
 
+/**
+ * The asset transforms this package ships, by the name a consumer writes.
+ *
+ * `packageBuild.assetTransform` takes either one of these names or a path to a
+ * module of the consumer's own. A name is the answer where every package wants
+ * the same behaviour — theming an icon to the reader's colour scheme is not a
+ * per-package decision, and a copy in each consumer is a copy that drifts.
+ *
+ * @type {Readonly<Record<string, string>>}
+ */
+export const BUILT_IN_ASSET_TRANSFORMS = Object.freeze({
+    "svg-theme": "./engine/svg-theme.mjs",
+});
+
+/**
+ * Where a declared `assetTransform` is loaded from.
+ *
+ * A built-in name resolves to the module this package ships. Anything else is a
+ * path, resolved against the repository root the way it always was — so a
+ * consumer with a transform of its own is unaffected.
+ *
+ * @param {string} declared - The authored value.
+ * @param {string} rootDir - The repository root.
+ * @returns {string} An absolute path to import.
+ */
+export function resolveAssetTransform(declared, rootDir) {
+    const builtIn = BUILT_IN_ASSET_TRANSFORMS[declared];
+    if (builtIn) {
+        return path.resolve(path.dirname(fileURLToPath(import.meta.url)), builtIn);
+    }
+    return path.resolve(rootDir, declared);
+}
+
+/**
+ * `package.json`'s `homepage`, checked against the address it must be.
+ *
+ * A package's Foundry manifest already derives its own `url` from
+ * `contentPackage` (`packageHomepage` in `manifest.mjs`); `homepage` states
+ * the same address a second time, in `package.json`, for the generated Hugo
+ * configuration to read a `baseURL` from without knowing where each
+ * repository keeps its own site configuration.
+ *
+ * Required unconditionally: every package publishes a site, so there is no
+ * package this does not apply to.
+ *
+ * **Not called by {@link resolvePackageBuildConfig}.** Every packaging
+ * command — `clean`, `deploy`, `manifest` and the rest — resolves through it,
+ * and none of them reads `homepage`: the Foundry manifest's own `url` is
+ * `packageHomepage(contentPackage)`, independent of it. The right caller is
+ * whatever reads `homepage` to write a site's `baseURL`.
+ *
+ * @param {string|null} homepage - The resolved `package.json` `homepage`, or
+ *   `null` when none is declared.
+ * @param {string} contentPackage - The resolved `contentPackage`.
+ * @returns {void}
+ */
+export function checkHomepage(homepage, contentPackage) {
+    if (homepage === null) {
+        fail(
+            "homepage",
+            "is not declared in `package.json`, and every package needs one " +
+                "to build its site's `baseURL` from. Add " +
+                `\`https://www.heroiclands.org/${contentPackage}/\``,
+        );
+    }
+
+    const suffix = `/${contentPackage}/`;
+    let pathname;
+    try {
+        pathname = new URL(homepage).pathname;
+    } catch {
+        fail("homepage", `is \`${homepage}\`, which is not an absolute URL`);
+    }
+    if (!pathname.endsWith(suffix)) {
+        fail(
+            "homepage",
+            `is \`${homepage}\`, but \`contentPackage\` is \`${contentPackage}\` — ` +
+                `a package's site is served at \`https://www.heroiclands.org${suffix}\`, ` +
+                `so \`package.json\`'s \`homepage\` must end \`${suffix}\``,
+        );
+    }
+}
+
 /**
  * Resolve a package-build configuration from an already-loaded shared one.
  *
@@ -703,9 +788,9 @@ export function resolvePackageBuildConfig(shared) {
         assetTransform:
             section.assetTransform === undefined ?
                 null
-            :   path.resolve(
-                    shared.rootDir,
+            :   resolveAssetTransform(
                     requireNonEmptyString(section.assetTransform, "packageBuild.assetTransform"),
+                    shared.rootDir,
                 ),
         manifest: normalizeManifest(section.manifest),
         manifestFlags: