relaton-cli 3.0.0.pre.alpha.1 → 3.0.0.pre.alpha.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a47c862db04dc7abfb595cf5bb150585dc09c6100047727ab33cf5c581c38dba
-  data.tar.gz: 2cb6efe9f2cc054b1825fed5a0c85b4988e74d93ecd7fdcb962af4efa6cd8eb9
+  metadata.gz: 40a33fcc5e9cb123ea61e9d73105524c896a8e79601431980b52589edb6123fd
+  data.tar.gz: 72cee8178b5ea4ba8a628f77ed04a9efa05f7d1890d6c0028a72d18eeb8939b6
 SHA512:
-  metadata.gz: 5433f5f5b96cd5e19e7fb17ee18012b275b6fe1468f38a61e4d6769b7d7b053dedc0d33925244660d8b8051197790a5f72805f0b84c4bc86ff0813e609841b21
-  data.tar.gz: 835cca13a812bf83da85d7251da06030da334bed0aa137ffc5d33b15d3482a12e1090c72585f85dfc6d89af8e1c7973f716106cbe07fb3d3fc3dc408b737ba28
+  metadata.gz: 046773f651afaf06a182b5217c8054b32bc2d34f4d4aedc68283741791347fc41b29603cad1d45fae318c0a961a7bc72cd3242b93785241c3cb56ead3436d0aa
+  data.tar.gz: 4fe86cad3ad78c8f317c26686f20bcb75780cac66ecac56cd0547a31b0aa3084a8e33d6fb129d9f9f1ddb47708bc27ed8962792addd900ca2eac6b194bd499f3

data/.gitignore

@@ -16,3 +16,7 @@
 Gemfile.lock
 .DS_Store
 .claude/
+
+# Frontend: source under frontend/src is committed; build output and deps are not.
+/frontend/node_modules/
+/frontend/dist/

data/CLAUDE.md

@@ -49,6 +49,32 @@ The executable `exe/relaton` calls `Relaton::Cli.start(ARGV)` which routes to `R
 
 Current fetch options that use this pattern: `--no-cache`, `--all-parts`, `--keep-year`, `--publication-date-before`, `--publication-date-after`.
 
+### Malformed-identifier handling (CLI is the friendly layer)
+
+The `relaton` library **raises** `Pubid::Errors::ParseError` when a reference
+can't be parsed — it does not swallow it, so API consumers of `Relaton::Db`
+handle it themselves. Every flavor raises that class, including ITU, whose own
+Parslet grammar (`Relaton::Itu::Pubid`) re-raises its failure as one. The
+**CLI** is the single place that turns it into a user-facing message:
+`fetch_document` rescues the marker module `Pubid::Errors::Error` — which also
+covers `Pubid::Errors::InvalidInputError` (input too long) — (alongside
+`Relaton::RequestError`) and returns `"<code>" is not a recognized
+standards identifier`, and `SubcommandCollection#fetch` — which bypasses
+`fetch_document` — rescues it too, emitting the same message via `Util.error`.
+The paths that actually reach pubid's parser are `relaton fetch` (→
+`Db#fetch`/`#fetch_std` → `processor.get`) and `relaton collection fetch` (→
+`Db#fetch`). `relaton db fetch` is a **cache-only** lookup (`fetch_db: true`
+short-circuits `check_bibliocache` to a cache-key read via `std_id`, which is
+plain string manipulation, never `processor.get`), so it never parses and can't
+raise a parse error — a malformed id there simply misses the cache and
+prints "No matching bibliographic entry found". `command.rb` and
+`subcommand_collection.rb` therefore `require "pubid"` at load time so the
+constant resolves even when an unrelated exception (e.g. an `ArgumentError` from
+`parse_date_option`) is the one propagating through the rescue chain:
+`require "relaton"` loads pubid lazily, so it may not be loaded yet. `Pubid::Errors`
+first shipped in pubid `2.0.0.pre.alpha.11`, which is why `relaton.gemspec`
+pins `~> 2.0.0.pre.alpha.11`.
+
 ### Core Data Classes
 
 - `lib/relaton/bibdata.rb` — `Relaton::Bibdata` wraps `RelatonBib::BibliographicItem`, adding URL type handling and serialization to XML/YAML/Hash. Uses `method_missing` to delegate to the underlying bibitem.
@@ -62,6 +88,383 @@ Current fetch options that use this pattern: `--no-cache`, `--all-parts`, `--kee
 - `lib/relaton/cli/yaml_convertor.rb` — YAML → XML conversion (includes processor detection via doctype)
 - `lib/relaton/cli/xml_to_html_renderer.rb` — Renders XML/YAML to HTML using Liquid templates from `templates/`
 
+### Data index site generator (`relaton index`)
+
+`relaton index [DATA-DIR]` builds the modern, browsable GitHub Pages index for
+`relaton-data-<flavor>` repos (e.g. https://relaton.github.io/relaton-data-bipm/),
+replacing the shared Jekyll theme (`relaton/jekyll-theme-relaton-data-index`) +
+`relaton/support` `data-deploy.yml` build. Pieces:
+
+**One delivery mode, sharded.** The shell carries **no document data** — only
+branding, the inlined bundle, and five `data-*` scalars describing the shard
+layout — so page weight is a function of what the reader opens, not of corpus
+size. Everything else is fetched:
+
+```
+_site/index.html          ~110 KB regardless of corpus size
+_site/search-0000.json …  summary records {r,c,t,s,d,u,l}, `--shard-size` each
+_site/detail-0000.json …  the rich fields, `--detail-shard-size` each
+```
+
+Two scope decisions are baked in here; both were deliberate, and re-litigating
+either without reading this will lead you back to a 150 MB page:
+
+1. **Crawler-indexability was dropped as a requirement.** The old `embedded`/`dom`
+   modes rendered one `.document` row per document precisely so crawlers could
+   read the corpus; that is what made the page O(corpus). relaton/relaton#48
+   listed it as an acceptance criterion, and it was given up knowingly — these
+   are machine-data repos whose consumers are relaton fetchers, and the Pages
+   site is a "what data is available" browser reached from the repo. Restoring it
+   means paginated `page/N/index.html` shells, not un-sharding the data.
+2. **Therefore the three `--mode`s collapsed to one.** `dom` existed *only* to be
+   crawlable, and as a data channel `data-*` attributes are strictly worse than
+   JSON (bigger, and lossy — 7 fields, no detail). `embedded`'s remaining
+   advantage was `file://` self-containment, which a fetched sidecar had already
+   given up. `static-json` was just "sharded, with one shard". `--mode` is
+   **removed**, not deprecated; see the note at the end of this section.
+
+- `lib/relaton/cli/index_site_generator.rb` — scans `DATA-DIR/**/*.{yaml,yml}`
+  (default `./data`, skips `index-v*.yaml`) **plus an auto-detected sibling
+  `static/` folder** (`STATIC_DIRNAME`, a sibling of `DATA-DIR` = `repo_root/static`;
+  `--no-static`/`static: false` opts out), normalizes each doc, and writes the
+  shell plus the shards into `--output` (default `_site`).
+  Both source dirs go through the same `SKIP_BASENAMES`/`document?` filters. De-dup
+  is **cross-folder only**: `data/` is scanned first, so a `static/` doc whose
+  normalized id already appeared in `data/` is skipped with a warning (data wins);
+  duplicates *within* a single folder are left as-is (preserving the pre-static
+  data-scan behavior), and a blank/empty normalized id never de-dups (so docid-less
+  title-only docs aren't collapsed). Static docs get correct `static/<path>`
+  raw-YAML links for free because `relative_path` is taken from `repo_root` (the
+  parent of both `data/` and `static/`). `static/` holds manually-curated bib
+  records the crawler can't fetch (ISO/IEC Directives, JCGM/GUM guides, NIST
+  research-library metadata) that are already part of the corpus (in `index-vN.yaml`).
+  **`each_document` is the streaming spine**: it yields normalized items instead
+  of accumulating them, and `ShardWriter` buffers at most one shard of each
+  family, so for the **search/detail** shard families peak RSS is O(shard size) —
+  measured at ~38 MB above baseline for a 20,000-document, 79 MB corpus. Nothing
+  may reintroduce a `collect_documents` that materializes the corpus for those.
+  The spec that guards this observes the property rather than asserting a method
+  is never called (that passes just as happily once the method is renamed): it
+  pins that a completed shard is on disk while the corpus is still being read.
+  **`MachineIndex` is the deliberate exception to the O(shard size) bound.**
+  Shard assignment needs the corpus size, so it is not knowable until the pass
+  ends — the class therefore retains one
+  `Row` per document. That is O(corpus), and fine: ~0.44 KB/row, i.e. ~76 MB at
+  177k rows. What it must NOT do is retain the pubid object, which costs
+  3.14 KB/row — 543 MB on the same corpus, past the ~250 MB where the process
+  was observed to hang. `Row` holds only `rendered`/`file`/`key`/`id_hash`, all
+  derived inside `add`; anything else needed from pubid must likewise be
+  computed there, because by write time the identifier is gone.
+
+**The machine-index wire contract.** `relaton index` also publishes a
+machine-consumable index — `index-vN.yaml`/`.zip`, `index/manifest.json` and
+`index/shard-NNNNN.json`. It is specified in `docs/data-repository-format.adoc`
+in the `relaton` repo; that file is the contract, this is why it is shaped that
+way. Four decisions, all settled deliberately — don't re-derive them:
+
+- **The name comes from the flavor's own `INDEXFILE`, never from the data.**
+  `index-vN` is *per-flavor structure versioning*, not a global generation
+  counter: 25 flavors read `index-v2`, IHO reads `index-v3`, JCGM reads
+  `index-v1` — and JCGM's v1 is structured, so the number is a name, not a
+  claim about the rows. An earlier revision derived it from a parse ratio
+  (`structured? ? "v3" : "v1"`), which both re-labelled v2 as v3 and let a crawl
+  that shifted the ratio by a few percent *rename the published file*, breaking
+  every pinned consumer with nothing but a 404. `--index-name` overrides, for a
+  corpus that is not a relaton flavor. Resolution is `Relaton::<Flavor>::INDEXFILE`
+  via `FLAVOR_NAMESPACES`, whose only entry is 3GPP (`Pubid::Tgpp` against
+  `Relaton::ThreeGpp`); every other flavor capitalizes identically in both gems.
+- **The shard key is `crc32(pubid.root.number.to_s) % N`, with no fallback.**
+  That expression is the one `Relaton::Index` sorts and bsearches on (`Type#
+  candidates_by_number`, `FileIO#deserialize_pubid`), and the identity is the
+  whole point: a client computing the key from its parsed query must land in the
+  bucket the rows are in. A rendered-id fallback for an empty root number
+  (added once to avoid a hot shard 0) breaks that — the client looks in shard 0,
+  the row is elsewhere, and the miss is indistinguishable from not-found because
+  empty shards are omitted. An identifier with no root number keys on `""` and
+  goes in shard 0, the same degeneracy the gem's bsearch already has.
+- **Every published row is structured, and the shards and the monolith agree.**
+  `row_record` and `write_monolith` both carry the pubid hash for every row.
+  They cannot differ: `FileIO#deserialize_id` calls `from_hash` on every row and
+  rejects the *whole* index on the first one it cannot deserialize, so one
+  plain-string row in the monolith would poison the file. That is also why
+  `--pubid-flavor` is **required** unless `--no-machine-index` — a machine index
+  with no parser could only be the string shape.
+- **An id the parser rejects is taken from the repo's committed index.** The
+  generator derives ids by parsing each document's *rendered* docid, while the
+  committed `index-vN.yaml` was built by the flavor's `DataFetcher` from source
+  metadata; five shipping corpora disagree on a handful of rows (ieee 69,
+  itu-r 47, iec 42, itu 3, nist 3). `committed_index` reads that file once and
+  `add` uses its hash **verbatim**, so the site's row is byte-identical to the
+  repo's. Only a row neither path resolves is dropped, counted in
+  `skipped_count`, and warned about.
+
+**The monolith's hand-rolled YAML writer must preserve type, not just text.**
+`write_monolith` avoids Psych (40 min on 177k rows), so `yaml_nested`/`yaml_scalar`
+own the whole round trip. Two shapes broke it, both invisible to specs that only
+build `{_type, number}` hashes: an **Array** fell through to `yaml_scalar`, whose
+`["IEC"].to_s` starts with `[` and was quoted into the String `"[\"IEC\"]"` —
+every ISO/IEC copublished id carries `copublishers`, and `from_hash` cannot cast
+it, so `FileIO` rejects the whole index; and a real **Boolean** was stringified
+then quoted (`d_prefix: "true"`, 31 of CIE's 1139 rows), so the published row no
+longer matched the repo's. Arrays are now written as a JSON flow sequence (valid
+YAML, unambiguous escaping); booleans and integers go out bare. Verify a writer
+change against a real corpus, comparing the site's monolith row-by-row to the
+repo's committed index — that is how both were found.
+
+**`purge_stale!` reads the previous manifest.** `STALE_GLOBS` only knows
+`index-v*`, but `--index-name` accepts any single file name (`INDEX_NAME`; a path
+segment or a leading dot is refused), so the previous build's `index/manifest.json`
+is the only record of which monolith to delete.
+
+**`Util.warn` is not the relaton logger — it is `Kernel#warn`.**
+`Relaton::Cli::Util` gets `info`/`error` from `Relaton::Bib::Util#method_missing`,
+but `warn` already exists as a private `Kernel` instance method, so the explicit
+receiver makes Ruby route it to `method_missing` *after* a private-method
+NoMethodError. The message does reach the logger, but `expect(Relaton::Cli::Util)
+.to receive(:warn)` never intercepts it. Mock `Relaton.logger_pool` instead:
+`expect(Relaton.logger_pool).to receive(:warn).with(/…/, "relaton-cli")`.
+  **`purge_stale!` is load-bearing, not hygiene**: with no manifest, a
+  `search-0034.json` left by a previous larger corpus is invisible (the shard
+  count on the mount node says 34), so it would sit in the deployed site forever.
+  **Per-flavor branding** beyond `--title` is
+  `--description` and `--favicon` (both `nil` by default — the per-flavor values the
+  deleted Jekyll `_config.yml`s carried, now supplied by the caller workflow). The
+  description becomes `<meta name="description">` and `data-description` on the
+  mount node; the favicon becomes `<link rel="icon">`
+  with its href passed through **verbatim** and a `type` sniffed from the extension
+  via `FAVICON_TYPES` (unknown extension → no `type`, the browser sniffs). The
+  generator copies **no assets** (only the shell + the shards), so a relative
+  favicon href is the caller's job to place in the deployed site — normally it's an
+  absolute URL. Both tags are emitted by `{%- if … %}` branches in `page.liquid`, so
+  a build with neither option is byte-identical to one that never had them —
+  `spec/index_fixtures/golden/index.html` pins the whole shell, so regenerate it
+  deliberately and read the diff when you touch that template.
+  `title`/`description`/`favicon` all go
+  through `presence`, which maps a blank string to `nil`: the deploy workflow
+  forwards unset inputs as `--favicon ""`, and an empty `href` would make the
+  browser re-fetch the page as its own icon.
+- `lib/relaton/cli/index_item_normalizer.rb` — doc-hash → `{id,title,doctype,
+  stage,date,link,yaml}` (the always-present summary core) **plus optional
+  detail-page fields** (`abstract, edition, languages, keywords, publisher,
+  contributors, docids, dates, relations`) added by `details`. Reads the doc's own rendered
+  fields (primary `docidentifier`, main/en `title`, `date[].at|on` preferring
+  `published`, `ext.doctype`), so **no pubid reconstruction** is needed. Detail
+  fields are dropped when empty (via `reject`), so a summary-only doc keeps the
+  compact 7-key shape. The generator then splits that one record in two:
+  `compact_record` projects the seven summary keys (renamed `r,c,t,s,d,u,l`) into
+  the search shards, and `detail_record` takes **everything else**, tagged with
+  `r`. The split is defined by one constant, `COMPACT_KEYS` — a key that isn't in
+  it is a detail field by definition, so a new normalizer field lands in the
+  detail shards automatically and never silently bloats the summary. The frontend
+  `types.ts` `IndexDocument` / `DetailRecord` interfaces mirror both halves.
+
+  **`relations` carries only `{type, id}`, and the id must render exactly like a
+  document's own.** The whole point of the field is that the detail page can match
+  the target against the rest of the corpus and turn it into a `?doc=` link, so
+  `relation_id` shares the primary-DocID logic that computes each document's `id`
+  (a relation's nested `bibitem` carries the same `docidentifier`/`primary`
+  shape). Don't "improve" it into a separate id renderer — the two would drift and
+  every link would silently degrade to plain text. It deliberately shares
+  `primary_docidentifier` and **not** `docidentifier` itself: the latter bottoms
+  out at the item's `id`, which on a relation bibitem is an internal XML anchor
+  (`CENISO/TS21003-7-2008/A1-2010` — CEN records carry exactly this). An anchor can
+  never match a corpus document, so the fallback order is docidentifier →
+  `formattedref` → `docnumber` → anchor; putting the anchor any earlier strands a
+  target that *is* present as plain text under a mangled label. No title/date is
+  carried: measured on
+  `relaton-data-iso` (79,402 docs) that is 83,074 relations, of which **99.9%
+  resolve in-dataset**; the remaining 48 are genuinely-absent `ISO/DIS …` drafts.
+  Two guards are load-bearing on real data — **self-references are dropped**
+  (BIPM's Metrologia articles relate to themselves once per carrier) and
+  identical `{type, id}` pairs are **de-duplicated**.
+
+  **Detail lookup is positional, verified by id.** A detail shard is a *dense*
+  array: slot `i` is either `null` (that document has no detail fields) or
+  `{"r": "<id>", …}`. A `null` slot is still written, because the frontend finds a
+  document's detail by arithmetic — `shard = ⌊i / detailShardSize⌋`,
+  `slot = i % detailShardSize`, where `i` is its position in the corpus — and
+  dropping empties would shift everything after it. Nothing stores that mapping,
+  which is why `loadDetail` checks the entry's `r` against the document id and
+  falls back to scanning the shard on mismatch: showing another document's
+  abstract would be worse than showing none. Keep the two families written from
+  the same single pass over `each_document`, or the arithmetic stops holding.
+
+  **Corpus position is carried, never derived from the array index.** The shard
+  loader stamps `pos` onto each document from its shard number
+  (`i * shardSize + j`), and `App.vue` uses `doc.pos` — falling back to `indexOf`
+  only for a document list supplied whole. This is not redundancy. A summary
+  shard that fails to load leaves a *hole*: every later document then sits at an
+  array index `shardSize` lower than its true position, which at the defaults
+  (5000 / 500) addresses a detail shard **ten files early**. The `r` check would
+  reject the wrong record, but its fallback scans only the shard it was sent to,
+  so it can never recover a cross-shard drift — the failure would be silent
+  summary-only detail panels for the entire rest of the corpus. Don't "simplify"
+  `doc.pos` back to `indexOf`.
+- `lib/relaton/cli/frontend_assets.rb` — reads the compiled bundle from
+  `frontend/dist/` (one relative path works for both gem and git checkout);
+  raises `BuildMissingError` with a "run `rake build_frontend`" message if absent.
+  `dist_dir` is overridable so specs point at a fixture dist (no Node build).
+- `templates/index/page.liquid` — the whole output HTML: head/branding, the
+  inlined IIFE + CSS, and the mount node carrying the five shard scalars
+  (`data-total`, `data-shards`, `data-shard-size`, `data-detail-shards`,
+  `data-detail-shard-size`). Those attributes are the **only** description of the
+  index's shape — there is deliberately no manifest file, which saves a
+  round-trip before the first shard can be requested and removes a failure mode
+  where the manifest 404s but the shards are fine. Shard file names are derived by
+  convention on both sides (`search-%04d.json` / `detail-%04d.json`) and resolve
+  relative to the page, which sits beside them. Rendered via a Liquid
+  `Environment` (not the deprecated global `Template.file_system=`).
+  (`_document.liquid` is gone with the crawler DOM.)
+- `--flavor <f>` / `--repo <org/name>` shallow-clone the data repo to a tempdir
+  and read its `data/`; the checked-out branch is auto-detected (bipm's is `v2`)
+  and used for the raw-YAML `--base-url`. Default (no flavor/repo) reads the local
+  folder — the in-Action path where the repo is already checked out.
+
+**Frontend (`frontend/`).** A Vite 8 **library-mode** project
+(`@vitejs/plugin-vue` + `@tailwindcss/vite` + Vue 3) that builds ONE
+`frontend/dist/app.iife.js` + `style.css` (the lutaml-xsd embedding pattern,
+modernized). `src/` is committed; `dist/` + `node_modules/` are gitignored and
+built at package time. The island (`src/App.vue`) does search / doctype+stage
+facets / sort / list-grid / dark-mode / copy-DocID / client pagination.
+
+**Hydration is one synchronous path.** `src/lib/hydrate.ts` `resolveIndex(el)`
+just reads the mount node — branding plus `readShardInfo` — and returns an empty
+document list; there is nothing to await, so the app paints immediately and the
+first shard request starts at boot. `src/lib/shards.ts` then owns all fetching:
+
+- `loadSearchShards(info, onBatch)` — a bounded worker pool (4) over the summary
+  shards. Two properties are load-bearing. **Order**: records are appended in
+  shard order no matter what order responses arrive in, because corpus position is
+  what the detail lookup is keyed to — a shard that resolves early waits for its
+  predecessors. **Throttling**: `App.vue` re-filters and re-sorts the entire array
+  on every change, so emitting per shard would re-sort a growing 166k-element
+  array dozens of times; emissions are coalesced into one per `flushMs` window,
+  with a forced final emit. A shard that 404s becomes `[]` rather than aborting
+  the load, and `res.ok` is checked because a 404 on Pages returns an HTML error
+  page that would otherwise throw inside `res.json()`.
+- `createDetailLoader(info)` — fetch-on-open with a per-shard cache, so paging
+  through neighbouring documents costs one request rather than one per document;
+  concurrent opens in the same shard share a single in-flight promise.
+
+`src/app.ts` holds `reactive()` `IndexData` and `Hydration` objects and mutates
+them as shards land — `createApp(App, props)` props are **static**, so reassigning
+a prop would never reach the component. The shard load is kicked off via
+`requestIdleCallback` so it stays off the first paint.
+The site `description` comes from `data-description` on the mount node; `App.vue`
+renders it as the header subtitle, falling back to the default "Please use the
+provided Relaton DocID…" sentence.
+
+**The loading gates are the subtle part.** While shards are arriving the corpus is
+knowingly incomplete, so anything that reads "absent" as "does not exist" has to
+wait for `hydration.loading` to clear:
+- an unresolved `?doc=` must **not** bounce to the list (that watcher previously
+  treated not-found as removed-document; on a large index every deep link would
+  break) — instead it calls `requestAll()` to hurry the load, and only drops the
+  param once loading finishes;
+- `?page=N` must not be clamped against a `pageCount` computed from a partial
+  corpus — `clampPage()` is deferred to a `watch(loading)`, and `onPopState`
+  skips clamping while loading;
+- the count line reads `hydration.total` (known up front from `data-total`) so it
+  says "12 of 166658", not a denominator that grows.
+The `hydration` and `loadDetail` props are **optional**: mounted without them the
+component behaves exactly as it did before, which is what keeps the pre-existing
+tests meaningful.
+Pagination is **viewport-adaptive**: `computePageSize()`
+sizes a page from the available height and grid/list row density (recomputed on
+resize — debounced — and on view-mode change, clamped `10..100`) rather than a
+fixed 100 rows, and the Prev/Next/page controls live in a shared
+`src/components/Pagination.vue` rendered **both above and below** the list. The
+URL-synced state is **two params**, both in `src/lib/url.ts` (History API,
+preserving each other + any other params, synced back by the `popstate`
+listener): the current page `?page=N` (page 1 omits it —
+`readPageFromUrl`/`writePageToUrl`) and the open **detail document** `?doc=<id>`
+(`readDocFromUrl`/`writeDocToUrl`) so a reload/share/bookmark restores it. User
+paging and opening/closing a detail use `history.pushState` (Back/Forward walk
+pages and in/out of the detail), while restore-clamp and filter-reset use
+`replaceState`. Search/facets/sort stay in-memory (view/theme still use the
+`localStorage` `relaton-index-*` keys).
+
+**Detail page.** Clicking a `DocumentRow` (its id/title/chevron — real
+`?doc=<id>` anchors, click-intercepted to SPA-navigate) opens
+`src/components/DocumentDetail.vue` in place of the list (`App.vue` swaps on the
+`selectedDoc` computed; an unknown `?doc=` falls back to the list). It renders the
+enriched fields (abstract, metadata grid, keywords, contributors, all identifiers,
+**relations**, all dates) and the landing + raw-YAML links, each block shown only
+when present so a summary-only doc degrades gracefully.
+
+**Relations link within the dataset.** A relation whose target DocID is a document
+in this corpus renders as a real `?doc=<id>` anchor (same `docHref` helper and
+click-intercept as `DocumentRow` — `lib/url.ts` owns that one builder, don't
+re-inline `URLSearchParams`); a target that isn't stays visible as plain text
+rather than linking to a page that would bounce back to the list. Resolution is
+**frontend-side against the loaded corpus**, not baked in at build time — the
+generator is a single streaming pass (`each_document`) that never holds the corpus,
+so it cannot know the full id set when it writes a detail shard. The consequence is
+the usual loading gate: while summary shards are still arriving a real target can
+look absent, so `DocumentDetail` takes a **`hasDoc(id)` predicate** rather than a
+built `Set` (a `computed` Set over 166k ids stays lazy behind the function, and only
+materializes for a document that actually has relations), and `App.vue` calls
+`hydration.requestAll()` when an opened document's detail carries relations. Links
+then upgrade from plain text to anchors as shards land. Pure
+filter/sort logic is `src/lib/filter.ts`. UI icons
+are inline Heroicons-outline SVGs via the shared `src/components/Icon.vue`
+(`<Icon name="…" />`, a name→path map — `fill=none stroke=currentColor` so they
+inherit text colour + dark mode), matching the CalConnect standards site — **not**
+Unicode emoji; add a new glyph to `Icon.vue`'s `PATHS` rather than inlining an
+emoji. Tests: `npm run typecheck` + `vitest`
+(filter/hydrate/shards/url/App/Icon/DocumentDetail via happy-dom).
+Anything touching the shard loaders needs a `fetch` stub — `src/test/fetch.ts`
+(`stubFetch` + `deferred`) is the shared one; `deferred` is how the
+out-of-order-response ordering test forces shard 1 to resolve before shard 0.
+
+**`npm run dev`.** `src/dev/main.ts` shards `sample-data.json` in memory, stubs
+`fetch` to serve it, and sets the mount-node scalars — i.e. the dev harness goes
+through the *real* hydration path rather than a shortcut that only exists there.
+Its shard sizes are deliberately tiny (3 / 2) so progressive loading is visible.
+
+**Build wiring.** `rake build_frontend` (`npm ci`/`install` + `npm run build`) is
+a prerequisite of `build`/`release`; the root `rake build_all` runs it before
+`gem build`. The gemspec ships the gitignored `frontend/dist/*` explicitly
+(`git ls-files` can't see it) and excludes the tracked `frontend/` sources.
+**Release note:** the release pipeline must have Node available or the gem ships
+without the bundle. The reusable Pages-deploy workflow (install relaton-cli →
+`relaton index` → deploy-pages) belongs in `relaton/support` — data repos call it
+via `uses: relaton/support/.github/workflows/data-deploy.yml@main` — not in this
+repo (a reusable workflow only runs from the calling repo's `.github/workflows/`,
+and an in-tree copy would also get packaged into the gem).
+
+**`--mode` removal and the caller contract.** `--mode` is **gone**, not
+deprecated. `relaton/support`'s `data-deploy.yml` passed it at two call sites,
+and every `relaton-data-*` caller runs `source: git` — building relaton-cli from
+`relaton/relaton` **main**, not a released gem — so there is no release gate:
+a stale caller breaks on its next deploy run, not at some later `gem push`.
+`/work/HANDOFFS/relaton__support__data-deploy-drop-mode-input.md` asks support to
+drop the input, and **must land first**.
+
+That is also why `Command.exit_on_failure?` now returns `true`. Thor's legacy
+default is to print a usage error and exit **0**, so a caller passing the removed
+flag would write no site at all and still go green — publishing an empty index
+across ~29 unattended Pages deploys with nothing red. An acceptance spec pins the
+loud failure. Don't remove `exit_on_failure?` to quiet a test.
+
+**Don't pre-compress the shards, and don't add a single archive.** GitHub Pages
+already gzips them in transit (measured: a 32.9 MB `search.json` transfers in
+4.8 MB, and the old 151 MB 3gpp page in 6.4 MB — the hand-off's headline byte
+counts were `curl` *without* `--compressed`, i.e. uncompressed sizes). Storing
+`.json.gz` would be served as `application/gzip` with no `Content-Encoding`, so
+the browser would not inflate it transparently: you would add a decompressor to
+the bundle to reach bytes Fastly already gives you, and forfeit brotli. A single
+zip (the `index-vN.zip` pattern the data repos use for their *machine* index) is
+worse still here — nothing renders until all of it arrives, which is exactly the
+O(corpus) first paint this design removes. The real remaining cost was never
+transfer; it is parse/DOM/heap, which compression does nothing for.
+
+**Known ceiling.** Sharding fixes *transfer and parse*, not *compute*. 166k
+documents is still ~40–60 MB of JS heap, and `applyFilters` runs a full
+`Intl.Collator` sort over the whole array on every keystroke. The real fix is a
+prebuilt inverted index or server-side search; that is a separate design, not
+something to bolt onto the shard loader.
+
 ### File Operations
 
 `lib/relaton/cli/relaton_file.rb` — Static methods for extract (pull bibdata from Metanorma XML), concatenate (combine files into a collection), and split (break a collection into individual files).

data/Gemfile

@@ -10,6 +10,11 @@ gem "relaton", path: "../.."
 # add openssl as an explicit gem to fix SSL verification issues in GHA.
 # gem "openssl"
 
+# `canon` comes in through `lutaml-model`. Keep it below 0.3.52, as the root
+# Gemfile does: from that version it installs `yeptris`, which replaces
+# `Object#to_yaml` and raises `NameError` on Windows. See the root Gemfile.
+gem "canon", "< 0.3.52"
+
 gem "byebug", "~> 11.0"
 gem "debug"
 gem "equivalent-xml", "~> 0.6"

data/Rakefile

@@ -3,4 +3,19 @@ require "rspec/core/rake_task"
 
 RSpec::Core::RakeTask.new(:spec)
 
+FRONTEND_DIR = File.expand_path("frontend", __dir__)
+
+desc "Build the JS/CSS frontend (Vite → frontend/dist/app.iife.js + style.css)"
+task :build_frontend do
+  Dir.chdir(FRONTEND_DIR) do
+    # `npm ci` when a lockfile is present (reproducible), else `npm install`.
+    sh(File.exist?("package-lock.json") ? "npm ci" : "npm install")
+    sh "npm run build"
+  end
+end
+
+# The gem must ship the compiled frontend, so build it before packaging/release.
+task build: :build_frontend
+task release: :build_frontend
+
 task default: :spec