@gmb/bitmark-parser 6.10.1 → 6.11.1

package/README.md

@@ -55,21 +55,43 @@ const bitInfo = info({ infoType: "list", format: "text" });
 console.log(version());
 ```
 
-### WASM feature modules (`full` / `bitmark-json`)
+### WASM variants (`full` / `browser-full` / `bitmark-json`)
 
-The package ships **two wasm builds of the same engine** and selects one at
+The package ships **three wasm builds of the same engine** and selects one at
 runtime — the API surface never changes:
 
 | Feature | Contents | Size (approx.) |
 | --- | --- | --- |
-| `full` (default) | everything | ~749 KB (~310 KB gzip) |
-| `bitmark-json` | bitmark ↔ JSON only: `convert`/`canonicalize`/`transform` (formats `auto`/`bitmark`/`json`, plus the `text` output), `info`, breakscape, text fragments | ~560 KB (~231 KB gzip) |
+| `full` (Node default) | everything: rich `info` metadata + built-in translations + the semantic `diff` | ~1082 KB (~423 KB gzip) |
+| `browser-full` (browser default) | the same conversions and `diff`, without either | ~935 KB (~374 KB gzip) |
+| `bitmark-json` | bitmark ↔ JSON only: `convert`/`canonicalize`/`transform` (formats `auto`/`bitmark`/`json`, plus the `text` output), `info`, breakscape, text fragments — no `diff` | ~705 KB (~277 KB gzip) |
 
 Not in `bitmark-json`: the markup formats (`html`, `xml`, `xml-niso-iec`, …)
 as input or output (they return an `error: … not supported in this build`
 string), the `mappingReport` option (same error), and `lex` (throws
 `UnsupportedFeatureError`).
 
+`full` and `browser-full` convert **identically** — they differ only in what
+`info` can report: the META fields (tag descriptions, group provenance and raw
+mapping patterns) and the built-in translations table. Bit titles, bit
+descriptions and the bit-group / resource-group catalogs are in every variant,
+in English; `register` supplies translations to the lean ones (see *Display
+names and languages*).
+
+Why the defaults differ: only the browser pays a download-latency cost for
+those strings, and a Node backend loading wasm from disk has the same
+requirements as the native CLI. But which channel an artifact ends up in is
+not knowable from the artifact — browser code is routinely package-managed and
+webpack-built — so both variants are available on both platforms and the
+defaults are conveniences only. **If you bundle the Node entry for the
+browser, select `browser-full` explicitly.**
+
+> **Migrating from ≤ 6.10:** `full` used to mean what `browser-full` means now.
+> Browser code that explicitly selected `"full"` should move to
+> `"browser-full"` to keep its download size close to what it was; leave it on
+> `"full"` to gain the metadata and built-in translations for ~49 KB gzip.
+> Nothing else changes — the capabilities that were in `full` are in both.
+
 ```js
 import { init, convert, variant } from "@gmb/bitmark-parser";
 
@@ -88,12 +110,16 @@ Notes:
 - **Node**: with no explicit `init`, the first call lazily loads `full`
   synchronously — existing zero-init usage is unchanged. `initSync({ feature })`
   selects a variant synchronously.
-- **Browser**: `init()` is required as before; only the selected variant's
-  `.wasm` is fetched (both small JS glue modules are in the bundle). Overlapping
-  `init` calls: same-feature calls coalesce; a different-feature call supersedes
-  an unfinished one (last call wins); every promise resolves once the
-  finally-active module is live. In the browser, `initSync(bytes, { feature })`
-  instantiates from bytes you supply.
+- **Browser**: `init()` is required as before, and loads `browser-full`; only
+  the selected variant's `.wasm` is fetched (all three small JS glue modules
+  are in the bundle). Overlapping `init` calls: same-feature calls coalesce; a
+  different-feature call supersedes an unfinished one (last call wins); every
+  promise resolves once the finally-active module is live. In the browser,
+  `initSync(bytes, { feature })` instantiates from bytes you supply.
+- **Supplying your own `.wasm`**: glue and module are a pair, so name the
+  feature the bytes belong to — `init({ feature, module_or_path })`. The
+  back-compat positional form `init(module_or_path)` stays on `full`, since
+  those bytes are `bitmark_wasm_bg.wasm`.
 - **Workers**: `transformParallel` workers load the active variant.
 - **Legacy API**: works on `bitmark-json` (it follows the active module) except
   its HTML-dependent members (e.g. `convertHtmlTable`), which throw.
@@ -124,8 +150,12 @@ Options:
   bit's config cannot resolve, at the bit body/footer — never inside cards or
   consumed by a chain) is appended after all real keys as an array of strings
   (a valueless `[@key]` → `true`); a key colliding with a configured key of
-  the bit is `_`-prefixed. Unknown properties are never converted back to
-  bitmark, and each occurrence warns whether or not it is included.
+  the bit is `_`-prefixed. A bare tag the bit does not declare (`[!…]`,
+  `[?…]`, `[#…]`, … on a bit whose config has no such tag) is an unknown
+  too, under the key a declaring bit would use (`[!x]` → `"instruction":
+  ["x"]`); a rejected resource attachment is not — it is still emitted, with
+  a warning. Unknown properties are never converted back to bitmark, and
+  each occurrence warns whether or not it is included.
   Also available on `bitmarkToObjects` and as the CLI's
   `--include-unknown-properties`
 - `mappingReport` — replace the converted output with a human-readable
@@ -153,7 +183,7 @@ Options:
 For bitmark input, output is a JSON array of entries with:
 
 - `bit` — serialized bit payload
-- `parser` — parser metadata, `errors` / `warnings`, and `infos` (informational notices for recoveries that are not necessarily wrong, e.g. text kept as text because it only looks like an unclosed tag)
+- `parser` — parser metadata, `errors` / `warnings`, and `infos` (informational notices for recoveries that are not necessarily wrong, e.g. text kept as text because it only looks like an unclosed tag). Each issue carries a stable machine-readable `code` (`"unknown-property"`, `"missing-required-tag"`, …) beside its human-readable `message`, plus `text` and `location`. **Branch on `code`** — a shipped code never changes meaning, whereas the message text is not contractual and may be reworded in any release
 - `bitmark` — source bitmark text
 
 #### canonicalize(input: string, options?: CanonicalizeOptions): string
@@ -162,7 +192,90 @@ Re-emit input in its own format in canonical form — the single same-format sur
 
 #### transform(input: string, options?: TransformOptions): string
 
-Apply path patches to each bit (per-bit transform pipeline; build patches with `patchEntry`). `transformParallel` is the worker-thread variant returning a `Promise<string>`.
+Apply a patch document to a document, bit by bit, and re-emit it.
+`transformParallel` is the worker-thread variant returning a `Promise<string>`.
+
+- `patch` — a `PatchDocument`, or the shorthand: a bare array of entries
+  applied to every bit (build entries with `patchEntry`). Either form may be
+  a JSON string
+- `preHook` / `postHook` — per-bit hooks; the pre-hook may return patches, an
+  output-format override or `drop`. Hooks see the original bits, in order,
+  before the document reshapes anything; a hook's patches apply after the
+  document's
+- `inputFormat`, `outputFormat`, `mode`, `pretty`, `indent`,
+  `spacesAroundValues` — as for `convert`
+
+A patch document addresses bits by their position (or id) in the input
+**before** any entry is applied, so it reads like a diff:
+
+```json
+{
+  "version": 1,
+  "entries": [
+    { "select": { "index": 3, "id": "mc-014" }, "patch": [{ "path": "title", "op": "set", "value": "New" }] },
+    { "select": { "index": 9 }, "remove": true },
+    { "insert": { "after": 4 }, "bit": { "type": "article", "body": "Inserted" } },
+    { "select": { "index": 7 }, "move": { "after": 12 } },
+    { "select": "all", "patch": [{ "lang": "de" }] }
+  ]
+}
+```
+
+`after` names an anchor — an input bit that is neither removed nor moved,
+or `-1` for the document start. `diff` produces such a document from two
+versions of a file (below).
+
+#### diff(a: string, b: string, options?: DiffOptions): string
+
+Semantic diff of two documents. Both sides are re-canonicalized and
+compared as bit JSON, so tag order, spacing, breakscaping and omitted
+defaults are not differences; every real change is rendered as the
+canonical bitmark an author would type. The inputs may be bitmark or JSON,
+and need not share a format.
+
+```
+@@ bit 1 -> 2 (id mc-014) [.multiple-choice] changed
+ instruction[0]
+-[!Pick one]
++[!Pick exactly one]
+ quizzes[0]
+ ====
+ [-Bonn]
+-[-Berlin]
++[+Berlin]
+@@ bit 4 (id hero) [.image] removed
+-[.image]
+-[@id:hero]
+-[&image:https://example.com/hero.png]
+```
+
+Bits are paired by `id`, then by `sourceBB` on the same `sourceRL`, then by
+exact content, then by text similarity; a bit whose position changed among
+the kept ones is reported `moved`. Empty output means the documents are
+semantically equal.
+
+- `outputFormat` — `"bitmark"` (default; the text above), `"json"` (one
+  object per aligned bit, see `diffBits`), or `"patch"` (a patch document
+  that turns `a` into `b` — feed it to `transform`)
+- `inputFormat` — applies to both sides (default: `"auto"`, sniffed per side)
+- `context` — unchanged lines kept around each change (default: `3`)
+- `locate` — add source line numbers to bit positions (bitmark inputs; default: `false`)
+- `similarity` — pairing threshold for bits with no id, box or exact match, `0..1` (default: `0.6`; `1` disables)
+- `bboxTolerance` — pixels per coordinate for the `sourceBB` stage (default: `8`)
+- `spacesAroundValues` — in the rendered bitmark lines (default: `0`)
+
+The text layout and the JSON shape are stable contracts. An unknown
+property is shown as a JSON line, since the generator cannot render it, and
+cannot be re-applied. Not available on the `bitmark-json` variant
+(`UnsupportedFeatureError`).
+
+#### diffBits(a: string, b: string, options?: DiffOptions): BitDiff[]
+
+The typed form of `diff`: one `BitDiff` per aligned bit, unchanged ones
+included so the alignment is complete — `status`
+(`unchanged | changed | moved | removed | added`), the `a`/`b` positions,
+`ops` (a patch turning the A bit into the B bit) and `revert`, the parser
+issue codes gained or lost, and the rendered `hunks`.
 
 #### countBits(input: string): number / splitBits(input: string): BitSlice[]
 
@@ -201,7 +314,8 @@ format-natural), nullability (absence as a distinct state — orthogonal to
 defaults), JSON keys, per-context overrides, and the card set structure. `"deprecated"` lists deprecated bits with their deprecation
 version and (separately) any migration target.
 
-- `infoType` — `"list"` (default), `"bit"`, `"all"`, or `"deprecated"`
+- `infoType` — `"list"` (default), `"bit"`, `"all"`, `"deprecated"`,
+  `"bit-groups"`, `"resource-groups"`, or `"languages"`
 - `format` — `"text"` (default) or `"json"`
 - `bit` — filter to a specific bit type (when `infoType` is `"bit"`)
 - `pretty` — prettify JSON output (default: `false`)
@@ -209,10 +323,151 @@ version and (separately) any migration target.
 - `includeDeprecated` — include deprecated bits in `"list"`/`"all"` (default: `false`)
 - `onlyDeprecated` — restrict `"list"`/`"all"` to deprecated bits (default: `false`)
 - `full` — complete detail for `"bit"`/`"all"` (default: `false` = compact view)
+- `language` — BCP-47 tag(s) for display names: one (`"de"`), several
+  (`["de", "fr"]` or `"de,fr"`), or `"all"` — exported as `ALL_LANGUAGES`
+  (default: English). More than one adds a `titles` map. See *Display names
+  and languages* below
+
+`"bit-groups"` / `"resource-groups"` return the search/filter catalogs —
+each group's key, translated title, description, optional aliases and
+`subgroupOf`, and its members. They are available in EVERY build: the group
+catalog and per-bit titles/descriptions are "descriptive" data, which the
+lean wasm variants carry too. Deprecated members are excluded by default and
+MARKED when included with `includeDeprecated: true`; a search index matching
+already-published content needs them, because a migrating bit is re-emitted
+under its target's name.
+
+To derive quiz categories, intersect a bit's `bitGroups` with the groups
+carrying `subgroupOf: "quizzes"` — `subgroupOf` is metadata and never implies
+membership, so every member of a subgroup also declares the parent.
+
+Only the META fields — TAG descriptions, group-inheritance provenance and raw
+mapping patterns (the `info-meta` cargo feature) — depend on the build. They
+are reported by the native CLI and the wasm `full` variant; `browser-full` and
+`bitmark-json` omit them to stay small. Those are absent structurally: the key
+is missing, never `null` or empty. Everything else `info` returns — including
+bit titles, bit descriptions and the group catalogs — is identical in every
+variant.
+
+#### Display names and languages
+
+`title` — on bits, bit groups and resource groups — is a DISPLAY name, meant
+to be shown. Pass `language` to get it in another language:
+
+```js
+info({ infoType: "bit-groups", format: "json", language: "de" });
+```
+
+Resolution narrows one subtag at a time (`de-CH` → `de`), then falls back to
+the English title, then to the technical key. A partially translated language
+is therefore safe: untranslated entries stay English rather than blank.
+**Technical identity is never translated** — `name`, `key`, tag names and
+member lists are the same in every language, so you can key off them freely.
+
+**Several languages at once.** Building a multilingual index should not mean
+walking every bit once per language, so `language` also takes a list, or
+`"all"` — exported as the `ALL_LANGUAGES` constant, so you can name it rather
+than retype the literal:
+
+```js
+import { info, ALL_LANGUAGES } from "@gmb/bitmark-parser";
+
+const groups = JSON.parse(
+  info({ infoType: "bit-groups", format: "json", language: ALL_LANGUAGES }),
+);
+groups[0].titles; // { en: "Cloze", de: "Lückentext", … }
 
-The WASM build omits bit/tag descriptions, group-inheritance provenance and
-raw mapping patterns (native-CLI-only, via the `info-meta` cargo feature) to
-keep the binary small.
+// equivalent, and the idiomatic JS spelling of a list
+info({ infoType: "bit-groups", format: "json", language: ["de", "fr"] });
+
+// what "all" expands to
+JSON.parse(info({ infoType: "languages", format: "json" })); // ["de","en",…]
+```
+
+**Getting every name in one call.** `bit-groups` carries each member's
+display name, so one request answers "every bit type, what it is called, and
+which categories it is in":
+
+```js
+const groups = JSON.parse(
+  info({ infoType: "bit-groups", format: "json", includeDeprecated: true, language: "all" }),
+);
+groups[0].bitTypes[0]; // { name: "assignment", title: "Assignment", titles: {…} }
+```
+
+That is ~132 KB. Do not reach for `infoType: "all"` to get names — it carries
+the full per-tag detail for every bit (~5 MB) and is the wrong tool for this.
+
+More than one language adds a **`titles`** map, and nothing else. It carries `en` when an English title exists, and is otherwise sparse: a requested language with no translation is absent rather than filled with English, so a gap stays visible.
+It is keyed by the tag you ASKED for — request `de-CH` where only `de` exists
+and you read `titles["de-CH"]`.
+
+A single tag (or none) renders exactly as it always has, with no `titles`. The
+shape follows your REQUEST rather than which language you picked: `"all"` adds
+`titles` even on a build that resolves nothing but English.
+
+`title` is the name in your PRIMARY language — the first one you named. Name
+none and it is English; ask for `"all"` and it is English too, because with
+everything already in `titles` there is no non-arbitrary "first". So `title`
+tracks how you asked: consistent for any consumer that asks the same way every
+time, which in practice is all of them — a UI names one language, an index
+asks for `"all"`.
+
+`title` stays optional, as its type says — it is the name in the language you
+asked for, and a few bits have one in some languages but not in English (the
+internal `_comment` and `_error`). Every other key is the same in every
+language.
+
+Which languages resolve depends on the build. The native CLI and the wasm
+`full` variant bake the table in; `browser-full` and `bitmark-json` do not,
+because ~76 KB of names is not something a browser should download without
+asking. Those variants — and any build whose translations you want to
+override — take the file at runtime:
+
+```js
+import { init, register, info } from "@gmb/bitmark-parser";
+import translations from "@gmb/bitmark-parser/translations" with { type: "json" };
+
+await init({ feature: "browser-full" });
+register({ type: "translations", data: JSON.stringify(translations) });
+info({ infoType: "bit", bit: "cloze", format: "json", language: "de" });
+```
+
+`register` REPLACES rather than merges: a second call supersedes the first
+entirely, and the built-in table (where there is one) stays underneath, so a
+key your file omits still resolves. It applies process-wide and survives an
+`init` variant swap. The `language` option stays per-call, because that is
+the axis that actually varies — one process may serve many.
+
+The same file backs all three routes: `@gmb/bitmark-parser/translations`
+(an opt-in subpath, so nothing pays for it unasked), the CLI's
+`--translations <file>` flag, and what the `full` builds bake in. It holds
+non-English names only — English lives in the config `title`, which every
+build carries.
+
+**The JSON shape is a stable contract**; a change to it is a semver-major
+release. Language changes VALUES only — the key set is identical for every
+language. Parse it and assert the matching exported type:
+
+```ts
+import { info, type DeprecatedInfo } from "@gmb/bitmark-parser";
+
+const deprecated = JSON.parse(
+  info({ infoType: "deprecated", format: "json" }),
+) as DeprecatedInfo;
+```
+
+Result types: `BitListInfo`, `BitInfo` (`CompactBitInfo | FullBitInfo`),
+`AllBitsInfo`, `DeprecatedInfo`, `BitGroupsInfo`, `ResourceGroupsInfo`, and
+`InfoErrorResult` — returned in place of
+a result when `infoType` is `"bit"` and the name is unknown. `info` itself
+returns a `string` because it renders a document (`format: "text"`, `pretty`
+and `indent` only mean anything for one).
+
+**The text output is NOT a contract** — it is for humans and may change in
+any release. Never parse it; every field it shows exists in the JSON form.
+The normative definition of the JSON is
+`.zen/specs/API-INF-info-output.tsp`.
 
 #### version(): string
 
@@ -376,9 +631,18 @@ bitmark convert standard.xml --input-format xml-niso-iec --output-format bitmark
 bitmark canonicalize input.bitmark
 bitmark canonicalize input.json --mode full
 
-# Apply path patches to each bit
+# Apply a patch document (per-bit patches by index/id, remove, insert, move;
+# a bare array of entries still patches every bit)
 bitmark transform input.json --patch patches.json   # alias: patch
 
+# Semantic diff of two documents, rendered as bitmark
+bitmark diff old.bitmark new.bitmark                       # unified-style text
+bitmark diff old.bitmark new.bitmark --stat --exit-code    # counts; exit 1 when they differ
+bitmark diff old.bitmark new.json --output-format json     # per-bit ops, revert, hunks
+bitmark diff old.bitmark new.bitmark --output-format patch > d.patch
+bitmark transform old.bitmark --patch d.patch --output-format bitmark   # reproduces new
+# --context N, --locate, --similarity F, --bbox-tolerance N, --color auto|always|never
+
 # Dump the lexer token stream
 bitmark lex input.bitmark
 
@@ -420,9 +684,11 @@ The package includes pre-built browser bundles with the WASM module.
 </script>
 ```
 
-The CDN bundle carries both feature modules' JS glue; `init({ feature:
-"bitmark-json" })` fetches only the smaller `.wasm`, and a later
-`init({ feature: "full" })` upgrades in place (see *WASM feature modules*).
+The CDN bundle carries all three variants' JS glue and fetches only the
+selected variant's `.wasm` (from `dist/browser/wasm/`). A bare `init()` loads
+`browser-full`; `init({ feature: "bitmark-json" })` fetches the smallest
+build, and a later `init({ feature: "full" })` upgrades in place (see *WASM
+variants*).
 
 ### Bundler (webpack / vite)