@concordance-wiki/nlp 0.3.0 → 0.4.0

package/README.md

@@ -71,6 +71,6 @@ Part of [Concordance](https://github.com/concordance-wiki/concordance), GNU GPL
 <details>
 <summary>Inside the package</summary>
 
-Language packs for `en` and `fr` built from data files (`locales/<locale>/pack.yaml`, `stopwords.txt` and the optional `suffixes.txt` of inflected-form endings; the default stopwords are the words that name nothing on their own in any corpus, from the articles to every form of the auxiliaries and light verbs, the adverbs of prose, quantifiers, numerals, time words and the nouns that only count or place, never a word that could be a business term or end a technical compound, which a project lists under `inference.stopwords`), BCP 47 locale resolution with fallback to the language, Unicode word segmentation and collation, a registry for packs shipped by plugins (`loadLanguagePack`, `languagePack`, `registerLanguagePack`, `resolveLocale`, `canonicalLocale`, `loadStopwords`), and text normalisation: the comparison form of a text (lower-cased, accents stripped, apostrophes unified, each word singularised by the pack's suffix rules; `comparisonForm`, `comparisonWords`, `singularize`) and word boundaries from Unicode segmentation (`wordBoundaries`, `isOnWordBoundaries`); and the recognition dictionary: `buildDictionary` keys every title and alias of the entities of a locale by comparison form, with priority to glossary sources (`glossarySources`), leaves out stopwords (`dictionaryStopwords` merges the pack's defaults with the files of `inference.stopwords`) and terms shorter than three characters unless `inference.short_terms` allows them, and flags the forms shared by several entities as homonyms with an `I-TERM-HOMONYM` finding; and the occurrence scan: `tokenize` cuts a text into words in comparison form with their spans, `buildAutomaton`, `scan` and `longestMatches` run a word-level Aho-Corasick over them (one pass per text; an expression contained in a longer recognised one is dropped, while expressions that only partly overlap are all kept), and `scanDocument` turns the matches of a dictionary in the paragraphs of a document into occurrences with file, line, position, the matched text as written, section, an 80-character context, the type announced by a recognised prefix (`type_prefixes`) and a confidence from the `glossary_occurrence` scale (`occurrenceConfidence` gives the per-occurrence increments up to the cap; `compareOccurrences` is their canonical order). The scan reads the units that `scannableText` of the ingestion package produces, so code blocks, inline code, URLs, frontmatter and link targets never yield an occurrence; the visible text of a link does. And the keyword discovery: `extractNgrams` reads the same units and yields every n-gram of one to four words (`keywordOptions` resolves `inference.ngrams`, `inference.candidate_score` and the lock's `rejected_terms`), leaving out those starting or ending with a stopword, made only of digits or shorter than three characters, each with its surface form, position and a 160-character context; `scoreCandidates` groups them by comparison form, drops the dictionary entries and the rejected terms (which still penalise the n-grams they contain), scores each candidate by C-value × IDF and keeps those with three occurrences in two distinct files, best score first with its display form and its mentions (`compareMentions`), each with its confidence in [0, 1] of being a term of the subject rather than a word of the language, the product of five signals rounded to four decimals (`confidenceOf`, `confidenceFactors`, `confidencePenalties`, the measures in `signals` and the lowering signals in `penalties`): spread `df / N`, 1 up to `maxSpread` (0.5) then linear down to 0 for a word in every file, read only in a corpus of `SPREAD_MIN_FILES` (10); burst `occurrences / df`, 1 from `BURST_PER_FILE` (1.5) else `0.5 + (burst − 1)`, read only from `BURST_MIN_FILES` (5); position `POSITION_BASE` (0.8) plus 0.2 times the share of appearances in the prominent texts given as `prominent` n-grams (headings, written links, frontmatter), capped at one; neighbourhood 1 when the expression is, or is a component of, a frequent n-gram that also holds a defined term, `NEIGHBOURHOOD_BASE` (0.9) otherwise; morphology `MORPHOLOGY_PENALTY` (0.8) when a word ends with an inflected-form suffix of the pack (`suffixes.txt`, `loadSuffixes`; a pack without the file penalises no form), 1 otherwise; `undefinedTermFindings` turns the candidates at or above the score threshold into `W-TERM-UNDEFINED` findings; `similarExpressions` picks, among keyed expressions, those of a similar form to a key (`similarForm`: one a contiguous part of the other, or half the distinct words of the longer one shared), the closest first, five at most (`SIMILAR_EXPRESSIONS_LIMIT`), for the leads of a keyword page; `definedExpressions` lists the n-grams the dictionary already defines that reach the publication threshold, the expressions whose keyword page a note took over. And the publication threshold: `publishKeywords` splits the candidates into the pages to generate (`keywords/<slug of the key>`, sorted by identifier; two keys with the same slug take numeric suffixes in score order), the expressions discarded by the counts of `inference.keyword_pages` (`keywordPublicationOptions`: three occurrences in two distinct files by default) and those withheld by its `min_confidence` (one half by default), both staying in the output for the search index without a page; `keywordEntities` turns each page into a `term` entity marked `keyword: true`, located on the first mention of its expression, with its occurrences, files, score and confidence as attributes. The counts of the three lists go into the build summary (`keywords.published`, `keywords.discarded`, `keywords.withheld` in core's `summarize`), printed by the build as `keyword pages`, `expressions under the threshold` and `expressions set aside by confidence` once the build runs the discovery.
+Language packs for `en` and `fr` built from data files (`locales/<locale>/pack.yaml`, `stopwords.txt` and the optional `suffixes.txt` of inflected-form endings; the default stopwords are the words that name nothing on their own in any corpus, from the articles to every form of the auxiliaries and light verbs, the adverbs of prose, quantifiers, numerals, time words and the nouns that only count or place, never a word that could be a business term or end a technical compound, which a project lists under `inference.stopwords`), BCP 47 locale resolution with fallback to the language, Unicode word segmentation and a collation computed from the options of the pack without the collation data of the runtime (`collation` of the core package, accents and case set aside, digits compared by value), a registry for packs shipped by plugins (`loadLanguagePack`, `languagePack`, `registerLanguagePack`, `resolveLocale`, `canonicalLocale`, `loadStopwords`), and text normalisation: the comparison form of a text (lower-cased, accents stripped, apostrophes unified, each word singularised by the pack's suffix rules; `comparisonForm`, `comparisonWords`, `singularize`) and word boundaries from Unicode segmentation (`wordBoundaries`, `isOnWordBoundaries`); and the recognition dictionary: `buildDictionary` keys every title and alias of the entities of a locale by comparison form, with priority to glossary sources (`glossarySources`), leaves out stopwords (`dictionaryStopwords` merges the pack's defaults with the files of `inference.stopwords`) and terms shorter than three characters unless `inference.short_terms` allows them, and flags the forms shared by several entities as homonyms with an `I-TERM-HOMONYM` finding; and the occurrence scan: `tokenize` cuts a text into words in comparison form with their spans, `buildAutomaton`, `scan` and `longestMatches` run a word-level Aho-Corasick over them (one pass per text; an expression contained in a longer recognised one is dropped, while expressions that only partly overlap are all kept), and `scanDocument` turns the matches of a dictionary in the paragraphs of a document into occurrences with file, line, position, the matched text as written, section, an 80-character context, the type announced by a recognised prefix (`type_prefixes`) and a confidence from the `glossary_occurrence` scale (`occurrenceConfidence` gives the per-occurrence increments up to the cap; `compareOccurrences` is their canonical order). The scan reads the units that `scannableText` of the ingestion package produces, so code blocks, inline code, URLs, frontmatter and link targets never yield an occurrence; the visible text of a link does. And the keyword discovery: `extractNgrams` reads the same units and yields every n-gram of one to four words (`keywordOptions` resolves `inference.ngrams`, `inference.candidate_score` and the lock's `rejected_terms`), leaving out those starting or ending with a stopword, made only of digits or shorter than three characters, each with its surface form, position and a 160-character context; `scoreCandidates` groups them by comparison form, drops the dictionary entries and the rejected terms (which still penalise the n-grams they contain), scores each candidate by C-value × IDF and keeps those with three occurrences in two distinct files, best score first with its display form and its mentions (`compareMentions`), each with its confidence in [0, 1] of being a term of the subject rather than a word of the language, the product of five signals rounded to four decimals (`confidenceOf`, `confidenceFactors`, `confidencePenalties`, the measures in `signals` and the lowering signals in `penalties`): spread `df / N`, 1 up to `maxSpread` (0.5) then linear down to 0 for a word in every file, read only in a corpus of `SPREAD_MIN_FILES` (10); burst `occurrences / df`, 1 from `BURST_PER_FILE` (1.5) else `0.5 + (burst − 1)`, read only from `BURST_MIN_FILES` (5); position `POSITION_BASE` (0.8) plus 0.2 times the share of appearances in the prominent texts given as `prominent` n-grams (headings, written links, frontmatter), capped at one; neighbourhood 1 when the expression is, or is a component of, a frequent n-gram that also holds a defined term, `NEIGHBOURHOOD_BASE` (0.9) otherwise; morphology `MORPHOLOGY_PENALTY` (0.8) when a word ends with an inflected-form suffix of the pack (`suffixes.txt`, `loadSuffixes`; a pack without the file penalises no form), 1 otherwise; `undefinedTermFindings` turns the candidates at or above the score threshold into `W-TERM-UNDEFINED` findings; `similarExpressions` picks, among keyed expressions, those of a similar form to a key (`similarForm`: one a contiguous part of the other, or half the distinct words of the longer one shared), the closest first, five at most (`SIMILAR_EXPRESSIONS_LIMIT`), for the leads of a keyword page; `definedExpressions` lists the n-grams the dictionary already defines that reach the publication threshold, the expressions whose keyword page a note took over. And the publication threshold: `publishKeywords` splits the candidates into the pages to generate (`keywords/<slug of the key>`, sorted by identifier; two keys with the same slug take numeric suffixes in score order), the expressions discarded by the counts of `inference.keyword_pages` (`keywordPublicationOptions`: three occurrences in two distinct files by default) and those withheld by its `min_confidence` (one half by default), both staying in the output for the search index without a page; `keywordEntities` turns each page into a `term` entity marked `keyword: true`, located on the first mention of its expression, with its occurrences, files, score and confidence as attributes. The counts of the three lists go into the build summary (`keywords.published`, `keywords.discarded`, `keywords.withheld` in core's `summarize`), printed by the build as `keyword pages`, `expressions under the threshold` and `expressions set aside by confidence` once the build runs the discovery.
 
 </details>

package/dist/dictionary/build.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"build.d.ts","sourceRoot":"","sources":["../../src/dictionary/build.ts"],"names":[],"mappings":"AAKA,OAAO,KAAK,EAAE,UAAU,EAAmB,gBAAgB,EAAoB,MAAM,YAAY,CAAC;AAElG,eAAO,MAAM,aAAa,mBAAmB,CAAC;AAK9C,MAAM,WAAW,oBAAoB;IACnC,QAAQ,EAAE,SAAS,gBAAgB,EAAE,CAAC;IACtC,MAAM,EAAE,MAAM,CAAC;IACf,yDAAyD;IACzD,eAAe,EAAE,WAAW,CAAC,MAAM,CAAC,CAAC;IACrC,uFAAuF;IACvF,SAAS,EAAE,WAAW,CAAC,MAAM,CAAC,CAAC;IAC/B,uEAAuE;IACvE,UAAU,EAAE,WAAW,CAAC,MAAM,CAAC,CAAC;CACjC;AA8BD;;;;GAIG;AACH,wBAAgB,eAAe,CAAC,KAAK,EAAE,oBAAoB,GAAG,UAAU,CAkCvE"}
+{"version":3,"file":"build.d.ts","sourceRoot":"","sources":["../../src/dictionary/build.ts"],"names":[],"mappings":"AAKA,OAAO,KAAK,EAAE,UAAU,EAAmB,gBAAgB,EAAoB,MAAM,YAAY,CAAC;AAElG,eAAO,MAAM,aAAa,mBAAmB,CAAC;AAK9C,MAAM,WAAW,oBAAoB;IACnC,QAAQ,EAAE,SAAS,gBAAgB,EAAE,CAAC;IACtC,MAAM,EAAE,MAAM,CAAC;IACf,yDAAyD;IACzD,eAAe,EAAE,WAAW,CAAC,MAAM,CAAC,CAAC;IACrC,uFAAuF;IACvF,SAAS,EAAE,WAAW,CAAC,MAAM,CAAC,CAAC;IAC/B,uEAAuE;IACvE,UAAU,EAAE,WAAW,CAAC,MAAM,CAAC,CAAC;CACjC;AA4CD;;;;GAIG;AACH,wBAAgB,eAAe,CAAC,KAAK,EAAE,oBAAoB,GAAG,UAAU,CAqCvE"}

package/dist/dictionary/build.js

@@ -10,12 +10,16 @@ function byCodeUnit(a, b) {
 function comparisonSet(forms, pack) {
     return new Set([...forms].map((form) => comparisonForm(form, pack)));
 }
-function homonymFinding(entry) {
+function homonymFinding(entry, first) {
     const ids = entry.targets.map((target) => target.id).sort(byCodeUnit);
+    const { entity, target } = first;
     return {
         check: HOMONYM_CHECK,
         severity: "info",
-        message: `"${entry.key}" is the title or an alias of ${String(ids.length)} entities: ${ids.join(", ")}`,
+        source: entity.source,
+        ...(entity.path === undefined ? {} : { path: entity.path }),
+        entity: entity.id,
+        message: `"${target.form}" is the title or an alias of ${String(ids.length)} entities: ${ids.join(", ")}`,
         remediation: "Occurrences link to each entity at half confidence. Give the entities distinct titles or aliases, or add a `## Not to be confused with` section to each note so that readers tell them apart.",
     };
 }
@@ -47,7 +51,7 @@ export function buildDictionary(input) {
         const targets = candidates.get(key) ?? new Map();
         if (!targets.has(entity.id)) {
             const priority = input.glossarySources.has(entity.source) ? 0 : 1;
-            targets.set(entity.id, { id: entity.id, kind, form, priority });
+            targets.set(entity.id, { entity, target: { id: entity.id, kind, form, priority } });
         }
         candidates.set(key, targets);
     };
@@ -61,11 +65,14 @@ export function buildDictionary(input) {
     const entries = new Map();
     const findings = [];
     for (const [key, byEntity] of [...candidates].sort(compareKeys)) {
-        const targets = [...byEntity.values()].sort(compareTargets);
+        const sorted = [...byEntity.values()].sort((a, b) => compareTargets(a.target, b.target));
+        const targets = sorted.map((candidate) => candidate.target);
         const entry = { key, targets, homonym: targets.length > 1 };
         entries.set(key, entry);
+        // The finding lands on the first candidate; a loop over one element, so that nothing is looked up twice.
         if (entry.homonym)
-            findings.push(homonymFinding(entry));
+            for (const first of sorted.slice(0, 1))
+                findings.push(homonymFinding(entry, first));
     }
     return { locale: input.locale, entries, findings };
 }

package/dist/dictionary/build.js.map

@@ -1 +1 @@
-{"version":3,"file":"build.js","sourceRoot":"","sources":["../../src/dictionary/build.ts"],"names":[],"mappings":"AAGA,OAAO,EAAE,YAAY,EAAE,MAAM,uBAAuB,CAAC;AACrD,OAAO,EAAE,cAAc,EAAE,MAAM,4BAA4B,CAAC;AAG5D,MAAM,CAAC,MAAM,aAAa,GAAG,gBAAgB,CAAC;AAE9C,4EAA4E;AAC5E,MAAM,aAAa,GAAG,CAAC,CAAC;AAaxB,sGAAsG;AACtG,SAAS,UAAU,CAAC,CAAS,EAAE,CAAS;IACtC,OAAO,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;AACvC,CAAC;AAED,SAAS,aAAa,CAAC,KAA0B,EAAE,IAAkB;IACnE,OAAO,IAAI,GAAG,CAAC,CAAC,GAAG,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,cAAc,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC,CAAC,CAAC;AACvE,CAAC;AAED,SAAS,cAAc,CAAC,KAAsB;IAC5C,MAAM,GAAG,GAAG,KAAK,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC;IACtE,OAAO;QACL,KAAK,EAAE,aAAa;QACpB,QAAQ,EAAE,MAAM;QAChB,OAAO,EAAE,IAAI,KAAK,CAAC,GAAG,iCAAiC,MAAM,CAAC,GAAG,CAAC,MAAM,CAAC,cAAc,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE;QACvG,WAAW,EACT,+LAA+L;KAClM,CAAC;AACJ,CAAC;AAED,gGAAgG;AAChG,SAAS,cAAc,CAAC,CAAmB,EAAE,CAAmB;IAC9D,OAAO,CAAC,CAAC,QAAQ,GAAG,CAAC,CAAC,QAAQ,IAAI,CAAC,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;AAC3D,CAAC;AACD,SAAS,WAAW,CAAC,CAAC,CAAC,CAAoB,EAAE,CAAC,CAAC,CAAoB;IACjE,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;AACxB,CAAC;AAED;;;;GAIG;AACH,MAAM,UAAU,eAAe,CAAC,KAA2B;IACzD,MAAM,IAAI,GAAG,YAAY,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC;IACxC,MAAM,SAAS,GAAG,aAAa,CAAC,KAAK,CAAC,SAAS,EAAE,IAAI,CAAC,CAAC;IACvD,MAAM,UAAU,GAAG,aAAa,CAAC,KAAK,CAAC,UAAU,EAAE,IAAI,CAAC,CAAC;IACzD,MAAM,UAAU,GAAG,IAAI,GAAG,EAAyC,CAAC;IAEpE,MAAM,QAAQ,GAAG,CAAC,MAAwB,EAAE,IAA8B,EAAE,IAAY,EAAE,EAAE;QAC1F,MAAM,GAAG,GAAG,cAAc,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;QACvC,IAAI,GAAG,KAAK,EAAE;YAAE,OAAO;QACvB,IAAI,GAAG,CAAC,MAAM,GAAG,aAAa,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,GAAG,CAAC;YAAE,OAAO;QAC/D,IAAI,SAAS,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,GAAG,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,SAAS,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;YAAE,OAAO;QACtF,MAAM,OAAO,GAAG,UAAU,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,IAAI,GAAG,EAA4B,CAAC;QAC3E,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,EAAE,CAAC;YAC5B,MAAM,QAAQ,GAAG,KAAK,CAAC,eAAe,CAAC,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;YAClE,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,EAAE,EAAE,EAAE,EAAE,MAAM,CAAC,EAAE,EAAE,IAAI,EAAE,IAAI,EAAE,QAAQ,EAAE,CAAC,CAAC;QAClE,CAAC;QACD,UAAU,CAAC,GAAG,CAAC,GAAG,EAAE,OAAO,CAAC,CAAC;IAC/B,CAAC,CAAC;IAEF,KAAK,MAAM,MAAM,IAAI,KAAK,CAAC,QAAQ,EAAE,CAAC;QACpC,IAAI,MAAM,CAAC,MAAM,KAAK,KAAK,CAAC,MAAM;YAAE,SAAS;QAC7C,QAAQ,CAAC,MAAM,EAAE,OAAO,EAAE,MAAM,CAAC,KAAK,CAAC,CAAC;QACxC,KAAK,MAAM,KAAK,IAAI,MAAM,CAAC,OAAO;YAAE,QAAQ,CAAC,MAAM,EAAE,OAAO,EAAE,KAAK,CAAC,CAAC;IACvE,CAAC;IAED,MAAM,OAAO,GAAG,IAAI,GAAG,EAA2B,CAAC;IACnD,MAAM,QAAQ,GAAc,EAAE,CAAC;IAC/B,KAAK,MAAM,CAAC,GAAG,EAAE,QAAQ,CAAC,IAAI,CAAC,GAAG,UAAU,CAAC,CAAC,IAAI,CAAC,WAAW,CAAC,EAAE,CAAC;QAChE,MAAM,OAAO,GAAG,CAAC,GAAG,QAAQ,CAAC,MAAM,EAAE,CAAC,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC;QAC5D,MAAM,KAAK,GAAoB,EAAE,GAAG,EAAE,OAAO,EAAE,OAAO,EAAE,OAAO,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC7E,OAAO,CAAC,GAAG,CAAC,GAAG,EAAE,KAAK,CAAC,CAAC;QACxB,IAAI,KAAK,CAAC,OAAO;YAAE,QAAQ,CAAC,IAAI,CAAC,cAAc,CAAC,KAAK,CAAC,CAAC,CAAC;IAC1D,CAAC;IACD,OAAO,EAAE,MAAM,EAAE,KAAK,CAAC,MAAM,EAAE,OAAO,EAAE,QAAQ,EAAE,CAAC;AACrD,CAAC"}
+{"version":3,"file":"build.js","sourceRoot":"","sources":["../../src/dictionary/build.ts"],"names":[],"mappings":"AAGA,OAAO,EAAE,YAAY,EAAE,MAAM,uBAAuB,CAAC;AACrD,OAAO,EAAE,cAAc,EAAE,MAAM,4BAA4B,CAAC;AAG5D,MAAM,CAAC,MAAM,aAAa,GAAG,gBAAgB,CAAC;AAE9C,4EAA4E;AAC5E,MAAM,aAAa,GAAG,CAAC,CAAC;AAaxB,sGAAsG;AACtG,SAAS,UAAU,CAAC,CAAS,EAAE,CAAS;IACtC,OAAO,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;AACvC,CAAC;AAED,SAAS,aAAa,CAAC,KAA0B,EAAE,IAAkB;IACnE,OAAO,IAAI,GAAG,CAAC,CAAC,GAAG,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,cAAc,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC,CAAC,CAAC;AACvE,CAAC;AAYD,SAAS,cAAc,CAAC,KAAsB,EAAE,KAAgB;IAC9D,MAAM,GAAG,GAAG,KAAK,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC;IACtE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE,GAAG,KAAK,CAAC;IACjC,OAAO;QACL,KAAK,EAAE,aAAa;QACpB,QAAQ,EAAE,MAAM;QAChB,MAAM,EAAE,MAAM,CAAC,MAAM;QACrB,GAAG,CAAC,MAAM,CAAC,IAAI,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,MAAM,CAAC,IAAI,EAAE,CAAC;QAC3D,MAAM,EAAE,MAAM,CAAC,EAAE;QACjB,OAAO,EAAE,IAAI,MAAM,CAAC,IAAI,iCAAiC,MAAM,CAAC,GAAG,CAAC,MAAM,CAAC,cAAc,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE;QACzG,WAAW,EACT,+LAA+L;KAClM,CAAC;AACJ,CAAC;AAED,gGAAgG;AAChG,SAAS,cAAc,CAAC,CAAmB,EAAE,CAAmB;IAC9D,OAAO,CAAC,CAAC,QAAQ,GAAG,CAAC,CAAC,QAAQ,IAAI,CAAC,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;AAC3D,CAAC;AACD,SAAS,WAAW,CAAC,CAAC,CAAC,CAAoB,EAAE,CAAC,CAAC,CAAoB;IACjE,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;AACxB,CAAC;AAED;;;;GAIG;AACH,MAAM,UAAU,eAAe,CAAC,KAA2B;IACzD,MAAM,IAAI,GAAG,YAAY,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC;IACxC,MAAM,SAAS,GAAG,aAAa,CAAC,KAAK,CAAC,SAAS,EAAE,IAAI,CAAC,CAAC;IACvD,MAAM,UAAU,GAAG,aAAa,CAAC,KAAK,CAAC,UAAU,EAAE,IAAI,CAAC,CAAC;IACzD,MAAM,UAAU,GAAG,IAAI,GAAG,EAAkC,CAAC;IAE7D,MAAM,QAAQ,GAAG,CAAC,MAAwB,EAAE,IAA8B,EAAE,IAAY,EAAE,EAAE;QAC1F,MAAM,GAAG,GAAG,cAAc,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;QACvC,IAAI,GAAG,KAAK,EAAE;YAAE,OAAO;QACvB,IAAI,GAAG,CAAC,MAAM,GAAG,aAAa,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,GAAG,CAAC;YAAE,OAAO;QAC/D,IAAI,SAAS,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,GAAG,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,KAAK,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,SAAS,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;YAAE,OAAO;QACtF,MAAM,OAAO,GAAG,UAAU,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,IAAI,GAAG,EAAqB,CAAC;QACpE,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,CAAC,EAAE,CAAC;YAC5B,MAAM,QAAQ,GAAG,KAAK,CAAC,eAAe,CAAC,GAAG,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;YAClE,OAAO,CAAC,GAAG,CAAC,MAAM,CAAC,EAAE,EAAE,EAAE,MAAM,EAAE,MAAM,EAAE,EAAE,EAAE,EAAE,MAAM,CAAC,EAAE,EAAE,IAAI,EAAE,IAAI,EAAE,QAAQ,EAAE,EAAE,CAAC,CAAC;QACtF,CAAC;QACD,UAAU,CAAC,GAAG,CAAC,GAAG,EAAE,OAAO,CAAC,CAAC;IAC/B,CAAC,CAAC;IAEF,KAAK,MAAM,MAAM,IAAI,KAAK,CAAC,QAAQ,EAAE,CAAC;QACpC,IAAI,MAAM,CAAC,MAAM,KAAK,KAAK,CAAC,MAAM;YAAE,SAAS;QAC7C,QAAQ,CAAC,MAAM,EAAE,OAAO,EAAE,MAAM,CAAC,KAAK,CAAC,CAAC;QACxC,KAAK,MAAM,KAAK,IAAI,MAAM,CAAC,OAAO;YAAE,QAAQ,CAAC,MAAM,EAAE,OAAO,EAAE,KAAK,CAAC,CAAC;IACvE,CAAC;IAED,MAAM,OAAO,GAAG,IAAI,GAAG,EAA2B,CAAC;IACnD,MAAM,QAAQ,GAAc,EAAE,CAAC;IAC/B,KAAK,MAAM,CAAC,GAAG,EAAE,QAAQ,CAAC,IAAI,CAAC,GAAG,UAAU,CAAC,CAAC,IAAI,CAAC,WAAW,CAAC,EAAE,CAAC;QAChE,MAAM,MAAM,GAAG,CAAC,GAAG,QAAQ,CAAC,MAAM,EAAE,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,cAAc,CAAC,CAAC,CAAC,MAAM,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC;QACzF,MAAM,OAAO,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC,SAAS,EAAE,EAAE,CAAC,SAAS,CAAC,MAAM,CAAC,CAAC;QAC5D,MAAM,KAAK,GAAoB,EAAE,GAAG,EAAE,OAAO,EAAE,OAAO,EAAE,OAAO,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;QAC7E,OAAO,CAAC,GAAG,CAAC,GAAG,EAAE,KAAK,CAAC,CAAC;QACxB,yGAAyG;QACzG,IAAI,KAAK,CAAC,OAAO;YACf,KAAK,MAAM,KAAK,IAAI,MAAM,CAAC,KAAK,CAAC,CAAC,EAAE,CAAC,CAAC;gBAAE,QAAQ,CAAC,IAAI,CAAC,cAAc,CAAC,KAAK,EAAE,KAAK,CAAC,CAAC,CAAC;IACxF,CAAC;IACD,OAAO,EAAE,MAAM,EAAE,KAAK,CAAC,MAAM,EAAE,OAAO,EAAE,QAAQ,EAAE,CAAC;AACrD,CAAC"}

package/dist/dictionary/stopwords.d.ts

@@ -8,7 +8,8 @@ export interface DictionaryStopwordsInput {
 }
 /**
  * The stopwords excluded from the dictionary of a locale: the pack's defaults plus every
- * file listed under `inference.stopwords`. A missing file is a configuration mistake.
+ * file listed under `inference.stopwords`. A missing file is a configuration mistake, which the
+ * commands report before a build starts.
  */
 export declare function dictionaryStopwords(input: DictionaryStopwordsInput): ReadonlySet<string>;
 /** The sources whose entities take priority: `inference.glossary_sources`, else those with `glossary: true`. */

package/dist/dictionary/stopwords.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"stopwords.d.ts","sourceRoot":"","sources":["../../src/dictionary/stopwords.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,wBAAwB,CAAC;AAKjE,MAAM,WAAW,wBAAwB;IACvC,MAAM,EAAE,MAAM,CAAC;IACf,MAAM,EAAE,MAAM,CAAC;IACf,2FAA2F;IAC3F,eAAe,EAAE,MAAM,CAAC;IACxB,EAAE,EAAE,UAAU,CAAC;CAChB;AAED;;;GAGG;AACH,wBAAgB,mBAAmB,CAAC,KAAK,EAAE,wBAAwB,GAAG,WAAW,CAAC,MAAM,CAAC,CAUxF;AAED,gHAAgH;AAChH,wBAAgB,eAAe,CAAC,MAAM,EAAE,MAAM,GAAG,WAAW,CAAC,MAAM,CAAC,CAInE"}
+{"version":3,"file":"stopwords.d.ts","sourceRoot":"","sources":["../../src/dictionary/stopwords.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,wBAAwB,CAAC;AAKjE,MAAM,WAAW,wBAAwB;IACvC,MAAM,EAAE,MAAM,CAAC;IACf,MAAM,EAAE,MAAM,CAAC;IACf,2FAA2F;IAC3F,eAAe,EAAE,MAAM,CAAC;IACxB,EAAE,EAAE,UAAU,CAAC;CAChB;AAED;;;;GAIG;AACH,wBAAgB,mBAAmB,CAAC,KAAK,EAAE,wBAAwB,GAAG,WAAW,CAAC,MAAM,CAAC,CAUxF;AAED,gHAAgH;AAChH,wBAAgB,eAAe,CAAC,MAAM,EAAE,MAAM,GAAG,WAAW,CAAC,MAAM,CAAC,CAInE"}

package/dist/dictionary/stopwords.js

@@ -3,7 +3,8 @@ import { languagePack } from "../locale/registry.js";
 import { loadStopwords } from "../locale/stopwords.js";
 /**
  * The stopwords excluded from the dictionary of a locale: the pack's defaults plus every
- * file listed under `inference.stopwords`. A missing file is a configuration mistake.
+ * file listed under `inference.stopwords`. A missing file is a configuration mistake, which the
+ * commands report before a build starts.
  */
 export function dictionaryStopwords(input) {
     const words = new Set(languagePack(input.locale).stopwords);

package/dist/dictionary/stopwords.js.map

@@ -1 +1 @@
-{"version":3,"file":"stopwords.js","sourceRoot":"","sources":["../../src/dictionary/stopwords.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AAIpC,OAAO,EAAE,YAAY,EAAE,MAAM,uBAAuB,CAAC;AACrD,OAAO,EAAE,aAAa,EAAE,MAAM,wBAAwB,CAAC;AAUvD;;;GAGG;AACH,MAAM,UAAU,mBAAmB,CAAC,KAA+B;IACjE,MAAM,KAAK,GAAG,IAAI,GAAG,CAAC,YAAY,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,SAAS,CAAC,CAAC;IAC5D,KAAK,MAAM,IAAI,IAAI,KAAK,CAAC,MAAM,CAAC,SAAS,EAAE,SAAS,IAAI,EAAE,EAAE,CAAC;QAC3D,MAAM,IAAI,GAAG,OAAO,CAAC,KAAK,CAAC,eAAe,EAAE,IAAI,CAAC,CAAC;QAClD,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC;YAC3B,MAAM,IAAI,KAAK,CAAC,4BAA4B,IAAI,gCAAgC,IAAI,IAAI,CAAC,CAAC;QAC5F,CAAC;QACD,KAAK,MAAM,IAAI,IAAI,aAAa,CAAC,KAAK,CAAC,EAAE,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;YAAE,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;IAC7E,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,gHAAgH;AAChH,MAAM,UAAU,eAAe,CAAC,MAAc;IAC5C,MAAM,QAAQ,GAAG,MAAM,CAAC,SAAS,EAAE,gBAAgB,CAAC;IACpD,IAAI,QAAQ,KAAK,SAAS;QAAE,OAAO,IAAI,GAAG,CAAC,QAAQ,CAAC,CAAC;IACrD,OAAO,IAAI,GAAG,CAAC,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,MAAM,CAAC,QAAQ,KAAK,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;AACjG,CAAC"}
+{"version":3,"file":"stopwords.js","sourceRoot":"","sources":["../../src/dictionary/stopwords.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAAE,MAAM,WAAW,CAAC;AAIpC,OAAO,EAAE,YAAY,EAAE,MAAM,uBAAuB,CAAC;AACrD,OAAO,EAAE,aAAa,EAAE,MAAM,wBAAwB,CAAC;AAUvD;;;;GAIG;AACH,MAAM,UAAU,mBAAmB,CAAC,KAA+B;IACjE,MAAM,KAAK,GAAG,IAAI,GAAG,CAAC,YAAY,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,SAAS,CAAC,CAAC;IAC5D,KAAK,MAAM,IAAI,IAAI,KAAK,CAAC,MAAM,CAAC,SAAS,EAAE,SAAS,IAAI,EAAE,EAAE,CAAC;QAC3D,MAAM,IAAI,GAAG,OAAO,CAAC,KAAK,CAAC,eAAe,EAAE,IAAI,CAAC,CAAC;QAClD,IAAI,CAAC,KAAK,CAAC,EAAE,CAAC,MAAM,CAAC,IAAI,CAAC,EAAE,CAAC;YAC3B,MAAM,IAAI,KAAK,CAAC,4BAA4B,IAAI,gCAAgC,IAAI,IAAI,CAAC,CAAC;QAC5F,CAAC;QACD,KAAK,MAAM,IAAI,IAAI,aAAa,CAAC,KAAK,CAAC,EAAE,CAAC,QAAQ,CAAC,IAAI,CAAC,CAAC;YAAE,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;IAC7E,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,gHAAgH;AAChH,MAAM,UAAU,eAAe,CAAC,MAAc;IAC5C,MAAM,QAAQ,GAAG,MAAM,CAAC,SAAS,EAAE,gBAAgB,CAAC;IACpD,IAAI,QAAQ,KAAK,SAAS;QAAE,OAAO,IAAI,GAAG,CAAC,QAAQ,CAAC,CAAC;IACrD,OAAO,IAAI,GAAG,CAAC,MAAM,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,MAAM,CAAC,QAAQ,KAAK,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;AACjG,CAAC"}

package/dist/dictionary/types.d.ts

@@ -3,6 +3,8 @@ import type { Finding } from "@concordance-wiki/core";
 export interface DictionarySource {
     id: string;
     source: string;
+    /** Path of the note in its source, so that a finding about the entity lands on it. */
+    path?: string;
     type: string;
     title: string;
     aliases: readonly string[];

package/dist/dictionary/types.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"types.d.ts","sourceRoot":"","sources":["../../src/dictionary/types.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,wBAAwB,CAAC;AAEtD,oGAAoG;AACpG,MAAM,WAAW,gBAAgB;IAC/B,EAAE,EAAE,MAAM,CAAC;IACX,MAAM,EAAE,MAAM,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,EAAE,MAAM,CAAC;IACd,OAAO,EAAE,SAAS,MAAM,EAAE,CAAC;IAC3B,MAAM,EAAE,MAAM,CAAC;CAChB;AAED,6FAA6F;AAC7F,MAAM,WAAW,gBAAgB;IAC/B,EAAE,EAAE,MAAM,CAAC;IACX,IAAI,EAAE,OAAO,GAAG,OAAO,CAAC;IACxB,+DAA+D;IAC/D,IAAI,EAAE,MAAM,CAAC;IACb,yDAAyD;IACzD,QAAQ,EAAE,MAAM,CAAC;CAClB;AAED,MAAM,WAAW,eAAe;IAC9B,0DAA0D;IAC1D,GAAG,EAAE,MAAM,CAAC;IACZ,8CAA8C;IAC9C,OAAO,EAAE,gBAAgB,EAAE,CAAC;IAC5B,4DAA4D;IAC5D,OAAO,EAAE,OAAO,CAAC;CAClB;AAED,MAAM,WAAW,UAAU;IACzB,MAAM,EAAE,MAAM,CAAC;IACf,gEAAgE;IAChE,OAAO,EAAE,WAAW,CAAC,MAAM,EAAE,eAAe,CAAC,CAAC;IAC9C,QAAQ,EAAE,OAAO,EAAE,CAAC;CACrB"}
+{"version":3,"file":"types.d.ts","sourceRoot":"","sources":["../../src/dictionary/types.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,wBAAwB,CAAC;AAEtD,oGAAoG;AACpG,MAAM,WAAW,gBAAgB;IAC/B,EAAE,EAAE,MAAM,CAAC;IACX,MAAM,EAAE,MAAM,CAAC;IACf,sFAAsF;IACtF,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,EAAE,MAAM,CAAC;IACd,OAAO,EAAE,SAAS,MAAM,EAAE,CAAC;IAC3B,MAAM,EAAE,MAAM,CAAC;CAChB;AAED,6FAA6F;AAC7F,MAAM,WAAW,gBAAgB;IAC/B,EAAE,EAAE,MAAM,CAAC;IACX,IAAI,EAAE,OAAO,GAAG,OAAO,CAAC;IACxB,+DAA+D;IAC/D,IAAI,EAAE,MAAM,CAAC;IACb,yDAAyD;IACzD,QAAQ,EAAE,MAAM,CAAC;CAClB;AAED,MAAM,WAAW,eAAe;IAC9B,0DAA0D;IAC1D,GAAG,EAAE,MAAM,CAAC;IACZ,8CAA8C;IAC9C,OAAO,EAAE,gBAAgB,EAAE,CAAC;IAC5B,4DAA4D;IAC5D,OAAO,EAAE,OAAO,CAAC;CAClB;AAED,MAAM,WAAW,UAAU;IACzB,MAAM,EAAE,MAAM,CAAC;IACf,gEAAgE;IAChE,OAAO,EAAE,WAAW,CAAC,MAAM,EAAE,eAAe,CAAC,CAAC;IAC9C,QAAQ,EAAE,OAAO,EAAE,CAAC;CACrB"}

package/dist/keywords/publish.d.ts

@@ -38,8 +38,9 @@ export declare function keywordPublicationOptions(config: Config): KeywordPublic
 /**
  * Splits the candidates into the pages to generate, the expressions the threshold
  * discards and those the confidence withholds. Two keys with the same slug would share an
- * address: the best-scored keeps the plain identifier and the next ones take a numeric
- * suffix, so that no page is lost.
+ * address: the first in code-unit order of the keys keeps the plain identifier and the next
+ * ones take a numeric suffix, so that no page is lost and the address of a key depends on the
+ * published keys alone, never on scores that move from one build to the next.
  */
 export declare function publishKeywords(candidates: readonly KeywordCandidate[], options: KeywordPublicationOptions): PublishedKeywords;
 /**

package/dist/keywords/publish.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"publish.d.ts","sourceRoot":"","sources":["../../src/keywords/publish.ts"],"names":[],"mappings":"AAAA,OAAO,EAAW,KAAK,MAAM,EAAE,KAAK,MAAM,EAAqB,MAAM,wBAAwB,CAAC;AAE9F,OAAO,KAAK,EAAE,gBAAgB,EAAE,cAAc,EAAE,MAAM,YAAY,CAAC;AAEnE,uFAAuF;AACvF,MAAM,WAAW,WAAW;IAC1B,oEAAoE;IACpE,EAAE,EAAE,MAAM,CAAC;IACX,GAAG,EAAE,MAAM,CAAC;IACZ,OAAO,EAAE,MAAM,CAAC;IAChB,WAAW,EAAE,MAAM,CAAC;IACpB,SAAS,EAAE,MAAM,CAAC;IAClB,KAAK,EAAE,MAAM,CAAC;IACd,UAAU,EAAE,MAAM,CAAC;IACnB,QAAQ,EAAE,cAAc,EAAE,CAAC;CAC5B;AAED,MAAM,WAAW,yBAAyB;IACxC,cAAc,EAAE,MAAM,CAAC;IACvB,oDAAoD;IACpD,QAAQ,EAAE,MAAM,CAAC;IACjB,oFAAoF;IACpF,aAAa,EAAE,MAAM,CAAC;CACvB;AAED,MAAM,WAAW,iBAAiB;IAChC,4BAA4B;IAC5B,SAAS,EAAE,WAAW,EAAE,CAAC;IACzB,6EAA6E;IAC7E,SAAS,EAAE,gBAAgB,EAAE,CAAC;IAC9B,gHAAgH;IAChH,QAAQ,EAAE,gBAAgB,EAAE,CAAC;CAC9B;AAED,MAAM,WAAW,sBAAsB;IACrC,MAAM,EAAE,MAAM,CAAC,QAAQ,CAAC,CAAC;CAC1B;AAED,eAAO,MAAM,0BAA0B,EAAE,yBAIxC,CAAC;AAEF,iEAAiE;AACjE,eAAO,MAAM,YAAY,SAAS,CAAC;AAanC,uHAAuH;AACvH,wBAAgB,yBAAyB,CAAC,MAAM,EAAE,MAAM,GAAG,yBAAyB,CAOnF;AASD;;;;;GAKG;AACH,wBAAgB,eAAe,CAC7B,UAAU,EAAE,SAAS,gBAAgB,EAAE,EACvC,OAAO,EAAE,yBAAyB,GACjC,iBAAiB,CA6BnB;AAUD;;;GAGG;AACH,wBAAgB,eAAe,CAC7B,KAAK,EAAE,SAAS,WAAW,EAAE,EAC7B,OAAO,EAAE,sBAAsB,GAC9B,MAAM,EAAE,CAmBV"}
+{"version":3,"file":"publish.d.ts","sourceRoot":"","sources":["../../src/keywords/publish.ts"],"names":[],"mappings":"AAAA,OAAO,EAAW,KAAK,MAAM,EAAE,KAAK,MAAM,EAAqB,MAAM,wBAAwB,CAAC;AAE9F,OAAO,KAAK,EAAE,gBAAgB,EAAE,cAAc,EAAE,MAAM,YAAY,CAAC;AAEnE,uFAAuF;AACvF,MAAM,WAAW,WAAW;IAC1B,oEAAoE;IACpE,EAAE,EAAE,MAAM,CAAC;IACX,GAAG,EAAE,MAAM,CAAC;IACZ,OAAO,EAAE,MAAM,CAAC;IAChB,WAAW,EAAE,MAAM,CAAC;IACpB,SAAS,EAAE,MAAM,CAAC;IAClB,KAAK,EAAE,MAAM,CAAC;IACd,UAAU,EAAE,MAAM,CAAC;IACnB,QAAQ,EAAE,cAAc,EAAE,CAAC;CAC5B;AAED,MAAM,WAAW,yBAAyB;IACxC,cAAc,EAAE,MAAM,CAAC;IACvB,oDAAoD;IACpD,QAAQ,EAAE,MAAM,CAAC;IACjB,oFAAoF;IACpF,aAAa,EAAE,MAAM,CAAC;CACvB;AAED,MAAM,WAAW,iBAAiB;IAChC,4BAA4B;IAC5B,SAAS,EAAE,WAAW,EAAE,CAAC;IACzB,6EAA6E;IAC7E,SAAS,EAAE,gBAAgB,EAAE,CAAC;IAC9B,gHAAgH;IAChH,QAAQ,EAAE,gBAAgB,EAAE,CAAC;CAC9B;AAED,MAAM,WAAW,sBAAsB;IACrC,MAAM,EAAE,MAAM,CAAC,QAAQ,CAAC,CAAC;CAC1B;AAED,eAAO,MAAM,0BAA0B,EAAE,yBAIxC,CAAC;AAEF,iEAAiE;AACjE,eAAO,MAAM,YAAY,SAAS,CAAC;AAanC,uHAAuH;AACvH,wBAAgB,yBAAyB,CAAC,MAAM,EAAE,MAAM,GAAG,yBAAyB,CAOnF;AASD;;;;;;GAMG;AACH,wBAAgB,eAAe,CAC7B,UAAU,EAAE,SAAS,gBAAgB,EAAE,EACvC,OAAO,EAAE,yBAAyB,GACjC,iBAAiB,CAiCnB;AAUD;;;GAGG;AACH,wBAAgB,eAAe,CAC7B,KAAK,EAAE,SAAS,WAAW,EAAE,EAC7B,OAAO,EAAE,sBAAsB,GAC9B,MAAM,EAAE,CAmBV"}

package/dist/keywords/publish.js

@@ -29,14 +29,15 @@ function reachesThreshold(candidate, options) {
 /**
  * Splits the candidates into the pages to generate, the expressions the threshold
  * discards and those the confidence withholds. Two keys with the same slug would share an
- * address: the best-scored keeps the plain identifier and the next ones take a numeric
- * suffix, so that no page is lost.
+ * address: the first in code-unit order of the keys keeps the plain identifier and the next
+ * ones take a numeric suffix, so that no page is lost and the address of a key depends on the
+ * published keys alone, never on scores that move from one build to the next.
  */
 export function publishKeywords(candidates, options) {
     const published = [];
     const discarded = [];
     const withheld = [];
-    const taken = new Map();
+    const publishable = [];
     for (const candidate of [...candidates].sort(compareByScore)) {
         if (!reachesThreshold(candidate, options)) {
             discarded.push(candidate);
@@ -46,6 +47,10 @@ export function publishKeywords(candidates, options) {
             withheld.push(candidate);
             continue;
         }
+        publishable.push(candidate);
+    }
+    const taken = new Map();
+    for (const candidate of publishable.toSorted((a, b) => byCodeUnit(a.key, b.key))) {
         const base = `keywords/${slugify(candidate.key)}`;
         const rank = (taken.get(base) ?? 0) + 1;
         taken.set(base, rank);

package/dist/keywords/publish.js.map

@@ -1 +1 @@
-{"version":3,"file":"publish.js","sourceRoot":"","sources":["../../src/keywords/publish.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAA+C,MAAM,wBAAwB,CAAC;AAsC9F,MAAM,CAAC,MAAM,0BAA0B,GAA8B;IACnE,cAAc,EAAE,CAAC;IACjB,QAAQ,EAAE,CAAC;IACX,aAAa,EAAE,GAAG;CACnB,CAAC;AAEF,iEAAiE;AACjE,MAAM,CAAC,MAAM,YAAY,GAAG,MAAM,CAAC;AAEnC,iEAAiE;AACjE,MAAM,cAAc,GAAG,OAAO,CAAC;AAE/B,SAAS,UAAU,CAAC,CAAS,EAAE,CAAS;IACtC,OAAO,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;AACvC,CAAC;AAED,SAAS,cAAc,CAAC,CAAmB,EAAE,CAAmB;IAC9D,OAAO,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,IAAI,UAAU,CAAC,CAAC,CAAC,GAAG,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC;AACvD,CAAC;AAED,uHAAuH;AACvH,MAAM,UAAU,yBAAyB,CAAC,MAAc;IACtD,MAAM,KAAK,GAAG,MAAM,CAAC,SAAS,EAAE,aAAa,IAAI,EAAE,CAAC;IACpD,OAAO;QACL,cAAc,EAAE,KAAK,CAAC,eAAe,IAAI,0BAA0B,CAAC,cAAc;QAClF,QAAQ,EAAE,KAAK,CAAC,SAAS,IAAI,0BAA0B,CAAC,QAAQ;QAChE,aAAa,EAAE,KAAK,CAAC,cAAc,IAAI,0BAA0B,CAAC,aAAa;KAChF,CAAC;AACJ,CAAC;AAED,SAAS,gBAAgB,CACvB,SAA2B,EAC3B,OAAkC;IAElC,OAAO,SAAS,CAAC,WAAW,IAAI,OAAO,CAAC,cAAc,IAAI,SAAS,CAAC,SAAS,IAAI,OAAO,CAAC,QAAQ,CAAC;AACpG,CAAC;AAED;;;;;GAKG;AACH,MAAM,UAAU,eAAe,CAC7B,UAAuC,EACvC,OAAkC;IAElC,MAAM,SAAS,GAAkB,EAAE,CAAC;IACpC,MAAM,SAAS,GAAuB,EAAE,CAAC;IACzC,MAAM,QAAQ,GAAuB,EAAE,CAAC;IACxC,MAAM,KAAK,GAAG,IAAI,GAAG,EAAkB,CAAC;IACxC,KAAK,MAAM,SAAS,IAAI,CAAC,GAAG,UAAU,CAAC,CAAC,IAAI,CAAC,cAAc,CAAC,EAAE,CAAC;QAC7D,IAAI,CAAC,gBAAgB,CAAC,SAAS,EAAE,OAAO,CAAC,EAAE,CAAC;YAC1C,SAAS,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;YAC1B,SAAS;QACX,CAAC;QACD,IAAI,SAAS,CAAC,UAAU,GAAG,OAAO,CAAC,aAAa,EAAE,CAAC;YACjD,QAAQ,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;YACzB,SAAS;QACX,CAAC;QACD,MAAM,IAAI,GAAG,YAAY,OAAO,CAAC,SAAS,CAAC,GAAG,CAAC,EAAE,CAAC;QAClD,MAAM,IAAI,GAAG,CAAC,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC;QACxC,KAAK,CAAC,GAAG,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;QACtB,SAAS,CAAC,IAAI,CAAC;YACb,EAAE,EAAE,IAAI,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,GAAG,IAAI,IAAI,MAAM,CAAC,IAAI,CAAC,EAAE;YACjD,GAAG,EAAE,SAAS,CAAC,GAAG;YAClB,OAAO,EAAE,SAAS,CAAC,OAAO;YAC1B,WAAW,EAAE,SAAS,CAAC,WAAW;YAClC,SAAS,EAAE,SAAS,CAAC,SAAS;YAC9B,KAAK,EAAE,SAAS,CAAC,KAAK;YACtB,UAAU,EAAE,SAAS,CAAC,UAAU;YAChC,QAAQ,EAAE,SAAS,CAAC,QAAQ;SAC7B,CAAC,CAAC;IACL,CAAC;IACD,OAAO,EAAE,SAAS,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,UAAU,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,SAAS,EAAE,QAAQ,EAAE,CAAC;AAClG,CAAC;AAED,SAAS,UAAU,CAAC,IAAiB;IACnC,MAAM,CAAC,KAAK,CAAC,GAAG,IAAI,CAAC,QAAQ,CAAC;IAC9B,IAAI,KAAK,KAAK,SAAS,EAAE,CAAC;QACxB,MAAM,IAAI,KAAK,CAAC,gBAAgB,IAAI,CAAC,EAAE,8BAA8B,CAAC,CAAC;IACzE,CAAC;IACD,OAAO,EAAE,IAAI,EAAE,KAAK,CAAC,MAAM,IAAI,EAAE,EAAE,IAAI,EAAE,KAAK,CAAC,IAAI,EAAE,IAAI,EAAE,KAAK,CAAC,IAAI,EAAE,CAAC;AAC1E,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,eAAe,CAC7B,KAA6B,EAC7B,OAA+B;IAE/B,OAAO,KAAK,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;QAC1B,EAAE,EAAE,IAAI,CAAC,EAAE;QACX,IAAI,EAAE,YAAY;QAClB,KAAK,EAAE,IAAI,CAAC,OAAO;QACnB,OAAO,EAAE,EAAE;QACX,MAAM,EAAE,OAAO,CAAC,MAAM;QACtB,MAAM,EAAE,cAAc;QACtB,WAAW,EAAE,SAAS;QACtB,KAAK,EAAE,MAAM;QACb,UAAU,EAAE;YACV,SAAS,EAAE,IAAI,CAAC,SAAS;YACzB,WAAW,EAAE,IAAI,CAAC,WAAW;YAC7B,KAAK,EAAE,IAAI,CAAC,KAAK;YACjB,UAAU,EAAE,IAAI,CAAC,UAAU;SAC5B;QACD,MAAM,EAAE,UAAU,CAAC,IAAI,CAAC;QACxB,OAAO,EAAE,IAAI;KACd,CAAC,CAAC,CAAC;AACN,CAAC"}
+{"version":3,"file":"publish.js","sourceRoot":"","sources":["../../src/keywords/publish.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,OAAO,EAA+C,MAAM,wBAAwB,CAAC;AAsC9F,MAAM,CAAC,MAAM,0BAA0B,GAA8B;IACnE,cAAc,EAAE,CAAC;IACjB,QAAQ,EAAE,CAAC;IACX,aAAa,EAAE,GAAG;CACnB,CAAC;AAEF,iEAAiE;AACjE,MAAM,CAAC,MAAM,YAAY,GAAG,MAAM,CAAC;AAEnC,iEAAiE;AACjE,MAAM,cAAc,GAAG,OAAO,CAAC;AAE/B,SAAS,UAAU,CAAC,CAAS,EAAE,CAAS;IACtC,OAAO,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;AACvC,CAAC;AAED,SAAS,cAAc,CAAC,CAAmB,EAAE,CAAmB;IAC9D,OAAO,CAAC,CAAC,KAAK,GAAG,CAAC,CAAC,KAAK,IAAI,UAAU,CAAC,CAAC,CAAC,GAAG,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC;AACvD,CAAC;AAED,uHAAuH;AACvH,MAAM,UAAU,yBAAyB,CAAC,MAAc;IACtD,MAAM,KAAK,GAAG,MAAM,CAAC,SAAS,EAAE,aAAa,IAAI,EAAE,CAAC;IACpD,OAAO;QACL,cAAc,EAAE,KAAK,CAAC,eAAe,IAAI,0BAA0B,CAAC,cAAc;QAClF,QAAQ,EAAE,KAAK,CAAC,SAAS,IAAI,0BAA0B,CAAC,QAAQ;QAChE,aAAa,EAAE,KAAK,CAAC,cAAc,IAAI,0BAA0B,CAAC,aAAa;KAChF,CAAC;AACJ,CAAC;AAED,SAAS,gBAAgB,CACvB,SAA2B,EAC3B,OAAkC;IAElC,OAAO,SAAS,CAAC,WAAW,IAAI,OAAO,CAAC,cAAc,IAAI,SAAS,CAAC,SAAS,IAAI,OAAO,CAAC,QAAQ,CAAC;AACpG,CAAC;AAED;;;;;;GAMG;AACH,MAAM,UAAU,eAAe,CAC7B,UAAuC,EACvC,OAAkC;IAElC,MAAM,SAAS,GAAkB,EAAE,CAAC;IACpC,MAAM,SAAS,GAAuB,EAAE,CAAC;IACzC,MAAM,QAAQ,GAAuB,EAAE,CAAC;IACxC,MAAM,WAAW,GAAuB,EAAE,CAAC;IAC3C,KAAK,MAAM,SAAS,IAAI,CAAC,GAAG,UAAU,CAAC,CAAC,IAAI,CAAC,cAAc,CAAC,EAAE,CAAC;QAC7D,IAAI,CAAC,gBAAgB,CAAC,SAAS,EAAE,OAAO,CAAC,EAAE,CAAC;YAC1C,SAAS,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;YAC1B,SAAS;QACX,CAAC;QACD,IAAI,SAAS,CAAC,UAAU,GAAG,OAAO,CAAC,aAAa,EAAE,CAAC;YACjD,QAAQ,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;YACzB,SAAS;QACX,CAAC;QACD,WAAW,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;IAC9B,CAAC;IACD,MAAM,KAAK,GAAG,IAAI,GAAG,EAAkB,CAAC;IACxC,KAAK,MAAM,SAAS,IAAI,WAAW,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,UAAU,CAAC,CAAC,CAAC,GAAG,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC;QACjF,MAAM,IAAI,GAAG,YAAY,OAAO,CAAC,SAAS,CAAC,GAAG,CAAC,EAAE,CAAC;QAClD,MAAM,IAAI,GAAG,CAAC,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC;QACxC,KAAK,CAAC,GAAG,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;QACtB,SAAS,CAAC,IAAI,CAAC;YACb,EAAE,EAAE,IAAI,KAAK,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,GAAG,IAAI,IAAI,MAAM,CAAC,IAAI,CAAC,EAAE;YACjD,GAAG,EAAE,SAAS,CAAC,GAAG;YAClB,OAAO,EAAE,SAAS,CAAC,OAAO;YAC1B,WAAW,EAAE,SAAS,CAAC,WAAW;YAClC,SAAS,EAAE,SAAS,CAAC,SAAS;YAC9B,KAAK,EAAE,SAAS,CAAC,KAAK;YACtB,UAAU,EAAE,SAAS,CAAC,UAAU;YAChC,QAAQ,EAAE,SAAS,CAAC,QAAQ;SAC7B,CAAC,CAAC;IACL,CAAC;IACD,OAAO,EAAE,SAAS,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,UAAU,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,SAAS,EAAE,QAAQ,EAAE,CAAC;AAClG,CAAC;AAED,SAAS,UAAU,CAAC,IAAiB;IACnC,MAAM,CAAC,KAAK,CAAC,GAAG,IAAI,CAAC,QAAQ,CAAC;IAC9B,IAAI,KAAK,KAAK,SAAS,EAAE,CAAC;QACxB,MAAM,IAAI,KAAK,CAAC,gBAAgB,IAAI,CAAC,EAAE,8BAA8B,CAAC,CAAC;IACzE,CAAC;IACD,OAAO,EAAE,IAAI,EAAE,KAAK,CAAC,MAAM,IAAI,EAAE,EAAE,IAAI,EAAE,KAAK,CAAC,IAAI,EAAE,IAAI,EAAE,KAAK,CAAC,IAAI,EAAE,CAAC;AAC1E,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,eAAe,CAC7B,KAA6B,EAC7B,OAA+B;IAE/B,OAAO,KAAK,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;QAC1B,EAAE,EAAE,IAAI,CAAC,EAAE;QACX,IAAI,EAAE,YAAY;QAClB,KAAK,EAAE,IAAI,CAAC,OAAO;QACnB,OAAO,EAAE,EAAE;QACX,MAAM,EAAE,OAAO,CAAC,MAAM;QACtB,MAAM,EAAE,cAAc;QACtB,WAAW,EAAE,SAAS;QACtB,KAAK,EAAE,MAAM;QACb,UAAU,EAAE;YACV,SAAS,EAAE,IAAI,CAAC,SAAS;YACzB,WAAW,EAAE,IAAI,CAAC,WAAW;YAC7B,KAAK,EAAE,IAAI,CAAC,KAAK;YACjB,UAAU,EAAE,IAAI,CAAC,UAAU;SAC5B;QACD,MAAM,EAAE,UAAU,CAAC,IAAI,CAAC;QACxB,OAAO,EAAE,IAAI;KACd,CAAC,CAAC,CAAC;AACN,CAAC"}

package/dist/locale/load-pack.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"load-pack.d.ts","sourceRoot":"","sources":["../../src/locale/load-pack.ts"],"names":[],"mappings":"AAOA,OAAO,KAAK,EAAE,YAAY,EAAoB,MAAM,WAAW,CAAC;AAahE,qBAAa,iBAAkB,SAAQ,KAAK;gBAC9B,SAAS,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM;CAI9C;AAoCD;;;;GAIG;AACH,wBAAgB,gBAAgB,CAAC,SAAS,EAAE,GAAG,GAAG,YAAY,CA4B7D"}
+{"version":3,"file":"load-pack.d.ts","sourceRoot":"","sources":["../../src/locale/load-pack.ts"],"names":[],"mappings":"AAcA,OAAO,KAAK,EAAE,YAAY,EAAoB,MAAM,WAAW,CAAC;AAahE,qBAAa,iBAAkB,SAAQ,KAAK;gBAC9B,SAAS,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM;CAI9C;AAkCD;;;;GAIG;AACH,wBAAgB,gBAAgB,CAAC,SAAS,EAAE,GAAG,GAAG,YAAY,CA4B7D"}

package/dist/locale/load-pack.js

@@ -1,6 +1,5 @@
 import { readFileSync } from "node:fs";
-import { describeSchemaError, formatIssue, readSchema } from "@concordance-wiki/core";
-import { Ajv2020 } from "ajv/dist/2020.js";
+import { collation, describeSchemaError, formatIssue, compiledSchema, } from "@concordance-wiki/core";
 import { parse } from "yaml";
 import { normalizer } from "./normalize.js";
 import { canonicalLocale } from "./tag.js";
@@ -13,8 +12,7 @@ export class LanguagePackError extends Error {
     }
 }
 function readDocument(directory) {
-    const ajv = new Ajv2020({ allErrors: true, strict: true });
-    const validate = ajv.compile(readSchema("language-pack"));
+    const validate = compiledSchema("language-pack");
     const document = parse(readFileSync(new URL("pack.yaml", directory), "utf8"));
     if (!validate(document)) {
         // The validator fills `errors` whenever it returns false.
@@ -23,7 +21,6 @@ function readDocument(directory) {
             .join("; ");
         throw new LanguagePackError(directory.pathname, detail);
     }
-    // Validated against the schema the type mirrors.
     return document;
 }
 function canonical(tag, directory) {
@@ -55,7 +52,7 @@ export function loadLanguagePack(directory) {
     const base = directory.href.endsWith("/") ? directory : new URL(`${directory.href}/`);
     const document = readDocument(base);
     const locale = canonical(document.locale, base);
-    const collator = new Intl.Collator(locale, document.collation);
+    const compare = collation(document.collation);
     const segmenter = new Intl.Segmenter(locale, { granularity: "word" });
     const normalize = normalizer(document.apostrophes ?? []);
     const plural = document.plural.map((rule) => ({
@@ -75,8 +72,8 @@ export function loadLanguagePack(directory) {
         stopwords: new Set(loadStopwords(readFileSync(new URL("stopwords.txt", base), "utf8"))),
         suffixes: new Set(loadSuffixes(optionalText(new URL("suffixes.txt", base)), normalize)),
         plural,
-        collator,
-        compare: (a, b) => collator.compare(a, b),
+        collation: document.collation,
+        compare,
     };
 }
 //# sourceMappingURL=load-pack.js.map

package/dist/locale/load-pack.js.map

@@ -1 +1 @@
-{"version":3,"file":"load-pack.js","sourceRoot":"","sources":["../../src/locale/load-pack.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,MAAM,SAAS,CAAC;AAEvC,OAAO,EAAE,mBAAmB,EAAE,WAAW,EAAE,UAAU,EAAe,MAAM,wBAAwB,CAAC;AACnG,OAAO,EAAE,OAAO,EAAoB,MAAM,kBAAkB,CAAC;AAC7D,OAAO,EAAE,KAAK,EAAE,MAAM,MAAM,CAAC;AAE7B,OAAO,EAAE,UAAU,EAAE,MAAM,gBAAgB,CAAC;AAE5C,OAAO,EAAE,eAAe,EAAE,MAAM,UAAU,CAAC;AAC3C,OAAO,EAAE,aAAa,EAAE,MAAM,gBAAgB,CAAC;AAC/C,OAAO,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC;AAU7C,MAAM,OAAO,iBAAkB,SAAQ,KAAK;IAC1C,YAAY,SAAiB,EAAE,MAAc;QAC3C,KAAK,CAAC,oBAAoB,SAAS,KAAK,MAAM,EAAE,CAAC,CAAC;QAClD,IAAI,CAAC,IAAI,GAAG,mBAAmB,CAAC;IAClC,CAAC;CACF;AAED,SAAS,YAAY,CAAC,SAAc;IAClC,MAAM,GAAG,GAAG,IAAI,OAAO,CAAC,EAAE,SAAS,EAAE,IAAI,EAAE,MAAM,EAAE,IAAI,EAAE,CAAC,CAAC;IAC3D,MAAM,QAAQ,GAAG,GAAG,CAAC,OAAO,CAAe,UAAU,CAAC,eAAe,CAAC,CAAC,CAAC;IACxE,MAAM,QAAQ,GAAY,KAAK,CAAC,YAAY,CAAC,IAAI,GAAG,CAAC,WAAW,EAAE,SAAS,CAAC,EAAE,MAAM,CAAC,CAAC,CAAC;IACvF,IAAI,CAAC,QAAQ,CAAC,QAAQ,CAAC,EAAE,CAAC;QACxB,0DAA0D;QAC1D,MAAM,MAAM,GAAI,QAAQ,CAAC,MAAwB;aAC9C,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,WAAW,CAAC,mBAAmB,CAAC,KAAK,EAAE,QAAQ,CAAC,EAAE,WAAW,CAAC,CAAC;aAC9E,IAAI,CAAC,IAAI,CAAC,CAAC;QACd,MAAM,IAAI,iBAAiB,CAAC,SAAS,CAAC,QAAQ,EAAE,MAAM,CAAC,CAAC;IAC1D,CAAC;IACD,iDAAiD;IACjD,OAAO,QAAwB,CAAC;AAClC,CAAC;AAED,SAAS,SAAS,CAAC,GAAW,EAAE,SAAc;IAC5C,IAAI,CAAC;QACH,OAAO,eAAe,CAAC,GAAG,CAAC,CAAC;IAC9B,CAAC;IAAC,MAAM,CAAC;QACP,MAAM,IAAI,iBAAiB,CAAC,SAAS,CAAC,QAAQ,EAAE,IAAI,GAAG,+BAA+B,CAAC,CAAC;IAC1F,CAAC;AACH,CAAC;AAED,4GAA4G;AAC5G,SAAS,YAAY,CAAC,GAAQ;IAC5B,IAAI,CAAC;QACH,OAAO,YAAY,CAAC,GAAG,EAAE,MAAM,CAAC,CAAC;IACnC,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,uFAAuF;QACvF,IAAK,KAA4B,CAAC,IAAI,KAAK,QAAQ;YAAE,OAAO,EAAE,CAAC;QAC/D,MAAM,KAAK,CAAC;IACd,CAAC;AACH,CAAC;AAED;;;;GAIG;AACH,MAAM,UAAU,gBAAgB,CAAC,SAAc;IAC7C,MAAM,IAAI,GAAG,SAAS,CAAC,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,GAAG,CAAC,GAAG,SAAS,CAAC,IAAI,GAAG,CAAC,CAAC;IACtF,MAAM,QAAQ,GAAG,YAAY,CAAC,IAAI,CAAC,CAAC;IACpC,MAAM,MAAM,GAAG,SAAS,CAAC,QAAQ,CAAC,MAAM,EAAE,IAAI,CAAC,CAAC;IAChD,MAAM,QAAQ,GAAG,IAAI,IAAI,CAAC,QAAQ,CAAC,MAAM,EAAE,QAAQ,CAAC,SAAS,CAAC,CAAC;IAC/D,MAAM,SAAS,GAAG,IAAI,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,EAAE,WAAW,EAAE,MAAM,EAAE,CAAC,CAAC;IACtE,MAAM,SAAS,GAAG,UAAU,CAAC,QAAQ,CAAC,WAAW,IAAI,EAAE,CAAC,CAAC;IACzD,MAAM,MAAM,GAAiB,QAAQ,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;QAC1D,MAAM,EAAE,IAAI,CAAC,MAAM;QACnB,QAAQ,EAAE,IAAI,CAAC,QAAQ;QACvB,SAAS,EAAE,IAAI,CAAC,UAAU,IAAI,IAAI,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC;KACrD,CAAC,CAAC,CAAC;IACJ,OAAO;QACL,MAAM;QACN,QAAQ,EAAE,QAAQ,CAAC,QAAQ;QAC3B,SAAS;QACT,OAAO,EAAE,CAAC,IAAI,EAAU,EAAE,CACxB,CAAC,GAAG,SAAS,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;YAC1C,IAAI,EAAE,IAAI,CAAC,OAAO;YAClB,KAAK,EAAE,IAAI,CAAC,KAAK;YACjB,UAAU,EAAE,IAAI,CAAC,UAAU,KAAK,IAAI;SACrC,CAAC,CAAC;QACL,SAAS,EAAE,IAAI,GAAG,CAAC,aAAa,CAAC,YAAY,CAAC,IAAI,GAAG,CAAC,eAAe,EAAE,IAAI,CAAC,EAAE,MAAM,CAAC,CAAC,CAAC;QACvF,QAAQ,EAAE,IAAI,GAAG,CAAC,YAAY,CAAC,YAAY,CAAC,IAAI,GAAG,CAAC,cAAc,EAAE,IAAI,CAAC,CAAC,EAAE,SAAS,CAAC,CAAC;QACvF,MAAM;QACN,QAAQ;QACR,OAAO,EAAE,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC,EAAE,CAAC,CAAC;KAC1C,CAAC;AACJ,CAAC"}
+{"version":3,"file":"load-pack.js","sourceRoot":"","sources":["../../src/locale/load-pack.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,MAAM,SAAS,CAAC;AAEvC,OAAO,EACL,SAAS,EACT,mBAAmB,EACnB,WAAW,EACX,cAAc,GAGf,MAAM,wBAAwB,CAAC;AAEhC,OAAO,EAAE,KAAK,EAAE,MAAM,MAAM,CAAC;AAE7B,OAAO,EAAE,UAAU,EAAE,MAAM,gBAAgB,CAAC;AAE5C,OAAO,EAAE,eAAe,EAAE,MAAM,UAAU,CAAC;AAC3C,OAAO,EAAE,aAAa,EAAE,MAAM,gBAAgB,CAAC;AAC/C,OAAO,EAAE,YAAY,EAAE,MAAM,eAAe,CAAC;AAU7C,MAAM,OAAO,iBAAkB,SAAQ,KAAK;IAC1C,YAAY,SAAiB,EAAE,MAAc;QAC3C,KAAK,CAAC,oBAAoB,SAAS,KAAK,MAAM,EAAE,CAAC,CAAC;QAClD,IAAI,CAAC,IAAI,GAAG,mBAAmB,CAAC;IAClC,CAAC;CACF;AAED,SAAS,YAAY,CAAC,SAAc;IAClC,MAAM,QAAQ,GAAG,cAAc,CAAe,eAAe,CAAC,CAAC;IAC/D,MAAM,QAAQ,GAAY,KAAK,CAAC,YAAY,CAAC,IAAI,GAAG,CAAC,WAAW,EAAE,SAAS,CAAC,EAAE,MAAM,CAAC,CAAC,CAAC;IACvF,IAAI,CAAC,QAAQ,CAAC,QAAQ,CAAC,EAAE,CAAC;QACxB,0DAA0D;QAC1D,MAAM,MAAM,GAAI,QAAQ,CAAC,MAAwB;aAC9C,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,WAAW,CAAC,mBAAmB,CAAC,KAAK,EAAE,QAAQ,CAAC,EAAE,WAAW,CAAC,CAAC;aAC9E,IAAI,CAAC,IAAI,CAAC,CAAC;QACd,MAAM,IAAI,iBAAiB,CAAC,SAAS,CAAC,QAAQ,EAAE,MAAM,CAAC,CAAC;IAC1D,CAAC;IACD,OAAO,QAAQ,CAAC;AAClB,CAAC;AAED,SAAS,SAAS,CAAC,GAAW,EAAE,SAAc;IAC5C,IAAI,CAAC;QACH,OAAO,eAAe,CAAC,GAAG,CAAC,CAAC;IAC9B,CAAC;IAAC,MAAM,CAAC;QACP,MAAM,IAAI,iBAAiB,CAAC,SAAS,CAAC,QAAQ,EAAE,IAAI,GAAG,+BAA+B,CAAC,CAAC;IAC1F,CAAC;AACH,CAAC;AAED,4GAA4G;AAC5G,SAAS,YAAY,CAAC,GAAQ;IAC5B,IAAI,CAAC;QACH,OAAO,YAAY,CAAC,GAAG,EAAE,MAAM,CAAC,CAAC;IACnC,CAAC;IAAC,OAAO,KAAK,EAAE,CAAC;QACf,uFAAuF;QACvF,IAAK,KAA4B,CAAC,IAAI,KAAK,QAAQ;YAAE,OAAO,EAAE,CAAC;QAC/D,MAAM,KAAK,CAAC;IACd,CAAC;AACH,CAAC;AAED;;;;GAIG;AACH,MAAM,UAAU,gBAAgB,CAAC,SAAc;IAC7C,MAAM,IAAI,GAAG,SAAS,CAAC,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,GAAG,CAAC,GAAG,SAAS,CAAC,IAAI,GAAG,CAAC,CAAC;IACtF,MAAM,QAAQ,GAAG,YAAY,CAAC,IAAI,CAAC,CAAC;IACpC,MAAM,MAAM,GAAG,SAAS,CAAC,QAAQ,CAAC,MAAM,EAAE,IAAI,CAAC,CAAC;IAChD,MAAM,OAAO,GAAG,SAAS,CAAC,QAAQ,CAAC,SAAS,CAAC,CAAC;IAC9C,MAAM,SAAS,GAAG,IAAI,IAAI,CAAC,SAAS,CAAC,MAAM,EAAE,EAAE,WAAW,EAAE,MAAM,EAAE,CAAC,CAAC;IACtE,MAAM,SAAS,GAAG,UAAU,CAAC,QAAQ,CAAC,WAAW,IAAI,EAAE,CAAC,CAAC;IACzD,MAAM,MAAM,GAAiB,QAAQ,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;QAC1D,MAAM,EAAE,IAAI,CAAC,MAAM;QACnB,QAAQ,EAAE,IAAI,CAAC,QAAQ;QACvB,SAAS,EAAE,IAAI,CAAC,UAAU,IAAI,IAAI,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC;KACrD,CAAC,CAAC,CAAC;IACJ,OAAO;QACL,MAAM;QACN,QAAQ,EAAE,QAAQ,CAAC,QAAQ;QAC3B,SAAS;QACT,OAAO,EAAE,CAAC,IAAI,EAAU,EAAE,CACxB,CAAC,GAAG,SAAS,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,CAAC;YAC1C,IAAI,EAAE,IAAI,CAAC,OAAO;YAClB,KAAK,EAAE,IAAI,CAAC,KAAK;YACjB,UAAU,EAAE,IAAI,CAAC,UAAU,KAAK,IAAI;SACrC,CAAC,CAAC;QACL,SAAS,EAAE,IAAI,GAAG,CAAC,aAAa,CAAC,YAAY,CAAC,IAAI,GAAG,CAAC,eAAe,EAAE,IAAI,CAAC,EAAE,MAAM,CAAC,CAAC,CAAC;QACvF,QAAQ,EAAE,IAAI,GAAG,CAAC,YAAY,CAAC,YAAY,CAAC,IAAI,GAAG,CAAC,cAAc,EAAE,IAAI,CAAC,CAAC,EAAE,SAAS,CAAC,CAAC;QACvF,MAAM;QACN,SAAS,EAAE,QAAQ,CAAC,SAAS;QAC7B,OAAO;KACR,CAAC;AACJ,CAAC"}

package/dist/locale/pack.d.ts

@@ -1,3 +1,4 @@
+import type { CollationOptions } from "@concordance-wiki/core";
 import type { Locale } from "@concordance-wiki/core";
 /** A suffix rule bringing a plural form back to its singular. */
 export interface PluralRule {
@@ -26,8 +27,8 @@ export interface LanguagePack {
     /** Endings of the inflected forms of the language, in comparison form; empty when the pack lists none. */
     suffixes: ReadonlySet<string>;
     plural: readonly PluralRule[];
-    /** Locale-aware, accent-insensitive and numeric, for every alphabetical index. */
-    collator: Intl.Collator;
+    /** What every alphabetical index of the locale sets aside, as the pack declares it: accents and case, digits compared by value, for the shipped packs. */
+    collation: CollationOptions;
     /** Usable directly as a sort comparator. */
     compare: (a: string, b: string) => number;
 }

package/dist/locale/pack.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"pack.d.ts","sourceRoot":"","sources":["../../src/locale/pack.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,wBAAwB,CAAC;AAErD,iEAAiE;AACjE,MAAM,WAAW,UAAU;IACzB,MAAM,EAAE,MAAM,CAAC;IACf,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,EAAE,MAAM,CAAC;CACnB;AAED,yDAAyD;AACzD,MAAM,WAAW,IAAI;IACnB,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,EAAE,MAAM,CAAC;IACd,wCAAwC;IACxC,UAAU,EAAE,OAAO,CAAC;CACrB;AAED,6GAA6G;AAC7G,MAAM,WAAW,YAAY;IAC3B,4BAA4B;IAC5B,MAAM,EAAE,MAAM,CAAC;IACf,8CAA8C;IAC9C,QAAQ,EAAE,MAAM,CAAC;IACjB,6FAA6F;IAC7F,SAAS,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,MAAM,CAAC;IACpC,6EAA6E;IAC7E,OAAO,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,IAAI,EAAE,CAAC;IAClC,SAAS,EAAE,WAAW,CAAC,MAAM,CAAC,CAAC;IAC/B,0GAA0G;IAC1G,QAAQ,EAAE,WAAW,CAAC,MAAM,CAAC,CAAC;IAC9B,MAAM,EAAE,SAAS,UAAU,EAAE,CAAC;IAC9B,kFAAkF;IAClF,QAAQ,EAAE,IAAI,CAAC,QAAQ,CAAC;IACxB,4CAA4C;IAC5C,OAAO,EAAE,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,EAAE,MAAM,KAAK,MAAM,CAAC;CAC3C"}
+{"version":3,"file":"pack.d.ts","sourceRoot":"","sources":["../../src/locale/pack.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,wBAAwB,CAAC;AAC/D,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,wBAAwB,CAAC;AAErD,iEAAiE;AACjE,MAAM,WAAW,UAAU;IACzB,MAAM,EAAE,MAAM,CAAC;IACf,QAAQ,EAAE,MAAM,CAAC;IACjB,SAAS,EAAE,MAAM,CAAC;CACnB;AAED,yDAAyD;AACzD,MAAM,WAAW,IAAI;IACnB,IAAI,EAAE,MAAM,CAAC;IACb,KAAK,EAAE,MAAM,CAAC;IACd,wCAAwC;IACxC,UAAU,EAAE,OAAO,CAAC;CACrB;AAED,6GAA6G;AAC7G,MAAM,WAAW,YAAY;IAC3B,4BAA4B;IAC5B,MAAM,EAAE,MAAM,CAAC;IACf,8CAA8C;IAC9C,QAAQ,EAAE,MAAM,CAAC;IACjB,6FAA6F;IAC7F,SAAS,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,MAAM,CAAC;IACpC,6EAA6E;IAC7E,OAAO,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,IAAI,EAAE,CAAC;IAClC,SAAS,EAAE,WAAW,CAAC,MAAM,CAAC,CAAC;IAC/B,0GAA0G;IAC1G,QAAQ,EAAE,WAAW,CAAC,MAAM,CAAC,CAAC;IAC9B,MAAM,EAAE,SAAS,UAAU,EAAE,CAAC;IAC9B,0JAA0J;IAC1J,SAAS,EAAE,gBAAgB,CAAC;IAC5B,4CAA4C;IAC5C,OAAO,EAAE,CAAC,CAAC,EAAE,MAAM,EAAE,CAAC,EAAE,MAAM,KAAK,MAAM,CAAC;CAC3C"}

package/dist/scan/occurrences.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"occurrences.d.ts","sourceRoot":"","sources":["../../src/scan/occurrences.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAmB,gBAAgB,EAAE,MAAM,wBAAwB,CAAC;AAC5F,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,mBAAmB,CAAC;AAQtD,OAAO,EAAqB,KAAK,UAAU,EAAE,MAAM,cAAc,CAAC;AAGlE,qFAAqF;AACrF,MAAM,WAAW,eAAe;IAC9B,IAAI,EAAE,MAAM,CAAC;IACb,cAAc,EAAE,MAAM,CAAC;IACvB,GAAG,EAAE,MAAM,CAAC;IACZ,cAAc,EAAE,MAAM,CAAC;IACvB,iBAAiB,EAAE,MAAM,CAAC;CAC3B;AAED,MAAM,WAAW,gBAAgB;IAC/B,IAAI,EAAE,MAAM,CAAC;IACb,qDAAqD;IACrD,IAAI,EAAE,MAAM,CAAC;IACb,kDAAkD;IAClD,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,uGAAuG;IACvG,IAAI,CAAC,EAAE,SAAS,UAAU,EAAE,CAAC;CAC9B;AAED,2FAA2F;AAC3F,MAAM,WAAW,eAAe;IAC9B,IAAI,EAAE,MAAM,CAAC;IACb,UAAU,EAAE,SAAS,gBAAgB,EAAE,CAAC;CACzC;AAED,MAAM,WAAW,UAAU;IACzB,uCAAuC;IACvC,GAAG,EAAE,MAAM,CAAC;IACZ,MAAM,EAAE,IAAI,CAAC,gBAAgB,EAAE,IAAI,GAAG,MAAM,CAAC,CAAC;IAC9C,MAAM,EAAE,MAAM,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,IAAI,EAAE,MAAM,CAAC;IACb,2DAA2D;IAC3D,QAAQ,EAAE,MAAM,CAAC;IACjB,8DAA8D;IAC9D,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,sHAAsH;IACtH,OAAO,EAAE,MAAM,CAAC;IAChB,4FAA4F;IAC5F,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,UAAU,EAAE,MAAM,CAAC;CACpB;AAED,MAAM,WAAW,iBAAiB;IAChC,QAAQ,EAAE,eAAe,CAAC;IAC1B,MAAM,EAAE,MAAM,CAAC;IACf,UAAU,EAAE,UAAU,CAAC;IACvB,IAAI,EAAE,YAAY,CAAC;IACnB,8EAA8E;IAC9E,YAAY,EAAE,QAAQ,CAAC,MAAM,CAAC,MAAM,EAAE,SAAS,MAAM,EAAE,CAAC,CAAC,CAAC;IAC1D,KAAK,EAAE,eAAe,CAAC;CACxB;AA4DD,8EAA8E;AAC9E,wBAAgB,kBAAkB,CAAC,CAAC,EAAE,UAAU,EAAE,CAAC,EAAE,UAAU,GAAG,MAAM,CAQvE;AAqCD;;;;;GAKG;AACH,wBAAgB,YAAY,CAAC,KAAK,EAAE,iBAAiB,GAAG,UAAU,EAAE,CAanE;AAED,kGAAkG;AAClG,wBAAgB,oBAAoB,CAAC,KAAK,EAAE,MAAM,EAAE,KAAK,EAAE,eAAe,GAAG,MAAM,CAElF"}
+{"version":3,"file":"occurrences.d.ts","sourceRoot":"","sources":["../../src/scan/occurrences.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAmB,gBAAgB,EAAE,MAAM,wBAAwB,CAAC;AAC5F,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,mBAAmB,CAAC;AAQtD,OAAO,EAAqB,KAAK,UAAU,EAAE,MAAM,cAAc,CAAC;AAGlE,qFAAqF;AACrF,MAAM,WAAW,eAAe;IAC9B,IAAI,EAAE,MAAM,CAAC;IACb,cAAc,EAAE,MAAM,CAAC;IACvB,GAAG,EAAE,MAAM,CAAC;IACZ,cAAc,EAAE,MAAM,CAAC;IACvB,iBAAiB,EAAE,MAAM,CAAC;CAC3B;AAED,MAAM,WAAW,gBAAgB;IAC/B,IAAI,EAAE,MAAM,CAAC;IACb,qDAAqD;IACrD,IAAI,EAAE,MAAM,CAAC;IACb,kDAAkD;IAClD,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,uGAAuG;IACvG,IAAI,CAAC,EAAE,SAAS,UAAU,EAAE,CAAC;CAC9B;AAED,2FAA2F;AAC3F,MAAM,WAAW,eAAe;IAC9B,IAAI,EAAE,MAAM,CAAC;IACb,UAAU,EAAE,SAAS,gBAAgB,EAAE,CAAC;CACzC;AAED,MAAM,WAAW,UAAU;IACzB,uCAAuC;IACvC,GAAG,EAAE,MAAM,CAAC;IACZ,MAAM,EAAE,IAAI,CAAC,gBAAgB,EAAE,IAAI,GAAG,MAAM,CAAC,CAAC;IAC9C,MAAM,EAAE,MAAM,CAAC;IACf,IAAI,EAAE,MAAM,CAAC;IACb,IAAI,EAAE,MAAM,CAAC;IACb,2DAA2D;IAC3D,QAAQ,EAAE,MAAM,CAAC;IACjB,8DAA8D;IAC9D,IAAI,EAAE,MAAM,CAAC;IACb,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,sHAAsH;IACtH,OAAO,EAAE,MAAM,CAAC;IAChB,4FAA4F;IAC5F,YAAY,CAAC,EAAE,MAAM,CAAC;IACtB,UAAU,EAAE,MAAM,CAAC;CACpB;AAED,MAAM,WAAW,iBAAiB;IAChC,QAAQ,EAAE,eAAe,CAAC;IAC1B,MAAM,EAAE,MAAM,CAAC;IACf,UAAU,EAAE,UAAU,CAAC;IACvB,IAAI,EAAE,YAAY,CAAC;IACnB,8EAA8E;IAC9E,YAAY,EAAE,QAAQ,CAAC,MAAM,CAAC,MAAM,EAAE,SAAS,MAAM,EAAE,CAAC,CAAC,CAAC;IAC1D,KAAK,EAAE,eAAe,CAAC;CACxB;AAoGD,8EAA8E;AAC9E,wBAAgB,kBAAkB,CAAC,CAAC,EAAE,UAAU,EAAE,CAAC,EAAE,UAAU,GAAG,MAAM,CAQvE;AAoCD;;;;;GAKG;AACH,wBAAgB,YAAY,CAAC,KAAK,EAAE,iBAAiB,GAAG,UAAU,EAAE,CAanE;AAED,kGAAkG;AAClG,wBAAgB,oBAAoB,CAAC,KAAK,EAAE,MAAM,EAAE,KAAK,EAAE,eAAe,GAAG,MAAM,CAElF"}

package/dist/scan/occurrences.js

@@ -25,20 +25,48 @@ function automatonOf(dictionary, pack) {
     return automaton;
 }
 /**
- * Comparison form of every prefix word to the type it announces; a word listed under several
- * types announces none. A prefix is a single word: the token right before the match.
+ * Comparison form of every prefix to the type it announces; a prefix listed under several types
+ * announces none. A prefix is one word or several, the tokens right before the match.
  */
 function prefixTypes(typePrefixes, pack) {
     const types = new Map();
+    let longest = 0;
     for (const [type, words] of Object.entries(typePrefixes)) {
         for (const prefix of words) {
-            const word = tokenize(prefix, pack)
-                .map((token) => token.word)
-                .join(" ");
+            const tokens = tokenize(prefix, pack).map((token) => token.word);
+            const word = tokens.join(" ");
+            longest = Math.max(longest, tokens.length);
             types.set(word, types.has(word) ? undefined : type);
         }
     }
-    return types;
+    return { types, longest };
+}
+const BLANK = /^\s*$/u;
+/** Whether nothing but spaces separates the tokens from one another and from the match. */
+function contiguous(text, tokens, matchStart) {
+    const ends = tokens.map((token) => token.end);
+    const starts = [...tokens.slice(1).map((token) => token.start), matchStart];
+    return ends.every((end, index) => BLANK.test(text.slice(end, starts[index])));
+}
+/**
+ * The type the words right before a match announce, the longest prefix first: `data object
+ * Resource` announces a data object where `object Resource` alone would not. A prefix separated
+ * from the mention by punctuation announces nothing: "of this type. Resource" is two sentences.
+ */
+function announcedType(scan, match, matchStart) {
+    const { tokens, prefixes, paragraph } = scan;
+    for (let length = prefixes.longest; length >= 1; length -= 1) {
+        const first = match.start - length;
+        if (first < 0)
+            continue;
+        const before = tokens.slice(first, match.start);
+        if (!contiguous(paragraph.text, before, matchStart))
+            continue;
+        const type = prefixes.types.get(before.map((token) => token.word).join(" "));
+        if (type !== undefined)
+            return type;
+    }
+    return undefined;
 }
 // The automaton only reports spans inside the tokens it read, so the slice is never empty.
 function spanOf(tokens, match) {
@@ -58,11 +86,10 @@ export function compareOccurrences(a, b) {
 }
 /** The occurrences of one match: one per target of the entry, at the same confidence. */
 function occurrencesOf(scan, match) {
-    const { input, paragraph, tokens, prefixes } = scan;
+    const { input, paragraph, tokens } = scan;
     const { scale } = input;
     const span = spanOf(tokens, match);
-    const before = tokens[match.start - 1];
-    const expectedType = before === undefined ? undefined : prefixes.get(before.word);
+    const expectedType = announcedType(scan, match, span.start);
     const announced = expectedType === undefined ? 0 : scale.type_prefix_bonus;
     const factor = match.key.homonym ? scale.homonym_factor : 1;
     const base = {

package/dist/scan/occurrences.js.map

@@ -1 +1 @@
-{"version":3,"file":"occurrences.js","sourceRoot":"","sources":["../../src/scan/occurrences.ts"],"names":[],"mappings":"AAEA,OAAO,EACL,cAAc,EACd,cAAc,EACd,IAAI,GAGL,MAAM,gBAAgB,CAAC;AACxB,OAAO,EAAE,iBAAiB,EAAmB,MAAM,cAAc,CAAC;AAClE,OAAO,EAAE,QAAQ,EAAc,MAAM,aAAa,CAAC;AAwDnD,SAAS,UAAU,CAAC,CAAS,EAAE,CAAS;IACtC,OAAO,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;AACvC,CAAC;AAED,sFAAsF;AACtF,MAAM,UAAU,GAAG,IAAI,OAAO,EAAiE,CAAC;AAEhG,SAAS,WAAW,CAAC,UAAsB,EAAE,IAAkB;IAC7D,IAAI,MAAM,GAAG,UAAU,CAAC,GAAG,CAAC,UAAU,CAAC,CAAC;IACxC,IAAI,MAAM,KAAK,SAAS,EAAE,CAAC;QACzB,MAAM,GAAG,IAAI,OAAO,EAAE,CAAC;QACvB,UAAU,CAAC,GAAG,CAAC,UAAU,EAAE,MAAM,CAAC,CAAC;IACrC,CAAC;IACD,IAAI,SAAS,GAAG,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;IACjC,IAAI,SAAS,KAAK,SAAS,EAAE,CAAC;QAC5B,+EAA+E;QAC/E,MAAM,QAAQ,GAAG,CAAC,GAAG,UAAU,CAAC,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC;YAChE,GAAG,EAAE,KAAK;YACV,KAAK,EAAE,QAAQ,CAAC,KAAK,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC;SAC5D,CAAC,CAAC,CAAC;QACJ,SAAS,GAAG,cAAc,CAAC,QAAQ,CAAC,CAAC;QACrC,MAAM,CAAC,GAAG,CAAC,IAAI,EAAE,SAAS,CAAC,CAAC;IAC9B,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED;;;GAGG;AACH,SAAS,WAAW,CAClB,YAAyD,EACzD,IAAkB;IAElB,MAAM,KAAK,GAAG,IAAI,GAAG,EAA8B,CAAC;IACpD,KAAK,MAAM,CAAC,IAAI,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,YAAY,CAAC,EAAE,CAAC;QACzD,KAAK,MAAM,MAAM,IAAI,KAAK,EAAE,CAAC;YAC3B,MAAM,IAAI,GAAG,QAAQ,CAAC,MAAM,EAAE,IAAI,CAAC;iBAChC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC;iBAC1B,IAAI,CAAC,GAAG,CAAC,CAAC;YACb,KAAK,CAAC,GAAG,CAAC,IAAI,EAAE,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;QACtD,CAAC;IACH,CAAC;IACD,OAAO,KAAK,CAAC;AACf,CAAC;AAED,2FAA2F;AAC3F,SAAS,MAAM,CACb,MAAwB,EACxB,KAAwB;IAExB,MAAM,IAAI,GAAG,MAAM,CAAC,KAAK,CAAC,KAAK,CAAC,KAAK,EAAE,KAAK,CAAC,GAAG,CAAC,CAAC;IAClD,OAAO;QACL,KAAK,EAAE,IAAI,CAAC,GAAG,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;QACpD,GAAG,EAAE,IAAI,CAAC,GAAG,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;KACjD,CAAC;AACJ,CAAC;AAED,8EAA8E;AAC9E,MAAM,UAAU,kBAAkB,CAAC,CAAa,EAAE,CAAa;IAC7D,OAAO,CACL,UAAU,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,IAAI,CAAC;QAC1B,CAAC,CAAC,IAAI,GAAG,CAAC,CAAC,IAAI;QACf,CAAC,CAAC,QAAQ,GAAG,CAAC,CAAC,QAAQ;QACvB,UAAU,CAAC,CAAC,CAAC,MAAM,CAAC,EAAE,EAAE,CAAC,CAAC,MAAM,CAAC,EAAE,CAAC;QACpC,UAAU,CAAC,CAAC,CAAC,GAAG,EAAE,CAAC,CAAC,GAAG,CAAC,CACzB,CAAC;AACJ,CAAC;AAUD,yFAAyF;AACzF,SAAS,aAAa,CAAC,IAAmB,EAAE,KAAgC;IAC1E,MAAM,EAAE,KAAK,EAAE,SAAS,EAAE,MAAM,EAAE,QAAQ,EAAE,GAAG,IAAI,CAAC;IACpD,MAAM,EAAE,KAAK,EAAE,GAAG,KAAK,CAAC;IACxB,MAAM,IAAI,GAAG,MAAM,CAAC,MAAM,EAAE,KAAK,CAAC,CAAC;IACnC,MAAM,MAAM,GAAG,MAAM,CAAC,KAAK,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC;IACvC,MAAM,YAAY,GAAG,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,QAAQ,CAAC,GAAG,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;IAClF,MAAM,SAAS,GAAG,YAAY,KAAK,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,iBAAiB,CAAC;IAC3E,MAAM,MAAM,GAAG,KAAK,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,KAAK,CAAC,cAAc,CAAC,CAAC,CAAC,CAAC,CAAC;IAC5D,MAAM,IAAI,GAA+B;QACvC,GAAG,EAAE,KAAK,CAAC,GAAG,CAAC,GAAG;QAClB,MAAM,EAAE,KAAK,CAAC,MAAM;QACpB,IAAI,EAAE,KAAK,CAAC,QAAQ,CAAC,IAAI;QACzB,IAAI,EAAE,SAAS,CAAC,IAAI;QACpB,QAAQ,EAAE,IAAI,CAAC,KAAK;QACpB,IAAI,EAAE,SAAS,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,GAAG,CAAC;QAChD,GAAG,CAAC,SAAS,CAAC,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,SAAS,CAAC,OAAO,EAAE,CAAC;QAC1E,OAAO,EAAE,iBAAiB,CAAC,SAAS,EAAE,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,GAAG,CAAC;QAC3D,GAAG,CAAC,YAAY,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,YAAY,EAAE,CAAC;QACvD,UAAU,EAAE,CAAC,KAAK,CAAC,IAAI,GAAG,SAAS,CAAC,GAAG,MAAM;KAC9C,CAAC;IACF,OAAO,KAAK,CAAC,GAAG,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;QACxC,GAAG,IAAI;QACP,MAAM,EAAE,EAAE,EAAE,EAAE,MAAM,CAAC,EAAE,EAAE,IAAI,EAAE,MAAM,CAAC,IAAI,EAAE;KAC7C,CAAC,CAAC,CAAC;AACN,CAAC;AAED;;;;;GAKG;AACH,MAAM,UAAU,YAAY,CAAC,KAAwB;IACnD,MAAM,EAAE,QAAQ,EAAE,UAAU,EAAE,IAAI,EAAE,GAAG,KAAK,CAAC;IAC7C,MAAM,SAAS,GAAG,WAAW,CAAC,UAAU,EAAE,IAAI,CAAC,CAAC;IAChD,MAAM,QAAQ,GAAG,WAAW,CAAC,KAAK,CAAC,YAAY,EAAE,IAAI,CAAC,CAAC;IACvD,MAAM,WAAW,GAAiB,EAAE,CAAC;IACrC,KAAK,MAAM,SAAS,IAAI,QAAQ,CAAC,UAAU,EAAE,CAAC;QAC5C,MAAM,MAAM,GAAG,QAAQ,CAAC,SAAS,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;QAC9C,MAAM,OAAO,GAAkB,EAAE,KAAK,EAAE,SAAS,EAAE,MAAM,EAAE,QAAQ,EAAE,CAAC;QACtE,KAAK,MAAM,KAAK,IAAI,cAAc,CAAC,IAAI,CAAC,SAAS,EAAE,MAAM,CAAC,CAAC,EAAE,CAAC;YAC5D,WAAW,CAAC,IAAI,CAAC,GAAG,aAAa,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC,CAAC;QACrD,CAAC;IACH,CAAC;IACD,OAAO,WAAW,CAAC,IAAI,CAAC,kBAAkB,CAAC,CAAC;AAC9C,CAAC;AAED,kGAAkG;AAClG,MAAM,UAAU,oBAAoB,CAAC,KAAa,EAAE,KAAsB;IACxE,OAAO,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC,GAAG,EAAE,KAAK,CAAC,IAAI,GAAG,KAAK,CAAC,cAAc,GAAG,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,CAAC;AAC9E,CAAC"}
+{"version":3,"file":"occurrences.js","sourceRoot":"","sources":["../../src/scan/occurrences.ts"],"names":[],"mappings":"AAEA,OAAO,EACL,cAAc,EACd,cAAc,EACd,IAAI,GAGL,MAAM,gBAAgB,CAAC;AACxB,OAAO,EAAE,iBAAiB,EAAmB,MAAM,cAAc,CAAC;AAClE,OAAO,EAAE,QAAQ,EAAc,MAAM,aAAa,CAAC;AAwDnD,SAAS,UAAU,CAAC,CAAS,EAAE,CAAS;IACtC,OAAO,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,GAAG,MAAM,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;AACvC,CAAC;AAED,sFAAsF;AACtF,MAAM,UAAU,GAAG,IAAI,OAAO,EAAiE,CAAC;AAEhG,SAAS,WAAW,CAAC,UAAsB,EAAE,IAAkB;IAC7D,IAAI,MAAM,GAAG,UAAU,CAAC,GAAG,CAAC,UAAU,CAAC,CAAC;IACxC,IAAI,MAAM,KAAK,SAAS,EAAE,CAAC;QACzB,MAAM,GAAG,IAAI,OAAO,EAAE,CAAC;QACvB,UAAU,CAAC,GAAG,CAAC,UAAU,EAAE,MAAM,CAAC,CAAC;IACrC,CAAC;IACD,IAAI,SAAS,GAAG,MAAM,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC;IACjC,IAAI,SAAS,KAAK,SAAS,EAAE,CAAC;QAC5B,+EAA+E;QAC/E,MAAM,QAAQ,GAAG,CAAC,GAAG,UAAU,CAAC,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC;YAChE,GAAG,EAAE,KAAK;YACV,KAAK,EAAE,QAAQ,CAAC,KAAK,CAAC,GAAG,EAAE,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC;SAC5D,CAAC,CAAC,CAAC;QACJ,SAAS,GAAG,cAAc,CAAC,QAAQ,CAAC,CAAC;QACrC,MAAM,CAAC,GAAG,CAAC,IAAI,EAAE,SAAS,CAAC,CAAC;IAC9B,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAUD;;;GAGG;AACH,SAAS,WAAW,CAClB,YAAyD,EACzD,IAAkB;IAElB,MAAM,KAAK,GAAG,IAAI,GAAG,EAA8B,CAAC;IACpD,IAAI,OAAO,GAAG,CAAC,CAAC;IAChB,KAAK,MAAM,CAAC,IAAI,EAAE,KAAK,CAAC,IAAI,MAAM,CAAC,OAAO,CAAC,YAAY,CAAC,EAAE,CAAC;QACzD,KAAK,MAAM,MAAM,IAAI,KAAK,EAAE,CAAC;YAC3B,MAAM,MAAM,GAAG,QAAQ,CAAC,MAAM,EAAE,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;YACjE,MAAM,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;YAC9B,OAAO,GAAG,IAAI,CAAC,GAAG,CAAC,OAAO,EAAE,MAAM,CAAC,MAAM,CAAC,CAAC;YAC3C,KAAK,CAAC,GAAG,CAAC,IAAI,EAAE,KAAK,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;QACtD,CAAC;IACH,CAAC;IACD,OAAO,EAAE,KAAK,EAAE,OAAO,EAAE,CAAC;AAC5B,CAAC;AAED,MAAM,KAAK,GAAG,QAAQ,CAAC;AAEvB,2FAA2F;AAC3F,SAAS,UAAU,CAAC,IAAY,EAAE,MAAwB,EAAE,UAAkB;IAC5E,MAAM,IAAI,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;IAC9C,MAAM,MAAM,GAAG,CAAC,GAAG,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,KAAK,CAAC,EAAE,UAAU,CAAC,CAAC;IAC5E,OAAO,IAAI,CAAC,KAAK,CAAC,CAAC,GAAG,EAAE,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC,IAAI,CAAC,KAAK,CAAC,GAAG,EAAE,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;AAChF,CAAC;AAED;;;;GAIG;AACH,SAAS,aAAa,CACpB,IAAmB,EACnB,KAAwB,EACxB,UAAkB;IAElB,MAAM,EAAE,MAAM,EAAE,QAAQ,EAAE,SAAS,EAAE,GAAG,IAAI,CAAC;IAC7C,KAAK,IAAI,MAAM,GAAG,QAAQ,CAAC,OAAO,EAAE,MAAM,IAAI,CAAC,EAAE,MAAM,IAAI,CAAC,EAAE,CAAC;QAC7D,MAAM,KAAK,GAAG,KAAK,CAAC,KAAK,GAAG,MAAM,CAAC;QACnC,IAAI,KAAK,GAAG,CAAC;YAAE,SAAS;QACxB,MAAM,MAAM,GAAG,MAAM,CAAC,KAAK,CAAC,KAAK,EAAE,KAAK,CAAC,KAAK,CAAC,CAAC;QAChD,IAAI,CAAC,UAAU,CAAC,SAAS,CAAC,IAAI,EAAE,MAAM,EAAE,UAAU,CAAC;YAAE,SAAS;QAC9D,MAAM,IAAI,GAAG,QAAQ,CAAC,KAAK,CAAC,GAAG,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC;QAC7E,IAAI,IAAI,KAAK,SAAS;YAAE,OAAO,IAAI,CAAC;IACtC,CAAC;IACD,OAAO,SAAS,CAAC;AACnB,CAAC;AAED,2FAA2F;AAC3F,SAAS,MAAM,CACb,MAAwB,EACxB,KAAwB;IAExB,MAAM,IAAI,GAAG,MAAM,CAAC,KAAK,CAAC,KAAK,CAAC,KAAK,EAAE,KAAK,CAAC,GAAG,CAAC,CAAC;IAClD,OAAO;QACL,KAAK,EAAE,IAAI,CAAC,GAAG,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;QACpD,GAAG,EAAE,IAAI,CAAC,GAAG,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC;KACjD,CAAC;AACJ,CAAC;AAED,8EAA8E;AAC9E,MAAM,UAAU,kBAAkB,CAAC,CAAa,EAAE,CAAa;IAC7D,OAAO,CACL,UAAU,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,CAAC,IAAI,CAAC;QAC1B,CAAC,CAAC,IAAI,GAAG,CAAC,CAAC,IAAI;QACf,CAAC,CAAC,QAAQ,GAAG,CAAC,CAAC,QAAQ;QACvB,UAAU,CAAC,CAAC,CAAC,MAAM,CAAC,EAAE,EAAE,CAAC,CAAC,MAAM,CAAC,EAAE,CAAC;QACpC,UAAU,CAAC,CAAC,CAAC,GAAG,EAAE,CAAC,CAAC,GAAG,CAAC,CACzB,CAAC;AACJ,CAAC;AAUD,yFAAyF;AACzF,SAAS,aAAa,CAAC,IAAmB,EAAE,KAAgC;IAC1E,MAAM,EAAE,KAAK,EAAE,SAAS,EAAE,MAAM,EAAE,GAAG,IAAI,CAAC;IAC1C,MAAM,EAAE,KAAK,EAAE,GAAG,KAAK,CAAC;IACxB,MAAM,IAAI,GAAG,MAAM,CAAC,MAAM,EAAE,KAAK,CAAC,CAAC;IACnC,MAAM,YAAY,GAAG,aAAa,CAAC,IAAI,EAAE,KAAK,EAAE,IAAI,CAAC,KAAK,CAAC,CAAC;IAC5D,MAAM,SAAS,GAAG,YAAY,KAAK,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,iBAAiB,CAAC;IAC3E,MAAM,MAAM,GAAG,KAAK,CAAC,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,KAAK,CAAC,cAAc,CAAC,CAAC,CAAC,CAAC,CAAC;IAC5D,MAAM,IAAI,GAA+B;QACvC,GAAG,EAAE,KAAK,CAAC,GAAG,CAAC,GAAG;QAClB,MAAM,EAAE,KAAK,CAAC,MAAM;QACpB,IAAI,EAAE,KAAK,CAAC,QAAQ,CAAC,IAAI;QACzB,IAAI,EAAE,SAAS,CAAC,IAAI;QACpB,QAAQ,EAAE,IAAI,CAAC,KAAK;QACpB,IAAI,EAAE,SAAS,CAAC,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,GAAG,CAAC;QAChD,GAAG,CAAC,SAAS,CAAC,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,SAAS,CAAC,OAAO,EAAE,CAAC;QAC1E,OAAO,EAAE,iBAAiB,CAAC,SAAS,EAAE,IAAI,CAAC,KAAK,EAAE,IAAI,CAAC,GAAG,CAAC;QAC3D,GAAG,CAAC,YAAY,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,YAAY,EAAE,CAAC;QACvD,UAAU,EAAE,CAAC,KAAK,CAAC,IAAI,GAAG,SAAS,CAAC,GAAG,MAAM;KAC9C,CAAC;IACF,OAAO,KAAK,CAAC,GAAG,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;QACxC,GAAG,IAAI;QACP,MAAM,EAAE,EAAE,EAAE,EAAE,MAAM,CAAC,EAAE,EAAE,IAAI,EAAE,MAAM,CAAC,IAAI,EAAE;KAC7C,CAAC,CAAC,CAAC;AACN,CAAC;AAED;;;;;GAKG;AACH,MAAM,UAAU,YAAY,CAAC,KAAwB;IACnD,MAAM,EAAE,QAAQ,EAAE,UAAU,EAAE,IAAI,EAAE,GAAG,KAAK,CAAC;IAC7C,MAAM,SAAS,GAAG,WAAW,CAAC,UAAU,EAAE,IAAI,CAAC,CAAC;IAChD,MAAM,QAAQ,GAAG,WAAW,CAAC,KAAK,CAAC,YAAY,EAAE,IAAI,CAAC,CAAC;IACvD,MAAM,WAAW,GAAiB,EAAE,CAAC;IACrC,KAAK,MAAM,SAAS,IAAI,QAAQ,CAAC,UAAU,EAAE,CAAC;QAC5C,MAAM,MAAM,GAAG,QAAQ,CAAC,SAAS,CAAC,IAAI,EAAE,IAAI,CAAC,CAAC;QAC9C,MAAM,OAAO,GAAkB,EAAE,KAAK,EAAE,SAAS,EAAE,MAAM,EAAE,QAAQ,EAAE,CAAC;QACtE,KAAK,MAAM,KAAK,IAAI,cAAc,CAAC,IAAI,CAAC,SAAS,EAAE,MAAM,CAAC,CAAC,EAAE,CAAC;YAC5D,WAAW,CAAC,IAAI,CAAC,GAAG,aAAa,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC,CAAC;QACrD,CAAC;IACH,CAAC;IACD,OAAO,WAAW,CAAC,IAAI,CAAC,kBAAkB,CAAC,CAAC;AAC9C,CAAC;AAED,kGAAkG;AAClG,MAAM,UAAU,oBAAoB,CAAC,KAAa,EAAE,KAAsB;IACxE,OAAO,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC,GAAG,EAAE,KAAK,CAAC,IAAI,GAAG,KAAK,CAAC,cAAc,GAAG,CAAC,KAAK,GAAG,CAAC,CAAC,CAAC,CAAC;AAC9E,CAAC"}

package/dist/text/context.d.ts

@@ -1,6 +1,7 @@
 /**
  * A window of `width` characters of the text centred on the span `start`–`end`, kept inside
- * the text, an ellipsis marking each cut.
+ * the text, an ellipsis marking each cut. A cut never falls inside a character: a surrogate
+ * pair or a letter and its combining marks stay together, the window growing by what it takes.
  */
 export declare function contextAround(text: string, start: number, end: number, width: number): string;
 //# sourceMappingURL=context.d.ts.map

package/dist/text/context.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"context.d.ts","sourceRoot":"","sources":["../../src/text/context.ts"],"names":[],"mappings":"AAEA;;;GAGG;AACH,wBAAgB,aAAa,CAAC,IAAI,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,GAAG,MAAM,CAK7F"}
+{"version":3,"file":"context.d.ts","sourceRoot":"","sources":["../../src/text/context.ts"],"names":[],"mappings":"AAUA;;;;GAIG;AACH,wBAAgB,aAAa,CAAC,IAAI,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,EAAE,GAAG,EAAE,MAAM,EAAE,KAAK,EAAE,MAAM,GAAG,MAAM,CAO7F"}

package/dist/text/context.js

@@ -1,12 +1,23 @@
 const ellipsis = "…";
+const MARK = /\p{M}/u;
+/** Whether cutting the text before this code unit would split a surrogate pair or orphan a combining mark. */
+function splits(text, at) {
+    const code = text.charCodeAt(at);
+    return (code >= 0xdc00 && code <= 0xdfff) || MARK.test(text.charAt(at));
+}
 /**
  * A window of `width` characters of the text centred on the span `start`–`end`, kept inside
- * the text, an ellipsis marking each cut.
+ * the text, an ellipsis marking each cut. A cut never falls inside a character: a surrogate
+ * pair or a letter and its combining marks stay together, the window growing by what it takes.
  */
 export function contextAround(text, start, end, width) {
     const centre = Math.floor((start + end) / 2);
-    const from = Math.max(0, Math.min(centre - width / 2, text.length - width));
-    const to = Math.min(text.length, from + width);
+    let from = Math.max(0, Math.min(Math.floor(centre - width / 2), text.length - width));
+    let to = Math.min(text.length, from + width);
+    while (from > 0 && splits(text, from))
+        from -= 1;
+    while (to < text.length && splits(text, to))
+        to += 1;
     return `${from > 0 ? ellipsis : ""}${text.slice(from, to)}${to < text.length ? ellipsis : ""}`;
 }
 //# sourceMappingURL=context.js.map

package/dist/text/context.js.map

@@ -1 +1 @@
-{"version":3,"file":"context.js","sourceRoot":"","sources":["../../src/text/context.ts"],"names":[],"mappings":"AAAA,MAAM,QAAQ,GAAG,GAAG,CAAC;AAErB;;;GAGG;AACH,MAAM,UAAU,aAAa,CAAC,IAAY,EAAE,KAAa,EAAE,GAAW,EAAE,KAAa;IACnF,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,KAAK,GAAG,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC;IAC7C,MAAM,IAAI,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,MAAM,GAAG,KAAK,GAAG,CAAC,EAAE,IAAI,CAAC,MAAM,GAAG,KAAK,CAAC,CAAC,CAAC;IAC5E,MAAM,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,EAAE,IAAI,GAAG,KAAK,CAAC,CAAC;IAC/C,OAAO,GAAG,IAAI,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE,EAAE,CAAC,GAAG,EAAE,GAAG,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC;AACjG,CAAC"}
+{"version":3,"file":"context.js","sourceRoot":"","sources":["../../src/text/context.ts"],"names":[],"mappings":"AAAA,MAAM,QAAQ,GAAG,GAAG,CAAC;AAErB,MAAM,IAAI,GAAG,QAAQ,CAAC;AAEtB,8GAA8G;AAC9G,SAAS,MAAM,CAAC,IAAY,EAAE,EAAU;IACtC,MAAM,IAAI,GAAG,IAAI,CAAC,UAAU,CAAC,EAAE,CAAC,CAAC;IACjC,OAAO,CAAC,IAAI,IAAI,MAAM,IAAI,IAAI,IAAI,MAAM,CAAC,IAAI,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC,CAAC,CAAC;AAC1E,CAAC;AAED;;;;GAIG;AACH,MAAM,UAAU,aAAa,CAAC,IAAY,EAAE,KAAa,EAAE,GAAW,EAAE,KAAa;IACnF,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,KAAK,GAAG,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC;IAC7C,IAAI,IAAI,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,KAAK,CAAC,MAAM,GAAG,KAAK,GAAG,CAAC,CAAC,EAAE,IAAI,CAAC,MAAM,GAAG,KAAK,CAAC,CAAC,CAAC;IACtF,IAAI,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,MAAM,EAAE,IAAI,GAAG,KAAK,CAAC,CAAC;IAC7C,OAAO,IAAI,GAAG,CAAC,IAAI,MAAM,CAAC,IAAI,EAAE,IAAI,CAAC;QAAE,IAAI,IAAI,CAAC,CAAC;IACjD,OAAO,EAAE,GAAG,IAAI,CAAC,MAAM,IAAI,MAAM,CAAC,IAAI,EAAE,EAAE,CAAC;QAAE,EAAE,IAAI,CAAC,CAAC;IACrD,OAAO,GAAG,IAAI,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,EAAE,EAAE,CAAC,GAAG,EAAE,GAAG,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC;AACjG,CAAC"}

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@concordance-wiki/nlp",
-  "version": "0.3.0",
+  "version": "0.4.0",
   "description": "Normalisation, Aho-Corasick, n-grams, C-value, MinHash and the language packs.",
   "license": "GPL-3.0-or-later",
   "repository": {
@@ -31,12 +31,13 @@
     "LICENSE"
   ],
   "dependencies": {
-    "ajv": "8.17.1",
-    "yaml": "2.8.1",
-    "@concordance-wiki/core": "0.3.0"
+    "ajv": "8.20.0",
+    "yaml": "2.9.1",
+    "@concordance-wiki/core": "0.4.0"
   },
   "devDependencies": {
-    "@concordance-wiki/ingest": "0.3.0"
+    "vitest": "4.1.11",
+    "@concordance-wiki/ingest": "0.4.0"
   },
   "scripts": {
     "typecheck": "tsc -b tsconfig.json"