@heroiclands/package-build 24.1.1 → 25.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (107) hide show
  1. package/CHANGELOG.md +36 -0
  2. package/README.md +1 -0
  3. package/bin/content-commands.mjs +30 -96
  4. package/bin/encoding-commands.mjs +146 -0
  5. package/bin/package-build.mjs +35 -24
  6. package/config.mjs +25 -0
  7. package/convert.mjs +182 -0
  8. package/docs/api.md +154 -11
  9. package/docs/authoring/links-and-markup.md +260 -473
  10. package/docs/commands.md +118 -119
  11. package/docs/configuration.md +31 -13
  12. package/docs/content-format.md +11 -4
  13. package/docs/diagnostics.md +43 -16
  14. package/docs/encoding-core.md +135 -0
  15. package/docs/guides/book.md +2 -2
  16. package/docs/guides/module-project.md +23 -1
  17. package/docs/guides/site.md +2 -2
  18. package/docs/guides/writing-in-invented-scripts.md +453 -569
  19. package/docs/reference/format-details.md +290 -380
  20. package/docs/reference/note-types.md +80 -62
  21. package/docs/source-preview.md +8 -0
  22. package/e2e.mjs +1 -1
  23. package/engine/base-compiler.mjs +27 -9
  24. package/engine/content-charset.mjs +21 -1
  25. package/engine/content-format.yaml +7 -2
  26. package/engine/content-index.mjs +6 -2
  27. package/engine/content-links.mjs +1 -0
  28. package/engine/content-scripts.mjs +658 -423
  29. package/engine/content-scroll.mjs +4 -4
  30. package/engine/content-spans.mjs +40 -9
  31. package/engine/content-verse.mjs +1 -1
  32. package/engine/converter-data.mjs +70 -0
  33. package/engine/dictionary-rows.mjs +2 -34
  34. package/engine/encoding-integrity.mjs +94 -0
  35. package/engine/encoding-language.mjs +506 -0
  36. package/engine/encoding-lexicon.mjs +699 -0
  37. package/engine/encoding-notes.mjs +186 -0
  38. package/engine/encoding-standard.mjs +42 -0
  39. package/engine/encoding-template.mjs +313 -0
  40. package/engine/encoding-terms.mjs +141 -0
  41. package/engine/encoding.mjs +946 -0
  42. package/engine/extension-attributes.mjs +2 -0
  43. package/engine/helpers.mjs +1 -1
  44. package/engine/index.mjs +10 -0
  45. package/engine/language-data.mjs +4 -1
  46. package/engine/language-terms.mjs +23 -12
  47. package/engine/{logographic.mjs → language-writings.mjs} +29 -38
  48. package/engine/markup-validation.mjs +1 -1
  49. package/engine/note-vocabulary.mjs +4 -158
  50. package/engine/pdf-render.mjs +31 -3
  51. package/engine/roman-characters.mjs +232 -0
  52. package/engine/roman-run.mjs +149 -0
  53. package/engine/roman-verse.mjs +95 -0
  54. package/engine/run-rows.mjs +29 -31
  55. package/engine/run-words.mjs +2 -2
  56. package/engine/script-emphasis-terms.mjs +3 -0
  57. package/engine/script-emphasis.mjs +3 -10
  58. package/engine/script-font-delivery.mjs +9 -12
  59. package/engine/script-fonts.mjs +2 -1
  60. package/engine/site-build.mjs +14 -5
  61. package/engine/site-preview.mjs +12 -2
  62. package/engine/sql-tables.mjs +36 -31
  63. package/engine/verse-metre.mjs +1 -1
  64. package/engine/word-forms.mjs +1 -1
  65. package/engine/word-inflections.mjs +48 -0
  66. package/engine/word-rows.mjs +1 -1
  67. package/engine/word-terms.mjs +16 -17
  68. package/package.json +11 -5
  69. package/types/config.d.mts +5 -0
  70. package/types/convert.d.mts +44 -0
  71. package/types/e2e.d.mts +1 -1
  72. package/types/engine/base-compiler.d.mts +4 -2
  73. package/types/engine/content-charset.d.mts +8 -0
  74. package/types/engine/content-index.d.mts +4 -1
  75. package/types/engine/content-scripts.d.mts +108 -42
  76. package/types/engine/converter-data.d.mts +11 -0
  77. package/types/engine/dictionary-rows.d.mts +1 -25
  78. package/types/engine/encoding-integrity.d.mts +14 -0
  79. package/types/engine/encoding-language.d.mts +36 -0
  80. package/types/engine/encoding-lexicon.d.mts +259 -0
  81. package/types/engine/encoding-notes.d.mts +39 -0
  82. package/types/engine/encoding-standard.d.mts +11 -0
  83. package/types/engine/encoding-template.d.mts +43 -0
  84. package/types/engine/encoding-terms.d.mts +105 -0
  85. package/types/engine/encoding.d.mts +155 -0
  86. package/types/engine/helpers.d.mts +1 -1
  87. package/types/engine/index.d.mts +5 -0
  88. package/types/engine/language-terms.d.mts +1 -1
  89. package/types/engine/{logographic.d.mts → language-writings.d.mts} +6 -6
  90. package/types/engine/markup-validation.d.mts +1 -1
  91. package/types/engine/pdf-render.d.mts +1 -1
  92. package/types/engine/roman-characters.d.mts +45 -0
  93. package/types/engine/roman-run.d.mts +101 -0
  94. package/types/engine/roman-verse.d.mts +23 -0
  95. package/types/engine/run-rows.d.mts +13 -20
  96. package/types/engine/script-emphasis-terms.d.mts +2 -0
  97. package/types/engine/script-emphasis.d.mts +1 -1
  98. package/types/engine/script-font-delivery.d.mts +2 -3
  99. package/types/engine/script-fonts.d.mts +5 -0
  100. package/types/engine/site-preview.d.mts +1 -1
  101. package/types/engine/word-inflections.d.mts +25 -0
  102. package/engine/map-lexicon.mjs +0 -1034
  103. package/engine/text-maps.mjs +0 -654
  104. package/engine/written-in.mjs +0 -204
  105. package/types/engine/map-lexicon.d.mts +0 -330
  106. package/types/engine/text-maps.d.mts +0 -189
  107. package/types/engine/written-in.d.mts +0 -76
@@ -12,9 +12,9 @@ reads as the language, text that is drawn in the language's own letters, and a
12
12
  way to keep both honest. This guide explains how the toolchain does that, from
13
13
  the model underneath to every finding it can report, for human authors and for
14
14
  agents. The reference pages state each rule exactly: [links and
15
- markup](../authoring/links-and-markup.md#marking-language-font-and-mapping),
16
- [format details](../reference/format-details.md#text-marked-with-a-language-a-font-or-a-map),
17
- [commands](../commands.md#package-build-transliterate-map-text),
15
+ markup](../authoring/links-and-markup.md#marking-language-font-and-encoding),
16
+ [format details](../reference/format-details.md#text-marked-with-a-language-font-or-encoding),
17
+ [commands](../commands.md#package-build-encode-text),
18
18
  [configuration](../configuration.md#languages) and
19
19
  [diagnostics](../diagnostics.md). This guide ties them into one account and
20
20
  works one example through from start to finish.
@@ -24,455 +24,182 @@ Semaron script. The lexicon and the notes in them are the guide's own sample.
24
24
 
25
25
  ## The model
26
26
 
27
- These ideas carry everything else.
28
-
29
- 1. **A note records Unicode characters, and nothing else.** Which keyboard
30
- typed them is the author's business. The markdown and the build neither
31
- know nor care.
32
- 2. **A font draws glyphs for code points.** A code point a font lacks draws a
33
- box, so the build refuses it before a box reaches a reader. A font states no
34
- sounds and no spelling, just as a Unicode block chart states none.
35
- 3. **Each script has code points of its own,** as Runic and Ogham have their
36
- own Unicode blocks. Semaron's `a` is not U+0061. Every face of one script
37
- draws the same code points, so changing the face changes only the look. The
38
- left-to-right scripts sit in plane 15, the Supplementary Private Use Area-A
39
- (Semaron from U+F0200). The right-to-left scripts sit in unassigned code
40
- points of plane 1 whose default direction is right to left, so their text
41
- runs right to left in any editor, browser or typesetter with no control
42
- characters.
43
- 4. **Sounds and spelling belong to the language.** A script is not a language:
44
- several languages are written in one script, as many Earth languages share
45
- the Latin alphabet. A mapping therefore belongs to a script and serves every
46
- language written in it. What differs between languages is their dictionaries.
47
- 5. **Language and presentation are separate.** `lang=` identifies the
48
- language, retaining the authored Roman spelling and the surrounding prose
49
- font. Language and vocabulary checks still apply.
50
- 6. **`.nativefont` requests native presentation.** It requires an explicit
51
- `lang=` on that same span, div or fence, then selects the language's
52
- configured native map and font. Each marked child makes its own choice;
53
- native presentation is not inherited by a language-only child.
54
- 7. **`font=` and `map=` override independently.** A native run takes any
55
- missing half from the language's `data.writtenIn`; if neither the run nor
56
- its language provides a font, the build reports the needed `font=` override.
57
- When the language supplies no map, native code points can be typed directly.
58
- A native font or map without `.nativefont` is an error. Ordinary Roman
59
- `font=mono`, `serif` and `sans` remain available without native opt-in.
60
- 8. **Native mapping uses the dictionary first.** A word the dictionary does
61
- not write is spelled by the mapping's rows. Every word must belong to the
62
- language where its configured note supplies a dictionary.
63
-
64
- A script that writes meanings rather than sounds is **logographic**. It has no
65
- sound rules, so every word is written from the dictionary and never guessed;
66
- see [logographic scripts](#logographic-scripts).
67
-
68
- One code path serves the build and its users. `package-build transliterate`
69
- converts a passage exactly as the build does, and `package-build lexicon` lists
70
- every word of a language with its native form; see
71
- [the commands](#the-commands).
72
-
73
- The pipeline for one marked run reads left to right:
74
-
75
- | Stage | Reads | Reports |
76
- | ------------- | ------------------------------------ | ------------------------------------------ |
77
- | Keys | `lang=`, `font=`, `map=` | an unknown language, font or map |
78
- | Dictionary | the language's `data.words[].native` | a native form the rows cannot capitalise |
79
- | Rows | the mapping's `data.map.rows` | nothing: a character with no row passes on |
80
- | Lexicon check | the language's words and register | a word the language does not hold |
81
- | Face coverage | the face's `fonts.json` code points | a character the face does not draw |
82
- | Element | the face, the tag and the direction | nothing |
27
+ A language defines its words, meanings and pronunciation. An encoding defines
28
+ which source sequences write which characters. A face draws those characters.
29
+ Keeping these facts in their owning sources gives each passage one consistent
30
+ conversion on the site, in Foundry and in the book.
83
31
 
84
- ## Language and native presentation
85
-
86
- The keys go in the attribute block that a span, a fenced div or a `verse`
87
- fence already takes:
88
-
89
- | Written | Meaning |
90
- | ---------------------------------------------------------------------- | -------------------------------------------------------------------------------- |
91
- | `[Daimela, helthos.]{lang=vyl}` | Vylari in authored Roman spelling and the page's font, with language validation. |
92
- | `[Zaron lonak]{lang=vyl .nativefont}` | Vylari mapped and drawn with its configured native defaults. |
93
- | `[Zaron lonak]{lang=vyl .nativefont font=semaron-kuriale map=semaron}` | Native Vylari with both defaults explicitly overridden. |
94
- | `[ad astra]{lang=la font=serif}` | A real language's BCP 47 tag, in the reader's serif. |
95
- | `::: {lang=vyl .nativefont}` … `:::` | Native presentation for a passage's unmarked paragraphs. |
96
- | a `verse` fence opened with `{lang=vyl .nativefont}` | Native presentation for a poem's unmarked lines. |
97
-
98
- The class is explicit on each construct. `[Zaron]{.nativefont}` is an error
99
- even inside a passage marked `lang=vyl`: add `lang=vyl` to that span. A native
100
- font such as `font=semaron-kuriale`, or any `map=`, requires `.nativefont`.
101
- Generic Roman `font=serif`, `sans` and `mono` do not.
102
-
103
- **`lang=`** is read in this order. First a short name declared in the
104
- configuration's `languages`; then a BCP 47 language tag for a real language
105
- (`la`, `grc`, `en-GB`), checked for its form alone. A declared name wins over a
106
- tag spelled the same way. A well-formed tag one edit from a declared name draws
107
- a warning that names the declared one, since `vly` is a valid tag for another
108
- language than `vyl`.
109
-
110
- **`font=`** is a face of `@heroiclands/fonts`, named by its class without
111
- `hl-` (`hl-semaron-kuriale` is `font=semaron-kuriale`), or one of the generic
112
- families `mono`, `serif` and `sans`. A package that sets text in a face adds the
113
- fonts to its own development dependencies:
114
-
115
- ```bash
116
- npm install --save-dev @heroiclands/fonts
117
- ```
118
-
119
- The installed package's `fonts.json` lists every face and every code point it
120
- draws, which is the list to write a mapping from. This prints Semaron Kuriale's:
32
+ | Fact | Source | Selection |
33
+ | -------------------------- | ----------------------------------------- | ------------------------------ |
34
+ | Language and vocabulary | Configuration and language note | `lang=` |
35
+ | Authored lexical spelling | Word, affix or grammatical `native` entry | Encoding and optional face |
36
+ | Encoding rows and controls | Encoding document frontmatter | `enc=` document Address |
37
+ | Font face | Installed `fonts.json` | `font=` or the encoding's font |
121
38
 
122
- ```bash
123
- jq -r '.families[] | select(.class == "hl-semaron-kuriale") | .codepoints[] | [.codepoint, .kind, .glyph, .name] | @tsv' \
124
- node_modules/@heroiclands/fonts/fonts.json
125
- ```
126
-
127
- Each line is a code point, its kind (`letter`, `digraph`, `numeral`, `point`, `mark`
128
- or `sign`), the glyph's name and the character's name; the glyph named `a` is the
129
- letter that writes the sound `a`, and `a.long` the long vowel. Those names are
130
- what a mapping's rows are written from.
131
-
132
- On the site `serif` and `mono` are the site's own body and code faces and
133
- `sans` is the reader's sans-serif; in Foundry they are the reader's generic
134
- families; in a book they are the book's own serif, sans and mono. The generic
135
- families are not checked for coverage.
39
+ Language identification alone retains Roman text. `enc=` selects an encoding
40
+ document, which infers its language and font. Unknown source characters are
41
+ errors; a face drawing a character does not make it a spelling mapping.
136
42
 
137
- **`map=`** is the `data.map.name` of a [mapping note](#mapping-notes). It
138
- rewrites the run before anything else; a run's `font=` is then one of the faces
139
- the mapping lists.
140
-
141
- The `languages` block of `package-build.config.yaml` declares the short names:
142
-
143
- ```yaml
144
- languages:
145
- vyl: { tag: art-x-vyl, note: skill-vylarilng }
146
- khel: { tag: art-x-khel, note: skill-khelathlng }
147
- ```
148
-
149
- A name is lowercase letters, digits and hyphens, starting with a letter. `tag`
150
- is a BCP 47 tag; a constructed language takes `art-x-<name>`. `note` is the
151
- address of the language's `skill` note of `subType: language`, whose
152
- `data.words` is the dictionary and whose `data.register` is the names register.
153
- Without `note`, `lang=` only tags: there is no dictionary and no lexicon check.
154
- `package-build lint` checks at the key that `note` names a language note.
155
-
156
- **A language names its script.** Most tongues are written in one script, so the
157
- language note says which, as one map and one face:
158
-
159
- ```yaml
160
- data:
161
- writtenIn: { map: semaron, font: semaron-kuriale }
162
- ```
163
-
164
- A language-only run stays Roman: `[Zaron lonak]{lang=vyl}`. Add
165
- `.nativefont` to select both defaults:
166
-
167
- ```markdown
168
- [Zaron lonak]{lang=vyl .nativefont}
169
- [Zaron lonak]{lang=vyl .nativefont font=semaron-kuriale map=semaron}
170
- ```
171
-
172
- Those two spans display the same native text. `font=` replaces only the font;
173
- `map=` replaces only the map. A missing font default produces an error naming the
174
- needed `font=` override, and incompatible map/font pairs are errors. A map
175
- is optional when the language supplies none; directly typed signs then
176
- reach the selected font unchanged. The build does
177
- not discard a default to make an override fit. This applies to spans, divs,
178
- verse fences, scroll fences and standalone wikilinks. The language note's
179
- keys are checked at their values:
180
-
181
- > `` `data.writtenIn.map` is `semoran`, which no mapping note's `data.map.name` declares; did you mean `semaron`? ``
182
-
183
- > `` `data.writtenIn.font` is `teng-chou`, which the map `semaron` is not drawn in; it is drawn in `semaron-kuriale` ``
184
-
185
- **Unmarked text follows its passage; marked children choose their own
186
- presentation.** A language-only child stays Roman, even in the same language
187
- as its native parent:
188
-
189
- ```markdown
190
- ::: {lang=vyl .nativefont}
191
- Zaron lonak, [Zaron lonak]{lang=vyl}, lokamen zaron.
192
- [Gar-Anlagh]{lang=khel .nativefont font=qalezu-hand map=qalezu}
193
- :::
194
- ```
43
+ ## Language and native presentation
195
44
 
196
- The first Vylari phrase and the unmarked text use Semaron; the language-only
197
- span gives the Roman reading. The Khelathi span opts in independently and uses
198
- its own map and font. Qalezu runs right to left inside the left-to-right line
199
- with no `dir=` written. An English gloss, `[an oath]{lang=en font=serif}`,
200
- uses the page's serif. A child's own native defaults and overrides are used;
201
- the parent's explicit map and font are not copied into the child.
45
+ `enc=` names an encoding document, with `doc` as its default Address type:
46
+ `[say.]{enc=aelwsemaronenc}` selects `doc-aelwsemaronenc`. That document supplies
47
+ its configured `language` and single `font`; an explicit `lang=` must identify
48
+ the same language, and an explicit `font=` must be that font. Another face is
49
+ a separate encoding document.
202
50
 
203
- **References make the same choice.** The first name below stays Roman; the
204
- second uses Vylari's native defaults:
51
+ `enc=default` leaves Roman text as written and is the default when no encoding
52
+ is selected. `lang=` still identifies the text and selects its language's
53
+ vocabulary; `font=mono`, `serif`, `sans` or an installed font selects its face
54
+ without encoding it. Language identification does not select an encoding.
205
55
 
206
56
  ```markdown
207
- [[lore-marendasdty|Marendas]]{lang=vyl}
208
- [[lore-marendasdty|Marendas]]{lang=vyl .nativefont}
57
+ [say.]{enc=aelwsemaronenc}
58
+ [say.]{enc=doc-aelwsemaronenc lang=aelw font=semaron-kuriale}
59
+ [say.]{lang=aelw enc=default}
209
60
  ```
210
61
 
211
- Only the displayed name is mapped and set in the native face. The destination,
212
- spoiler and `.show` behavior, draft cue and book page references are preserved.
213
- Keep adjacent prose and its spaces in sibling spans; a longer native passage
214
- cannot contain an embedded link.
215
-
216
- **What a run may hold.** A marked span or verse line is plain text: no
217
- emphasis (other than the stars a face's run takes, see [emphasis in a
218
- script](#emphasis-in-a-script)), link, image, footnote, wikilink, code or HTML.
219
- A standalone resolved reference may carry the language and native class, as
220
- above; only its label is set. A marked div holds paragraphs and line breaks only. The text is read as written,
221
- so `[`, `]`, `"`, `<<`, `>>`, `!` and `@ # $ % ^ &` are characters; a bracket
222
- pair is markup only when `(` makes a link, `{` a span or a second `[` a
223
- wikilink, and an unbalanced bracket takes a backslash, `\[`. A code fence is
224
- verbatim: `[…]{…}` inside it stays literal text. Text marked with a real
225
- language's tag alone keeps its Markdown when that language is English; a span
226
- with a non-English effective language cannot contain actual bold or italic
227
- markup, even when the language is real.
62
+ Class-only emphasis and name children belong to the parent's complete logical
63
+ text; encoding preserves their rendered wrappers. A child with its own run
64
+ keys makes a separate presentation choice. `enc=default` inside an encoded
65
+ parent keeps the inherited language but does not inherit its inferred script
66
+ face. An explicitly authored font remains a font choice.
228
67
 
229
- The language, font and map keys are errors on a named block such as a secret block. Put a marked
230
- div inside the block instead.
68
+ A name's braces use the shared name controls. Resolved references encode only
69
+ their visible labels, retaining URLs and presentation cues. These keys apply
70
+ to spans, divs, verse fences and scroll fences. A scroll's selected font must
71
+ belong to a script whose note declares `direction: vertical-rl`.
231
72
 
232
- ## Roman text and directly typed signs
73
+ ## Roman source and literal characters
233
74
 
234
- Language-only text retains its spelling and the surrounding prose font:
75
+ Language-only text retains the exact spelling and surrounding prose font:
235
76
 
236
77
  ```markdown
237
- [Zaron lonak]{lang=vyl}
78
+ [Zaron lonak]{lang=vyl enc=default}
238
79
  [Zaron lonak]{lang=vyl font=serif}
239
80
  ```
240
81
 
241
- The second span explicitly chooses the ordinary serif. Neither selects the
242
- language's native defaults. A configured language's existing vocabulary and
243
- language checks still apply; presentation does not turn those checks off.
244
-
245
- Text already typed in a script's code points also opts in explicitly:
246
-
247
- ```markdown
248
- The lintel is cut with [󰈹󰈰]{lang=und .nativefont font=semaron-kuriale}.
249
- ```
250
-
251
- Here `und` identifies an unspecified language with no configured map. The
252
- font receives the typed code points unchanged. If a map is selected, rows
253
- leave characters with no matching row unchanged. The selected face must draw
254
- every resulting character, except whitespace, or the build
255
- reports the source character. Keyboard layouts and Emacs input methods for
256
- the scripts ship under the fonts package's `keyboards/`; an editor font that
257
- carries the script draws its code points where they are typed. Typing Roman
258
- and mapping it keeps the source readable and diffable.
259
-
260
- ## Mapping notes
261
-
262
- A mapping note is a `doc` note whose `data.map` lists the rewrites. **There is
263
- one mapping per script,** serving every language written in it. The note below
264
- is complete; the rows are the Semaron letters this guide's sample uses.
265
-
266
- **`assets/content/Lore/Semaron_Spelling.md`**
267
-
268
- ```markdown
269
- ---
270
- shortcode: mapsemaron
271
- name: { full: Semaron Spelling }
272
- type: doc
273
- subType: reference
274
- data:
275
- map:
276
- name: semaron
277
- vowels: "aeiou"
278
- rows:
279
- - ["a", "󰈰"] # U+F0230
280
- - ["d", "󰈲"] # U+F0232
281
- - ["e", "󰈳"] # U+F0233
282
- - ["h", "󰈶"] # U+F0236
283
- - ["i", "󰈷"] # U+F0237
284
- - ["k", "󰈹"] # U+F0239
285
- - ["l", "󰈺"] # U+F023A
286
- - ["m", "󰈻"] # U+F023B
287
- - ["n", "󰈼"] # U+F023C
288
- - ["o", "󰈽"] # U+F023D
289
- - ["r", "󰈿"] # U+F023F
290
- - ["s", "󰉀"] # U+F0240
291
- - ["t", "󰉁"] # U+F0241
292
- - ["u", "󰉂"] # U+F0242
293
- - ["z", "󰉆"] # U+F0246
294
- - ["aa", "󰉈"] # U+F0248
295
- - ["th", "󰉐"] # U+F0250
296
- - ["Th", "󰉑"] # U+F0251
297
- - ["L", "󰈊"] # U+F020A
298
- - ["T", "󰈑"] # U+F0211
299
- - ["Z", "󰈖"] # U+F0216
300
- - { from: "y", to: "󰉅", before: vowel } # U+F0245
301
- - ["y", "󰈷"] # U+F0237
302
- - [".", "󰉨"] # U+F0268
303
- - [",", "󰉨"] # U+F0268
304
- fonts: [{ font: semaron-kuriale }]
305
- ---
306
-
307
- Semaron Spelling writes roman spelling in the Semaron script. A passage marked with
308
- `map=semaron` is converted by the rows of this note before it is set, so the note's
309
- source stays in roman letters and the page shows the script. Every language written
310
- in Semaron shares these rows.
311
- ```
312
-
313
- The keys of `data.map`:
314
-
315
- | Key | Meaning |
316
- | ------------- | -------------------------------------------------------------------------------------------- |
317
- | `name` | What `map=` names it by: lowercase letters, digits and `-`, unique in the package. Required. |
318
- | `rows` | The rewrites, in authored order. Required. |
319
- | `fonts` | The faces the mapping is drawn in, each by its slug, at least one. Required. |
320
- | `vowels` | The characters that `before: vowel` reads as vowels. Any other letter is a consonant. |
321
- | `logographic` | `true` for a script that writes meanings; see [logographic scripts](#logographic-scripts). |
322
-
323
- **Rows.** A row is `[from, to]`, or `{ from, to, before, at }` with the two
324
- conditions optional. `from` is one or more characters; `to` is any number of
325
- them, none included. A mapping that drops a language's vowels writes `[a, ""]`.
326
- `rows` is a list, never a keyed map, so `A` and `a` are two rows.
327
-
328
- **How a mapping reads text.**
329
-
330
- - **The longest match wins.** In the note above `th` is one letter, and `t` and
331
- `h` are read only where `th` does not match. Among rows of one length, a row
332
- whose condition holds comes before a row with none, and otherwise the earlier
333
- row wins.
334
- - **A row whose condition fails is skipped.** `before: vowel` holds when the next
335
- character is one of `vowels`; `before: consonant` when it is any other
336
- letter; neither holds at the end of the text. `at: start` holds at the first
337
- character of the run or after white space or punctuation; `at: end` holds at
338
- the last character or before white space or punctuation. In the note above,
339
- `y` is the consonant letter before a vowel and the vowel `i` elsewhere.
340
- - **Capitals and marks need rows of their own.** A row matches the character as
341
- written, so `L` and `l` are two rows, and a script with no capitals gives both
342
- the same code point. The full stop and the comma are characters too: a run
343
- that holds one needs a row for it, or the face-coverage check reports it.
344
- - **A character no row matches passes through unchanged,** and the face-coverage
345
- check then reports it. A mapping for a face that draws nothing on roman code
346
- points lists every letter the spelling uses.
347
-
348
- **Fonts.** `fonts` lists each face the mapping may be used with, as a list of
349
- `{ font, rows }`. A face may carry rows of its own, which a run in that face
350
- tries first (longest match among them) before the shared rows. That is the way
351
- to write a sound differently in a variant face. The generic families `mono`,
352
- `serif` and `sans` are not listed. The lint checks that every listed face draws
353
- every code point that its own rows, the shared rows it can reach and the
354
- lexicons' native forms for it write.
355
-
356
- **Code points above U+FFFF in YAML.** A double-quoted YAML string writes a
357
- code point by its escape, and the left-to-right scripts sit in plane 15, so
358
- they take the eight-digit `\U`: `"\U000F0230"` is U+F0230, and the four-digit
359
- `\u` reaches only code points up to U+FFFF. Both forms mean the same thing.
360
- `package-build format --write` (Prettier) rewrites such an escape into the
361
- character itself, so a mapping note on disk usually holds the raw Private Use
362
- characters, as the note above does. Those read only in one of the fonts
363
- package's faces or its editor fonts, and elsewhere show as boxes. The
364
- convention is a trailing comment, `# U+F0230`, on every row, which keeps the
365
- note readable in any editor and survives formatting. A row that writes several
366
- code points lists them all in its comment.
367
-
368
- **Where the checks sit.** Every key is checked at its own line and column:
369
-
370
- > `` `data.map.name` is `Vyl_Semaron`; a map's name is lowercase letters, digits and `-` ``
371
-
372
- > `` `data.map.rows[4]` has `before: consonant`, and the map states no `vowels` to read it by ``
373
-
374
- > `` `data.map.rows[1]` has an empty `from`; a row matches one or more characters ``
375
-
376
- > `` `data.map.fonts` is empty; a map lists each face it is drawn in ``
377
-
378
- > `` `data.map.rows[5]` writes U+F0239, which the font `semaron-kuriale`, listed in `data.map.fonts`, does not draw ``
379
-
380
- A row with a fault is left out of the map and the rest apply.
381
-
382
- ## How a mapping treats punctuation
383
-
384
- The toolchain is used with one convention for marks: **mood, questions and
385
- quotation are carried by words, not marks,** unless a language's note rules
386
- otherwise. A mapping therefore lists the marks its script has and nothing more.
387
- In the sample, a comma and a full stop are the script's one point:
388
-
389
- ```yaml
390
- - [".", "\U000F0268"] # U+F0268
391
- - [",", "\U000F0268"] # U+F0268
392
- ```
393
-
394
- (Formatting rewrites the escapes, as above.) A character with no row passes
395
- through unchanged, and the face does not draw it, so a question mark in a
396
- mapped run is an error that names the mark and the mapping:
397
-
398
- > `` `?` (U+003F) is not drawn by the font `semaron-kuriale`, and the map `semaron` has no row for it ``
399
-
400
- The remedy is to write the question or the quotation in words, as the language
401
- does, and leave the mark out of the run. A language whose note rules
402
- otherwise gets rows for its marks in the mapping, and the check stops
403
- reporting them. The same holds for the digits, the hyphen and the catch: a row
404
- for each makes the mark part of the script; none leaves it out.
82
+ The language note's `data.roman` lists additions to the shared standard:
83
+ letters in their accepted cases, accented letters, catch signs and digits.
84
+ List each character once in NFC spelling. Standard punctuation and whitespace
85
+ are supplied automatically. A language that lists uppercase letters accepts
86
+ them; the build changes no authored case.
87
+
88
+ The terminals are `.` for a statement, `?` for a question, `‼` for an imperative
89
+ and `!` for an exclamation. None is required. A comma pauses within a sentence;
90
+ use it instead of `;` or `:`, and use a period between complete sentences.
91
+ Quotation pairs alternate `«…»`, `‹…›`, then `«…»` at successive depths.
92
+ English double quotes, `<<` and `>>` are findings. A closing quote is not a
93
+ terminal for the enclosing sentence.
94
+
95
+ An em dash interrupts a sentence. An ellipsis at a sentence's start resumes
96
+ one; later it abandons one. An interruption or abandonment takes an immediate
97
+ terminal unless it ends the run. Write `…`, not three periods. Markdown
98
+ escapes are removed before encoding, and diagnostics retain the source position.
99
+
100
+ Every `Zs` space separator and U+0009 through U+000D count as whitespace. A run
101
+ of them is one word separator, which the encoding writes by its own rule.
102
+ Newlines in prose are whitespace. In a verse fence, line endings supply line
103
+ boundaries, blank lines supply stanza boundaries, and `--- {.canto}` on its
104
+ own line supplies a canto boundary. Do not type a verse slash `/` inside the
105
+ fence; an inline quotation of verse may use it. Verse indentation is layout,
106
+ not a punctuation character added to the encoded words.
107
+
108
+ A `.name` child in a language run reaches the encoding with standard `{…}`
109
+ name boundaries. An `.emph` child contributes the same letters with emphasis
110
+ presentation. Both inherit the surrounding native writing when they state no
111
+ run keys.
112
+
113
+ Write an exact native character through codage, such as `⟦U+F0230⟧`, or a name
114
+ that the chosen encoding defines. Literal codes are immutable; ordinary letter
115
+ and punctuation rules do not reinterpret their contents. A setting-script
116
+ character pasted directly into a configured language run is a finding. The
117
+ same restriction applies to a face that can draw it.
118
+
119
+ A `!` touching the `[` of another span is Markdown image syntax. Separate the
120
+ terminal from the following span, or escape the terminal.
121
+
122
+ ## Encodings
123
+
124
+ An encoding is a `doc` note. Its `data.language` is a configured language
125
+ alias, `data.font` is one installed font slug, and `data.mappings` maps Roman
126
+ strings to a font glyph code or a sequence of codes. `data.rules` holds the
127
+ closed executable rules. Optional `data.accepts` states accepted characters
128
+ or ranges and `data.vowels` states contextual vowels. The note body explains
129
+ the spelling to a reader.
130
+
131
+ The font supplies each glyph's `code` beside its `codepoint` in `fonts.json`.
132
+ Codes belong to the selected font; the note never repeats a code table.
133
+ Unknown fields, glyph codes, languages and fonts are positioned findings.
134
+ An encoding's output must decode unambiguously, including across adjacent
135
+ mappings and spaces. The compiler refuses indistinguishable spellings rather
136
+ than choosing a guessed reading. See [the executable rules](../encoding-core.md).
137
+
138
+ Mappings match longest first; contextual before-vowel rules precede their
139
+ unconditional alternatives. A case rule may allow lowercase and initial-capital
140
+ groups while refusing other mixed or uppercase group spellings. Encoding keeps
141
+ authored case and never rewrites a word's Roman spelling.
142
+
143
+ Outside literal markers, a run of Unicode Zs characters or U+0009 through
144
+ U+000D writes one space. Other whitespace needs an explicit mapping. Standard
145
+ Roman punctuation, verse and name controls are inherited from the shared
146
+ standard; an explicit mapping can give a standard source its font glyph.
147
+
148
+ ### Literal codes
149
+
150
+ Inside an encoded run, `⟦…⟧` writes the selected font's glyph codes or exact
151
+ Unicode scalars: `[⟦MAN⟧]{enc=khelzethuenc}` where `MAN` is defined, and
152
+ `[⟦U+00E1⟧]{enc=vylsemaronenc}` for an exact character. A Unicode code takes four to
153
+ six uppercase hexadecimal digits, U+0000 through U+10FFFF excluding
154
+ surrogates. Codes come only from the font; Unicode database
155
+ names are not inferred. Whitespace between codes writes nothing. Nested or
156
+ unmatched markers, unknown codes and standard marks absent from the font
157
+ code table are errors. Without an encoding, a literal marker is a finding.
158
+
159
+ Each outer marker writes its own invisible control. Payload occurrences of
160
+ those controls or the literal escape are quoted with U+E0115. Decoding
161
+ unquotes the payload before reading code rows and uses `U+XXXX` for scalars
162
+ with no code name. Variation selectors U+E0100 through U+E01EF are valid
163
+ encoded characters and carry distinctions the visible font does not draw.
405
164
 
406
165
  ## The dictionary first
407
166
 
408
- The sound rules belong to the script and the exceptions belong to the language.
409
- With `lang=` naming a language that has a `note`, the build reads the
410
- language's `data.words` before the rows.
411
-
412
- **How a run is split into words.** A word is a run of letters, combining marks
413
- and digits, with an apostrophe (`'` or `’`), a hyphen or an en dash joining two
414
- such runs into one word. `k'aralzon` is one word and `zaron-lonak` is one word
415
- for lookup. Everything between words is spelled by the rows.
416
-
417
- **How a word is written.** The word is looked up as written, and then in
418
- lowercase. A word that states a `native` form for the run's mapping (for the
419
- run's face where an entry names one, else the one for no face) is written by it
420
- whole. Every other word, with every derived form of an exception word, is
421
- spelled by the rows. There are no stem-level exceptions: the plural of an
422
- exception word is another word, and the rows spell it.
423
-
424
- **A native form states only an exception.** `native` is a list of
425
- `{ map, font, text }` on a word:
167
+ A language's configured note supplies `data.words`, including the nearest
168
+ complete inherited definition from its `data.family` chain. Grammar and names
169
+ remain the current language's own. Native entries use `{enc, font?, text}`.
426
170
 
427
171
  ```yaml
428
- - form: zaron
429
- class: n
430
- field: Law
431
- gloss: an oath
432
- native:
433
- - { map: semaron, text: "\U000F0246\U000F0248\U000F023F\U000F023D\U000F023C" }
172
+ words:
173
+ - form: zaron
174
+ class: n
175
+ field: Law
176
+ gloss: an oath
177
+ native:
178
+ - { enc: vylsemaronenc, text: "\U000F024F\U000F0230\U000F023F\U000F023C" }
434
179
  ```
435
180
 
436
- `map` is the mapping's `name`, `font` optionally limits the form to one face, and
437
- `text` is the word as the script writes it. A form that the rows already give is
438
- redundant, and the lint says so:
439
-
440
- > `` `data.words[0].native[0]` states what the rows already give: the map `semaron` spells `zaron` so in the font `semaron-kuriale`; a native form is stated only where the rows spell the word differently ``
441
-
442
- so a dictionary holds its exceptions and nothing else. The other checks of a
443
- native form (an unknown map, a face the map is not for, a second form for one
444
- map, a character the face does not draw) report at the entry:
445
-
446
- > `` `data.words[0].native[0].map` is `semron`, which is not a map: a native form names the `data.map.name` of a mapping note; did you mean `semaron`? ``
447
-
448
- > `` `data.words[0].native[2]` states a second native form for the map `semaron`; the first is `data.words[0].native[0]` ``
449
-
450
- > `` `data.words[0].native[0]` holds U+F024F, which the font `semaron-kuriale`, listed in the map `semaron`, does not draw ``
451
-
452
- **Capitals.** A capitalised word that is found only in lowercase takes its
453
- capital from the rows: the rows spell the first sound as written and in
454
- lowercase, and where the native form begins with the lowercase spelling, that
455
- beginning is replaced by the capital. A native form that does not begin the way
456
- the rows begin the word cannot be capitalised, and the capitalised word is an
457
- error at its place in the run:
181
+ Every encoding tries the word's own writing, then complete native parts,
182
+ affixes and explicitly named grammatical endings, then its rows. A face-specific
183
+ entry takes precedence. A native entry repeating what all applicable rows give
184
+ is redundant and is reported at that entry.
458
185
 
459
- > `` `Orlan` is capitalised, and the native form the lexicon of `vyl` states for `orlan` under the map `semaron` does not begin as the rows spell its first letter, so the rows cannot capitalise it; state `Orlan` as a word of its own with its native form ``
186
+ Exact case and exact variants are tried before folded lookup. A lowercase
187
+ native exception can take an initial capital where the rows identify its first
188
+ sound in both cases and its native text begins with the lowercase result.
189
+ That prefix is replaced by the capital result. An ambiguous or uninferable
190
+ capital needs an explicitly authored case-specific entry. Explicit entries
191
+ retain authority; their pronunciation or glyph choices are not inferred.
460
192
 
461
- The fix is the one the message names: a lexicon word `Orlan` with its own
462
- native form, which the lookup as written finds first.
463
-
464
- A character the face does not draw inside a native form reports at the word:
465
-
466
- > `` `zaron` is written U+F024F by the lexicon of `vyl` for the map `semaron`, which the font `semaron-kuriale` does not draw ``
467
-
468
- A native run whose explicit `lang=` is a bare BCP 47 tag, and a declared
469
- language with no `note` are spelled by the rows alone.
193
+ A derived form uses a native grammatical ending only when the complete stem
194
+ and ending writings exist. Otherwise the rows spell the complete form. Missing
195
+ parts never invent signs. A bare encoding without a language uses its rows and
196
+ inline codes alone.
470
197
 
471
198
  ## The checks
472
199
 
473
200
  A run with `lang=`, whose language has a `note`, is checked word by
474
201
  word. This applies to Roman language-only runs and native runs alike;
475
- `.nativefont` selects presentation and does not turn vocabulary checks off. The comparison is case folded and NFC, with `’` read as `'`. A word passes
202
+ Encoding selection does not turn vocabulary checks off. The comparison is case folded and NFC, with `’` read as `'`. A word passes
476
203
  when it is any of these:
477
204
 
478
205
  1. **A lexicon word:** a `data.words[].form` of the language. A form listed in a
@@ -535,25 +262,24 @@ Bare `other`, `kept`, `ancient` and `translation` rows are not checked.
535
262
  noun stated with its own plural and the plural the rule gives both pass, and no
536
263
  check asks that a word takes the right ending.
537
264
 
538
- **Face coverage** follows the mapping. A character left over after the rows
539
- reports once per run, at the start of the matched `from`:
265
+ **Encoding coverage** refuses an unknown character at its source position:
540
266
 
541
- > `` `x` (U+0078) is not drawn by the font `semaron-kuriale`, and the map `semaron` has no row for it ``
267
+ > ``"x" (U+0078) is not in the `semaron` encoding``
542
268
 
543
269
  and a row that writes a code point the face lacks reports where it is applied:
544
270
 
545
- > `` `ch` maps to U+F0299 in the map `semaron`, which the font `semaron-kuriale` does not draw ``
271
+ > `` `ch` encodes to U+F0299 in `semaron`, which the font `semaron-kuriale` does not draw ``
546
272
 
547
273
  **Keys that name nothing.** A key that names nothing is the one finding, and
548
274
  the run is not then checked against a face or a lexicon it cannot reach.
549
275
 
550
276
  > `` `semaron-kurial` is not a font: `font=` names an installed face, or `mono`, `serif` or `sans`; did you mean `semaron-kuriale`? ``
551
277
 
552
- > `` `semron` is not a map: `map=` names the `data.map.name` of a mapping note; did you mean `semaron`? ``
278
+ > `` `semron` is not an encoding document ``
553
279
 
554
280
  > `` `q` is not a declared language (`languages` in package-build.config.yaml) or a BCP 47 tag ``
555
281
 
556
- > `` `thuravark-carved` is not a font of the map `semaron`; it is for `semaron-kuriale` ``
282
+ > `` `thuravark-carved` is not a font of encoding `semaron`; it is for `semaron-kuriale` ``
557
283
 
558
284
  > `` `semaron-kuriale` is not a font: `font=` names `mono`, `serif` or `sans`, or a face of @heroiclands/fonts, which this package has not installed ``
559
285
 
@@ -616,7 +342,7 @@ With the note above, a run accepts:
616
342
 
617
343
  A near miss such as _alathe_ is still an unknown word. To see every form the
618
344
  rules give, each beside the word it comes from, run
619
- `package-build lexicon vyl --map semaron --derived`.
345
+ `package-build lexicon vyl --enc vylsemaronenc --derived`.
620
346
 
621
347
  The keys are checked where they sit:
622
348
 
@@ -630,100 +356,71 @@ The keys are checked where they sit:
630
356
 
631
357
  > `` `data.affixes[1].attachesTo[0]` is `verb`, which is not a word class; did you mean `v`? ``
632
358
 
633
- ## Logographic scripts
634
-
635
- **A script that writes meanings is written from the dictionary, never by
636
- sound.** Its mapping note states `logographic: true`, and every word of a run
637
- under it comes from the language's lexicon. The rows write only the text
638
- between words: spaces, points and anything else with no letter. A word never
639
- falls through to the rows, so a word the lexicon gives no writing is an error
640
- rather than a guess.
641
-
642
- A word is written by the first of these that gives a writing, matched case
643
- folded because the script has no capitals:
644
-
645
- 1. **Its own `native` entry** for the map (the run's face where an entry names
646
- one, else the one for no face), or the entry of a set compound written its
647
- own way.
648
- 2. **Its parts, joined.** The roots of the entry's `built` list in order; each
649
- piece is a lexicon word or an affix of `data.affixes` with a `native` entry of
650
- its own, resolved the same way down. A noun's plural and a verb's present
651
- forms are the stem's writing followed by the ending's, which
652
- `data.grammar.native` states once per ending: `plural`, or one of the persons
653
- `I`, `you`, `heShe`, `we`, `youAll` and `they`.
654
- 3. **Its hyphenated parts,** each by these same steps, so `Gar-halzunet` is
655
- `Gar-`'s writing followed by `halzunet`'s.
656
- 4. **Otherwise an error** at the word.
657
-
658
- Forms outside the parts rule (a suffix that replaces a vowel, an agreeing form,
659
- a participle) have no writing of their own. They are coined as lexicon words
660
- with a writing, or written in signs.
359
+ ## Native spellings and parts
360
+
361
+ Every encoding uses the same order for a language word: its own `native`
362
+ entry for that encoding and face; otherwise the writings of all its parts;
363
+ otherwise the encoding's rows. A face-specific entry precedes an entry for no
364
+ face. Exact lexical forms precede folded lookup, so separately authored
365
+ capitalized spellings keep their own native text. A flat variant is a lexical
366
+ form; it does not assign a grammatical role by itself.
661
367
 
662
368
  ```yaml
663
- affixes:
664
- - form: "-unet"
665
- gloss: "a thing made of"
666
- native:
667
- - { map: zethu, text: "𞯇" }
668
- grammar:
669
- plural:
670
- - { ending: "", becomes: "im" }
671
- native:
672
- - { ending: plural, map: zethu, text: "𞯡" }
673
369
  words:
674
370
  - form: halzi
675
371
  class: n
676
- field: The body
677
372
  gloss: heart
373
+ field: Words
678
374
  native:
679
- - { map: zethu, text: "𞪻" }
375
+ - { enc: zethu, text: "\U0001EABB" }
680
376
  - form: halzunet
681
377
  class: n
682
- field: The body
683
378
  gloss: the account
379
+ field: Words
684
380
  built: [halzi, -unet]
381
+ affixes:
382
+ - form: -unet
383
+ gloss: a thing made of
384
+ native:
385
+ - { enc: zethu, text: "\U0001EBC7" }
685
386
  ```
686
387
 
687
- `[halzi halzunet halziim]{lang=khel font=khelathi-zethu-glyphs map=zethu .nativefont}` is
688
- written U+1EABB, then U+1EABB U+1EBC7, then U+1EABB U+1EBE1, with the spaces the
689
- rows write between them. An affix's `native` and `data.grammar.native` are read
690
- only under a logographic map; naming any other map is an error at the entry.
691
-
692
- **Ambiguity.** One roman form that stands for entries written differently is an
693
- error naming every entry's gloss, and so is one form with a writing for one
694
- sense and none for another. Nothing is guessed. List the words with
695
- `package-build lexicon khel --map zethu`: the third column says where each
696
- writing comes from, `own` for a stored writing, `parts` for one the pieces give,
697
- `none` for a word with no writing, and `ambiguous` for a form a run cannot
698
- write. An ambiguous word's second column is still that entry's own writing, so
699
- pick the entry by its gloss and write those signs directly in the run. Signs
700
- typed in a run hold no letter, so they are no word and pass through as typed.
701
-
702
- **Foreign names.** A registered name is written like any other word, so a name
703
- with no writing is an error. A foreign name is written deliberately: its signs
704
- typed directly, or through a mapping whose rows read a sign-by-sign notation
705
- (sign ids and joiners, as in Manuel de Codage input), which an author writes the
706
- way any other mapping is written. A native run with an explicit bare BCP 47
707
- tag such as `lang=und` has no configured dictionary and uses the rows alone;
708
- it still needs `.nativefont`, an explicit map for conversion and an explicit
709
- font where no language default supplies one.
710
-
711
- **An unknown word gets two errors:** the lexicon check reports it is no word of
712
- the language, and the writing check reports it has no writing.
713
-
714
- > `` `zamlu` has no `zethu` writing in the `khel` lexicon, and its parts do not give one; state one, or write the word directly in signs ``
715
-
716
- > `` `nelgir` stands for 2 words of the `khel` lexicon that the map `zethu` writes differently, “a god” and “god”; write the word directly in signs, choosing by its sense in `package-build lexicon khel --map zethu` ``
717
-
718
- > `` `halzi` has no `zethu` writing: the map `zethu` is logographic, and the run names no `lang=` whose lexicon writes it; mark the run with its language, or write the word directly in signs ``
719
-
720
- > `` `data.grammar.native[3]` writes the ending `they`, and `data.grammar.verbs` states no `persons` ``
388
+ A built part is an exact word or affix, then the same form with joining
389
+ hyphens removed. Parts can themselves be built. A plural or present form can
390
+ use the stem's writing followed by its explicitly named ending from
391
+ `grammar.native`, whose entries use `{ending, enc, font?, text}`. These
392
+ writings work under every encoding. Missing parts do not invent signs, and
393
+ recursive parts terminate without a writing. Entries with genuinely different
394
+ writings remain an ambiguity naming every matching gloss.
395
+
396
+ An encoding accepting no letters, such as `zethu` or `chalish-pilam`, needs a
397
+ lexical writing or literal codes for a word. The refusal names the word,
398
+ language and encoding and points to that word.
399
+
400
+ ### Reading script text
401
+
402
+ Without a language, decoding is exactly the encoding's longest-output match.
403
+ With a language, explicit native and composed writings are tried first,
404
+ longest first; a shared writing uses the first listed sense. Literal marker
405
+ spans decode through their inline codes. Unknown unmarked signs in a script
406
+ accepting no letters become `⟦codes⟧` with a finding.
407
+
408
+ For an unmarked word, the lexicon can recover its unwritten sounds from the
409
+ plain decoded skeleton. A unique skeleton gives its form; a shared skeleton
410
+ gives the first authored form. Each word's citation precedes its variants and derived forms, before the next word's citation. Registered names retain their authored order. Complete names are matched longest first across word separators and receive one pair of braces. A known ordinary word keeps its encoded case without a false unknown-word finding. An unknown skeleton keeps the
411
+ plain decoding and produces a located finding. This is decided per word: a
412
+ passage can mix typed words with encoded words. A word carrying any invisible
413
+ mark keeps exact decoding without skeleton recovery; an explicit authored
414
+ native exception still has precedence. Stress marks are preserved by exact
415
+ rows. Names are represented with braces according to the language's rules.
721
416
 
722
417
  ## Emphasis in a script
723
418
 
724
- Inside a run set in a face, `*…*` and `**…**` are written as in English. No
725
- script marks a stressed word by a bold or a slant, so the build sets emphasis by
726
- the scribal convention the text's language or script states:
419
+ Inside a language run, mark a stressed word as `[word]{.emph}`. The span
420
+ inherits its surrounding language, face and encoding, so its letters follow
421
+ the same writing rules as the rest of the run. In English, `.emph` equals
422
+ `*…*`, and `.strong` equals `**…**`. Non-English spans refuse `.strong` and
423
+ Markdown emphasis. The build sets `.emph` by the language's or script's convention:
727
424
 
728
425
  | Convention | Site and Foundry | Book |
729
426
  | ---------- | -------------------------------------------------- | --------------------- |
@@ -746,12 +443,11 @@ data:
746
443
  script:
747
444
  fonts: [semaron-kuriale]
748
445
  emphasis: rubric
749
- strongEmphasis: overline
750
446
  ```
751
447
 
752
448
  `fonts` lists the faces of the script (a face belongs to one script),
753
- `emphasis` sets both `*…*` and `**…**`, and `strongEmphasis` optionally sets
754
- `**…**` alone. A tongue that states nothing takes its script's convention.
449
+ `emphasis` sets `.emph` and accepted English emphasis. A tongue that states
450
+ nothing takes its script's convention.
755
451
 
756
452
  **Non-English spans do not take Markdown emphasis.** Bold and italic markup in
757
453
  a span with an effective non-English language is an error, whether it has a
@@ -760,13 +456,12 @@ English (`en`) and English BCP 47 tags remain eligible. Escaped literal
760
456
  delimiters are not emphasis. This restriction applies throughout the note.
761
457
 
762
458
  `data.emphasis: words` also makes emphasis an error: it says the language marks
763
- stress with a word, so stars in its runs are refused:
459
+ stress with a word, so `.emph` and stars in its runs are refused:
764
460
 
765
461
  > `` `ban` marks emphasis with a word, not a mark; write the word ``
766
462
 
767
- Underscores and strikethrough are errors in a face. A stressed word is best
768
- enclosed whole: a star inside a word splits it, so a row or lexicon word that
769
- crosses the star is not matched.
463
+ A strikethrough is an error in a face. Mark a whole stressed word: a mark
464
+ inside a word splits the text, so a row or lexicon word crossing it cannot match.
770
465
 
771
466
  **The `--rubric` token.** The theme's `--rubric` is a warm red for its dark
772
467
  ground, and the book prints its own. A rubric carries emphasis by colour alone,
@@ -792,8 +487,8 @@ the contents of a fence byte for byte, which is why a poem is a fence and not
792
487
  paragraphs: indentation, line breaks and spans survive formatting untouched.
793
488
 
794
489
  ````markdown
795
- ```verse {lang=vyl font=semaron-kuriale map=semaron .nativefont}
796
- Lokazor lonak,
490
+ ```verse {lang=vyl font=semaron-kuriale enc=vylsemaronenc}
491
+ Lokazor lonak.
797
492
  lokamen zaron.
798
493
  ```
799
494
  ````
@@ -816,20 +511,20 @@ fills it, and the last row begins a new stanza:
816
511
 
817
512
  ````markdown
818
513
  ```verse {numstep=1 lang=vyl cols=3}
819
- 1 [Lokazor lonak zaron.]{lang=vyl .nativefont}
514
+ 1 [Lokazor lonak zaron.]{lang=vyl enc=vylsemaronenc}
820
515
  3 [He walks the oath road.]{lang=en}
821
- 3 [Lokazor lonak zaron.]{lang=vyl}
516
+ 3 [Lokazor lonak zaron.]{lang=vyl enc=default}
822
517
 
823
- 1 [Lokamen lonak zaron.]{lang=vyl .nativefont}
518
+ 1 [Lokamen lonak zaron.]{lang=vyl enc=vylsemaronenc}
824
519
  2 [Nothing to say]{lang=en}
825
520
  3 [I walk the oath road.]{lang=en}
826
- 3 [Lokamen lonak zaron.]{lang=vyl}
521
+ 3 [Lokamen lonak zaron.]{lang=vyl enc=default}
827
522
 
828
523
  ***
829
524
 
830
- 1 [alathos lokazor.]{lang=vyl .nativefont}
525
+ 1 [alathos lokazor.]{lang=vyl enc=vylsemaronenc}
831
526
  3 [That one walks.]{lang=en}
832
- 3 [alathos lokazor.]{lang=vyl}
527
+ 3 [alathos lokazor.]{lang=vyl enc=default}
833
528
  ```
834
529
  ````
835
530
 
@@ -935,7 +630,7 @@ paragraphs, and each paragraph starts at the top of a fresh column. `rows=`
935
630
  sets the characters in a column and `cols=` the columns in a panel:
936
631
 
937
632
  ````markdown
938
- ```scroll {rows=25 cols=12 lang=tsu map=tengchou .nativefont}
633
+ ```scroll {rows=25 cols=12 lang=tsu enc=tsutengchouenc}
939
634
  chen teng chou hing
940
635
  tan tsou dou rou
941
636
 
@@ -943,8 +638,8 @@ gen chei chou tsuen
943
638
  ```
944
639
  ````
945
640
 
946
- Where the Tantsou note states `writtenIn: { map: tengchou, font: teng-chou }`,
947
- ` ```scroll {rows=25 cols=12 lang=tsu .nativefont} ` is enough.
641
+ The Tantsou encoding document names its language and the `teng-chou` font;
642
+ its Address selects both. A language-only scroll keeps Roman text.
948
643
 
949
644
  The site, the preview and Foundry set it in columns, in a panel that scrolls
950
645
  sideways from the right. The book draws it as panel images from the face's
@@ -1004,38 +699,35 @@ package's own output: the site copies `fonts.css`, the WOFF2 files and the
1004
699
  licence to `fonts/heroiclands/<version>/` beside the pages and links them from
1005
700
  every page; Foundry stages the same files; the book adds the TrueType files to
1006
701
  the typesetter's font search path. A package that has not installed the fonts, or
1007
- whose notes set no text in a face, ships none of it.
702
+ whose notes select no native face, ships none of it.
1008
703
 
1009
704
  ## The commands
1010
705
 
1011
- `package-build transliterate <map> [--lang <language>] [--font <slug>] [--hex] [text..]`
1012
- maps a passage exactly as the build maps a run marked
1013
- `lang=<language> .nativefont map=<map> font=<slug>`. The text is the words given; `-` or none reads standard input.
1014
- `--font` defaults to the language's `writtenIn` face where the mapping is drawn
1015
- in it, and is otherwise required when the mapping lists more than one face.
1016
- `--hex` prints `U+XXXX` code points separated by spaces.
1017
-
1018
- ```console
1019
- $ package-build transliterate semaron --lang vyl --hex zaron lonak
1020
- U+F0246 U+F0248 U+F023F U+F023D U+F023C U+0020 U+F023A U+F023D U+F023C U+F0230 U+F0239
1021
- $ package-build transliterate semaron --hex zaron
1022
- U+F0246 U+F0230 U+F023F U+F023D U+F023C
706
+ `package-build encode --enc <name|file> [--in|-i <file>] [--out|-o <file>] [text..]`
707
+ encodes one string argument or the UTF-8 file named by `--in`; `--in -` reads
708
+ standard input. `package-build decode` takes the same options and reads code
709
+ points back through the encoding. An encoding is a document Address, with `doc` as the default type.
710
+ These commands apply the bare encoding. The [browser conversion API](../api.md#browser-conversion)
711
+ uses the same implementation with a language's vocabulary and lexical recovery.
712
+
713
+ ```sh
714
+ package-build encode --enc vylsemaronenc "zaron lonak"
715
+ package-build encode --enc khelqalezuenc --in roman.txt --out native.txt
716
+ package-build decode --enc khelqalezuenc --in native.txt --out roman.txt
717
+ package-build encode --enc khelzethuenc "⟦MAN⟧"
1023
718
  ```
1024
719
 
1025
- The first run writes `zaron` by its exception, and the second, with no
1026
- `--lang`, spells it by the rows.
1027
-
1028
- `package-build lexicon <language> [--map <map>] [--font <slug>] [--derived] [--json]`
1029
- lists every word of the language with its native form under the map, or under
1030
- the language's `writtenIn` script when `--map` is left out, tab-separated: the form,
1031
- the native form, `exception` or `rows`, the class and the gloss. `--derived`
720
+ `package-build lexicon <language> [--enc <name>] [--font <slug>] [--derived] [--json]`
721
+ lists every word of the language with its native form under the encoding, or under
722
+ the explicitly selected encoding document, tab-separated: the form,
723
+ the native form, `own`, `parts`, `rows` or `none`, the class and the gloss. `--derived`
1032
724
  adds the forms the language's rules derive, each with the word it comes from,
1033
- `--json` prints an array instead, and under a logographic map the third column
725
+ `--json` prints an array instead, and the third column
1034
726
  is `own`, `parts`, `none` or `ambiguous`.
1035
727
 
1036
728
  ```console
1037
- $ package-build lexicon vyl --map semaron | cut -f1,3-
1038
- zaron exception n an oath
729
+ $ package-build lexicon vyl --enc vylsemaronenc | cut -f1,3-
730
+ zaron own n an oath
1039
731
  lonak rows n a road
1040
732
  lokathen rows v to walk
1041
733
  alathos rows pron he, that one
@@ -1049,8 +741,8 @@ thiruthrimar rows n a sworn witness
1049
741
 
1050
742
  This builds a small package from nothing: a language, a script, a couplet and a
1051
743
  poem. It is the guide's own test: every file below is written into a project and
1052
- linted, and the findings printed are the ones the lint reports. The mapping note
1053
- is the one from [mapping notes](#mapping-notes).
744
+ linted, and the findings printed are the ones the lint reports. The sample selects the
745
+ installed Semaron encoding.
1054
746
 
1055
747
  Install the fonts, then write the configuration. A package that declares
1056
748
  dependencies in its configuration fetches their indexes with
@@ -1114,7 +806,7 @@ name: { full: Vylari Language }
1114
806
  type: skill
1115
807
  subType: language
1116
808
  data:
1117
- writtenIn: { map: semaron, font: semaron-kuriale }
809
+ roman: "ABDEFGHIKLMNOPRSTUVWYZabdefghiklmnoprstuvwyzÁÉÍÓÚÝáéíóúýËëÀà0123456789"
1118
810
  core: true
1119
811
  fields:
1120
812
  - { name: Law, holds: "Oaths, seals and roads." }
@@ -1152,7 +844,7 @@ data:
1152
844
  gloss: an oath
1153
845
  concept: oath-noun
1154
846
  native:
1155
- - { map: semaron, text: "󰉆󰉈󰈿󰈽󰈼" }
847
+ - { enc: vylsemaronenc, text: "󰉆󰉈󰈿󰈽󰈼" }
1156
848
  - { form: lonak, class: n, field: Law, gloss: a road, concept: road-noun }
1157
849
  - { form: lokathen, class: v, field: Law, gloss: to walk, concept: walk-verb }
1158
850
  - { form: alathos, class: pron, field: Law, gloss: "he, that one" }
@@ -1162,6 +854,196 @@ The imperial tongue of the sample setting, written in the Semaron script and
1162
854
  read aloud at its court. The lexicon here is a small sample made for the guide.
1163
855
  ```
1164
856
 
857
+ **`assets/content/Docs/Vylari_Encoding.md`**
858
+
859
+ ```markdown
860
+ ---
861
+ type: doc
862
+ subType: reference
863
+ shortcode: vylsemaronenc
864
+ name:
865
+ full: Vylari in Semaron
866
+ data:
867
+ language: vyl
868
+ font: semaron-kuriale
869
+ accepts:
870
+ - a-z
871
+ - A-Z
872
+ - 0-9
873
+ - "'"
874
+ - "-"
875
+ - "`"
876
+ - Á
877
+ - É
878
+ - Í
879
+ - Ó
880
+ - Ú
881
+ - á
882
+ - é
883
+ - í
884
+ - ó
885
+ - ú
886
+ - ’
887
+ vowels: aeiouAEIOUáéíóúÁÉÍÓÚ
888
+ mappings:
889
+ "0": DIGIT_ZERO
890
+ "1": DIGIT_ONE
891
+ "2": DIGIT_TWO
892
+ "3": DIGIT_THREE
893
+ "4": DIGIT_FOUR
894
+ "5": DIGIT_FIVE
895
+ "6": DIGIT_SIX
896
+ "7": DIGIT_SEVEN
897
+ "8": DIGIT_EIGHT
898
+ "9": DIGIT_NINE
899
+ a: ANKRON_MIN
900
+ A: ANKRON_MAJ
901
+ e: ELZA_MIN
902
+ E: ELZA_MAJ
903
+ i: ILTHERA_MIN
904
+ I: ILTHERA_MAJ
905
+ o: ORNETHE_MIN
906
+ O: ORNETHE_MAJ
907
+ u: ULMERA_MIN
908
+ U: ULMERA_MAJ
909
+ b: BRUTHEKA_MIN
910
+ B: BRUTHEKA_MAJ
911
+ d: DEKAZU_MIN
912
+ D: DEKAZU_MAJ
913
+ f: FUNELA_MIN
914
+ F: FUNELA_MAJ
915
+ g: GULMOS_MIN
916
+ G: GULMOS_MAJ
917
+ h: HARLEN_MIN
918
+ H: HARLEN_MAJ
919
+ j: JA_MIN
920
+ J: JA_MAJ
921
+ k: KALPEZA_MIN
922
+ K: KALPEZA_MAJ
923
+ l: LIZELA_MIN
924
+ L: LIZELA_MAJ
925
+ m: MEZARA_MIN
926
+ M: MEZARA_MAJ
927
+ n: NELAMIS_MIN
928
+ N: NELAMIS_MAJ
929
+ p: POZERA_MIN
930
+ P: POZERA_MAJ
931
+ q: QORETHA_MIN
932
+ Q: QORETHA_MAJ
933
+ r: RONETHOS_MIN
934
+ R: RONETHOS_MAJ
935
+ s: SIRETHA_MIN
936
+ S: SIRETHA_MAJ
937
+ t: TALOTHOS_MIN
938
+ T: TALOTHOS_MAJ
939
+ v: VARTHOS_MIN
940
+ V: VARTHOS_MAJ
941
+ w: WA_MIN
942
+ W: WA_MAJ
943
+ y: YALZEMIS_MIN
944
+ Y: YALZEMIS_MAJ
945
+ z: ZARGON_MIN
946
+ Z: ZARGON_MAJ
947
+ ch: CHEROLA_MIN
948
+ Ch: CHEROLA_MAJ
949
+ sh: SHELMORA_MIN
950
+ Sh: SHELMORA_MAJ
951
+ th: THOLERA_MIN
952
+ Th: THOLERA_MAJ
953
+ dh: DHANETHE_MIN
954
+ Dh: DHANETHE_MAJ
955
+ kh: KHORVENA_MIN
956
+ Kh: KHORVENA_MAJ
957
+ gh: GHORMAS_MIN
958
+ Gh: GHORMAS_MAJ
959
+ zh: ZHAREVA_MIN
960
+ Zh: ZHAREVA_MAJ
961
+ ts: TSALEMA_MIN
962
+ Ts: TSALEMA_MAJ
963
+ ng: ENGARIS_MIN
964
+ Ng: ENGARIS_MAJ
965
+ .: POINT
966
+ á:
967
+ - ANKRON_MIN
968
+ - STRESS
969
+ é:
970
+ - ELZA_MIN
971
+ - STRESS
972
+ í:
973
+ - ILTHERA_MIN
974
+ - STRESS
975
+ ó:
976
+ - ORNETHE_MIN
977
+ - STRESS
978
+ ú:
979
+ - ULMERA_MIN
980
+ - STRESS
981
+ Á:
982
+ - ANKRON_MAJ
983
+ - STRESS
984
+ É:
985
+ - ELZA_MAJ
986
+ - STRESS
987
+ Í:
988
+ - ILTHERA_MAJ
989
+ - STRESS
990
+ Ó:
991
+ - ORNETHE_MAJ
992
+ - STRESS
993
+ Ú:
994
+ - ULMERA_MAJ
995
+ - STRESS
996
+ rules:
997
+ normalization: NFC
998
+ matching: longest-first
999
+ case:
1000
+ single-letter: preserve
1001
+ consonant-group: lowercase-or-initial-uppercase
1002
+ other-group-cases: forbidden
1003
+ stress:
1004
+ modifier: STRESS
1005
+ position: after-stressed-vowel
1006
+ printing: false
1007
+ effect: stress-only
1008
+ changes-sound: false
1009
+ changes-length: false
1010
+ vowel-group-position: first-vowel
1011
+ vowel-groups:
1012
+ reading: established
1013
+ allow-splitting: false
1014
+ unrecognized-adjacent-vowels: separate
1015
+ groups:
1016
+ - ai
1017
+ - au
1018
+ - ei
1019
+ - oi
1020
+ - ou
1021
+ - ae
1022
+ - ie
1023
+ - uo
1024
+ - aa
1025
+ - ii
1026
+ - uu
1027
+ diaeresis:
1028
+ allowed: false
1029
+ contextual-mappings:
1030
+ - match: ny
1031
+ before: vowel
1032
+ emit: NYELIS_MIN
1033
+ - match: Ny
1034
+ before: vowel
1035
+ emit: NYELIS_MAJ
1036
+ y:
1037
+ kind: consonant
1038
+ sound: y in yes
1039
+ vowel-reading: false
1040
+ grave-accent:
1041
+ allowed: false
1042
+ ---
1043
+
1044
+ These mappings write the sample vocabulary in Semaron Kuriale. They preserve its letter case and use the font's published glyph codes. Contextual groups take their stated before-vowel spelling, and stress is retained after the accented vowel so that the written distinction can be read back exactly.
1045
+ ```
1046
+
1165
1047
  A note with a run. The couplet uses a word the lexicon does not hold:
1166
1048
 
1167
1049
  **`assets/content/Lore/Oath_Road.md`**
@@ -1177,7 +1059,7 @@ The lintel over the door of the oath road is cut with a couplet, and the
1177
1059
  stonecutter set the couplet in the script of the court, so that every traveller
1178
1060
  could read it as the court reads it.
1179
1061
 
1180
- [Lokazor lonak thiruthrimar, lokamen zaron.]{lang=vyl font=semaron-kuriale map=semaron .nativefont}
1062
+ [Lokazor lonak thiruthrimar. lokamen zaron.]{lang=vyl font=semaron-kuriale enc=vylsemaronenc}
1181
1063
  ```
1182
1064
 
1183
1065
  Linting reports the unknown word, at its first character:
@@ -1198,7 +1080,7 @@ name: { full: Vylari Language }
1198
1080
  type: skill
1199
1081
  subType: language
1200
1082
  data:
1201
- writtenIn: { map: semaron, font: semaron-kuriale }
1083
+ roman: "ABDEFGHIKLMNOPRSTUVWYZabdefghiklmnoprstuvwyzÁÉÍÓÚÝáéíóúýËëÀà0123456789"
1202
1084
  core: true
1203
1085
  fields:
1204
1086
  - { name: Law, holds: "Oaths, seals and roads." }
@@ -1236,7 +1118,7 @@ data:
1236
1118
  gloss: an oath
1237
1119
  concept: oath-noun
1238
1120
  native:
1239
- - { map: semaron, text: "󰉆󰉈󰈿󰈽󰈼" }
1121
+ - { enc: vylsemaronenc, text: "󰉆󰉈󰈿󰈽󰈼" }
1240
1122
  - { form: lonak, class: n, field: Law, gloss: a road, concept: road-noun }
1241
1123
  - { form: lokathen, class: v, field: Law, gloss: to walk, concept: walk-verb }
1242
1124
  - { form: alathos, class: pron, field: Law, gloss: "he, that one" }
@@ -1247,14 +1129,16 @@ The imperial tongue of the sample setting, written in the Semaron script and
1247
1129
  read aloud at its court. The lexicon here is a small sample made for the guide.
1248
1130
  ```
1249
1131
 
1250
- The lint is now clean, and the conversion can be read. `package-build transliterate`
1251
- prints the code points the build writes for the couplet:
1132
+ The lint is now clean, and the conversion can be read. `package-build encode`
1133
+ checks a passage through the encoding's rows:
1252
1134
 
1253
- ```console
1254
- $ package-build transliterate semaron --lang vyl --hex "Lokazor lonak, lokamen zaron."
1255
- U+F020A U+F023D U+F0239 U+F0230 U+F0246 U+F023D U+F023F U+0020 U+F023A U+F023D U+F023C U+F0230 U+F0239 U+F0268 U+0020 U+F023A U+F023D U+F0239 U+F0230 U+F023B U+F0233 U+F023C U+0020 U+F0246 U+F0248 U+F023F U+F023D U+F023C U+F0268
1135
+ ```sh
1136
+ package-build encode --enc vylsemaronenc "Lokazor lonak. lokamen zaron."
1256
1137
  ```
1257
1138
 
1139
+ The build also applies the language's recorded native writings when setting
1140
+ its runs.
1141
+
1258
1142
  The site sets the run as
1259
1143
 
1260
1144
  ```html
@@ -1264,8 +1148,8 @@ The site sets the run as
1264
1148
  ```
1265
1149
 
1266
1150
  where the characters are the code points above: `Lokazor` begins with the
1267
- capital `L` of the rows, `zaron` is the exception's long vowel, and the comma
1268
- and full stop are the script's one point.
1151
+ capital `L` of the rows, `zaron` is the exception's long vowel, and each period
1152
+ writes the script's pause point.
1269
1153
 
1270
1154
  A poem in columns and a poem with its metre checked sit on one note. Both parse
1271
1155
  with no finding: the first sets native Vylari in column one and English over Roman
@@ -1285,20 +1169,20 @@ beside its translation, and the second keeps the metre that the court keeps, so
1285
1169
  that a line with the wrong count is reported to the author who wrote it.
1286
1170
 
1287
1171
  ```verse {numstep=1 lang=vyl cols=3}
1288
- 1 [Lokazor lonak zaron.]{lang=vyl .nativefont}
1172
+ 1 [Lokazor lonak zaron.]{lang=vyl enc=vylsemaronenc}
1289
1173
  3 [He walks the oath road.]{lang=en}
1290
- 3 [Lokazor lonak zaron.]{lang=vyl}
1174
+ 3 [Lokazor lonak zaron.]{lang=vyl enc=default}
1291
1175
 
1292
- 1 [Lokamen lonak zaron.]{lang=vyl .nativefont}
1176
+ 1 [Lokamen lonak zaron.]{lang=vyl enc=vylsemaronenc}
1293
1177
  2 [Nothing to say]{lang=en}
1294
1178
  3 [I walk the oath road.]{lang=en}
1295
- 3 [Lokamen lonak zaron.]{lang=vyl}
1179
+ 3 [Lokamen lonak zaron.]{lang=vyl enc=default}
1296
1180
 
1297
1181
  ***
1298
1182
 
1299
- 1 [alathos lokazor.]{lang=vyl .nativefont}
1183
+ 1 [alathos lokazor.]{lang=vyl enc=vylsemaronenc}
1300
1184
  3 [That one walks.]{lang=en}
1301
- 3 [alathos lokazor.]{lang=vyl}
1185
+ 3 [alathos lokazor.]{lang=vyl enc=default}
1302
1186
  ```
1303
1187
 
1304
1188
  ```verse {lang=vyl scan=trimeter}
@@ -1341,7 +1225,7 @@ The door is marked with the same words as the lintel. Although Semaron's
1341
1225
  convention is rubric, this Vylari span uses no Markdown emphasis: spans with a
1342
1226
  non-English language cannot contain bold or italic markup.
1343
1227
 
1344
- [Lokazor lonak, zaron.]{lang=vyl font=semaron-kuriale map=semaron .nativefont}
1228
+ [Lokazor lonak. zaron.]{lang=vyl font=semaron-kuriale enc=vylsemaronenc}
1345
1229
  ```
1346
1230
 
1347
1231
  Finally, the core report lists what the language still lacks:
@@ -1354,6 +1238,6 @@ Vylari Language (skill-vylarilng): 1 of 4 core concepts have no word
1354
1238
  ```
1355
1239
 
1356
1240
  Everything in this package comes from the guide. A new script follows the same
1357
- steps: a mapping note for its face, `languages` entries for the languages
1241
+ steps: an encoding document for its language and font, `languages` entries for the languages
1358
1242
  written in it, a language note with the dictionary, and runs marked with the
1359
1243
  language key and native presentation class.