@kensio/pinyinjs 1.5.2 → 1.6.0

package/CHANGELOG.md

@@ -9,6 +9,169 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
 
 ## Unreleased
 
+## 1.6.0
+
+### Added
+
+- **Hanzi and pinyin together, rather than one instead of the other.**
+  `convertToAnnotatedHtml` writes the characters with their reading above them,
+  as `<ruby>` markup — the output every other mode could not give, and what a
+  learner's text, a subtitle and a dictionary entry all want. `convert` and
+  `convertToHtml` write the reading _instead of_ the characters.
+  `toAnnotatedHtml(pieces)` renders pieces you already have, and the command
+  line has `pinyinjs annotate`.
+
+  Each `<rt>` holds exactly what `convertToHtml` would have written, so tone
+  classes and uncertainty marking work inside an annotation as they do outside
+  one. `<rp>` gives the parenthesised fallback, the `<ruby>` carries `lang="zh"`
+  and the syllables inside it keep `zh-Latn-CN-pinyin`, since on an English page
+  a screen reader needs to be told both.
+
+  **A base is not always one character**, which is what makes this harder than
+  it looks and what every naive per-character annotation gets wrong. 玩儿 is two
+  characters and the one syllable `wánr`; 95% is `bǎifēnzhījiǔshíwǔ` over three
+  written characters in the other order; 1988 is `yī jiǔ bā bā` over one. All
+  are annotated whole. `ConvertedPiece` gains a `source` — the characters a
+  piece reads, or undefined where it reads on into the ones named before it —
+  which is what makes that expressible.
+
+  Pinyin orthography stays out of the hanzi it annotates: no word spaces, since
+  Chinese is not written with them; the hyphen of `gāngān-jìngjìng` sits in the
+  reading, being a boundary inside one word rather than a gap between two; and
+  the base keeps 。 where the conversion writes a full stop.
+
+- **`readings`: a conversion takes readings the caller asserts.** No rule
+  settles every polyphone, and some texts are genuinely ambiguous —
+  孩子越长越漂亮 grows where 头发越长越漂亮 lengthens, and nothing in the
+  characters says which. An application that knows its own content can now say
+  so, rather than the library guessing on its behalf.
+
+  ```ts
+  convert(dictionary, "这篇文章不太长。", { readings: { 太长: "tài cháng" } });
+  ```
+
+  A **word** hint asserts something about the text it names and rewrites exactly
+  those characters, so a bare `长` leaves 校长 as `xiàozhǎng` — the dictionary
+  knowing that word is better evidence about the stretch than a remark about one
+  of its characters, and that is what makes a corrections table safe to
+  accumulate. Naming the whole word does reach it, and keeps it whole:
+  `{ 银行: "yín xíng" }` gives `yínxíng`.
+
+  A **positional** hint asserts something about one character of one text, and
+  nothing outranks it, the enclosing word included. Positions count code points
+  from the start of the text, across any non-Han runs in it. The list form takes
+  both kinds together.
+
+  Hints enter as lattice edges rather than as rules, because the rules layer
+  cannot invent a reading no source attests — which is exactly what a hint is,
+  the difference being that a caller attests it. Spacing is untouched, an
+  unmarked syllable is 轻声, and a hint that cannot be parsed throws rather than
+  being skipped: a correction that silently does nothing is worse than one that
+  fails.
+
+### Fixed
+
+- **Polyphones defaulted to whichever reading a dictionary happened to print
+  first.** Unihan's `kTGHZ2013` ranked above `kMandarin`, and that field
+  prefixes every reading with its entry number in 《現代漢語規範詞典》 — 勒 is
+  `212.050:lè 212.100:lēi` — so its order is a page order, and taking the first
+  reading meant taking whichever one sorted earlier by pinyin. 殷 read `yān` and
+  楷 `jiē`. The dictionary-indexed fields now rank last: they are a good
+  _source_ of readings and a meaningless _ranking_ of them.
+
+  `kHanyuPinlu` still leads where it lists more than one reading, because there
+  it has counted them against each other, but where it lists one it counted
+  nothing — it is saying "this occurred", not "this leads" — and a lone reading
+  now ranks last too. Only ten characters have one that disagrees with
+  `kMandarin`, and eight of those are simply pre-1985: the corpus behind the
+  field, 《現代漢語頻率詞典》, predates the 普通话异读词审音表, and the 132
+  occurrences carrying 绩 as `jī` are the old standard rather than a case for
+  it.
+
+  354 readings fixed against 94 broken over 25 characters, the largest being 贾
+  `gǔ`→`jiǎ`, 苔 `tāi`→`tái`, 町 `dīng`→`tǐng`, 殷 `yān`→`yīn` and 芥
+  `gài`→`jiè`. CPP's polyphone score is 90.34%, up from 89.05%, and the gold
+  corpus does not move. What broke is mostly `kMandarin` being wrong with
+  nothing behind it — 媛 `yuán`→`yuàn`, 桔 `jú`→`jié`, 耶 `yē`→`yé`, 缪
+  `miào`→`móu` — which needs a third opinion with corpus evidence behind it
+  rather than an override, since here the sources disagree. 䘚 is now paired
+  with 卒, so `core` and `standard` each gained a key.
+
+- **`full` read 特徵 as `tèzhǐ` where `standard` read it right.**
+  `large_pinyin.txt` is a 简体 corpus, and where it carries a 繁體 headword
+  anyway it has read it as though the characters were 简体, taking each at face
+  value instead of as the 简体 character it stands for: 徵 as itself rather than
+  as 征, 沈 as the surname rather than as 沉, so 沈溺 came out `shěnnì`. Of the
+  2,854 corpus headwords carrying a 繁體-only character, 72 are ones CC-CEDICT
+  pairs with a 简体 form and 13 of those read wrongly.
+
+  The reading was only half of it. A corpus headword becomes an entry of its
+  own, and an entry outranks the 繁體 key that the traditional derivation hangs
+  on the 简体 entry — so the word was in the dictionary twice, and which copy
+  answered depended on the tier. `standard` excludes the phrase tail and was
+  right; `full`, the default, was wrong. Loading more dictionary made the answer
+  worse, which is also the upgrade path the docs describe, of converting with
+  `standard` and reloading as `full`.
+
+  A headword CC-CEDICT knows _only_ as another word's 繁體 spelling now
+  contributes no entry, and the word keeps the reading CC-CEDICT pairs it with.
+  68 entries drop out and their keys are re-claimed from the 简体 entry, so
+  `full` loses 2 keys; 简→繁→简 round trips gain 38 against 5 lost on 繁→简→繁,
+  both unchanged as percentages. The 2,782 headwords CC-CEDICT does not pair are
+  rare or mixed-script words whose characters do read them, and are left alone.
+  Three of the words are build assertions now, this being the kind of defect
+  that comes back when a rolling source is refreshed and nothing is watching.
+
+- **长 read `zhǎng` wherever no word covered it**, so 这篇文章不太长 came out
+  `bú tài zhǎng` and 要多长时间 `duō zhǎng shíjiān`. The stored default is not
+  the defect — Unihan ranks the readings `zhǎng(1879) cháng(1179)` and names
+  `zhǎng` in `kMandarin`, and that ranking is what reads 署长, 团长, 公安局长 and
+  总会长 correctly off a bare 长 at the end of a title. What was missing is that
+  nothing could prefer the alternate at a position no word covers.
+
+  A degree adverb settles it, and only from the left: a growing 长 is a verb and
+  no 很, 太, 最 or 多 can modify one, while what follows an adjectival 长 is a
+  noun, a particle or the end of the sentence — which is what follows half the
+  verbs too. 得, 着 and the 越…越 correlative are the exceptions, and are guarded;
+  了 and 的 are deliberately not, since after an adverb they are the sentence
+  particle and the attributive, making 时间太长了 and 很长的道路 both `cháng`.
+
+  Measured over the same 88,866 lines of Tatoeba and zh.wikipedia the other
+  edge rules were sized against, 260 长 decode as a word of their own and this
+  moves 75 to `cháng`, all 75 correctly. On CPP's 40 hand-labelled 长 the
+  character goes 85.00% to 87.50%; overall CPP is unmoved at 90.34%, as expected
+  of one character in 20,139.
+
+- **越长越高 read `yuè cháng yuè gāo`.** 越长 is a key read `yuè cháng` and the
+  only one of its shape — 越大, 越高, 越好 and 越快 are all absent, so 越高越好
+  decodes as two words while 越长越高 reached for a word nothing else in the
+  paradigm has. It carries no part of speech, which is how a reading somebody
+  asserted is held rather than a word anybody counted, and only
+  `large_pinyin.txt` has it. Where the far half of the correlative names
+  something growing produces — 高, 大, 胖, 壮, 结实 — the edge is dropped and the
+  character's own `zhǎng` stands.
+
+  Unlike everything else here this is a heuristic with no corpus behind it: 越长
+  occurs three times in the 88,866 lines and all three are 越来越长 or 说的越长.
+  It forbids rather than forces, so `cháng` stays on as a rival one bucket
+  dearer and the syllable still reports itself uncertain — 越长越X is genuinely
+  ambiguous, 孩子越长越漂亮 growing where 头发越长越漂亮 lengthens, which is why
+  漂亮 is not in the list. 时间越长越好 and 头发越长 are untouched.
+
+## 1.5.3
+
+### Fixed
+
+- **The changelog inside the package described five releases as unreleased.**
+  Every tarball from 1.1.0 to 1.5.2 shipped a `## Unreleased` section holding
+  everything since 1.0.0, so a reader who opened the copy they installed —
+  which [the API page](docs/api/#stability) points them at — had no way to tell
+  which version any of it arrived in. The entries are filed under the versions
+  that shipped them now. No code changes; this release exists to put the
+  corrected file on the registry.
+
+## 1.5.2
+
 ### Fixed
 
 - **Third-tone sandhi ran over the syllables rather than over the feet.** The
@@ -52,28 +215,11 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
   written, so 相親 keeps the `xiàngqīn` its dominant sense carries. CC-CEDICT
   now supplies 335 deltas rather than 349.
 
-- **240 dictionary entries could not be looked up at all.** Recognising Hong
-  Kong glyph forms on the lookup path means a key written in a form the path
-  normalises _away from_ can never be found, and the merge derived 繁體 forms
-  per word from Unihan's variant lists with no single standard behind them — so
-  it wrote 裏 where the corpus overwhelmingly writes 裡, and 上樑, 中峯 and 义藴
-  became keys nothing could reach. The merge now derives canonical glyph forms,
-  and keys a headword's canonical spelling alongside it where the phrase corpus
-  wrote one with a 繁體 variant. **No key is now unreachable**, and Hong Kong
-  spellings resolve slightly better than before: 5,499 of the 9,978 keys with a
-  distinct Hong Kong form, against 5,439.
+## 1.5.1
 
-  Two things had to be got right alongside it. The evidence for "简体 writes
-  this character" comes from CC-CEDICT entries whose two columns differ, not
-  from the phrase corpus, which is nominally 简体 but carries 峯, 藴 and 枱 of
-  its own — counting it cut the table from 50 mappings to 32 and stopped Hong
-  Kong text converting at all. And a 繁體 form differing from its 简体 one only
-  by glyph normalisation is not evidence about script, or the alias key would
-  put that entry's 简体 characters into the 繁體-only set, which is how 发
-  briefly stopped being 简体-only.
+Tooling and test-suite work only; nothing a consumer installs changed.
 
-  `toScript` drops the pass that was papering over this. Key counts move:
-  723,149 for `full`, 97,997 for `standard`, 16,975 for `core`.
+## 1.5.0
 
 ### Added
 
@@ -91,6 +237,10 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
   command line — leaves it off, for a caller who would rather declare the
   language once on a wrapper of their own than repeat it on every syllable.
 
+## 1.4.0
+
+### Added
+
 - **`pinyinjs script`,** script conversion at the command line, with `--to` for
   the target and `--from-script` where the text is too short for detection to
   tell. The lines carry the converted text and nothing else so it pipes;
@@ -168,6 +318,35 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
   before any dictionary has loaded; `pnpm build:data` re-derives them from
   OpenCC and fails the build if the two have drifted.
 
+### Fixed
+
+- **240 dictionary entries could not be looked up at all.** Recognising Hong
+  Kong glyph forms on the lookup path means a key written in a form the path
+  normalises _away from_ can never be found, and the merge derived 繁體 forms
+  per word from Unihan's variant lists with no single standard behind them — so
+  it wrote 裏 where the corpus overwhelmingly writes 裡, and 上樑, 中峯 and 义藴
+  became keys nothing could reach. The merge now derives canonical glyph forms,
+  and keys a headword's canonical spelling alongside it where the phrase corpus
+  wrote one with a 繁體 variant. **No key is now unreachable**, and Hong Kong
+  spellings resolve slightly better than before: 5,499 of the 9,978 keys with a
+  distinct Hong Kong form, against 5,439.
+
+  Two things had to be got right alongside it. The evidence for "简体 writes
+  this character" comes from CC-CEDICT entries whose two columns differ, not
+  from the phrase corpus, which is nominally 简体 but carries 峯, 藴 and 枱 of
+  its own — counting it cut the table from 50 mappings to 32 and stopped Hong
+  Kong text converting at all. And a 繁體 form differing from its 简体 one only
+  by glyph normalisation is not evidence about script, or the alias key would
+  put that entry's 简体 characters into the 繁體-only set, which is how 发
+  briefly stopped being 简体-only.
+
+  `toScript` drops the pass that was papering over this. Key counts move:
+  723,149 for `full`, 97,997 for `standard`, 16,975 for `core`.
+
+## 1.3.0
+
+### Added
+
 - **`slug`,** hanzi to a URL-safe slug: 我想学中文。 is
   `wo3-xiang3-xue2-zhong1wen2`. Built on the decode rather than on a finished
   string, which is the whole point of it — 银行 is `yin2hang2` and not
@@ -179,30 +358,20 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
   The options generalise it past URLs to search keys and identifiers, which the
   [slug page](docs/slug/) tabulates. `pinyinjs slug` at the command line.
 
+## 1.2.2
+
+### Added
+
 - **`TEACHING_JIAO` and `COUNTED_MEASURE`,** two more lattice rules, and
   `QUANTITY_CHARACTERS`, the 汉字 a quantity is written with that the second of
   them asks about. See the fixes below for what they do and what they measure.
+
 - **`counts` on the numeral options.** Whether the number stands immediately in
   front of something it counts, which is what makes a lone 2 两:
   `numeralHanzi(2, { counts: true })` is `两` and `numeralHanzi(2)` is `二`.
   Unlike `liang` it is not a variable choice — 二个 is simply wrong — but
   nothing about the number says whether it is counting, so the caller says.
   `convert` sets it from the character after the digits.
-- **`SYLLABLE_TONES` and `isAttestedTone`.** Which tones each of the 424
-  syllables is actually written in, extracted from the merged dictionary and
-  held to it by a build assertion. 424 syllables in five tones would be 2,120
-  combinations and only 1,708 of them occur, so a fifth of that grid is empty.
-- **`Syllable.originalTone`,** the tone a neutral syllable had before it was
-  reduced. Gwoyeu Romatzyh writes the neutral tone as a dot in front of the
-  syllable _in its original tonal spelling_, so 没有 méiyou is `mei.yeou`, and
-  nothing else in a pinyin syllable records what that tone was. Optional, never
-  inferred, and used by GR alone.
-- **Two more outside sources for the transcription tables.** 148 words from
-  en.wiktionary's Mandarin pronunciation blocks, in bopomofo, Wade-Giles, Yale
-  and GR with their tones, and all 50 rows of Wikipedia's _Help:IPA/Mandarin_
-  key. The 417-row syllabary already checked was toneless, so it could say
-  nothing about tone marks, the neutral tone, 儿化 or how a word is joined,
-  which is where the four fixes below were found.
 
 ### Fixed
 
@@ -222,6 +391,7 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
   `jiāo`, three of them wrongly — 统一教创始人, 方法教深思 and 做到了教政分离,
   where a nominal compound takes an object's shape. CPP's polyphone score is
   89.05%, up from 89.04%.
+
 - **A 量词 was swallowed by the word behind it.** 三个人 was `sān gèrén`, three
   _personals_, because 个人 is a common noun and nothing weighed it against the
   个 belonging to the 三 in front of it. It is now `sān gè rén`, and so are
@@ -237,6 +407,7 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
   and moves 53 decodes, three of them wrongly — 一批评, 这一名词 and 六七股灾,
   where the 一 and the 六七 count nothing. Exactly one reading changes in the
   whole corpus and it is a fix: 已经下了两天雨了 read 天雨 as `tiān yù`.
+
 - **The Han after a number was decoded without it.** 2个人 was `liǎng gèrén`
   while 两个人 written out was already `liǎng gè rén`, because a Han run was
   decoded on its own and had no idea a number preceded it. A run is now decoded
@@ -245,6 +416,7 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
   take that context as a fourth argument; where a reading would straddle the
   join — 1点儿事, whose 一点儿 is one `yìdiǎnr` over both sides of it — the run
   is decoded alone, as before.
+
 - **A lone 2 in front of a 量词 was read 二.** 我们买了2个西瓜 came out as
   `wǒmen mǎile èr gè xīguā`, and a 2 standing immediately in front of what it
   counts is 两: it is now `liǎng gè`, and 2岁 is `liǎng suì`, 2点 `liǎng diǎn`
@@ -255,43 +427,22 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
   the 第 in front of it marks (第2次 is `dì èr cì`), and 十 and 百, which follow
   二 as 20 and 200 already do. A 2 inside a larger number is untouched: 12个 is
   still `shí'èr gè`.
+
+## 1.2.1
+
+### Fixed
+
 - **A decimal lost the grouping of its counted part.** 一共75.5元 was
   `yígòng qī shí wǔ diǎn wǔ yuán`, with the 75 in loose syllables, and is now
   `yígòng qīshíwǔ diǎn wǔ yuán`, the same word 一共75元 already got. Only what
   follows the 点 is read a digit at a time, and that part is unchanged: 3.14 is
   still `sān diǎn yī sì`, and sandhi still stops at the point.
+
 - **A decimal point was read as a full stop.** 一共75.5元 came back as
   `Yígòng …`, capitalised as though the text were a sentence, and a conversion
   keeping its digits capitalised again after the point: 我有75.5个。was
   `Wǒ yǒu75.5Gè.` A stop between two digits now ends no sentence.
-- **Bopomofo wrote the tone mark after the 儿化 ㄦ.** 哪儿 nǎr was `ㄋㄚㄦˇ` and
-  is now `ㄋㄚˇㄦ`: the mark belongs to the nucleus and the suffix is not part of
-  what it marks. A mark written after the ㄦ is still read.
-- **Wade-Giles put the tone digit on the 儿化 suffix,** so 花儿 huār was
-  `hua-êrh¹`, a first-tone 兒. It is now `hua¹-'rh`, with the digit on the
-  syllable the tone belongs to and the suffix written as the reduced `'rh` that
-  keeps it apart from 女儿 nǚ'ér's `nü³-êrh²`.
-- **Gwoyeu Romatzyh wrote 儿化 as a plain `-l` suffix.** GR fuses it into the
-  rime instead, and the rules are now implemented as _Spelling in Gwoyeu
-  Romatzyh_ gives them: 玩儿 wánr is `wal` rather than `wanl`, 事儿 shìr is
-  `shell`, 今儿 jīnr is `jiel`. The fusion is many-to-one, so `jiel` is 今儿 and
-  鸡儿 both and 128 forms no longer round-trip, every one of them a form the
-  language does not write.
-- **Gwoyeu Romatzyh wrote `.mhe` for a neutral 么.** The `-h-` is the first tone
-  of a sonorant initial, and a neutral syllable is in no tone at all; the basic
-  form goes behind the dot, so 什么 shénme is `shern.me`.
-- **The IPA module said it followed _Help:IPA/Mandarin_ and did not.** It
-  follows the broader IPA column of _Comparison of Standard Chinese
-  transcription systems_, which is where its ground truth comes from; the two
-  pages differ at the medials, -ang, the empty rhyme and the diphthongs. No
-  output changed, only the claim.
-- **`--system ipa` capitalised the IPA.** 我去银行。他姓王。 came out as
-  `Uo˨˩˦ ... Tʰa˥ ɕiŋ˥˩ Uaŋ˧˥`, with the sentence capital and the proper noun
-  carried over from the pinyin as though a transcription spelled words. IPA
-  letters are symbols rather than an alphabet, since `[T]` is not `[t]` in a
-  larger size but a symbol the IPA does not have, so the capitals are now dropped for
-  IPA and bopomofo and kept for the three romanisations. `toTranscription`
-  takes `{ capitals: false }` for callers writing their own.
+
 - **A compound lost the zh-TW reading its own parts carry.** 垃圾分類 read
   `lājīfēnlèi` under `zh-TW`, because CC-CEDICT marks `Taiwan pr.` on 垃圾 and
   垃圾桶 and on no other compound, and the decoder prefers the longer word, so
@@ -303,6 +454,7 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
   it, and it has to be a whole word rather than a bare character, since 會's `huǐ`
   would rewrite 三合會 as `sānhéhuǐ` and 3,743 entries would compose that way
   against the 101 that whole words compose.
+
 - **A character's zh-TW delta was often a choice between its 普通话 senses.**
   从容地 read `cōngróng dì` and 他都知道 read `tā dū zhīdào` under `zh-TW`,
   because 地 was stored with a Taiwan reading of `dì` and 都 with `dū`. Neither
@@ -315,12 +467,75 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
   only read off a sense matching the reading the entry settled on, which drops
   16 more: the note on 著's chess-move sense, which reads `zhāo`, had been
   giving the aspect particle 着 a 國語 reading of `zhuó`.
+
 - **A locale delta overwrote a reading it was not measured against.** The delta
   describes the entry's own 普通话 reading, so a polyphone the decode read some
   other way, such as 得 forced to the modal `děi` rather than the particle `de`
   its entry stores, now keeps what the decode worked out instead of being
   rewritten from the entry.
 
+## 1.2.0
+
+### Fixed
+
+- **`--system ipa` capitalised the IPA.** 我去银行。他姓王。 came out as
+  `Uo˨˩˦ ... Tʰa˥ ɕiŋ˥˩ Uaŋ˧˥`, with the sentence capital and the proper noun
+  carried over from the pinyin as though a transcription spelled words. IPA
+  letters are symbols rather than an alphabet, since `[T]` is not `[t]` in a
+  larger size but a symbol the IPA does not have, so the capitals are now dropped for
+  IPA and bopomofo and kept for the three romanisations. `toTranscription`
+  takes `{ capitals: false }` for callers writing their own.
+
+## 1.1.0
+
+### Added
+
+- **`SYLLABLE_TONES` and `isAttestedTone`.** Which tones each of the 424
+  syllables is actually written in, extracted from the merged dictionary and
+  held to it by a build assertion. 424 syllables in five tones would be 2,120
+  combinations and only 1,708 of them occur, so a fifth of that grid is empty.
+
+- **`Syllable.originalTone`,** the tone a neutral syllable had before it was
+  reduced. Gwoyeu Romatzyh writes the neutral tone as a dot in front of the
+  syllable _in its original tonal spelling_, so 没有 méiyou is `mei.yeou`, and
+  nothing else in a pinyin syllable records what that tone was. Optional, never
+  inferred, and used by GR alone.
+
+- **Two more outside sources for the transcription tables.** 148 words from
+  en.wiktionary's Mandarin pronunciation blocks, in bopomofo, Wade-Giles, Yale
+  and GR with their tones, and all 50 rows of Wikipedia's _Help:IPA/Mandarin_
+  key. The 417-row syllabary already checked was toneless, so it could say
+  nothing about tone marks, the neutral tone, 儿化 or how a word is joined,
+  which is where the four fixes below were found.
+
+### Fixed
+
+- **Bopomofo wrote the tone mark after the 儿化 ㄦ.** 哪儿 nǎr was `ㄋㄚㄦˇ` and
+  is now `ㄋㄚˇㄦ`: the mark belongs to the nucleus and the suffix is not part of
+  what it marks. A mark written after the ㄦ is still read.
+
+- **Wade-Giles put the tone digit on the 儿化 suffix,** so 花儿 huār was
+  `hua-êrh¹`, a first-tone 兒. It is now `hua¹-'rh`, with the digit on the
+  syllable the tone belongs to and the suffix written as the reduced `'rh` that
+  keeps it apart from 女儿 nǚ'ér's `nü³-êrh²`.
+
+- **Gwoyeu Romatzyh wrote 儿化 as a plain `-l` suffix.** GR fuses it into the
+  rime instead, and the rules are now implemented as _Spelling in Gwoyeu
+  Romatzyh_ gives them: 玩儿 wánr is `wal` rather than `wanl`, 事儿 shìr is
+  `shell`, 今儿 jīnr is `jiel`. The fusion is many-to-one, so `jiel` is 今儿 and
+  鸡儿 both and 128 forms no longer round-trip, every one of them a form the
+  language does not write.
+
+- **Gwoyeu Romatzyh wrote `.mhe` for a neutral 么.** The `-h-` is the first tone
+  of a sonorant initial, and a neutral syllable is in no tone at all; the basic
+  form goes behind the dot, so 什么 shénme is `shern.me`.
+
+- **The IPA module said it followed _Help:IPA/Mandarin_ and did not.** It
+  follows the broader IPA column of _Comparison of Standard Chinese
+  transcription systems_, which is where its ground truth comes from; the two
+  pages differ at the medials, -ang, the empty rhyme and the diphthongs. No
+  output changed, only the claim.
+
 ### Changed
 
 - **Reading a romanisation narrows on the tone that was written.**
@@ -333,6 +548,7 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
   candidates alone, since that says the tone is wrong rather than the spelling.
   Over the phrase corpus, taking the first candidate recovers 82.66% of
   syllables when the tone digit is written, against 79.05% without it.
+
 - Eight Wade-Giles forms no longer round-trip, and one to three in each of the
   other systems, because they are forms the language does not write: `lo` in the
   four contour tones, a first-tone 兒, and a neutral 誒. Every attested
@@ -393,14 +609,14 @@ decoder internals and the build pipeline.
 
 ### Accuracy at 1.0.0
 
-Over the 126-case gold corpus, which is committed to the repository rather than
+Over the 130-case gold corpus, which is committed to the repository rather than
 to the published package, and scored by `pnpm accuracy`:
 
 |                  |   lattice | greedy baseline |
 | ---------------- | --------: | --------------: |
-| exact match      | **97.6%** |           88.9% |
-| reading accuracy |     99.7% |           98.4% |
-| spacing (F1)     | **99.8%** |           96.5% |
+| exact match      | **97.7%** |           86.9% |
+| reading accuracy |     99.7% |           97.9% |
+| spacing (F1)     | **99.8%** |           95.7% |
 
 Every figure in that table is asserted against the scorer by
 `src/changelog.test.ts`, because a number nothing executes goes stale, which is

package/README.md

@@ -59,6 +59,7 @@ běijīng     běijīng   ㄅㄟˇ ㄐㄧㄥ     pei³-ching¹ běijīng   beeij
 | ------------ | --------------------------------------------------- |
 | `convert`    | hanzi to pinyin                                     |
 | `html`       | the same, as HTML                                   |
+| `annotate`   | hanzi with its pinyin above, as ruby HTML           |
 | `slug`       | hanzi to a URL-safe slug                            |
 | `script`     | 简体 ↔ 繁體 conversion                              |
 | `explain`    | each syllable, how settled it was, and what it beat |
@@ -143,9 +144,9 @@ Load the dictionary once and keep it; it is immutable and safe to share.
 
 | Tier       | Entries | Download (brotli) | Contains               |
 | ---------- | ------: | ----------------: | ---------------------- |
-| `core`     |  16,730 |             70 KB | single characters only |
-| `standard` |  66,730 |            376 KB | the most common words  |
-| `full`     | 461,623 |          2,381 KB | every word             |
+| `core`     |  16,727 |             70 KB | single characters only |
+| `standard` |  66,727 |            376 KB | the most common words  |
+| `full`     | 461,555 |          2,381 KB | every word             |
 
 `full` is the default. The tiers are nested, so a page can load `standard`
 first, convert with it, and reload as `full` arrives.
@@ -349,6 +350,45 @@ included, so write your own:
 Takes any `convert` option, plus `toneClasses: false`, `markUncertain: false`
 and `lang: false`. `toHtml(pieces, options)` renders pieces you already have.
 
+## Annotated output: hanzi and pinyin together
+
+`convertToHtml` writes the pinyin _instead of_ the hanzi. To keep both — a
+learner's text, a subtitle, a dictionary entry — annotate it instead:
+
+```ts
+import { convertToAnnotatedHtml } from "@kensio/pinyinjs";
+
+convertToAnnotatedHtml(dictionary, "银行");
+// <ruby lang="zh">银<rp>(</rp><rt><span class="py-syllable py-tone-2"
+//       lang="zh-Latn-CN-pinyin">yín</span></rt><rp>)</rp></ruby>…
+```
+
+The markup is `<ruby>`, which browsers lay out natively — the reading sits
+above the characters, and `<rp>` puts it in parentheses where it does not. Each
+`<rt>` holds exactly what `convertToHtml` would have written, so tone colours
+and uncertainty marking work inside an annotation as they do outside one.
+
+**A base is not always one character**, and this is where per-character
+annotation goes wrong. 玩儿 is two characters and the one syllable `wánr`, and a
+read number reverses on the way, so both are annotated whole:
+
+```ts
+convertToAnnotatedHtml(dictionary, "玩儿"); // one <ruby>, 玩儿 over wánr
+convertToAnnotatedHtml(dictionary, "95%"); // one <ruby>, 95% over bǎifēnzhījiǔshíwǔ
+```
+
+Ruby needs no styling to work, but the reading is small by default:
+
+```css
+ruby rt {
+  font-size: 0.5em;
+}
+```
+
+`toAnnotatedHtml(pieces, options)` renders pieces you already have, and
+`ConvertedPiece.source` is what makes that possible: the characters each piece
+reads, or undefined where it reads on into the ones before it.
+
 ## Look words up
 
 ```ts
@@ -598,6 +638,7 @@ This is orthography and not translation — 软件 becomes 軟件, never 軟體.
 | `convert(dictionary, text, options?)`                | hanzi → pinyin                                    |
 | `convertPieces(dictionary, text, ...)`               | the same, per syllable, with confidence           |
 | `convertToHtml(dictionary, text, ...)`               | the same, as HTML                                 |
+| `convertToAnnotatedHtml(dictionary, text, ...)`      | hanzi and pinyin together, as ruby HTML           |
 | `slug(dictionary, text, options?)`                   | hanzi → a URL-safe slug                           |
 | `joinPieces(pieces)` / `toHtml(pieces)`              | render pieces                                     |
 | `isUncertain(confidence)`                            | was this syllable a guess?                        |