@kensio/pinyinjs 1.5.1 → 1.5.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +175 -67
- package/README.md +4 -1
- package/data/core.entries +14 -14
- package/data/full.entries +14 -14
- package/data/manifest.json +6 -6
- package/data/standard.entries +14 -14
- package/dist/cli/arguments.d.ts.map +1 -1
- package/dist/cli/arguments.js.map +1 -1
- package/dist/cli/colour.js +1 -1
- package/dist/cli/commands.d.ts.map +1 -1
- package/dist/cli/commands.js +10 -3
- package/dist/cli/commands.js.map +1 -1
- package/dist/cli/environment.d.ts.map +1 -1
- package/dist/cli/environment.js +0 -1
- package/dist/cli/environment.js.map +1 -1
- package/dist/cli/main.js +0 -0
- package/dist/decode/constituents.d.ts +26 -0
- package/dist/decode/constituents.d.ts.map +1 -0
- package/dist/decode/constituents.js +57 -0
- package/dist/decode/constituents.js.map +1 -0
- package/dist/decode/convert.d.ts.map +1 -1
- package/dist/decode/convert.js +11 -2
- package/dist/decode/convert.js.map +1 -1
- package/dist/decode/sandhi.d.ts +18 -1
- package/dist/decode/sandhi.d.ts.map +1 -1
- package/dist/decode/sandhi.js +105 -8
- package/dist/decode/sandhi.js.map +1 -1
- package/dist/dictionary/artifact.d.ts.map +1 -1
- package/dist/dictionary/artifact.js +0 -1
- package/dist/dictionary/artifact.js.map +1 -1
- package/dist/dictionary/dictionary.d.ts.map +1 -1
- package/dist/dictionary/dictionary.js +0 -1
- package/dist/dictionary/dictionary.js.map +1 -1
- package/dist/dictionary/merge.d.ts.map +1 -1
- package/dist/dictionary/merge.js +38 -2
- package/dist/dictionary/merge.js.map +1 -1
- package/dist/dictionary/node-source.d.ts.map +1 -1
- package/dist/dictionary/node-source.js +1 -4
- package/dist/dictionary/node-source.js.map +1 -1
- package/dist/index.d.ts +1 -1
- package/dist/index.d.ts.map +1 -1
- package/dist/numerals/numerals.d.ts.map +1 -1
- package/dist/numerals/numerals.js +0 -1
- package/dist/numerals/numerals.js.map +1 -1
- package/dist/numerals/text.d.ts.map +1 -1
- package/dist/numerals/text.js +0 -1
- package/dist/numerals/text.js.map +1 -1
- package/dist/sources/cedict.d.ts +28 -5
- package/dist/sources/cedict.d.ts.map +1 -1
- package/dist/sources/cedict.js +30 -1
- package/dist/sources/cedict.js.map +1 -1
- package/package.json +31 -37
package/CHANGELOG.md
CHANGED
|
@@ -9,30 +9,68 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
|
|
|
9
9
|
|
|
10
10
|
## Unreleased
|
|
11
11
|
|
|
12
|
+
## 1.5.3
|
|
13
|
+
|
|
12
14
|
### Fixed
|
|
13
15
|
|
|
14
|
-
- **
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
wrote one with a 繁體 variant. **No key is now unreachable**, and Hong Kong
|
|
22
|
-
spellings resolve slightly better than before: 5,499 of the 9,978 keys with a
|
|
23
|
-
distinct Hong Kong form, against 5,439.
|
|
16
|
+
- **The changelog inside the package described five releases as unreleased.**
|
|
17
|
+
Every tarball from 1.1.0 to 1.5.2 shipped a `## Unreleased` section holding
|
|
18
|
+
everything since 1.0.0, so a reader who opened the copy they installed —
|
|
19
|
+
which [the API page](docs/api/#stability) points them at — had no way to tell
|
|
20
|
+
which version any of it arrived in. The entries are filed under the versions
|
|
21
|
+
that shipped them now. No code changes; this release exists to put the
|
|
22
|
+
corrected file on the registry.
|
|
24
23
|
|
|
25
|
-
|
|
26
|
-
this character" comes from CC-CEDICT entries whose two columns differ, not
|
|
27
|
-
from the phrase corpus, which is nominally 简体 but carries 峯, 藴 and 枱 of
|
|
28
|
-
its own — counting it cut the table from 50 mappings to 32 and stopped Hong
|
|
29
|
-
Kong text converting at all. And a 繁體 form differing from its 简体 one only
|
|
30
|
-
by glyph normalisation is not evidence about script, or the alias key would
|
|
31
|
-
put that entry's 简体 characters into the 繁體-only set, which is how 发
|
|
32
|
-
briefly stopped being 简体-only.
|
|
24
|
+
## 1.5.2
|
|
33
25
|
|
|
34
|
-
|
|
35
|
-
|
|
26
|
+
### Fixed
|
|
27
|
+
|
|
28
|
+
- **Third-tone sandhi ran over the syllables rather than over the feet.** The
|
|
29
|
+
rule is usually stated as "a third tone before another third tone is said as
|
|
30
|
+
a second", and a left-to-right scan of a reading applying exactly that is
|
|
31
|
+
wrong about as often as it is right: sandhi's domain is the prosodic foot,
|
|
32
|
+
which is built out of structure ([Shih 1986][1]). 這家銀行的行長很喜歡旅行
|
|
33
|
+
came out `hángzháng hén xǐhuan`, running the subject into the predicate, and
|
|
34
|
+
紙老虎 came out `zhíláohǔ`.
|
|
35
|
+
|
|
36
|
+
Three passes now, innermost first, each reading what the one before it left.
|
|
37
|
+
Inside a constituent, every third tone but the last lowers. Between the
|
|
38
|
+
constituents of a word, so that 展覽館 is 展覽 + 館 and stays `zhánlánguǎn`
|
|
39
|
+
while 紙老虎 is 紙 + 老虎 and becomes `zhǐláohǔ` — the dictionary is asked
|
|
40
|
+
where a word divides, and only proposes one where both halves are words. And
|
|
41
|
+
between a monosyllabic word and the word after it, which it leans on: 我也很好
|
|
42
|
+
is `wó yé hén hǎo`, but 行長 and 老闆 keep their second syllable.
|
|
43
|
+
|
|
44
|
+
`applySandhi` takes a `SandhiGrouping` for this, and without one still treats
|
|
45
|
+
the whole reading as a single word. `convert` passes what the decoder found;
|
|
46
|
+
the `sandhi` command splits its argument on whitespace.
|
|
47
|
+
|
|
48
|
+
What is given up is the monosyllable leaning backwards — 保管好 is
|
|
49
|
+
`báoguán hǎo` and comes out `báoguǎn hǎo` — which needs to know which way it
|
|
50
|
+
attaches, a question about syntax rather than about the words.
|
|
51
|
+
|
|
52
|
+
[1]: https://www.researchgate.net/publication/36071823_The_Prosodic_Domain_of_Tone_Sandhi_in_Chinese
|
|
53
|
+
|
|
54
|
+
- **14 characters read wrongly under `zh-TW`.** CC-CEDICT states a `Taiwan pr.`
|
|
55
|
+
note either as a definition of its own, where it qualifies the headword's
|
|
56
|
+
reading, or parenthesised inside one sense, where it qualifies that sense —
|
|
57
|
+
and the merge took both as facts about the headword. 從's `zòng` is the 侍從
|
|
58
|
+
and 從兄弟 senses, all of them bound forms, so `我從北京來` read
|
|
59
|
+
`wǒ zòng Běijīng lái`; 會 took `huǐ` from 一會兒 and `很會說話` followed it.
|
|
60
|
+
A note now has to cover the sense the entry leads with, which is what a
|
|
61
|
+
character means with nothing to narrow it — 和's `hàn` is the conjunction, and
|
|
62
|
+
it stays. Also fixed for 勞, 燥, 行, 勝, 匹, 多, 抵, 枕, 比, 玩, 署 and 聽.
|
|
63
|
+
Compounds keep their own notes, so 肉燥麵 is still `ròusào miàn`.
|
|
64
|
+
|
|
65
|
+
Only a character is tested this way; a word is reached only where it is
|
|
66
|
+
written, so 相親 keeps the `xiàngqīn` its dominant sense carries. CC-CEDICT
|
|
67
|
+
now supplies 335 deltas rather than 349.
|
|
68
|
+
|
|
69
|
+
## 1.5.1
|
|
70
|
+
|
|
71
|
+
Tooling and test-suite work only; nothing a consumer installs changed.
|
|
72
|
+
|
|
73
|
+
## 1.5.0
|
|
36
74
|
|
|
37
75
|
### Added
|
|
38
76
|
|
|
@@ -50,6 +88,10 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
|
|
|
50
88
|
command line — leaves it off, for a caller who would rather declare the
|
|
51
89
|
language once on a wrapper of their own than repeat it on every syllable.
|
|
52
90
|
|
|
91
|
+
## 1.4.0
|
|
92
|
+
|
|
93
|
+
### Added
|
|
94
|
+
|
|
53
95
|
- **`pinyinjs script`,** script conversion at the command line, with `--to` for
|
|
54
96
|
the target and `--from-script` where the text is too short for detection to
|
|
55
97
|
tell. The lines carry the converted text and nothing else so it pipes;
|
|
@@ -127,6 +169,35 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
|
|
|
127
169
|
before any dictionary has loaded; `pnpm build:data` re-derives them from
|
|
128
170
|
OpenCC and fails the build if the two have drifted.
|
|
129
171
|
|
|
172
|
+
### Fixed
|
|
173
|
+
|
|
174
|
+
- **240 dictionary entries could not be looked up at all.** Recognising Hong
|
|
175
|
+
Kong glyph forms on the lookup path means a key written in a form the path
|
|
176
|
+
normalises _away from_ can never be found, and the merge derived 繁體 forms
|
|
177
|
+
per word from Unihan's variant lists with no single standard behind them — so
|
|
178
|
+
it wrote 裏 where the corpus overwhelmingly writes 裡, and 上樑, 中峯 and 义藴
|
|
179
|
+
became keys nothing could reach. The merge now derives canonical glyph forms,
|
|
180
|
+
and keys a headword's canonical spelling alongside it where the phrase corpus
|
|
181
|
+
wrote one with a 繁體 variant. **No key is now unreachable**, and Hong Kong
|
|
182
|
+
spellings resolve slightly better than before: 5,499 of the 9,978 keys with a
|
|
183
|
+
distinct Hong Kong form, against 5,439.
|
|
184
|
+
|
|
185
|
+
Two things had to be got right alongside it. The evidence for "简体 writes
|
|
186
|
+
this character" comes from CC-CEDICT entries whose two columns differ, not
|
|
187
|
+
from the phrase corpus, which is nominally 简体 but carries 峯, 藴 and 枱 of
|
|
188
|
+
its own — counting it cut the table from 50 mappings to 32 and stopped Hong
|
|
189
|
+
Kong text converting at all. And a 繁體 form differing from its 简体 one only
|
|
190
|
+
by glyph normalisation is not evidence about script, or the alias key would
|
|
191
|
+
put that entry's 简体 characters into the 繁體-only set, which is how 发
|
|
192
|
+
briefly stopped being 简体-only.
|
|
193
|
+
|
|
194
|
+
`toScript` drops the pass that was papering over this. Key counts move:
|
|
195
|
+
723,149 for `full`, 97,997 for `standard`, 16,975 for `core`.
|
|
196
|
+
|
|
197
|
+
## 1.3.0
|
|
198
|
+
|
|
199
|
+
### Added
|
|
200
|
+
|
|
130
201
|
- **`slug`,** hanzi to a URL-safe slug: 我想学中文。 is
|
|
131
202
|
`wo3-xiang3-xue2-zhong1wen2`. Built on the decode rather than on a finished
|
|
132
203
|
string, which is the whole point of it — 银行 is `yin2hang2` and not
|
|
@@ -138,30 +209,20 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
|
|
|
138
209
|
The options generalise it past URLs to search keys and identifiers, which the
|
|
139
210
|
[slug page](docs/slug/) tabulates. `pinyinjs slug` at the command line.
|
|
140
211
|
|
|
212
|
+
## 1.2.2
|
|
213
|
+
|
|
214
|
+
### Added
|
|
215
|
+
|
|
141
216
|
- **`TEACHING_JIAO` and `COUNTED_MEASURE`,** two more lattice rules, and
|
|
142
217
|
`QUANTITY_CHARACTERS`, the 汉字 a quantity is written with that the second of
|
|
143
218
|
them asks about. See the fixes below for what they do and what they measure.
|
|
219
|
+
|
|
144
220
|
- **`counts` on the numeral options.** Whether the number stands immediately in
|
|
145
221
|
front of something it counts, which is what makes a lone 2 两:
|
|
146
222
|
`numeralHanzi(2, { counts: true })` is `两` and `numeralHanzi(2)` is `二`.
|
|
147
223
|
Unlike `liang` it is not a variable choice — 二个 is simply wrong — but
|
|
148
224
|
nothing about the number says whether it is counting, so the caller says.
|
|
149
225
|
`convert` sets it from the character after the digits.
|
|
150
|
-
- **`SYLLABLE_TONES` and `isAttestedTone`.** Which tones each of the 424
|
|
151
|
-
syllables is actually written in, extracted from the merged dictionary and
|
|
152
|
-
held to it by a build assertion. 424 syllables in five tones would be 2,120
|
|
153
|
-
combinations and only 1,708 of them occur, so a fifth of that grid is empty.
|
|
154
|
-
- **`Syllable.originalTone`,** the tone a neutral syllable had before it was
|
|
155
|
-
reduced. Gwoyeu Romatzyh writes the neutral tone as a dot in front of the
|
|
156
|
-
syllable _in its original tonal spelling_, so 没有 méiyou is `mei.yeou`, and
|
|
157
|
-
nothing else in a pinyin syllable records what that tone was. Optional, never
|
|
158
|
-
inferred, and used by GR alone.
|
|
159
|
-
- **Two more outside sources for the transcription tables.** 148 words from
|
|
160
|
-
en.wiktionary's Mandarin pronunciation blocks, in bopomofo, Wade-Giles, Yale
|
|
161
|
-
and GR with their tones, and all 50 rows of Wikipedia's _Help:IPA/Mandarin_
|
|
162
|
-
key. The 417-row syllabary already checked was toneless, so it could say
|
|
163
|
-
nothing about tone marks, the neutral tone, 儿化 or how a word is joined,
|
|
164
|
-
which is where the four fixes below were found.
|
|
165
226
|
|
|
166
227
|
### Fixed
|
|
167
228
|
|
|
@@ -181,6 +242,7 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
|
|
|
181
242
|
`jiāo`, three of them wrongly — 统一教创始人, 方法教深思 and 做到了教政分离,
|
|
182
243
|
where a nominal compound takes an object's shape. CPP's polyphone score is
|
|
183
244
|
89.05%, up from 89.04%.
|
|
245
|
+
|
|
184
246
|
- **A 量词 was swallowed by the word behind it.** 三个人 was `sān gèrén`, three
|
|
185
247
|
_personals_, because 个人 is a common noun and nothing weighed it against the
|
|
186
248
|
个 belonging to the 三 in front of it. It is now `sān gè rén`, and so are
|
|
@@ -196,6 +258,7 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
|
|
|
196
258
|
and moves 53 decodes, three of them wrongly — 一批评, 这一名词 and 六七股灾,
|
|
197
259
|
where the 一 and the 六七 count nothing. Exactly one reading changes in the
|
|
198
260
|
whole corpus and it is a fix: 已经下了两天雨了 read 天雨 as `tiān yù`.
|
|
261
|
+
|
|
199
262
|
- **The Han after a number was decoded without it.** 2个人 was `liǎng gèrén`
|
|
200
263
|
while 两个人 written out was already `liǎng gè rén`, because a Han run was
|
|
201
264
|
decoded on its own and had no idea a number preceded it. A run is now decoded
|
|
@@ -204,6 +267,7 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
|
|
|
204
267
|
take that context as a fourth argument; where a reading would straddle the
|
|
205
268
|
join — 1点儿事, whose 一点儿 is one `yìdiǎnr` over both sides of it — the run
|
|
206
269
|
is decoded alone, as before.
|
|
270
|
+
|
|
207
271
|
- **A lone 2 in front of a 量词 was read 二.** 我们买了2个西瓜 came out as
|
|
208
272
|
`wǒmen mǎile èr gè xīguā`, and a 2 standing immediately in front of what it
|
|
209
273
|
counts is 两: it is now `liǎng gè`, and 2岁 is `liǎng suì`, 2点 `liǎng diǎn`
|
|
@@ -214,43 +278,22 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
|
|
|
214
278
|
the 第 in front of it marks (第2次 is `dì èr cì`), and 十 and 百, which follow
|
|
215
279
|
二 as 20 and 200 already do. A 2 inside a larger number is untouched: 12个 is
|
|
216
280
|
still `shí'èr gè`.
|
|
281
|
+
|
|
282
|
+
## 1.2.1
|
|
283
|
+
|
|
284
|
+
### Fixed
|
|
285
|
+
|
|
217
286
|
- **A decimal lost the grouping of its counted part.** 一共75.5元 was
|
|
218
287
|
`yígòng qī shí wǔ diǎn wǔ yuán`, with the 75 in loose syllables, and is now
|
|
219
288
|
`yígòng qīshíwǔ diǎn wǔ yuán`, the same word 一共75元 already got. Only what
|
|
220
289
|
follows the 点 is read a digit at a time, and that part is unchanged: 3.14 is
|
|
221
290
|
still `sān diǎn yī sì`, and sandhi still stops at the point.
|
|
291
|
+
|
|
222
292
|
- **A decimal point was read as a full stop.** 一共75.5元 came back as
|
|
223
293
|
`Yígòng …`, capitalised as though the text were a sentence, and a conversion
|
|
224
294
|
keeping its digits capitalised again after the point: 我有75.5个。was
|
|
225
295
|
`Wǒ yǒu75.5Gè.` A stop between two digits now ends no sentence.
|
|
226
|
-
|
|
227
|
-
is now `ㄋㄚˇㄦ`: the mark belongs to the nucleus and the suffix is not part of
|
|
228
|
-
what it marks. A mark written after the ㄦ is still read.
|
|
229
|
-
- **Wade-Giles put the tone digit on the 儿化 suffix,** so 花儿 huār was
|
|
230
|
-
`hua-êrh¹`, a first-tone 兒. It is now `hua¹-'rh`, with the digit on the
|
|
231
|
-
syllable the tone belongs to and the suffix written as the reduced `'rh` that
|
|
232
|
-
keeps it apart from 女儿 nǚ'ér's `nü³-êrh²`.
|
|
233
|
-
- **Gwoyeu Romatzyh wrote 儿化 as a plain `-l` suffix.** GR fuses it into the
|
|
234
|
-
rime instead, and the rules are now implemented as _Spelling in Gwoyeu
|
|
235
|
-
Romatzyh_ gives them: 玩儿 wánr is `wal` rather than `wanl`, 事儿 shìr is
|
|
236
|
-
`shell`, 今儿 jīnr is `jiel`. The fusion is many-to-one, so `jiel` is 今儿 and
|
|
237
|
-
鸡儿 both and 128 forms no longer round-trip, every one of them a form the
|
|
238
|
-
language does not write.
|
|
239
|
-
- **Gwoyeu Romatzyh wrote `.mhe` for a neutral 么.** The `-h-` is the first tone
|
|
240
|
-
of a sonorant initial, and a neutral syllable is in no tone at all; the basic
|
|
241
|
-
form goes behind the dot, so 什么 shénme is `shern.me`.
|
|
242
|
-
- **The IPA module said it followed _Help:IPA/Mandarin_ and did not.** It
|
|
243
|
-
follows the broader IPA column of _Comparison of Standard Chinese
|
|
244
|
-
transcription systems_, which is where its ground truth comes from; the two
|
|
245
|
-
pages differ at the medials, -ang, the empty rhyme and the diphthongs. No
|
|
246
|
-
output changed, only the claim.
|
|
247
|
-
- **`--system ipa` capitalised the IPA.** 我去银行。他姓王。 came out as
|
|
248
|
-
`Uo˨˩˦ ... Tʰa˥ ɕiŋ˥˩ Uaŋ˧˥`, with the sentence capital and the proper noun
|
|
249
|
-
carried over from the pinyin as though a transcription spelled words. IPA
|
|
250
|
-
letters are symbols rather than an alphabet, since `[T]` is not `[t]` in a
|
|
251
|
-
larger size but a symbol the IPA does not have, so the capitals are now dropped for
|
|
252
|
-
IPA and bopomofo and kept for the three romanisations. `toTranscription`
|
|
253
|
-
takes `{ capitals: false }` for callers writing their own.
|
|
296
|
+
|
|
254
297
|
- **A compound lost the zh-TW reading its own parts carry.** 垃圾分類 read
|
|
255
298
|
`lājīfēnlèi` under `zh-TW`, because CC-CEDICT marks `Taiwan pr.` on 垃圾 and
|
|
256
299
|
垃圾桶 and on no other compound, and the decoder prefers the longer word, so
|
|
@@ -262,6 +305,7 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
|
|
|
262
305
|
it, and it has to be a whole word rather than a bare character, since 會's `huǐ`
|
|
263
306
|
would rewrite 三合會 as `sānhéhuǐ` and 3,743 entries would compose that way
|
|
264
307
|
against the 101 that whole words compose.
|
|
308
|
+
|
|
265
309
|
- **A character's zh-TW delta was often a choice between its 普通话 senses.**
|
|
266
310
|
从容地 read `cōngróng dì` and 他都知道 read `tā dū zhīdào` under `zh-TW`,
|
|
267
311
|
because 地 was stored with a Taiwan reading of `dì` and 都 with `dū`. Neither
|
|
@@ -274,12 +318,75 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
|
|
|
274
318
|
only read off a sense matching the reading the entry settled on, which drops
|
|
275
319
|
16 more: the note on 著's chess-move sense, which reads `zhāo`, had been
|
|
276
320
|
giving the aspect particle 着 a 國語 reading of `zhuó`.
|
|
321
|
+
|
|
277
322
|
- **A locale delta overwrote a reading it was not measured against.** The delta
|
|
278
323
|
describes the entry's own 普通话 reading, so a polyphone the decode read some
|
|
279
324
|
other way, such as 得 forced to the modal `děi` rather than the particle `de`
|
|
280
325
|
its entry stores, now keeps what the decode worked out instead of being
|
|
281
326
|
rewritten from the entry.
|
|
282
327
|
|
|
328
|
+
## 1.2.0
|
|
329
|
+
|
|
330
|
+
### Fixed
|
|
331
|
+
|
|
332
|
+
- **`--system ipa` capitalised the IPA.** 我去银行。他姓王。 came out as
|
|
333
|
+
`Uo˨˩˦ ... Tʰa˥ ɕiŋ˥˩ Uaŋ˧˥`, with the sentence capital and the proper noun
|
|
334
|
+
carried over from the pinyin as though a transcription spelled words. IPA
|
|
335
|
+
letters are symbols rather than an alphabet, since `[T]` is not `[t]` in a
|
|
336
|
+
larger size but a symbol the IPA does not have, so the capitals are now dropped for
|
|
337
|
+
IPA and bopomofo and kept for the three romanisations. `toTranscription`
|
|
338
|
+
takes `{ capitals: false }` for callers writing their own.
|
|
339
|
+
|
|
340
|
+
## 1.1.0
|
|
341
|
+
|
|
342
|
+
### Added
|
|
343
|
+
|
|
344
|
+
- **`SYLLABLE_TONES` and `isAttestedTone`.** Which tones each of the 424
|
|
345
|
+
syllables is actually written in, extracted from the merged dictionary and
|
|
346
|
+
held to it by a build assertion. 424 syllables in five tones would be 2,120
|
|
347
|
+
combinations and only 1,708 of them occur, so a fifth of that grid is empty.
|
|
348
|
+
|
|
349
|
+
- **`Syllable.originalTone`,** the tone a neutral syllable had before it was
|
|
350
|
+
reduced. Gwoyeu Romatzyh writes the neutral tone as a dot in front of the
|
|
351
|
+
syllable _in its original tonal spelling_, so 没有 méiyou is `mei.yeou`, and
|
|
352
|
+
nothing else in a pinyin syllable records what that tone was. Optional, never
|
|
353
|
+
inferred, and used by GR alone.
|
|
354
|
+
|
|
355
|
+
- **Two more outside sources for the transcription tables.** 148 words from
|
|
356
|
+
en.wiktionary's Mandarin pronunciation blocks, in bopomofo, Wade-Giles, Yale
|
|
357
|
+
and GR with their tones, and all 50 rows of Wikipedia's _Help:IPA/Mandarin_
|
|
358
|
+
key. The 417-row syllabary already checked was toneless, so it could say
|
|
359
|
+
nothing about tone marks, the neutral tone, 儿化 or how a word is joined,
|
|
360
|
+
which is where the four fixes below were found.
|
|
361
|
+
|
|
362
|
+
### Fixed
|
|
363
|
+
|
|
364
|
+
- **Bopomofo wrote the tone mark after the 儿化 ㄦ.** 哪儿 nǎr was `ㄋㄚㄦˇ` and
|
|
365
|
+
is now `ㄋㄚˇㄦ`: the mark belongs to the nucleus and the suffix is not part of
|
|
366
|
+
what it marks. A mark written after the ㄦ is still read.
|
|
367
|
+
|
|
368
|
+
- **Wade-Giles put the tone digit on the 儿化 suffix,** so 花儿 huār was
|
|
369
|
+
`hua-êrh¹`, a first-tone 兒. It is now `hua¹-'rh`, with the digit on the
|
|
370
|
+
syllable the tone belongs to and the suffix written as the reduced `'rh` that
|
|
371
|
+
keeps it apart from 女儿 nǚ'ér's `nü³-êrh²`.
|
|
372
|
+
|
|
373
|
+
- **Gwoyeu Romatzyh wrote 儿化 as a plain `-l` suffix.** GR fuses it into the
|
|
374
|
+
rime instead, and the rules are now implemented as _Spelling in Gwoyeu
|
|
375
|
+
Romatzyh_ gives them: 玩儿 wánr is `wal` rather than `wanl`, 事儿 shìr is
|
|
376
|
+
`shell`, 今儿 jīnr is `jiel`. The fusion is many-to-one, so `jiel` is 今儿 and
|
|
377
|
+
鸡儿 both and 128 forms no longer round-trip, every one of them a form the
|
|
378
|
+
language does not write.
|
|
379
|
+
|
|
380
|
+
- **Gwoyeu Romatzyh wrote `.mhe` for a neutral 么.** The `-h-` is the first tone
|
|
381
|
+
of a sonorant initial, and a neutral syllable is in no tone at all; the basic
|
|
382
|
+
form goes behind the dot, so 什么 shénme is `shern.me`.
|
|
383
|
+
|
|
384
|
+
- **The IPA module said it followed _Help:IPA/Mandarin_ and did not.** It
|
|
385
|
+
follows the broader IPA column of _Comparison of Standard Chinese
|
|
386
|
+
transcription systems_, which is where its ground truth comes from; the two
|
|
387
|
+
pages differ at the medials, -ang, the empty rhyme and the diphthongs. No
|
|
388
|
+
output changed, only the claim.
|
|
389
|
+
|
|
283
390
|
### Changed
|
|
284
391
|
|
|
285
392
|
- **Reading a romanisation narrows on the tone that was written.**
|
|
@@ -292,6 +399,7 @@ which a dictionary rebuild can change, and the artifact format under `data/`.
|
|
|
292
399
|
candidates alone, since that says the tone is wrong rather than the spelling.
|
|
293
400
|
Over the phrase corpus, taking the first candidate recovers 82.66% of
|
|
294
401
|
syllables when the tone digit is written, against 79.05% without it.
|
|
402
|
+
|
|
295
403
|
- Eight Wade-Giles forms no longer round-trip, and one to three in each of the
|
|
296
404
|
other systems, because they are forms the language does not write: `lo` in the
|
|
297
405
|
four contour tones, a first-tone 兒, and a neutral 誒. Every attested
|
|
@@ -352,14 +460,14 @@ decoder internals and the build pipeline.
|
|
|
352
460
|
|
|
353
461
|
### Accuracy at 1.0.0
|
|
354
462
|
|
|
355
|
-
Over the
|
|
463
|
+
Over the 126-case gold corpus, which is committed to the repository rather than
|
|
356
464
|
to the published package, and scored by `pnpm accuracy`:
|
|
357
465
|
|
|
358
466
|
| | lattice | greedy baseline |
|
|
359
467
|
| ---------------- | --------: | --------------: |
|
|
360
|
-
| exact match | **97.6%** | 88.
|
|
361
|
-
| reading accuracy | 99.7% | 98.
|
|
362
|
-
| spacing (F1) | **99.8%** | 96.
|
|
468
|
+
| exact match | **97.6%** | 88.9% |
|
|
469
|
+
| reading accuracy | 99.7% | 98.4% |
|
|
470
|
+
| spacing (F1) | **99.8%** | 96.5% |
|
|
363
471
|
|
|
364
472
|
Every figure in that table is asserted against the scorer by
|
|
365
473
|
`src/changelog.test.ts`, because a number nothing executes goes stale, which is
|
package/README.md
CHANGED
|
@@ -532,6 +532,9 @@ applySandhi(niHao, { thirdTone: true }); // ní hǎo
|
|
|
532
532
|
|
|
533
533
|
Third-tone sandhi is off by default: standard orthography writes 你好 as
|
|
534
534
|
`nǐ hǎo` even though it is said `ní hǎo`. Turn it on for transcribing speech.
|
|
535
|
+
Its domain is the prosodic foot rather than the syllable string, so 展览馆 is
|
|
536
|
+
`zhánlánguǎn` and 纸老虎 `zhǐláohǔ`, and 老板很好 keeps the 板 that a
|
|
537
|
+
left-to-right scan would lower.
|
|
535
538
|
|
|
536
539
|
## Scripts and locales
|
|
537
540
|
|
|
@@ -624,7 +627,7 @@ pnpm polyphones # score them against 20,139 hand-labelled polyphones
|
|
|
624
627
|
pnpm build:data # rebuild data/ from the upstream sources
|
|
625
628
|
```
|
|
626
629
|
|
|
627
|
-
`pnpm check` must pass:
|
|
630
|
+
`pnpm check` must pass: oxlint including the type-aware rules, `tsc` with
|
|
628
631
|
`exactOptionalPropertyTypes` and `noUncheckedIndexedAccess`, and vitest at 95%
|
|
629
632
|
coverage thresholds.
|
|
630
633
|
|
package/data/core.entries
CHANGED
|
@@ -262,7 +262,7 @@ jin1 zg
|
|
|
262
262
|
jie4 nr
|
|
263
263
|
bing1 zg
|
|
264
264
|
reng2 zg
|
|
265
|
-
cong2
|
|
265
|
+
cong2 p cong1
|
|
266
266
|
san3 zg
|
|
267
267
|
lun2 g
|
|
268
268
|
cang1 nr
|
|
@@ -328,7 +328,7 @@ tang3 zg
|
|
|
328
328
|
zhong4 ng
|
|
329
329
|
you1 n
|
|
330
330
|
huo3 n huo5
|
|
331
|
-
hui4
|
|
331
|
+
hui4 v kuai4
|
|
332
332
|
yu3 v
|
|
333
333
|
yun2 zg
|
|
334
334
|
san3 n
|
|
@@ -1110,7 +1110,7 @@ shao4 g
|
|
|
1110
1110
|
zhu1 zg
|
|
1111
1111
|
li4 nr p
|
|
1112
1112
|
jin4 n jing4
|
|
1113
|
-
lao2
|
|
1113
|
+
lao2 vn
|
|
1114
1114
|
juan4 zg
|
|
1115
1115
|
kou3
|
|
1116
1116
|
wa1 zg
|
|
@@ -1212,7 +1212,7 @@ suan3 zg
|
|
|
1212
1212
|
du2
|
|
1213
1213
|
jiu4
|
|
1214
1214
|
jue2
|
|
1215
|
-
pi3
|
|
1215
|
+
pi3 q
|
|
1216
1216
|
qu1 n ou1
|
|
1217
1217
|
yi1 n
|
|
1218
1218
|
ke1 zg
|
|
@@ -1435,7 +1435,7 @@ dun1 dun4 m
|
|
|
1435
1435
|
fen1 v
|
|
1436
1436
|
e2
|
|
1437
1437
|
han2 v
|
|
1438
|
-
ting1
|
|
1438
|
+
ting1 v ting5
|
|
1439
1439
|
keng1 zg hang2
|
|
1440
1440
|
shun3 v
|
|
1441
1441
|
qi3 v
|
|
@@ -2464,7 +2464,7 @@ wai4 f
|
|
|
2464
2464
|
yuan4
|
|
2465
2465
|
mao3 t
|
|
2466
2466
|
su4 n
|
|
2467
|
-
duo1
|
|
2467
|
+
duo1 m
|
|
2468
2468
|
duo1 zg
|
|
2469
2469
|
ye4 t
|
|
2470
2470
|
gou4 zg
|
|
@@ -4537,7 +4537,7 @@ yang1
|
|
|
4537
4537
|
bao4 v
|
|
4538
4538
|
ni3
|
|
4539
4539
|
ye4 zg
|
|
4540
|
-
di3
|
|
4540
|
+
di3 zg
|
|
4541
4541
|
chi4
|
|
4542
4542
|
mo3 v ma1,mo4
|
|
4543
4543
|
mei4 zg
|
|
@@ -5558,7 +5558,7 @@ xi1 vg
|
|
|
5558
5558
|
hu4
|
|
5559
5559
|
ya1 zg
|
|
5560
5560
|
dou3 zg
|
|
5561
|
-
zhen3
|
|
5561
|
+
zhen3 n
|
|
5562
5562
|
yao1 zg
|
|
5563
5563
|
lin2 ng
|
|
5564
5564
|
rui4 zg
|
|
@@ -6428,7 +6428,7 @@ ai3 zg
|
|
|
6428
6428
|
jie3
|
|
6429
6429
|
du2 zg
|
|
6430
6430
|
yu4 nz
|
|
6431
|
-
bi3
|
|
6431
|
+
bi3 p
|
|
6432
6432
|
bi4 v
|
|
6433
6433
|
bi4 g
|
|
6434
6434
|
pi2 zg
|
|
@@ -7619,7 +7619,7 @@ jing5 zg
|
|
|
7619
7619
|
ying2
|
|
7620
7620
|
yu4 ag
|
|
7621
7621
|
tai4
|
|
7622
|
-
zao4
|
|
7622
|
+
zao4 ag
|
|
7623
7623
|
can4
|
|
7624
7624
|
sui4 n
|
|
7625
7625
|
zong3 zg
|
|
@@ -7980,7 +7980,7 @@ bin1 x fen1
|
|
|
7980
7980
|
yue4 zg
|
|
7981
7981
|
jue2 zg
|
|
7982
7982
|
jue2 g
|
|
7983
|
-
wan2
|
|
7983
|
+
wan2 v
|
|
7984
7984
|
jian1 zg
|
|
7985
7985
|
mei2 n
|
|
7986
7986
|
pin2
|
|
@@ -10409,7 +10409,7 @@ zhi4 v zhi5
|
|
|
10409
10409
|
an3
|
|
10410
10410
|
fa2
|
|
10411
10411
|
lan3 v
|
|
10412
|
-
shu3
|
|
10412
|
+
shu3 ng
|
|
10413
10413
|
si1
|
|
10414
10414
|
pi2 g
|
|
10415
10415
|
ma4
|
|
@@ -10688,7 +10688,7 @@ zhen1 g
|
|
|
10688
10688
|
zuo4 g
|
|
10689
10689
|
pei1 n
|
|
10690
10690
|
jia3 zg
|
|
10691
|
-
sheng4
|
|
10691
|
+
sheng4 v sheng1
|
|
10692
10692
|
zhi1 x
|
|
10693
10693
|
bao1 n
|
|
10694
10694
|
qu1 zg
|
|
@@ -12161,7 +12161,7 @@ er4
|
|
|
12161
12161
|
ka1
|
|
12162
12162
|
mie4
|
|
12163
12163
|
xi4
|
|
12164
|
-
xing2
|
|
12164
|
+
xing2 zg hang2,heng2,hang4
|
|
12165
12165
|
yan3 v
|
|
12166
12166
|
kan4 zg
|
|
12167
12167
|
yuan4 zg
|
package/data/full.entries
CHANGED
|
@@ -41153,7 +41153,7 @@ reng2 zhi2 zhuo2 de5
|
|
|
41153
41153
|
|
|
41154
41154
|
|
|
41155
41155
|
|
|
41156
|
-
cong2
|
|
41156
|
+
cong2 p cong1
|
|
41157
41157
|
|
|
41158
41158
|
i
|
|
41159
41159
|
d
|
|
@@ -45842,7 +45842,7 @@ huo3 chang2
|
|
|
45842
45842
|
n
|
|
45843
45843
|
n
|
|
45844
45844
|
|
|
45845
|
-
hui4
|
|
45845
|
+
hui4 v kuai4
|
|
45846
45846
|
t
|
|
45847
45847
|
|
|
45848
45848
|
|
|
@@ -93086,7 +93086,7 @@ jing4 feng1 n
|
|
|
93086
93086
|
jing4 gu3 feng1 ji1 n
|
|
93087
93087
|
|
|
93088
93088
|
|
|
93089
|
-
lao2
|
|
93089
|
+
lao2 vn
|
|
93090
93090
|
|
|
93091
93091
|
|
|
93092
93092
|
v
|
|
@@ -97023,7 +97023,7 @@ jiu4
|
|
|
97023
97023
|
|
|
97024
97024
|
jue2
|
|
97025
97025
|
xi4
|
|
97026
|
-
pi3
|
|
97026
|
+
pi3 q
|
|
97027
97027
|
|
|
97028
97028
|
|
|
97029
97029
|
|
|
@@ -119479,7 +119479,7 @@ han2 ling2 zi3
|
|
|
119479
119479
|
i
|
|
119480
119480
|
|
|
119481
119481
|
i
|
|
119482
|
-
ting1
|
|
119482
|
+
ting1 v ting5
|
|
119483
119483
|
ting1 shang4 qu5
|
|
119484
119484
|
|
|
119485
119485
|
v
|
|
@@ -148162,7 +148162,7 @@ su4 xing1 mei4 dan4 i
|
|
|
148162
148162
|
su4 yuan4 de2 chang2
|
|
148163
148163
|
|
|
148164
148164
|
|
|
148165
|
-
duo1
|
|
148165
|
+
duo1 m
|
|
148166
148166
|
|
|
148167
148167
|
l
|
|
148168
148168
|
|
|
@@ -248064,7 +248064,7 @@ bao4 guan1 zhi2 yue4 i
|
|
|
248064
248064
|
he1
|
|
248065
248065
|
ni3
|
|
248066
248066
|
ye4 zg
|
|
248067
|
-
di3
|
|
248067
|
+
di3 zg
|
|
248068
248068
|
|
|
248069
248069
|
v
|
|
248070
248070
|
|
|
@@ -305806,7 +305806,7 @@ dou3 zg
|
|
|
305806
305806
|
|
|
305807
305807
|
|
|
305808
305808
|
xin2
|
|
305809
|
-
zhen3
|
|
305809
|
+
zhen3 n
|
|
305810
305810
|
s
|
|
305811
305811
|
|
|
305812
305812
|
|
|
@@ -329379,7 +329379,7 @@ yu4 zi3 yun4 sun1
|
|
|
329379
329379
|
|
|
329380
329380
|
|
|
329381
329381
|
|
|
329382
|
-
bi3
|
|
329382
|
+
bi3 p
|
|
329383
329383
|
nr
|
|
329384
329384
|
i
|
|
329385
329385
|
|
|
@@ -374383,7 +374383,7 @@ xue2
|
|
|
374383
374383
|
lan2
|
|
374384
374384
|
tai4
|
|
374385
374385
|
|
|
374386
|
-
zao4
|
|
374386
|
+
zao4 ag
|
|
374387
374387
|
|
|
374388
374388
|
|
|
374389
374389
|
|
|
@@ -387007,7 +387007,7 @@ jue2 zg
|
|
|
387007
387007
|
|
|
387008
387008
|
men2
|
|
387009
387009
|
jue2
|
|
387010
|
-
wan2
|
|
387010
|
+
wan2 v
|
|
387011
387011
|
|
|
387012
387012
|
n
|
|
387013
387013
|
i
|
|
@@ -469185,7 +469185,7 @@ fa2 mo4 kuan3 v
|
|
|
469185
469185
|
lan3 v
|
|
469186
469186
|
|
|
469187
469187
|
|
|
469188
|
-
shu3
|
|
469188
|
+
shu3 ng
|
|
469189
469189
|
|
|
469190
469190
|
|
|
469191
469191
|
|
|
@@ -480909,7 +480909,7 @@ jia3 zi3
|
|
|
480909
480909
|
|
|
480910
480910
|
|
|
480911
480911
|
n
|
|
480912
|
-
sheng4
|
|
480912
|
+
sheng4 v sheng1
|
|
480913
480913
|
i
|
|
480914
480914
|
sheng4 bu4 jiao1 bai4 bu4 nei3
|
|
480915
480915
|
|
|
@@ -513419,7 +513419,7 @@ xi4
|
|
|
513419
513419
|
|
|
513420
513420
|
|
|
513421
513421
|
|
|
513422
|
-
xing2
|
|
513422
|
+
xing2 zg hang2,heng2,hang4
|
|
513423
513423
|
|
|
513424
513424
|
|
|
513425
513425
|
l
|
package/data/manifest.json
CHANGED
|
@@ -5,24 +5,24 @@
|
|
|
5
5
|
"keys": "core.keys",
|
|
6
6
|
"readings": "core.entries",
|
|
7
7
|
"frequencies": "core.freq",
|
|
8
|
-
"bytes":
|
|
9
|
-
"compressedBytes":
|
|
8
|
+
"bytes": 217984,
|
|
9
|
+
"compressedBytes": 71263
|
|
10
10
|
},
|
|
11
11
|
"standard": {
|
|
12
12
|
"entries": 66727,
|
|
13
13
|
"keys": "standard.keys",
|
|
14
14
|
"readings": "standard.entries",
|
|
15
15
|
"frequencies": "standard.freq",
|
|
16
|
-
"bytes":
|
|
17
|
-
"compressedBytes":
|
|
16
|
+
"bytes": 1399234,
|
|
17
|
+
"compressedBytes": 387749
|
|
18
18
|
},
|
|
19
19
|
"full": {
|
|
20
20
|
"entries": 461623,
|
|
21
21
|
"keys": "full.keys",
|
|
22
22
|
"readings": "full.entries",
|
|
23
23
|
"frequencies": "full.freq",
|
|
24
|
-
"bytes":
|
|
25
|
-
"compressedBytes":
|
|
24
|
+
"bytes": 9349648,
|
|
25
|
+
"compressedBytes": 2448267
|
|
26
26
|
}
|
|
27
27
|
},
|
|
28
28
|
"script": {
|