xjyutping 1.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,280 @@
1
+ # Changelog
2
+
3
+ All notable changes to xjyutping, the Python package, are recorded here. The
4
+ format follows [Keep a Changelog 1.1.0](https://keepachangelog.com/en/1.1.0/),
5
+ and versions follow [Semantic Versioning 2.0.0](https://semver.org/spec/v2.0.0.html).
6
+
7
+ This package is the Python port of the LaTeX package
8
+ [xjyutping](https://github.com/Beyond3345/xjyutping-tex), which has its own
9
+ version numbers and its own changelog. That changelog also holds the full
10
+ history of how both packages were built, including the review rounds whose
11
+ results this package inherits through the shared data. This file has the
12
+ Python releases and the handover notes for the Python side.
13
+
14
+ The version is written in three places: `version` in `pyproject.toml`,
15
+ `__version__` in `src/xjyutping/__init__.py` (`test_version` checks that the
16
+ two agree), and `README.md`. A release bumps all three and gets an entry in
17
+ Part I.
18
+
19
+ ---
20
+
21
+ # Part I: Releases
22
+
23
+ ## [Unreleased]
24
+
25
+ Nothing yet. Add entries here under `### Added`, `### Changed`, `### Fixed`
26
+ and so on.
27
+
28
+ ## [1.1.0] - 2026-09-28
29
+
30
+ ### Added
31
+
32
+ - **More vocabulary:** the data of xjyutping-tex 1.2.0.
33
+ - 2,291 words from CC-Canto and the Cantonese readings of CC-CEDICT, as
34
+ distributed with Jyut Dictionary. They are added only where they change a
35
+ reading and pass consistency checks against rime-cantonese; 480 more
36
+ were excluded by hand.
37
+ - 640 characters that the other sources lack, with readings from the book
38
+ data of 粵音資料集叢 (jyutnet/cantonese-books-data, by 石見田).
39
+
40
+ On the corpora audited earlier nothing got worse; on new test sentences,
41
+ 24 readings improved and 3 changed neutrally (see the LaTeX package's
42
+ changelog, Part II, section 7.4).
43
+ - `test_added_vocabulary`, which checks new words, the words that must not
44
+ capture their neighbours, the two hand fixes and a book character.
45
+ - README: an **Acknowledgements and attributions** section.
46
+ - The inspiration: [xpinyin](https://github.com/lxneng/xpinyin) by Eric Lo
47
+ ([lxneng](https://lxneng.com)), whose API this package follows.
48
+ - Visual Jyutping and the Visual Cantonese Fonts, which inspired the
49
+ `fancy` and `discord` styles.
50
+ - Every data source: LSHK, rime-cantonese, 粵音資料集叢, Jyut Dictionary
51
+ with CC-Canto, the CC-CEDICT readings and CC-CEDICT, and OpenCC.
52
+ - This `CHANGELOG.md` and a `.gitignore`. Up to 1.0.0 both packages shared
53
+ one changelog in the folder that held them; the history is kept in
54
+ xjyutping-tex's.
55
+
56
+ ### Changed
57
+
58
+ - **Readings**, as in xjyutping-tex 1.2.0, for example 機長 zoeng2, 營業額
59
+ ngaak2, 自由行 hang4, 研討會 wui2, 消防隊 deoi2, 冠狀病毒 gun1, 智能卡
60
+ kaat1. 化合物 is now faa3 hap6 mat6 and 會否 wui5 fau2.
61
+ - **Licence of the data files** in `src/xjyutping/data/`: CC BY-SA 4.0, since
62
+ the added word lists are CC BY-SA 3.0 (share-alike). Before this, the data
63
+ carried CC BY 4.0. `LICENSE` lists every source. The code stays MIT.
64
+ - `tests/parity.tex` compiles with XeLaTeX or LuaLaTeX.
65
+ - `tests/parity_expected.txt` was regenerated from xjyutping-tex 1.2.0 and
66
+ changed in one line: 隨着 is now one word, with the same readings.
67
+ - README: shorter, with installing from PyPI or GitHub, and links to the LaTeX
68
+ package's repository. The PyPI summary is now "Convert Traditional Chinese
69
+ text to Cantonese Jyutping (粵拼), with pronunciation chosen from context".
70
+ - **Packaging for PyPI:** `pyproject.toml` declares the licence as the SPDX
71
+ expression `MIT AND CC-BY-SA-4.0` with `license-files` (PEP 639, hence
72
+ setuptools 77 or later to build), and has keywords, classifiers and the
73
+ project URLs (repository, issues, changelog, the LaTeX package).
74
+ `MANIFEST.in` adds `CHANGELOG.md` and the parity files to the sdist. The
75
+ sdist and wheel pass `twine check`.
76
+
77
+ ## [1.0.0] - 2026-09-28
78
+
79
+ ### Added
80
+
81
+ - **First release** of the Python package `xjyutping`: the Python port of
82
+ the LaTeX package, with the API of
83
+ [xpinyin](https://github.com/lxneng/xpinyin). Python 3.8 or later, no
84
+ dependencies. Class `Jyutping(data_dir=None)`:
85
+ - `get_jyutping(chars, splitter='-', tone_marks='numbers', convert='lower')`
86
+ - `get_jyutpings(chars, splitter, tone_marks, convert, n=10)`: the reading
87
+ in context first, then the other readings.
88
+ - `annotate(text, tone_marks)`: one `(char, reading or None)` pair per
89
+ input character.
90
+ - `segment(text)`: `Segment(text, readings, type)` with the types w/u/m/s
91
+ of the TeX debug log.
92
+ - `set_jyutping(text, reading)`, the equivalent of `\setjyutping`.
93
+ - `get_initial(char, full=False)` and `get_initials(chars, splitter, full)`.
94
+ - The static methods `decode_jyutping` and `convert_jyutping`.
95
+ - **Tone styles:** `'numbers'` (default), `None`, `'fancy'` and `'discord'`.
96
+ The last two use the exact code points of Visual Jyutping's web and
97
+ Discord tone maps: ˉ¹ ˊ² ˗₃ ˎ₄ ˏ₅ ˍ₆ and ˉ¹ ⸍² -₃ ⸜₄ ⸝₅ ˍ₆.
98
+ - **Same readings as xjyutping-tex 1.1.0.** It uses the same data
99
+ (`src/xjyutping/data/*.tsv`, generated by that package's
100
+ `tools/build-data.py`) and the same run splitting, segmentation, variant
101
+ folding, run-final readings and user overrides.
102
+ - **Tests:** 38 pytest tests, including a parity test against a frozen TeX
103
+ debug log (`tests/parity.tex`, `parity_corpus.txt`,
104
+ `parity_expected.txt`). There are 109 runs in the log and no mismatches.
105
+ - MIT licence for the code; the data carried CC BY 4.0 (LSHK,
106
+ rime-cantonese) and Apache-2.0 (OpenCC).
107
+
108
+ ---
109
+
110
+ # Part II: Handover notes
111
+
112
+ ## 0. Start here
113
+
114
+ **Where things are:**
115
+
116
+ | Path | What it is |
117
+ | --- | --- |
118
+ | `src/xjyutping/__init__.py` | The whole package: `Jyutping`, `Segment`, `TONE_MARKS`. |
119
+ | `src/xjyutping/data/*.tsv` | Generated data (see below); never edit by hand. |
120
+ | `tests/test_xjyutping.py` | pytest suite (39 tests). |
121
+ | `tests/parity_corpus.txt`, `parity_expected.txt`, `parity.tex` | The parity test with the LaTeX package: the corpus, the frozen TeX debug log, and the document that regenerates the log. |
122
+ | `pyproject.toml`, `MANIFEST.in` | Packaging (setuptools; the TSV files are package data; the sdist includes the parity files). |
123
+ | `README.md`, `LICENSE`, `CHANGELOG.md` | Documentation, licences (code MIT, data CC BY-SA 4.0), this file. |
124
+
125
+ **The data** is generated, together with the LaTeX package's `.def` files,
126
+ by `tools/build-data.py` in
127
+ [xjyutping-tex](https://github.com/Beyond3345/xjyutping-tex). It writes here
128
+ when this repository sits next to xjyutping-tex (or with
129
+ `--py-data DIR`). The files are:
130
+
131
+ - `chars.tsv`: character, default reading, other readings, polyphone flag;
132
+ - `finals.tsv`: reading at the end of a run;
133
+ - `variants.tsv`: variant, canonical character;
134
+ - `words.tsv`: word, readings.
135
+
136
+ The sources and every rule for choosing readings are described in that
137
+ repository's README and changelog.
138
+
139
+ **Invariants to keep:**
140
+
141
+ 1. **The same readings as the LaTeX package.**
142
+ `test_parity_with_tex_package` compares every segment, reading and type of
143
+ `tests/parity_corpus.txt` with the LaTeX package's debug log.
144
+ - When the data or the segmentation changes on purpose, regenerate
145
+ `parity_expected.txt`: compile `tests/parity.tex` with
146
+ `TEXINPUTS=<xjyutping-tex>: LUAINPUTS=<xjyutping-tex>:` and
147
+ `max_print_line=10000`, then copy the `xjyutping>` lines. Check the diff
148
+ by hand.
149
+ 2. **Reading fixes belong in xjyutping-tex's `tools/build-data.py`**, never
150
+ in the TSV files or the code.
151
+ 3. **The algorithm mirrors the TeX package** (section 2). A change on one
152
+ side needs the same change on the other.
153
+ 4. **Python 3.8 compatibility:** no `match`, no `X | Y` unions evaluated at
154
+ runtime, no 3.9+ string methods.
155
+ 5. **Versioning** follows SemVer:
156
+ - MAJOR: an incompatible change of the API (a method, argument or return
157
+ value);
158
+ - MINOR: a backward-compatible feature, or new vocabulary (1.1.0);
159
+ - PATCH: bug fixes and reading corrections.
160
+
161
+ **How to test:**
162
+
163
+ ```bash
164
+ python3 -m pip install --upgrade pip && pip install -e '.[test]' && pytest tests
165
+ ```
166
+
167
+ Without installing, run `PYTHONPATH=src python3 -m pytest tests`. All tests
168
+ must pass, including `test_parity_with_tex_package`. To check a release,
169
+ build the sdist and wheel and run the tests from the extracted sdist.
170
+
171
+ ## 1. How the port was built (with xjyutping-tex 1.1.0)
172
+
173
+ The port was written by an agent against a frozen copy of the LaTeX package
174
+ 1.0.0. It took xpinyin's API and reproduced the LaTeX package's algorithm:
175
+ runs, segmentation by dynamic programming, word and character lookup, and
176
+ user settings. The parity with TeX was then checked on a 668-character
177
+ corpus: 0 mismatches.
178
+
179
+ Three independent reviewers then checked it, each from one angle:
180
+
181
+ - reading parity with TeX, on a new 1 021-character corpus: 119 runs
182
+ compared line for line;
183
+ - API, packaging and docs: the README examples run as doctests; wheel and
184
+ sdist contents;
185
+ - edge cases and tone styles.
186
+
187
+ Every finding was reproduced before it was fixed. The author's own 35 tests
188
+ had passed.
189
+
190
+ - **P-1. CR line ends** (reported by all three reviewers). A lone CR was not
191
+ counted as a line end, so `'\r\r'`, a blank line to TeX, did not end a
192
+ run. That changed segmentation and the run-final 呢.
193
+ - Fix: `_runs` counts `\n`, and `\r` not followed by `\n`.
194
+ - Test: `test_runs_cr_line_ends`, with expected values from a TeX log.
195
+ - **P-2. The sdist lacked the parity files,** so its own test suite failed
196
+ (2 tests, FileNotFoundError). Fix: `MANIFEST.in` includes `tests/*.txt`
197
+ and `*.tex`.
198
+ - **P-3. The README's editable install failed** with pip 21.2.4, since
199
+ editable installs need pip 21.3 or later. The README now upgrades pip
200
+ first.
201
+ - **P-4. `license = {text = "MIT"}` in `pyproject.toml`** triggered a
202
+ setuptools deprecation that becomes a build error after 2027-02-18.
203
+ Removed; `License-File` and the classifier remain.
204
+ - **P-5. The README's run-final example showed a word-list entry** instead
205
+ of the run-final rule. It was replaced with 呢張床好平,好麻煩呢?
206
+ - **P-6. `set_jyutping` on a character missing from the data did nothing.**
207
+ TeX reads such a character without context and with type `u`. It is now
208
+ its own run of type `u`; test `test_set_jyutping_char_not_in_data`.
209
+ - **P-7. `set_jyutping` rejected whitespace** that `\setjyutping` ignores.
210
+ It now strips spaces, tabs, CR and LF; test
211
+ `test_set_jyutping_ignores_spaces`.
212
+
213
+ Timing on the development Mac (Python 3.9):
214
+
215
+ - `Jyutping()` takes about 0.08 s;
216
+ - `annotate` on 10 000 characters takes about 0.02 s.
217
+
218
+ ## 2. How the port maps to the LaTeX package
219
+
220
+ | TeX (`xjyutping.sty`) | Python (`src/xjyutping/__init__.py`) |
221
+ | --- | --- |
222
+ | a character is Chinese iff `\xjp@c@<char>` exists | `c in self._chars` |
223
+ | runs: spaces and single line breaks between characters continue a run; `\par`/blank line ends it | `_runs` (ASCII space, tab, CR, LF; two line ends end the run; U+3000 ends it) |
224
+ | `\__xjyutping_flush:` (DP, 100000 per segment + 1 per single, `<=` so longer final word wins; bound `\xjp@e@` raw or canonical) | `_segment_run` (same costs and comparison; bound `_longest`, computed from `words.tsv` at load) |
225
+ | `\__xjyutping_emit:nn` word: raw key, else canonical; type `u` if `\xjp@uw@` | same, `self._user_words` |
226
+ | `\__xjyutping_lookup:nn` single: user, then `\xjp@f@` at run end, then default; type m/s from `\xjp@m@` | same (`_user_chars`, `_finals`, polyphone flag in `chars.tsv`) |
227
+ | `\__xjyutping_set:nnn` / `\__xjyutping_set_word:nn` | `set_jyutping` (raw and canonical keys, longest bound) |
228
+ | `debug` log `銀ngan4:w…` | `segment()` |
229
+
230
+ Differences by design:
231
+
232
+ - Python has no markup, so it has nothing like the command table, footnote
233
+ runs, `\xjyutping` manual readings or plain mode. The parity corpus
234
+ therefore has TeX's commands resolved into the runs TeX makes of them.
235
+ - `chars.tsv` lists the other readings of every character (for
236
+ `get_jyutpings`), not only of flagged polyphones.
237
+ - `Segment` has no field for the other readings. The parity test reads them
238
+ from `_chars`; users get them through `get_jyutpings`.
239
+ - `get_jyutping` drops whitespace between Chinese characters, U+3000
240
+ included, although U+3000 ends a run for segmentation.
241
+
242
+ ## 3. Version 1.1.0
243
+
244
+ Data only, plus documentation.
245
+
246
+ - The data of xjyutping-tex 1.2.0 was copied in by its build: 640 more
247
+ characters, 2 291 more words and the two hand fixes.
248
+ - The parity file was regenerated with both TeX engines, which agree.
249
+ - `test_added_vocabulary` was added. The README examples still run
250
+ unchanged (24 doctests). `python -m doctest README.md` counts the closing
251
+ code fence as expected output, so strip the fences first:
252
+
253
+ ```bash
254
+ PYTHONPATH=src python3 -c "import doctest,re; t=re.sub(r'(?m)^\x60{3}.*$','',open('README.md',encoding='utf8').read()); r=doctest.DocTestRunner(optionflags=doctest.NORMALIZE_WHITESPACE); r.run(doctest.DocTestParser().get_doctest(t,{},'README','README.md',0)); print(r.summarize())"
255
+ ```
256
+ - The code did not change.
257
+ - **Release to PyPI** (the name `xjyutping` was free on 2026-09-28):
258
+
259
+ ```bash
260
+ python3 -m pip install --upgrade build twine
261
+ python3 -m build
262
+ python3 -m twine check dist/*
263
+ python3 -m twine upload dist/*
264
+ ```
265
+
266
+ The 1.1.0 wheel and sdist were built this way and checked: `twine check`
267
+ passes, the wheel installs into a clean virtual environment, and the 39
268
+ tests pass from the extracted sdist. `twine upload` asks for a PyPI API
269
+ token (user name `__token__`); upload to TestPyPI first with
270
+ `--repository testpypi` if in doubt.
271
+
272
+ ## 4. Open issues
273
+
274
+ - **Readings that need more context than a word list gives** stay wrong
275
+ until a user sets them. The list is in the LaTeX package's changelog, Part
276
+ II, section 4, item 14, still open after its 1.2.0 (section 7.7), e.g. 為
277
+ wai4/wai6, 同行, 種花, 長得, 重未.
278
+ - **The book characters' licence.** The 640 characters from 粵音資料集叢
279
+ come from data published without a licence statement; see `LICENSE` and
280
+ the LaTeX package's changelog, section 7.4.
@@ -0,0 +1,48 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 the xjyutping authors
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
22
+
23
+ -------------------------------------------------------------------------------
24
+ Third-party material
25
+
26
+ The files in src/xjyutping/data/ are generated by tools/build-data.py of
27
+ the LaTeX package (https://github.com/Beyond3345/xjyutping-tex) and are
28
+ distributed under Creative Commons Attribution-ShareAlike 4.0
29
+ (CC BY-SA 4.0, https://creativecommons.org/licenses/by-sa/4.0/), because
30
+ they adapt share-alike material. They are built from:
31
+
32
+ * the LSHK Cantonese Pronunciation List of Characters for Computers
33
+ (jyutping-table), Creative Commons Attribution 4.0 (CC BY 4.0);
34
+ * rime-cantonese jyut6ping3.chars and jyut6ping3.words, Creative Commons
35
+ Attribution 4.0 (CC BY 4.0);
36
+ * CC-Canto, Copyright (c) 2015-17 Pleco Inc., and the CC-CEDICT Cantonese
37
+ readings, Copyright (c) 2015 Pleco Software Inc., both Creative Commons
38
+ Attribution-ShareAlike 3.0 (CC BY-SA 3.0), as distributed with Jyut
39
+ Dictionary (https://github.com/aaronhktan/jyut-dict);
40
+ * OpenCC HKVariants.txt and TWVariants.txt, Apache License 2.0;
41
+ * readings of 640 characters from the book data of 粵音資料集叢 by 石見田
42
+ (https://github.com/jyutnet/cantonese-books-data), published without a
43
+ licence statement and used with attribution.
44
+
45
+ The 'fancy' and 'discord' tone symbols (TONE_MARKS in
46
+ src/xjyutping/__init__.py) come from Visual Jyutping
47
+ (https://github.com/VincentTam/visual-jyutping), MIT License,
48
+ Copyright (c) 2024 Vincent Tam.
@@ -0,0 +1,2 @@
1
+ include CHANGELOG.md
2
+ recursive-include tests *.txt *.tex
@@ -0,0 +1,245 @@
1
+ Metadata-Version: 2.4
2
+ Name: xjyutping
3
+ Version: 1.1.0
4
+ Summary: Convert Traditional Chinese text to Cantonese Jyutping (粵拼), with pronunciation chosen from context
5
+ License-Expression: MIT AND CC-BY-SA-4.0
6
+ Project-URL: Homepage, https://github.com/Beyond3345/xjyutping-py
7
+ Project-URL: Repository, https://github.com/Beyond3345/xjyutping-py
8
+ Project-URL: Issues, https://github.com/Beyond3345/xjyutping-py/issues
9
+ Project-URL: Changelog, https://github.com/Beyond3345/xjyutping-py/blob/main/CHANGELOG.md
10
+ Project-URL: LaTeX package, https://github.com/Beyond3345/xjyutping-tex
11
+ Keywords: jyutping,cantonese,romanization,romanisation,粵拼,粵語,chinese
12
+ Classifier: Development Status :: 4 - Beta
13
+ Classifier: Intended Audience :: Developers
14
+ Classifier: Intended Audience :: Education
15
+ Classifier: Natural Language :: Chinese (Traditional)
16
+ Classifier: Operating System :: OS Independent
17
+ Classifier: Programming Language :: Python :: 3
18
+ Classifier: Programming Language :: Python :: 3 :: Only
19
+ Classifier: Topic :: Text Processing :: Linguistic
20
+ Requires-Python: >=3.8
21
+ Description-Content-Type: text/markdown
22
+ License-File: LICENSE
23
+ Provides-Extra: test
24
+ Requires-Dist: pytest>=6.2; extra == "test"
25
+ Dynamic: license-file
26
+
27
+ # xjyutping (Python)
28
+
29
+ Version 1.1.0 (2026-09-28). Versions follow
30
+ [Semantic Versioning](https://semver.org), and the history is in
31
+ [`CHANGELOG.md`](https://github.com/Beyond3345/xjyutping-py/blob/main/CHANGELOG.md).
32
+
33
+ xjyutping converts Traditional Chinese text to Cantonese Jyutping (粵拼).
34
+ Pronunciation is determined by looking at the context and cross-referencing
35
+ it with a list of about 104 000 words, so 行 is *hong4* in 銀行 but *haang4*
36
+ in 行路. It is the Python version of the LaTeX package
37
+ [xjyutping](https://github.com/Beyond3345/xjyutping-tex) and gives the same
38
+ readings. It was inspired by [xpinyin](https://github.com/lxneng/xpinyin),
39
+ which converts Chinese characters to Mandarin pinyin, and follows its API.
40
+
41
+ ## Install
42
+
43
+ Python 3.8 or later, no dependencies.
44
+
45
+ ```bash
46
+ pip install xjyutping
47
+ ```
48
+
49
+ or from GitHub with `pip install git+https://github.com/Beyond3345/xjyutping-py`.
50
+ For development (editable installs need pip 21.3 or later):
51
+
52
+ ```bash
53
+ python3 -m pip install --upgrade pip && pip install -e '.[test]' && pytest tests
54
+ ```
55
+
56
+ ## Usage
57
+
58
+ ```pycon
59
+ >>> from xjyutping import Jyutping
60
+ >>> j = Jyutping()
61
+ >>> j.get_jyutping('我哋去銀行') # default splitter is '-'
62
+ 'ngo5-dei6-heoi3-ngan4-hong4'
63
+ >>> j.get_jyutping('行路返屋企', ' ')
64
+ 'haang4 lou6 faan1 uk1 kei2'
65
+ >>> j.get_jyutping('香港', '')
66
+ 'hoeng1gong2'
67
+ >>> j.get_jyutping('香港', convert='capitalize') # or 'upper'; default 'lower'
68
+ 'Hoeng1-Gong2'
69
+ >>> j.get_jyutping('Apple發布iOS7') # other text is kept as it is
70
+ 'Apple-faat3-bou3-iOS7'
71
+ >>> j.get_jyutping('你好 世界') # spaces between characters are dropped
72
+ 'nei5-hou2-sai3-gaai3'
73
+ >>> j.annotate('銀行 ATM') # one pair per input character
74
+ [('銀', 'ngan4'), ('行', 'hong4'), (' ', None), ('A', None), ('T', None), ('M', None)]
75
+ ```
76
+
77
+ ### Tone styles
78
+
79
+ `tone_marks` works with `get_jyutping`, `get_jyutpings` and `annotate`:
80
+
81
+ | `tone_marks` | 詩史試時市事 |
82
+ | --- | --- |
83
+ | `'numbers'` (default) | si1 si2 si3 si4 si5 si6 |
84
+ | `None` | si si si si si si |
85
+ | `'fancy'` | siˉ¹ siˊ² si˗₃ siˎ₄ siˏ₅ siˍ₆ |
86
+ | `'discord'` | siˉ¹ si⸍² si-₃ si⸜₄ si⸝₅ siˍ₆ |
87
+
88
+ `'fancy'` and `'discord'` use the tone symbols of
89
+ [Visual Jyutping](https://github.com/VincentTam/visual-jyutping), from its web
90
+ and Discord versions. Any other value raises `ValueError`.
91
+
92
+ ```pycon
93
+ >>> j.get_jyutping('行路', ' ', tone_marks='fancy')
94
+ 'haangˎ₄ louˍ₆'
95
+ >>> Jyutping.decode_jyutping('hou2', 'fancy')
96
+ 'houˊ²'
97
+ ```
98
+
99
+ ### Context and your own readings
100
+
101
+ Text is split into runs of Chinese characters, and each run is split into
102
+ the fewest words from the word list. A word takes its reading from the list,
103
+ and a character on its own takes your reading or else its default. 呢 reads
104
+ *ni1* inside a run but *ne1* at its end. Variant shapes such as 為/爲 and
105
+ 裡/裏 find the same words.
106
+
107
+ ```pycon
108
+ >>> j.get_jyutping('校長喺學校長大')
109
+ 'haau6-zoeng2-hai2-hok6-haau6-zoeng2-daai6'
110
+ >>> j.get_jyutping('呢張床好平,好麻煩呢?', ' ')
111
+ 'ni1 zoeng1 cong4 hou2 peng4 , hou2 maa4 faan4 ne1 ?'
112
+ >>> j.segment('佢重話聽日嚟')
113
+ [Segment(text='佢', readings=['keoi5'], type='s'), Segment(text='重', readings=['cung5'], type='m'), Segment(text='話', readings=['waa6'], type='s'), Segment(text='聽日', readings=['ting1', 'jat6'], type='w'), Segment(text='嚟', readings=['lai4'], type='m')]
114
+ ```
115
+
116
+ `segment` shows how a text was read, like the LaTeX package's `debug`
117
+ option. The types are `w` for a word from the list, `u` for your setting, `m`
118
+ for a guessed reading of a character with several, and `s` for a character
119
+ with one reading.
120
+
121
+ `set_jyutping` works like `\setjyutping` in LaTeX. One character gets a new
122
+ default reading, and several characters (one syllable each) become a word. A
123
+ syllable count that doesn't match raises `ValueError`. Settings belong to the
124
+ `Jyutping` instance.
125
+
126
+ ```pycon
127
+ >>> j.set_jyutping('重話', 'zung6 waa6')
128
+ >>> j.get_jyutping('佢重話聽日嚟')
129
+ 'keoi5-zung6-waa6-ting1-jat6-lai4'
130
+ >>> j.set_jyutping('重', 'zung6')
131
+ >>> j.get_jyutping('佢重未嚟')
132
+ 'keoi5-zung6-mei6-lai4'
133
+ ```
134
+
135
+ ### Combinations
136
+
137
+ `get_jyutpings` lists up to `n` (default 10) combinations of readings, with
138
+ the reading in context first.
139
+
140
+ ```pycon
141
+ >>> j.get_jyutpings('行')
142
+ ['haang4', 'hang4', 'hong4', 'hong2', 'hang6']
143
+ >>> j.get_jyutpings('銀行', n=3)
144
+ ['ngan4-hong4', 'ngan4-haang4', 'ngan4-hang4']
145
+ >>> j.get_jyutpings('行', tone_marks=None)
146
+ ['haang', 'hang', 'hong']
147
+ ```
148
+
149
+ ### Initials
150
+
151
+ `get_initial` and `get_initials` give the first letter of each reading in
152
+ capitals, and leave other text unchanged. With `full=True` they give the
153
+ whole initial (`NG`, `GW`, `KW` ...), or an empty string for a syllable
154
+ without one, such as 屋 *uk1* or 唔 *m4*.
155
+
156
+ ```pycon
157
+ >>> j.get_initials('我哋去廣州')
158
+ 'N-D-H-G-Z'
159
+ >>> j.get_initials('我哋去廣州', full=True)
160
+ 'NG-D-H-GW-Z'
161
+ >>> j.get_initials('唔五屋'), j.get_initials('唔五屋', full=True)
162
+ ('M-N-U', '--')
163
+ ```
164
+
165
+ ## API
166
+
167
+ | | |
168
+ | --- | --- |
169
+ | `Jyutping(data_dir=None)` | Loads the data (from the package unless `data_dir` is given). |
170
+ | `get_jyutping(chars, splitter='-', tone_marks='numbers', convert='lower')` | Jyutping of a text. |
171
+ | `get_jyutpings(chars, splitter='-', tone_marks='numbers', convert='lower', n=10)` | Combinations of readings. |
172
+ | `annotate(text, tone_marks='numbers')` | `[(char, reading or None), ...]`, one per character. |
173
+ | `segment(text)` | `[Segment(text, readings, type), ...]` for the Chinese text. |
174
+ | `set_jyutping(text, reading)` | Your reading for a character or a word. |
175
+ | `get_initial(char, full=False)`, `get_initials(chars, splitter='-', full=False)` | Initials. |
176
+ | `Jyutping.decode_jyutping(syllable, tone_marks)`, `Jyutping.convert_jyutping(syllable, convert)` | Format one syllable. |
177
+
178
+ ## Data
179
+
180
+ The data in `src/xjyutping/data/` (30 089 characters, about 104 000 words) is
181
+ generated by `tools/build-data.py` in
182
+ [xjyutping-tex](https://github.com/Beyond3345/xjyutping-tex), from the
183
+ sources credited below. Fix readings there, not in the TSV files.
184
+
185
+ ## Tests
186
+
187
+ `pytest tests` runs the tests. One of them checks that the readings match
188
+ the LaTeX package's debug log (`tests/parity_expected.txt`), which
189
+ `tests/parity.tex` regenerates.
190
+
191
+ ## Acknowledgements and attributions
192
+
193
+ This project was inspired by [xpinyin](https://github.com/lxneng/xpinyin) by
194
+ Eric Lo ([lxneng](https://lxneng.com)), a Python package that converts
195
+ Chinese characters to pinyin. Its API is the model for this one. The LaTeX
196
+ version of xjyutping was inspired by the LaTeX package
197
+ [xpinyin](https://ctan.org/pkg/xpinyin) by Qing Lee (李清).
198
+
199
+ The `'fancy'` and `'discord'` styles use the tone symbols of
200
+ [Visual Jyutping](https://github.com/VincentTam/visual-jyutping) by Vincent
201
+ Tam, which was itself inspired by the Visual Cantonese Fonts (粵語字體) by Jon
202
+ Chui / A3I Ltd. ([canto.hk](https://canto.hk),
203
+ [documentation](https://docs.visual-fonts.com),
204
+ [source](https://github.com/jkwchui/visual-fonts-starlight-docs)). Thank you
205
+ both.
206
+
207
+ Thanks also to the authors of the data sources:
208
+
209
+ * the Jyutping Workgroup of the Linguistic Society of Hong Kong, for the
210
+ *Cantonese Pronunciation List of Characters for Computers*
211
+ (電腦用漢字粵語拼音表,
212
+ [lshk-org/jyutping-table](https://github.com/lshk-org/jyutping-table),
213
+ CC BY 4.0), and to Prof Lu Qin and Dr Cheung Kwan Hin of the Hong Kong
214
+ Polytechnic University and Nathan Hammond, who are thanked there;
215
+ * the Cantonese Computational Linguistics Infrastructure Development
216
+ Workgroup (CanCLID) and contributors, for
217
+ [rime-cantonese](https://github.com/rime/rime-cantonese) (粵語拼音輸入方案,
218
+ CC BY 4.0), which gives the default readings and most of the words;
219
+ * 石見田, for 粵音資料集叢 ([jyut.net](https://jyut.net/about), data at
220
+ [jyutnet/cantonese-books-data](https://github.com/jyutnet/cantonese-books-data)),
221
+ and the authors and editors of the dictionaries it digitises: 廣州話正音字典
222
+ (2004), 廣州話標準音字彙 (1988), 粵語同音字典 (1974/1996), 粵語查音識字字典
223
+ (1985), 同音字彙 (1971), 部身字典 (1967), *The Student's Cantonese-English
224
+ Dictionary* (1947), 粵音韻彙 (1941), 道字典 (1941), 道漢字音 (1939),
225
+ 民眾識字粵語拼音字彙 (1931), 廣話國語一貫未定稿 (1916) and
226
+ 分部分音廣話九聲字宗 (1914);
227
+ * Aaron Tan, for [Jyut Dictionary](https://github.com/aaronhktan/jyut-dict),
228
+ which distributes CC-Canto (© 2015–17 Pleco Inc.,
229
+ [cantonese.org](https://cantonese.org), CC BY-SA 3.0) and the Cantonese
230
+ readings for CC-CEDICT (© 2015 Pleco Software Inc., CC BY-SA 3.0), and MDBG
231
+ and the contributors of [CC-CEDICT](https://cc-cedict.org);
232
+ * Carbo Kuo (BYVoid) and contributors, for
233
+ [OpenCC](https://github.com/BYVoid/OpenCC) (Apache-2.0), whose variant
234
+ tables let variant shapes find the same words.
235
+
236
+ ## Licence
237
+
238
+ The code is under the MIT licence
239
+ ([`LICENSE`](https://github.com/Beyond3345/xjyutping-py/blob/main/LICENSE)),
240
+ and the tone symbols in `TONE_MARKS` come from Visual Jyutping (MIT). The data
241
+ files in `src/xjyutping/data/` are under
242
+ [CC BY-SA 4.0](https://creativecommons.org/licenses/by-sa/4.0/), because they
243
+ adapt the CC BY-SA 3.0 word lists above. The readings of the 640 characters
244
+ from 粵音資料集叢 come from data published without a licence and are used with
245
+ attribution.