hatsuon 0.1.0__tar.gz

hatsuon-0.1.0/.gitignore

@@ -0,0 +1,19 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+.venv/
+dist/
+build/
+.pytest_cache/
+.ruff_cache/
+
+# Never commit audio or downloaded corpora (licences forbid redistribution)
+data/
+*.wav
+*.flac
+*.mp3
+*.m4a
+*.onnx
+
+# Claude
+CLAUDE.md

hatsuon-0.1.0/DATA_SOURCES.md

@@ -0,0 +1,18 @@
+# Data sources
+
+Corpora used for calibration and evaluation (SPEC Appendix B). Nothing listed here
+is redistributed by this repository: audio lives in the git-ignored `data/`
+directory and is fetched by `tools/fetch_data.sh`. Licence summaries are
+paraphrases for planning, not legal advice — re-read the official page before use.
+
+| Source | Official page | Licence (summary) | Read on | Used for |
+|---|---|---|---|---|
+| JSUT v1.1 | https://sites.google.com/site/shinnosuketakamichi/publication/jsut | Text CC BY-SA 4.0. Audio: academic / non-commercial research (incl. inside companies) / personal use; no redistribution (small excerpts allowed); commercial use by contacting the authors. | 2026-11-04 | F0 and duration sanity checks, local regression fixtures |
+| JVS v1 | https://sites.google.com/site/shinnosuketakamichi/research-topics/jvs_corpus | Tags CC BY-SA 4.0. Audio: same terms as JSUT (non-commercial research, no redistribution). | 2026-11-04 | Per-phone GOP calibration, duration norms, native false-alarm rate |
+| jsut-label | https://github.com/sarulab-speech/jsut-label | CC BY-SA 4.0 | 2026-11-04 | Ground-truth accent nucleus for basic5000 (front-end accent accuracy) |
+
+## Models
+
+| Model | Page | Licence | Read on | Notes |
+|---|---|---|---|---|
+| japanese-hubert-base-phoneme-ctc-v4 | https://huggingface.co/prj-beatrice/japanese-hubert-base-phoneme-ctc-v4 | Apache-2.0 | 2026-11-04 | Fine-tuned from rinna/japanese-hubert-base on ReazonSpeech (CDLA-Sharing-1.0, Art. 30-4 restricted — see SPEC B.2). Not redistributed; users export it locally with `hatsuon-export`. |

hatsuon-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Công Anh Dũng
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

hatsuon-0.1.0/PKG-INFO

@@ -0,0 +1,121 @@
+Metadata-Version: 2.5
+Name: hatsuon
+Version: 0.1.0
+Summary: Japanese pronunciation scoring: mora-level accuracy, special morae and pitch accent.
+Author: Công Anh Dũng
+License-Expression: MIT
+License-File: LICENSE
+Keywords: gop,japanese,mora,pitch-accent,pronunciation,speech
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Education
+Classifier: Natural Language :: Japanese
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Topic :: Multimedia :: Sound/Audio :: Speech
+Requires-Python: >=3.10
+Requires-Dist: numpy>=1.24
+Provides-Extra: all
+Requires-Dist: librosa>=0.10; extra == 'all'
+Requires-Dist: onnxruntime>=1.17; extra == 'all'
+Requires-Dist: pyopenjtalk>=0.3; extra == 'all'
+Requires-Dist: soundfile>=0.12; extra == 'all'
+Provides-Extra: audio
+Requires-Dist: librosa>=0.10; extra == 'audio'
+Requires-Dist: soundfile>=0.12; extra == 'audio'
+Provides-Extra: dev
+Requires-Dist: build>=1.2; extra == 'dev'
+Requires-Dist: pytest>=8; extra == 'dev'
+Requires-Dist: ruff>=0.5; extra == 'dev'
+Requires-Dist: twine>=5; extra == 'dev'
+Provides-Extra: export
+Requires-Dist: onnx>=1.15; extra == 'export'
+Requires-Dist: torch>=2.2; extra == 'export'
+Requires-Dist: transformers>=4.40; extra == 'export'
+Provides-Extra: frontend
+Requires-Dist: pyopenjtalk>=0.3; extra == 'frontend'
+Provides-Extra: onnx
+Requires-Dist: onnxruntime>=1.17; extra == 'onnx'
+Description-Content-Type: text/markdown
+
+# hatsuon
+
+Japanese pronunciation scoring for read-aloud speech, on the CPU. It scores
+accuracy per mora, the special morae (っ, ん, long vowels) and Tokyo pitch accent.
+
+> Status: **alpha (0.1)**. Thresholds were tuned on native speech only (JSUT/JVS),
+> and no L2 learner data has been used yet. Treat the scores as feedback, not as a
+> grade. Design and tuning records: [docs/SPEC.md](docs/SPEC.md) and
+> [`calibration/default.json`](src/hatsuon/calibration/default.json).
+
+## Install
+
+```bash
+pip install "hatsuon[all]"
+```
+
+The acoustic model is **not** shipped in the package. Convert it once on your
+machine. This step needs PyTorch and transformers only for the conversion itself:
+
+```bash
+pip install "hatsuon[export]"
+hatsuon-export            # writes ~/.cache/hatsuon/models/... (ONNX, int8)
+```
+
+`hatsuon-export --out DIR` writes the model elsewhere; set `HATSUON_MODEL_DIR=DIR`
+to use it. `--no-quantize` keeps FP32 weights.
+
+## Use
+
+```python
+import hatsuon
+
+r = hatsuon.score("learner.wav", "東京に行きます。")
+print(r.overall, r.accuracy, r.completeness, r.pitch, r.timing, r.confidence)
+for w in r.words:
+    print(w.surface, w.reading, w.score, w.accent.value)
+    for m in w.morae:
+        print("  ", m.kana, m.status.value, round(m.score), m.duration, m.pitch_status)
+print([x.value for x in r.warnings])
+```
+
+- `audio` can be a path, encoded bytes, or a `(samples, sample_rate)` tuple.
+- Ambiguous readings: `reading="きょう"` (whole text) or `reading={"今日": "きょう"}`.
+- Accent overrides: `accent={"箸": 1}` (0 = heiban, n = fall after mora n).
+- `pitch=False` or `timing=False` skips those stages.
+- `debug=True` attaches F0, the free decode and per-phone GOP to `r.debug`.
+- Soft problems such as `no_speech`, `content_mismatch` or `pitch_unreliable`
+  come back in `r.warnings` and lower `r.confidence`.
+- Hard errors raise `hatsuon.AudioError` or `hatsuon.TextError`.
+
+## Known limitations (0.1)
+
+- **Native-only calibration.** We have not yet measured mispronunciation recall
+  on learners. On 20 held-out JVS speakers, 0.6% of correct native morae fall
+  below the `mispronounced` threshold. Synthetic one-mora substitutions (the
+  text is changed, the audio is not) are caught 94% of the time: 98% for
+  consonants and 91% for vowels. Real learner errors are subtler than these.
+- **Pitch accent:**
+  - Native speakers often place the F0 peak one mora late, so a fall up to one
+    mora after the nucleus is accepted. As a result, a learner who puts the
+    nucleus one mora too late is not caught.
+  - About 9% of native heiban phrases are flagged with a false fall.
+- **Readings come from OpenJTalk.** When it misreads a word (proper nouns,
+  命 → イノチ when the speaker says めい), a correct reading shows up as
+  `deleted` morae. Pass `reading={"命": "めい"}` in those cases.
+- Read speech only. Not for spontaneous speech or singing.
+
+## Development
+
+```bash
+pip install -e ".[all,dev]"
+pytest && ruff check .
+bash tools/fetch_data.sh                  # JSUT, JVS, jsut-label into data/ (see DATA_SOURCES.md)
+python tools/eval_accent.py --n 300       # pitch-accent accuracy on JSUT
+python tools/calibrate.py collect && python tools/calibrate.py fit --write
+```
+
+## License
+
+MIT. Corpora and the acoustic model have their own licences and are never
+redistributed. See [DATA_SOURCES.md](DATA_SOURCES.md).

hatsuon-0.1.0/README.md

@@ -0,0 +1,81 @@
+# hatsuon
+
+Japanese pronunciation scoring for read-aloud speech, on the CPU. It scores
+accuracy per mora, the special morae (っ, ん, long vowels) and Tokyo pitch accent.
+
+> Status: **alpha (0.1)**. Thresholds were tuned on native speech only (JSUT/JVS),
+> and no L2 learner data has been used yet. Treat the scores as feedback, not as a
+> grade. Design and tuning records: [docs/SPEC.md](docs/SPEC.md) and
+> [`calibration/default.json`](src/hatsuon/calibration/default.json).
+
+## Install
+
+```bash
+pip install "hatsuon[all]"
+```
+
+The acoustic model is **not** shipped in the package. Convert it once on your
+machine. This step needs PyTorch and transformers only for the conversion itself:
+
+```bash
+pip install "hatsuon[export]"
+hatsuon-export            # writes ~/.cache/hatsuon/models/... (ONNX, int8)
+```
+
+`hatsuon-export --out DIR` writes the model elsewhere; set `HATSUON_MODEL_DIR=DIR`
+to use it. `--no-quantize` keeps FP32 weights.
+
+## Use
+
+```python
+import hatsuon
+
+r = hatsuon.score("learner.wav", "東京に行きます。")
+print(r.overall, r.accuracy, r.completeness, r.pitch, r.timing, r.confidence)
+for w in r.words:
+    print(w.surface, w.reading, w.score, w.accent.value)
+    for m in w.morae:
+        print("  ", m.kana, m.status.value, round(m.score), m.duration, m.pitch_status)
+print([x.value for x in r.warnings])
+```
+
+- `audio` can be a path, encoded bytes, or a `(samples, sample_rate)` tuple.
+- Ambiguous readings: `reading="きょう"` (whole text) or `reading={"今日": "きょう"}`.
+- Accent overrides: `accent={"箸": 1}` (0 = heiban, n = fall after mora n).
+- `pitch=False` or `timing=False` skips those stages.
+- `debug=True` attaches F0, the free decode and per-phone GOP to `r.debug`.
+- Soft problems such as `no_speech`, `content_mismatch` or `pitch_unreliable`
+  come back in `r.warnings` and lower `r.confidence`.
+- Hard errors raise `hatsuon.AudioError` or `hatsuon.TextError`.
+
+## Known limitations (0.1)
+
+- **Native-only calibration.** We have not yet measured mispronunciation recall
+  on learners. On 20 held-out JVS speakers, 0.6% of correct native morae fall
+  below the `mispronounced` threshold. Synthetic one-mora substitutions (the
+  text is changed, the audio is not) are caught 94% of the time: 98% for
+  consonants and 91% for vowels. Real learner errors are subtler than these.
+- **Pitch accent:**
+  - Native speakers often place the F0 peak one mora late, so a fall up to one
+    mora after the nucleus is accepted. As a result, a learner who puts the
+    nucleus one mora too late is not caught.
+  - About 9% of native heiban phrases are flagged with a false fall.
+- **Readings come from OpenJTalk.** When it misreads a word (proper nouns,
+  命 → イノチ when the speaker says めい), a correct reading shows up as
+  `deleted` morae. Pass `reading={"命": "めい"}` in those cases.
+- Read speech only. Not for spontaneous speech or singing.
+
+## Development
+
+```bash
+pip install -e ".[all,dev]"
+pytest && ruff check .
+bash tools/fetch_data.sh                  # JSUT, JVS, jsut-label into data/ (see DATA_SOURCES.md)
+python tools/eval_accent.py --n 300       # pitch-accent accuracy on JSUT
+python tools/calibrate.py collect && python tools/calibrate.py fit --write
+```
+
+## License
+
+MIT. Corpora and the acoustic model have their own licences and are never
+redistributed. See [DATA_SOURCES.md](DATA_SOURCES.md).