locale-ai 0.4.0 → 0.5.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -4
- package/dist/catalog.js +10 -0
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -111,19 +111,21 @@ then declare it in `locale-ai.config.json`:
|
|
|
111
111
|
|
|
112
112
|
300 Tatoeba test sentences per language, int8, beam 4 unless noted; reproduce with `bench/` (see `bench/README.md`). `opus-mt-tc-big-en-ko` is broken upstream (garbage output even in PyTorch) and is not recommended.
|
|
113
113
|
|
|
114
|
+
Also new: **pl** (chrF 62.6 / BLEU 40.3, `>>pol<<`), hosted at [khabouss/opus-mt-en-sla-onnx](https://huggingface.co/khabouss/opus-mt-en-sla-onnx) — no tc-big `en-pl` checkpoint exists, so this uses Helsinki-NLP's multi-target Slavic model instead.
|
|
115
|
+
|
|
114
116
|
## Languages
|
|
115
117
|
|
|
116
|
-
From English: af, ar, ca, cs, da, de, es, fi, fr, gl, hi, hu, id, it, nl, oc, pt (pt-BR, pt-PT), ro, ru, sv, uk, vi, xh, zh (zh-CN, zh-TW). Other source languages are supported directly or through English (`locale-ai models list --from fr`).
|
|
118
|
+
From English: af, ar, ca, cs, da, de, es, fi, fr, gl, hi, hu, id, it, nl, oc, pl, pt (pt-BR, pt-PT), ro, ru, sv, uk, vi, xh, zh (zh-CN, zh-TW). Other source languages are supported directly or through English (`locale-ai models list --from fr`).
|
|
117
119
|
|
|
118
120
|
Measured on a sample app (see `test/fixtures`):
|
|
119
121
|
|
|
120
122
|
| Quality (built-in models) | Languages |
|
|
121
123
|
|---|---|
|
|
122
124
|
| Good — light review | fr, es, de, pt-BR, nl |
|
|
123
|
-
| Usable — review before shipping | it, ar, ru, zh-CN, zh-TW |
|
|
125
|
+
| Usable — review before shipping | it, ar, ru, pl, zh-CN, zh-TW |
|
|
124
126
|
| Untested | the other pairs in `locale-ai models list` |
|
|
125
127
|
|
|
126
|
-
`en-fr` is already on the tc-big model (chrF 72.5 / BLEU 55.9) as a built-in, no config needed. With the other tc-big models above, it, ar and ru move up a tier and tr
|
|
128
|
+
`en-fr` is already on the tc-big model (chrF 72.5 / BLEU 55.9) as a built-in, no config needed. With the other tc-big models above, it, ar and ru move up a tier, and tr and pl become available. Not available yet: ja, ko and others; he works with a converted `opus-mt-en-he` (weaker). Moroccan Arabic (Darija): the tc-big en-ar model has a `>>ary<<` token, but its output is close to Modern Standard Arabic — not supported.
|
|
127
129
|
|
|
128
130
|
## How it works
|
|
129
131
|
|
|
@@ -150,5 +152,5 @@ npm run build # → dist/
|
|
|
150
152
|
|
|
151
153
|
## Next steps
|
|
152
154
|
|
|
153
|
-
1. **More languages**: ja, ko
|
|
155
|
+
1. **More languages**: ja, ko have no usable OPUS-MT model yet; a different model family would be needed.
|
|
154
156
|
2. **More formats**: `.po` (gettext), ARB (Flutter), Apple `.strings`. (JSON and YAML are supported today.)
|
package/dist/catalog.js
CHANGED
|
@@ -47,6 +47,16 @@ export const CATALOG = [
|
|
|
47
47
|
targets: ["ru", "uk"],
|
|
48
48
|
prefix: (t) => (t.lang === "ru" ? ">>rus<<" : ">>ukr<<"),
|
|
49
49
|
},
|
|
50
|
+
{
|
|
51
|
+
// New: Polish, not supported by any built-in model before this. The tc-big
|
|
52
|
+
// family has no en-pl checkpoint; this is Helsinki-NLP's multi-target Slavic
|
|
53
|
+
// model, used for its >>pol<< target only.
|
|
54
|
+
id: "en-pl",
|
|
55
|
+
repo: "khabouss/opus-mt-en-sla-onnx", // chrF 62.6, BLEU 40.3
|
|
56
|
+
sources: ["en"],
|
|
57
|
+
targets: ["pl"],
|
|
58
|
+
prefix: () => ">>pol<<",
|
|
59
|
+
},
|
|
50
60
|
{
|
|
51
61
|
// Note: the model card lists Moroccan Arabic ("ary") but its output stays close
|
|
52
62
|
// to Modern Standard Arabic, so Darija is not treated as supported.
|