locale-ai 0.2.2 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (3) hide show
  1. package/README.md +21 -7
  2. package/dist/catalog.js +30 -4
  3. package/package.json +1 -1
package/README.md CHANGED
@@ -1,5 +1,11 @@
1
1
  # locale-ai
2
2
 
3
+ [![npm version](https://img.shields.io/npm/v/locale-ai.svg)](https://www.npmjs.com/package/locale-ai)
4
+ [![npm downloads](https://img.shields.io/npm/dm/locale-ai.svg)](https://www.npmjs.com/package/locale-ai)
5
+ [![CI](https://github.com/khabouss/locale-ai/actions/workflows/ci.yml/badge.svg)](https://github.com/khabouss/locale-ai/actions/workflows/ci.yml)
6
+ [![license](https://img.shields.io/npm/l/locale-ai.svg)](LICENSE)
7
+ [![node](https://img.shields.io/node/v/locale-ai.svg)](package.json)
8
+
3
9
  Translate your i18n JSON locale files **locally, on CPU, for free**. No API keys, no Python, nothing leaves your machine.
4
10
 
5
11
  ```bash
@@ -10,9 +16,9 @@ npx locale-ai translate --from en --to fr,es,de
10
16
  locale-ai 0.2.0
11
17
  Found 3 en files in locales/en/
12
18
  Models to download (one time, then cached for every project):
13
- en-fr 114 MB Xenova/opus-mt-en-fr
19
+ en-fr 345 MB khabouss/opus-mt-tc-big-en-fr-onnx
14
20
  ...
15
- Download 346 MB? (Y/n)
21
+ Download 577 MB? (Y/n)
16
22
  ✓ fr 43 strings translated, 3 files written in 2.7s, 1 plural forms added
17
23
  ✓ es 43 strings translated, 3 files written in 2.5s, 1 plural forms added
18
24
  ✓ de 43 strings translated, 3 files written in 2.1s
@@ -27,6 +33,15 @@ Download 346 MB? (Y/n)
27
33
  - **Incremental**: only missing keys are translated, existing translations are kept, re-runs are no-ops. `--overwrite` re-translates everything.
28
34
  - **Structure**: key order, numbers, booleans, arrays, indentation and trailing newline are preserved.
29
35
 
36
+ ## Why locale-ai?
37
+
38
+ - **No API keys, no cloud bill.** Google/DeepL/OpenAI-based translation tools work well but mean an account, a key, a per-character cost, and your strings leaving your machine. locale-ai runs the model on your own CPU.
39
+ - **Built for i18n files, not plain text.** General MT tools translate strings; they don't know that `{count}`, `<0>bold</0>`, or `$t(common.footer.tagline)` must survive untouched. locale-ai is built around i18next/ICU conventions specifically, so placeholders, plurals, and nested references come out intact instead of mangled or dropped.
40
+ - **Incremental by default.** Add one new key to `en.json` and only that key gets translated — existing translations in `fr.json`/`es.json`/etc. are left alone, so re-runs are cheap and safe to automate in CI.
41
+ - **Honest about quality.** The [Languages](#languages) section tells you which pairs are review-light vs. which need a human pass, instead of presenting every language as equally production-ready.
42
+
43
+ **When *not* to use it**: if you need professional/legal-grade translation, or languages outside the [supported list](#languages), use a human translator or a larger cloud model instead — locale-ai is built for keeping UI strings in sync across languages during development, with a review pass before shipping.
44
+
30
45
  ## Commands
31
46
 
32
47
  ```bash
@@ -83,11 +98,10 @@ then declare it in `locale-ai.config.json`:
83
98
  - `prefix` is required for multi-target models (`convert` lists the model's `>>xx<<` tokens).
84
99
  - Project models take priority over built-ins for their language pairs and can be used for pivoting (e.g. `ja → en → tr`).
85
100
 
86
- **Recommended for quality**: the `opus-mt-tc-big-en-*` models (fr, es, it, pt, ar, tr, and `zle` for ru/uk). They are ~3x larger (270–370 MB int8) and ~1.5x slower, and clearly better:
101
+ **Recommended for quality**: the `opus-mt-tc-big-en-*` models (es, it, pt, ar, tr, and `zle` for ru/uk). They are ~3x larger (270–370 MB int8) and ~1.5x slower, and clearly better. `en-fr` already uses one of these as its built-in model (see [Languages](#languages)), hosted at [khabouss/opus-mt-tc-big-en-fr-onnx](https://huggingface.co/khabouss/opus-mt-tc-big-en-fr-onnx) — no conversion needed for it.
87
102
 
88
103
  | Target | Built-in model, chrF / BLEU | tc-big model, chrF / BLEU |
89
104
  |---|---|---|
90
- | fr | 68.9 / 53.7 | **72.5 / 55.9** |
91
105
  | es | 72.9 / 56.9 | **73.6 / 57.4** |
92
106
  | it | 73.8 / 54.9 | **75.3 / 58.2** |
93
107
  | pt | 68.3 / 44.4 | **70.6 / 47.7** (`>>por<<`, or `>>pob<<` for pt-BR) |
@@ -109,7 +123,7 @@ Measured on a sample app (see `test/fixtures`):
109
123
  | Usable — review before shipping | it, ar, ru, zh-CN, zh-TW |
110
124
  | Untested | the other pairs in `locale-ai models list` |
111
125
 
112
- With the tc-big models above, it, ar and ru move up a tier and tr becomes available. Not available yet: ja, ko, pl and others; he works with a converted `opus-mt-en-he` (weaker). Moroccan Arabic (Darija): the tc-big en-ar model has a `>>ary<<` token, but its output is close to Modern Standard Arabic — not supported.
126
+ `en-fr` is already on the tc-big model (chrF 72.5 / BLEU 55.9) as a built-in, no config needed. With the other tc-big models above, it, ar and ru move up a tier and tr becomes available. Not available yet: ja, ko, pl and others; he works with a converted `opus-mt-en-he` (weaker). Moroccan Arabic (Darija): the tc-big en-ar model has a `>>ary<<` token, but its output is close to Modern Standard Arabic — not supported.
113
127
 
114
128
  ## How it works
115
129
 
@@ -127,7 +141,7 @@ With the tc-big models above, it, ar and ru move up a tier and tr becomes availa
127
141
  ```bash
128
142
  npm install
129
143
  npm test # unit tests (no models)
130
- npm run test:e2e # real models, downloads ~115 MB once
144
+ npm run test:e2e # real models, downloads ~345 MB once (en-fr, tc-big)
131
145
  node bench/eval.ts --dir <model dir> --lang fr --beams 4 --out out.json # quality benchmark (see bench/README.md)
132
146
  npm run dev -- translate --to fr # run from source (Node ≥ 22.18 strips types)
133
147
  npm run build # → dist/
@@ -135,7 +149,7 @@ npm run build # → dist/
135
149
 
136
150
  ## Next steps
137
151
 
138
- 1. **Host the tc-big models** (converted ONNX) on Hugging Face so they can become the built-in defaults instead of a manual conversion.
152
+ 1. **Host the tc-big models** (converted ONNX) on Hugging Face so they can become the built-in defaults instead of a manual conversion. `en-fr` is done ([khabouss/opus-mt-tc-big-en-fr-onnx](https://huggingface.co/khabouss/opus-mt-tc-big-en-fr-onnx)); es, it, pt, ar, ru/uk, tr remain.
139
153
  2. **Smaller install.** `onnxruntime-node` is ~550 MB unpacked: it ships binaries for every platform plus a 260 MB CUDA provider we never use.
140
154
  3. **More languages**: ja, ko, pl have no usable OPUS-MT model yet; a different model family would be needed.
141
155
  4. **More formats**: YAML, `.po`, ARB, Apple `.strings`.
package/dist/catalog.js CHANGED
@@ -8,8 +8,8 @@ const pair = (src, tgt) => ({
8
8
  });
9
9
  const DIRECT = [
10
10
  "af-en", "ar-en", "cs-en", "da-de", "da-en", "de-en", "de-es", "de-fr",
11
- "en-af", "en-cs", "en-da", "en-de", "en-es", "en-fi", "en-fr", "en-hi",
12
- "en-hu", "en-id", "en-it", "en-nl", "en-ro", "en-ru", "en-sv", "en-uk",
11
+ "en-af", "en-cs", "en-da", "en-de", "en-es", "en-fi", "en-hi",
12
+ "en-hu", "en-id", "en-it", "en-nl", "en-ro", "en-sv",
13
13
  "en-xh", "es-de", "es-en", "es-fr", "es-it", "es-ru", "et-en", "fi-de",
14
14
  "fi-en", "fr-de", "fr-en", "fr-es", "fr-ro", "fr-ru", "hi-en", "hu-en",
15
15
  "id-en", "it-en", "it-es", "it-fr", "ja-en", "ko-en", "nl-en", "nl-fr",
@@ -22,10 +22,36 @@ export const CATALOG = [
22
22
  const [s, t] = p.split("-");
23
23
  return pair(s, t);
24
24
  }),
25
+ // tc-big: larger OPUS-MT checkpoints, clearly better than the defaults (see
26
+ // bench/README.md). Converted and hosted per the README's "Add a language,
27
+ // or a better model" section.
28
+ { ...pair("en", "fr"), repo: "khabouss/opus-mt-tc-big-en-fr-onnx" }, // chrF 72.5 vs 68.9, BLEU 55.9 vs 53.7
29
+ { ...pair("en", "es"), repo: "khabouss/opus-mt-tc-big-en-es-onnx" }, // chrF 73.6 vs 72.9, BLEU 57.4 vs 56.9
30
+ { ...pair("en", "it"), repo: "khabouss/opus-mt-tc-big-en-it-onnx" }, // chrF 75.3 vs 73.8, BLEU 58.2 vs 54.9
31
+ // New: Turkish, not supported by any built-in model before this.
32
+ { ...pair("en", "tr"), repo: "khabouss/opus-mt-tc-big-en-tr-onnx" }, // chrF 67.4, BLEU 42.1
25
33
  {
26
- // Note: the model card lists Moroccan Arabic ("ary") but its vocabulary has no
27
- // >>ary<< token, so Darija is not supported.
34
+ // Multi-target: >>por<< (European) / >>pob<< (Brazilian). No individual built-in
35
+ // existed before; pt was only reachable via the weaker en-ROMANCE model.
36
+ id: "en-pt",
37
+ repo: "khabouss/opus-mt-tc-big-en-pt-onnx", // chrF 70.6 vs 68.3, BLEU 47.7 vs 44.4
38
+ sources: ["en"],
39
+ targets: ["pt"],
40
+ prefix: (t) => (t.region === "BR" ? ">>pob<<" : ">>por<<"),
41
+ },
42
+ {
43
+ // Multi-target East Slavic model, replaces the separate en-ru/en-uk built-ins.
44
+ id: "en-zle",
45
+ repo: "khabouss/opus-mt-tc-big-en-zle-onnx", // ru: chrF 68.2 vs 65.7, BLEU 48.1 vs 46.0
46
+ sources: ["en"],
47
+ targets: ["ru", "uk"],
48
+ prefix: (t) => (t.lang === "ru" ? ">>rus<<" : ">>ukr<<"),
49
+ },
50
+ {
51
+ // Note: the model card lists Moroccan Arabic ("ary") but its output stays close
52
+ // to Modern Standard Arabic, so Darija is not treated as supported.
28
53
  ...pair("en", "ar"),
54
+ repo: "khabouss/opus-mt-tc-big-en-ar-onnx", // chrF 49.7 vs 47.8, BLEU 19.9 vs 16.6 (greedy)
29
55
  prefix: () => ">>ara<<",
30
56
  },
31
57
  {
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "locale-ai",
3
- "version": "0.2.2",
3
+ "version": "0.3.0",
4
4
  "description": "Translate your i18n locale files locally on CPU. Free, offline, no API keys.",
5
5
  "author": "Taha KHABOUSS",
6
6
  "repository": {