locale-ai 0.6.3 → 0.6.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +6 -2
- package/dist/cli.js +76 -3
- package/dist/engine.js +32 -0
- package/dist/glossary.js +2 -0
- package/dist/locales.js +138 -34
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -31,11 +31,11 @@ Download 577 MB? (Y/n)
|
|
|
31
31
|
- **Layouts**: `locales/en/*.json` (nested folders too) → `locales/fr/...`, or `messages/en.json` → `messages/fr.json`. `.json`, `.yaml`, `.yml`, `.po` (gettext), `.arb` (Flutter) and `.strings` (Apple) are all supported, mixed freely in the same project. Common folders (`locales`, `src/locales`, `public/locales`, `src/i18n`, `messages`, `lang`, ...) are found automatically, or pass `--input`.
|
|
32
32
|
- **Placeholders are never translated or lost**: `{name}`, `{{count}}`, `%s`, `%1$s`, `%(name)s`, `%{name}`, `${name}`, `:attribute`, `$t(key)`, `@:key`, `<b>…</b>`, `<0>…</0>`, ` `, URLs, e-mails.
|
|
33
33
|
- **ICU MessageFormat**: `{count, plural, one {# file} other {# files}}`, `select`, nesting. Only the text is translated.
|
|
34
|
-
- **i18next plurals**: adds the forms the target language needs (`_few`, `_many` for Arabic, Russian, Polish...), so the app doesn't fall back to the English string — each added form copies the `_other` translation verbatim rather than inflecting it, so review and adjust the wording of `_zero`/`_two`/`_few`/`_many` by hand for languages where those forms use a different noun/adjective form.
|
|
34
|
+
- **i18next plurals**: adds the forms the target language needs (`_few`, `_many` for Arabic, Russian, Polish...), so the app doesn't fall back to the English string — each added form copies the `_other` translation verbatim rather than inflecting it, so review and adjust the wording of `_zero`/`_two`/`_few`/`_many` by hand for languages where those forms use a different noun/adjective form. The `_one` form is translated with its `{{count}}` temporarily replaced by a literal "1", since the model otherwise sometimes picks the target's plural noun form despite the English source being grammatically singular.
|
|
35
35
|
- **Incremental**: only missing keys are translated, existing translations are kept, re-runs are no-ops. `--overwrite` re-translates everything.
|
|
36
36
|
- **Structure**: key order, numbers, booleans, arrays, indentation and trailing newline are preserved.
|
|
37
37
|
|
|
38
|
-
**Format notes**: for `.po`, `msgid` is the stable key (never changed) and `msgstr` is the translated value, as gettext expects — the English `.po` file's `msgstr` must hold the source text (most generators already fill it; an empty `msgstr` is treated as empty, not inherited from `msgid`)
|
|
38
|
+
**Format notes**: for `.po`, `msgid` is the stable key (never changed) and `msgstr` is the translated value, as gettext expects — the English `.po` file's `msgstr` must hold the source text (most generators already fill it; an empty `msgstr` is treated as empty, not inherited from `msgid`). `msgid_plural`/`msgstr[n]` plural entries are translated too (the singular and plural source forms), then expanded to however many `msgstr[n]` the target language's plural rules need (6 for Arabic, 1 for Japanese, ...) — not full grammatical inflection for in-between categories (`few`/`many`/...), which get the plural-sourced translation verbatim, the same simplification `addPluralForms` makes for i18next. `Plural-Forms` in the header is computed for the target language, not copied from the source; an existing target file's header is otherwise left as you last edited it (a hand-fixed `Plural-Forms`, `Last-Translator`, ...) instead of being regenerated on every rerun. For `.arb`, `@key` metadata and `@@locale` are never translated; `@@locale` is set to the target locale on output.
|
|
39
39
|
|
|
40
40
|
## Why locale-ai?
|
|
41
41
|
|
|
@@ -50,6 +50,7 @@ Download 577 MB? (Y/n)
|
|
|
50
50
|
|
|
51
51
|
```bash
|
|
52
52
|
locale-ai translate --to fr,pt-BR [--from en] [--input dir] [--dry-run] [--overwrite] [--fast] [--report file] [-y]
|
|
53
|
+
locale-ai check [--from en] [--input dir] # CI: fails if a translated locale has missing keys or placeholder mismatches
|
|
53
54
|
locale-ai models list [--from en] # supported language pairs
|
|
54
55
|
locale-ai models installed
|
|
55
56
|
locale-ai models install en-de # pre-download (e.g. in CI)
|
|
@@ -68,6 +69,7 @@ Models are cached once per machine in `~/.cache/locale-ai` (`LOCALE_AI_CACHE` to
|
|
|
68
69
|
```json
|
|
69
70
|
{
|
|
70
71
|
"doNotTranslate": ["Acme", "Acme Pro"],
|
|
72
|
+
"doNotTranslateKeys": ["brand.name", "config.statusEnum"],
|
|
71
73
|
"glossary": {
|
|
72
74
|
"fr": { "Workspace": "Espace de travail" },
|
|
73
75
|
"de": { "Sign in": "Einloggen" }
|
|
@@ -77,6 +79,8 @@ Models are cached once per machine in `~/.cache/locale-ai` (`LOCALE_AI_CACHE` to
|
|
|
77
79
|
|
|
78
80
|
`glossary` entries replace whole strings. `models` adds your own translation models (next section). A built-in glossary covers ~60 standard UI labels ("Sign in", "Settings", "Log out", ...) for fr, es, de, it, pt-BR, nl and ar, because one-to-three word labels are where sentence-level MT is weakest.
|
|
79
81
|
|
|
82
|
+
`doNotTranslate` protects a term everywhere it appears in any string. `doNotTranslateKeys` instead protects a key path (dot/`[i]` notation, e.g. `brand.name`, `items[0]`) — the whole value at that path, and everything nested under it if it's an object or array, is copied to every target verbatim. Use it for an enum array (`"statusEnum": ["ACTIVE", "PENDING"]`, which you'd otherwise have to list value-by-value in `doNotTranslate`) or a field a model might respell or inflect even when the word is listed in `doNotTranslate` elsewhere in the same string.
|
|
83
|
+
|
|
80
84
|
## Add a language, or a better model
|
|
81
85
|
|
|
82
86
|
Any [MarianMT / OPUS-MT](https://huggingface.co/Helsinki-NLP) model on Hugging Face can be used. Convert it once (needs Python):
|
package/dist/cli.js
CHANGED
|
@@ -9,14 +9,19 @@ import { CATALOG, customEntries, findById, reachableTargets, resolveRoute, } fro
|
|
|
9
9
|
import { translateStrings } from "./engine.js";
|
|
10
10
|
import { CONFIG_FILE, glossaryLookup, loadConfig, protectRegex } from "./glossary.js";
|
|
11
11
|
import { intlCode, parseLocale } from "./lang.js";
|
|
12
|
-
import { addPluralForms, buildTarget, detectLayout, pendingStrings, readLocaleDoc, sourceFiles, targetFile, } from "./locales.js";
|
|
12
|
+
import { addPluralForms, buildTarget, detectLayout, existingTargets, flattenStrings, keyPathProtect, pendingStrings, pluralOneStrings, readLocaleDoc, sourceFiles, targetFile, } from "./locales.js";
|
|
13
13
|
import { cacheRoot, install, isInstalled, loadTranslator, localModelProblem, planInstall, readManifest, uninstall, } from "./models.js";
|
|
14
|
+
import { mask } from "./placeholders.js";
|
|
14
15
|
/** Read from package.json so `npm version` is the single source of truth. */
|
|
15
16
|
const VERSION = JSON.parse(fs.readFileSync(fileURLToPath(new URL("../package.json", import.meta.url)), "utf8")).version;
|
|
16
17
|
const HELP = `locale-ai ${VERSION} — translate i18n locale files locally, on CPU, for free.
|
|
17
18
|
|
|
18
19
|
Usage:
|
|
19
20
|
locale-ai translate --to <locales> [options]
|
|
21
|
+
locale-ai check [--from <locale>]
|
|
22
|
+
Validate that already-translated locales have the same keys and
|
|
23
|
+
placeholders as the source, without translating. Exits non-zero
|
|
24
|
+
on a mismatch (CI-friendly).
|
|
20
25
|
locale-ai models list [--from <locale>]
|
|
21
26
|
locale-ai models installed
|
|
22
27
|
locale-ai models install <from-to | model-id> [--yes]
|
|
@@ -153,8 +158,10 @@ async function cmdTranslate(argv) {
|
|
|
153
158
|
const config = loadConfig(process.cwd(), values.config);
|
|
154
159
|
const catalog = catalogFor(config);
|
|
155
160
|
const protect = protectRegex(config);
|
|
161
|
+
const isProtectedKey = keyPathProtect(config.doNotTranslateKeys);
|
|
156
162
|
const layout = detectLayout(process.cwd(), from.tag, values.input);
|
|
157
163
|
const sources = sourceFiles(layout, from.tag).map((f) => ({ file: f, doc: readLocaleDoc(f, from.tag) }));
|
|
164
|
+
const singularHints = new Set(sources.flatMap(({ doc }) => [...pluralOneStrings(doc.data)]));
|
|
158
165
|
log(`locale-ai ${VERSION}`);
|
|
159
166
|
log(`Found ${sources.length} ${from.tag} file${sources.length === 1 ? "" : "s"} in ${rel(layout.root)}/` +
|
|
160
167
|
(layout.kind === "dir" ? `${from.tag}/` : ""));
|
|
@@ -172,7 +179,13 @@ async function cmdTranslate(argv) {
|
|
|
172
179
|
const files = sources.map(({ file, doc }) => {
|
|
173
180
|
const target = targetFile(layout, from.tag, locale.tag, file);
|
|
174
181
|
const existing = fs.existsSync(target) ? readLocaleDoc(target, locale.tag).data : undefined;
|
|
175
|
-
return {
|
|
182
|
+
return {
|
|
183
|
+
source: file,
|
|
184
|
+
target,
|
|
185
|
+
doc,
|
|
186
|
+
existing,
|
|
187
|
+
pending: pendingStrings(doc.data, existing, values.overwrite, isProtectedKey),
|
|
188
|
+
};
|
|
176
189
|
});
|
|
177
190
|
const unique = new Set(files.flatMap((f) => f.pending)).size;
|
|
178
191
|
jobs.push({ locale, route, files, unique });
|
|
@@ -214,6 +227,7 @@ async function cmdTranslate(argv) {
|
|
|
214
227
|
batchSize,
|
|
215
228
|
protect,
|
|
216
229
|
lookup: glossaryLookup(from, job.locale, config),
|
|
230
|
+
singularHints,
|
|
217
231
|
onProgress: makeProgress((d, t) => `${job.locale.tag}: translating ${bar(d, t)} ${d}/${t}`),
|
|
218
232
|
});
|
|
219
233
|
endStatus();
|
|
@@ -230,7 +244,7 @@ async function cmdTranslate(argv) {
|
|
|
230
244
|
let written = 0;
|
|
231
245
|
let pluralsAdded = 0;
|
|
232
246
|
for (const f of job.files) {
|
|
233
|
-
const built = buildTarget(f.doc.data, f.existing, (s) => translations.get(s) ?? s, values.overwrite);
|
|
247
|
+
const built = buildTarget(f.doc.data, f.existing, (s) => translations.get(s) ?? s, values.overwrite, isProtectedKey);
|
|
234
248
|
const { doc, added } = addPluralForms(built, intlCode(job.locale));
|
|
235
249
|
pluralsAdded += added;
|
|
236
250
|
const before = fs.existsSync(f.target) ? fs.readFileSync(f.target, "utf8") : undefined;
|
|
@@ -370,11 +384,70 @@ async function cmdModels(argv) {
|
|
|
370
384
|
return sub ? 1 : 0;
|
|
371
385
|
}
|
|
372
386
|
}
|
|
387
|
+
/**
|
|
388
|
+
* Validates already-translated locales against the source without translating:
|
|
389
|
+
* same key structure, and the same placeholders in each matching string. Meant for
|
|
390
|
+
* CI, so a mismatch (a key a translator dropped, a {name} a translation lost) fails
|
|
391
|
+
* the build instead of shipping silently.
|
|
392
|
+
*/
|
|
393
|
+
async function cmdCheck(argv) {
|
|
394
|
+
const { values } = parseArgs({
|
|
395
|
+
args: argv,
|
|
396
|
+
options: {
|
|
397
|
+
from: { type: "string", default: "en" },
|
|
398
|
+
input: { type: "string" },
|
|
399
|
+
},
|
|
400
|
+
});
|
|
401
|
+
const from = parseLocale(values.from);
|
|
402
|
+
const layout = detectLayout(process.cwd(), from.tag, values.input);
|
|
403
|
+
const sources = sourceFiles(layout, from.tag).map((f) => ({ file: f, doc: readLocaleDoc(f, from.tag) }));
|
|
404
|
+
const targets = existingTargets(layout, from.tag);
|
|
405
|
+
if (!targets.length) {
|
|
406
|
+
log(`No translated locales found next to ${from.tag}.`);
|
|
407
|
+
return 0;
|
|
408
|
+
}
|
|
409
|
+
let problems = 0;
|
|
410
|
+
for (const to of targets) {
|
|
411
|
+
let missing = 0;
|
|
412
|
+
let mismatched = 0;
|
|
413
|
+
for (const { file, doc } of sources) {
|
|
414
|
+
const target = targetFile(layout, from.tag, to, file);
|
|
415
|
+
if (!fs.existsSync(target)) {
|
|
416
|
+
missing += flattenStrings(doc.data).length;
|
|
417
|
+
log(`✗ ${to}: ${rel(target)} is missing entirely`);
|
|
418
|
+
continue;
|
|
419
|
+
}
|
|
420
|
+
const srcFlat = flattenStrings(doc.data);
|
|
421
|
+
const tgtByPath = new Map(flattenStrings(readLocaleDoc(target, to).data).map((e) => [e.path, e.value]));
|
|
422
|
+
for (const { path: p, value: srcVal } of srcFlat) {
|
|
423
|
+
const tgtVal = tgtByPath.get(p);
|
|
424
|
+
if (tgtVal === undefined) {
|
|
425
|
+
missing++;
|
|
426
|
+
log(`✗ ${to}: ${rel(target)} missing key "${p}"`);
|
|
427
|
+
continue;
|
|
428
|
+
}
|
|
429
|
+
const srcTokens = mask(srcVal, "natural").tokens.toSorted();
|
|
430
|
+
const tgtTokens = mask(tgtVal, "natural").tokens.toSorted();
|
|
431
|
+
if (srcTokens.join("\u0001") !== tgtTokens.join("\u0001")) {
|
|
432
|
+
mismatched++;
|
|
433
|
+
log(`✗ ${to}: ${rel(target)} "${p}" placeholders differ: source [${srcTokens.join(", ")}], target [${tgtTokens.join(", ")}]`);
|
|
434
|
+
}
|
|
435
|
+
}
|
|
436
|
+
}
|
|
437
|
+
if (missing || mismatched)
|
|
438
|
+
problems++;
|
|
439
|
+
else
|
|
440
|
+
log(`✓ ${to}`);
|
|
441
|
+
}
|
|
442
|
+
return problems ? 1 : 0;
|
|
443
|
+
}
|
|
373
444
|
async function main() {
|
|
374
445
|
const [cmd, ...rest] = process.argv.slice(2);
|
|
375
446
|
switch (cmd) {
|
|
376
447
|
case "translate":
|
|
377
448
|
return cmdTranslate(rest);
|
|
449
|
+
case "check":
|
|
450
|
+
return cmdCheck(rest);
|
|
378
451
|
case "models":
|
|
379
452
|
return cmdModels(rest);
|
|
380
453
|
case "-v":
|
package/dist/engine.js
CHANGED
|
@@ -208,6 +208,36 @@ async function splitFallback(units, hops, opts, progress, issues) {
|
|
|
208
208
|
issues.push({ source: unit.owner, kind: "placeholder-fallback", output: unit.result });
|
|
209
209
|
}
|
|
210
210
|
}
|
|
211
|
+
const COUNT_PLACEHOLDER = /\{\{\s*count\s*\}\}/;
|
|
212
|
+
/**
|
|
213
|
+
* A "_one" key's source is grammatically singular ("You have {{count}} file"), but the
|
|
214
|
+
* masked marker that stands in for {{count}} carries no number of its own, so the model
|
|
215
|
+
* sometimes still produces the target's PLURAL noun form (confirmed: en→fr "fileCount_one"
|
|
216
|
+
* came back "Vous avez {{count}} fichiers"). Retranslating with a literal "1" in place of
|
|
217
|
+
* {{count}} gives the model the real grammatical cue; only applied when {{count}} is the
|
|
218
|
+
* string's only placeholder (anything else needs the normal masking pipeline to survive),
|
|
219
|
+
* and only kept when the "1" survives translation intact enough to restore {{count}}.
|
|
220
|
+
*/
|
|
221
|
+
async function fixSingularGrammar(translations, singularHints, remaining, hops, opts, progress) {
|
|
222
|
+
for (const s of singularHints) {
|
|
223
|
+
if (!remaining.has(s) || translations.get(s) === undefined)
|
|
224
|
+
continue;
|
|
225
|
+
const tokens = mask(s, "natural").tokens;
|
|
226
|
+
if (tokens.length !== 1 || !COUNT_PLACEHOLDER.test(tokens[0]))
|
|
227
|
+
continue;
|
|
228
|
+
const m = COUNT_PLACEHOLDER.exec(s);
|
|
229
|
+
const pinned = s.slice(0, m.index) + "1" + s.slice(m.index + m[0].length);
|
|
230
|
+
const prepared = prepare(pinned);
|
|
231
|
+
const out = (await runModels(new Set([prepared]), hops, opts, progress)).get(prepared) ?? "";
|
|
232
|
+
if (isDegenerate(pinned, out))
|
|
233
|
+
continue;
|
|
234
|
+
const post = postprocess(pinned, out, opts.targetLang);
|
|
235
|
+
const restored = /(?<![\d.])1(?![\d.])/.exec(post);
|
|
236
|
+
if (!restored)
|
|
237
|
+
continue; // can't confirm the "1" survived: keep the original translation
|
|
238
|
+
translations.set(s, post.slice(0, restored.index) + m[0] + post.slice(restored.index + 1));
|
|
239
|
+
}
|
|
240
|
+
}
|
|
211
241
|
export async function translateStrings(strings, hops, opts) {
|
|
212
242
|
const issues = [];
|
|
213
243
|
const progress = { done: 0, total: 0 };
|
|
@@ -236,5 +266,7 @@ export async function translateStrings(strings, hops, opts) {
|
|
|
236
266
|
const parts = p.units.map((u) => u.result ?? u.text);
|
|
237
267
|
translations.set(p.source, p.split ? p.split.rebuild(parts) : parts[0]);
|
|
238
268
|
}
|
|
269
|
+
if (opts.singularHints?.size)
|
|
270
|
+
await fixSingularGrammar(translations, opts.singularHints, new Set(remaining), hops, opts, progress);
|
|
239
271
|
return { translations, issues };
|
|
240
272
|
}
|
package/dist/glossary.js
CHANGED
|
@@ -84,6 +84,8 @@ export function loadConfig(cwd, file) {
|
|
|
84
84
|
const cfg = JSON.parse(fs.readFileSync(p, "utf8"));
|
|
85
85
|
if (cfg.doNotTranslate && !Array.isArray(cfg.doNotTranslate))
|
|
86
86
|
throw new Error('"doNotTranslate" must be an array');
|
|
87
|
+
if (cfg.doNotTranslateKeys && !Array.isArray(cfg.doNotTranslateKeys))
|
|
88
|
+
throw new Error('"doNotTranslateKeys" must be an array');
|
|
87
89
|
if (cfg.models && !Array.isArray(cfg.models))
|
|
88
90
|
throw new Error('"models" must be an array');
|
|
89
91
|
return { ...cfg, dir: path.dirname(p) };
|
package/dist/locales.js
CHANGED
|
@@ -82,6 +82,23 @@ export function targetFile(layout, from, to, src) {
|
|
|
82
82
|
return path.join(layout.root, `${to}${layout.ext}`);
|
|
83
83
|
return path.join(layout.root, to, path.relative(path.join(layout.root, from), src));
|
|
84
84
|
}
|
|
85
|
+
/** Locale tags already translated alongside `from` in this layout (for `locale-ai check`). */
|
|
86
|
+
export function existingTargets(layout, from) {
|
|
87
|
+
if (layout.kind === "file") {
|
|
88
|
+
const ext = layout.ext;
|
|
89
|
+
return fs
|
|
90
|
+
.readdirSync(layout.root)
|
|
91
|
+
.filter((f) => f.endsWith(ext) && f.slice(0, -ext.length) !== from)
|
|
92
|
+
.map((f) => f.slice(0, -ext.length))
|
|
93
|
+
.sort();
|
|
94
|
+
}
|
|
95
|
+
return fs
|
|
96
|
+
.readdirSync(layout.root, { withFileTypes: true })
|
|
97
|
+
.filter((e) => e.isDirectory() && !e.name.startsWith(".") && e.name !== "node_modules" && e.name !== from)
|
|
98
|
+
.map((e) => e.name)
|
|
99
|
+
.filter((name) => listLocaleFiles(path.join(layout.root, name)).length > 0)
|
|
100
|
+
.sort();
|
|
101
|
+
}
|
|
85
102
|
const finalNewline = (text) => text === "" || text.endsWith("\n");
|
|
86
103
|
const withNewline = (text, keep) => text.replace(/\n*$/, keep ? "\n" : "");
|
|
87
104
|
function readJsonDoc(file, text) {
|
|
@@ -242,6 +259,10 @@ function pluralFormsHeader(lang) {
|
|
|
242
259
|
const branches = Array.from({ length: n - 1 }, (_, i) => `n==${i + 1} ? ${i} :`).join(" ");
|
|
243
260
|
return `nplurals=${n}; plural=(${branches} ${n - 1});`;
|
|
244
261
|
}
|
|
262
|
+
/** The `nplurals=N` count from pluralFormsHeader, for expanding msgstr[] to the right length. */
|
|
263
|
+
function pluralCount(lang) {
|
|
264
|
+
return Number(/nplurals=(\d+)/.exec(pluralFormsHeader(lang))[1]);
|
|
265
|
+
}
|
|
245
266
|
function parsePo(text) {
|
|
246
267
|
const entries = [];
|
|
247
268
|
let leading = [];
|
|
@@ -318,12 +339,26 @@ function parsePo(text) {
|
|
|
318
339
|
flushEntry();
|
|
319
340
|
return { entries, leading };
|
|
320
341
|
}
|
|
342
|
+
/** Synthetic data keys for a plural entry's two translatable forms. */
|
|
343
|
+
const pluralOneKey = (e) => poKey(e) + "\u0005one";
|
|
344
|
+
const pluralOtherKey = (e) => poKey(e) + "\u0005other";
|
|
321
345
|
function readPoDoc(file, text) {
|
|
322
346
|
const { entries, leading } = parsePo(text);
|
|
323
347
|
const data = newObj();
|
|
324
|
-
for (const e of entries)
|
|
325
|
-
if (
|
|
348
|
+
for (const e of entries) {
|
|
349
|
+
if (e.isHeader)
|
|
350
|
+
continue;
|
|
351
|
+
if (e.msgidPlural === undefined)
|
|
326
352
|
data[poKey(e)] = e.msgstr;
|
|
353
|
+
else {
|
|
354
|
+
// Like msgstr for a singular entry, msgstrPlural holds this FILE's own text: the
|
|
355
|
+
// English source form in the source .po, the translated forms in a target .po
|
|
356
|
+
// (msgid/msgid_plural are the stable keys, identical across every locale).
|
|
357
|
+
const forms = e.msgstrPlural ?? [];
|
|
358
|
+
data[pluralOneKey(e)] = forms[0] ?? "";
|
|
359
|
+
data[pluralOtherKey(e)] = forms[forms.length - 1] ?? forms[0] ?? "";
|
|
360
|
+
}
|
|
361
|
+
}
|
|
327
362
|
const keepNewline = finalNewline(text);
|
|
328
363
|
const serialize = (d, targetTag, existingText) => {
|
|
329
364
|
const obj = isObj(d) ? d : newObj();
|
|
@@ -347,7 +382,19 @@ function readPoDoc(file, text) {
|
|
|
347
382
|
}
|
|
348
383
|
else if (e.msgidPlural !== undefined) {
|
|
349
384
|
lines.push(`msgid "${escapeC(e.msgid)}"`, `msgid_plural "${escapeC(e.msgidPlural)}"`);
|
|
350
|
-
|
|
385
|
+
const oneKey = pluralOneKey(e);
|
|
386
|
+
const otherKey = pluralOtherKey(e);
|
|
387
|
+
seen.add(oneKey);
|
|
388
|
+
seen.add(otherKey);
|
|
389
|
+
const one = typeof obj[oneKey] === "string" ? obj[oneKey] : e.msgid;
|
|
390
|
+
const other = typeof obj[otherKey] === "string" ? obj[otherKey] : e.msgidPlural;
|
|
391
|
+
const n = pluralCount(targetTag);
|
|
392
|
+
// Not full grammatical inflection (same simplification as addPluralForms for
|
|
393
|
+
// i18next): index 0 gets the singular-sourced form, every other index the
|
|
394
|
+
// plural-sourced one. A language with only one category (ja, zh, ...) gets
|
|
395
|
+
// just the plural form, since it has no separate singular count.
|
|
396
|
+
for (let i = 0; i < n; i++)
|
|
397
|
+
lines.push(`msgstr[${i}] "${escapeC(n === 1 ? other : i === 0 ? one : other)}"`);
|
|
351
398
|
}
|
|
352
399
|
else {
|
|
353
400
|
const key = poKey(e);
|
|
@@ -390,47 +437,104 @@ const isObj = (v) => typeof v === "object" && v !== null && !Array.isArray(v);
|
|
|
390
437
|
const newObj = () => Object.create(null);
|
|
391
438
|
/** A string needs translating when the target has no non-empty string at the same path. */
|
|
392
439
|
const needsTranslation = (existing, overwrite) => overwrite || typeof existing !== "string" || existing === "";
|
|
393
|
-
|
|
394
|
-
|
|
395
|
-
|
|
396
|
-
|
|
397
|
-
|
|
398
|
-
|
|
399
|
-
|
|
400
|
-
|
|
401
|
-
|
|
402
|
-
|
|
403
|
-
|
|
404
|
-
|
|
440
|
+
/** Flattens a locale tree into key-path -> string-value pairs (for `locale-ai check`). */
|
|
441
|
+
export function flattenStrings(data, prefix = "", out = []) {
|
|
442
|
+
if (typeof data === "string")
|
|
443
|
+
out.push({ path: prefix || "(root)", value: data });
|
|
444
|
+
else if (Array.isArray(data))
|
|
445
|
+
data.forEach((v, i) => flattenStrings(v, `${prefix}[${i}]`, out));
|
|
446
|
+
else if (isObj(data))
|
|
447
|
+
for (const k of Object.keys(data))
|
|
448
|
+
flattenStrings(data[k], prefix ? `${prefix}.${k}` : k, out);
|
|
449
|
+
return out;
|
|
450
|
+
}
|
|
451
|
+
/**
|
|
452
|
+
* True when `path` is, or is nested under, one of `doNotTranslateKeys` — the same
|
|
453
|
+
* dot/`[i]` notation flattenStrings produces (e.g. "brand.name", "config.statusEnum").
|
|
454
|
+
* Protecting a path that points at an object or array protects every string nested
|
|
455
|
+
* under it too, which is what makes an enum array ("statusEnum": ["ACTIVE", ...])
|
|
456
|
+
* protectable without enumerating each value as a doNotTranslate term.
|
|
457
|
+
*/
|
|
458
|
+
export function keyPathProtect(doNotTranslateKeys) {
|
|
459
|
+
const keys = (doNotTranslateKeys ?? []).filter(Boolean);
|
|
460
|
+
if (!keys.length)
|
|
461
|
+
return undefined;
|
|
462
|
+
return (path) => keys.some((k) => path === k || path.startsWith(`${k}.`) || path.startsWith(`${k}[`));
|
|
463
|
+
}
|
|
464
|
+
export function pendingStrings(source, existing, overwrite, isProtected) {
|
|
465
|
+
const out = [];
|
|
466
|
+
const visit = (s, e, path) => {
|
|
467
|
+
if (typeof s === "string") {
|
|
468
|
+
if (!isProtected?.(path) && needsTranslation(e, overwrite))
|
|
469
|
+
out.push(s);
|
|
470
|
+
}
|
|
471
|
+
else if (Array.isArray(s)) {
|
|
472
|
+
s.forEach((v, i) => visit(v, Array.isArray(e) ? e[i] : undefined, `${path}[${i}]`));
|
|
473
|
+
}
|
|
474
|
+
else if (isObj(s)) {
|
|
475
|
+
for (const k of Object.keys(s))
|
|
476
|
+
visit(s[k], isObj(e) ? e[k] : undefined, path ? `${path}.${k}` : k);
|
|
477
|
+
}
|
|
478
|
+
};
|
|
479
|
+
visit(source, existing, "");
|
|
405
480
|
return out;
|
|
406
481
|
}
|
|
407
482
|
/**
|
|
408
483
|
* Builds the target document: source structure and key order, existing
|
|
409
484
|
* translations kept (unless overwrite), new strings from `translate`, and keys
|
|
410
|
-
* that only exist in the target preserved at the end.
|
|
485
|
+
* that only exist in the target preserved at the end. A string at a protected
|
|
486
|
+
* path (see keyPathProtect) is always copied from the source verbatim.
|
|
411
487
|
*/
|
|
412
|
-
export function buildTarget(source, existing, translate, overwrite) {
|
|
413
|
-
|
|
414
|
-
|
|
415
|
-
|
|
416
|
-
|
|
417
|
-
|
|
418
|
-
}
|
|
419
|
-
if (isObj(source)) {
|
|
420
|
-
const out = newObj();
|
|
421
|
-
for (const k of Object.keys(source)) {
|
|
422
|
-
out[k] = buildTarget(source[k], isObj(existing) ? existing[k] : undefined, translate, overwrite);
|
|
488
|
+
export function buildTarget(source, existing, translate, overwrite, isProtected) {
|
|
489
|
+
const visit = (s, e, path) => {
|
|
490
|
+
if (typeof s === "string") {
|
|
491
|
+
if (isProtected?.(path))
|
|
492
|
+
return s;
|
|
493
|
+
return needsTranslation(e, overwrite) ? translate(s) : e;
|
|
423
494
|
}
|
|
424
|
-
if (
|
|
425
|
-
|
|
426
|
-
|
|
427
|
-
|
|
428
|
-
|
|
429
|
-
|
|
430
|
-
|
|
495
|
+
if (Array.isArray(s))
|
|
496
|
+
return s.map((v, i) => visit(v, Array.isArray(e) ? e[i] : undefined, `${path}[${i}]`));
|
|
497
|
+
if (isObj(s)) {
|
|
498
|
+
const out = newObj();
|
|
499
|
+
for (const k of Object.keys(s))
|
|
500
|
+
out[k] = visit(s[k], isObj(e) ? e[k] : undefined, path ? `${path}.${k}` : k);
|
|
501
|
+
if (isObj(e))
|
|
502
|
+
for (const k of Object.keys(e))
|
|
503
|
+
if (!(k in out))
|
|
504
|
+
out[k] = e[k];
|
|
505
|
+
return out;
|
|
506
|
+
}
|
|
507
|
+
return s;
|
|
508
|
+
};
|
|
509
|
+
return visit(source, existing, "");
|
|
431
510
|
}
|
|
432
511
|
const CATEGORIES = ["zero", "one", "two", "few", "many", "other"];
|
|
433
512
|
const PLURAL_KEY = /^(.+?)(_ordinal)?_(zero|one|two|few|many|other)$/;
|
|
513
|
+
/**
|
|
514
|
+
* Source strings at a cardinal i18next "_one" plural key ("itemCount_one": "{{count}}
|
|
515
|
+
* item"). These are grammatically singular in English, but the model sometimes infers
|
|
516
|
+
* the TARGET's plural form anyway because {{count}} reads as an opaque marker rather
|
|
517
|
+
* than a concrete number — the engine uses this set to retranslate with a real "1" in
|
|
518
|
+
* place of {{count}} and recover the correct grammatical number (see engine.ts).
|
|
519
|
+
*/
|
|
520
|
+
export function pluralOneStrings(data, out = new Set()) {
|
|
521
|
+
const visit = (v) => {
|
|
522
|
+
if (Array.isArray(v)) {
|
|
523
|
+
v.forEach(visit);
|
|
524
|
+
return;
|
|
525
|
+
}
|
|
526
|
+
if (!isObj(v))
|
|
527
|
+
return;
|
|
528
|
+
for (const k of Object.keys(v)) {
|
|
529
|
+
const m = PLURAL_KEY.exec(k);
|
|
530
|
+
if (m && !m[2] && m[3] === "one" && typeof v[k] === "string")
|
|
531
|
+
out.add(v[k]);
|
|
532
|
+
visit(v[k]);
|
|
533
|
+
}
|
|
534
|
+
};
|
|
535
|
+
visit(data);
|
|
536
|
+
return out;
|
|
537
|
+
}
|
|
434
538
|
/**
|
|
435
539
|
* i18next stores plurals as `key_one` / `key_other`. Languages such as Arabic,
|
|
436
540
|
* Russian or Polish need more forms (`_few`, `_many` ...) and i18next does not fall
|