locale-ai 0.6.16 → 0.6.18
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +4 -3
- package/dist/cli.js +6 -3
- package/dist/locales.js +279 -11
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -28,15 +28,16 @@ Download 577 MB? (Y/n)
|
|
|
28
28
|
|
|
29
29
|
## What it handles
|
|
30
30
|
|
|
31
|
-
- **Layouts**: `locales/en/*.json` (nested folders too) → `locales/fr/...`, or `messages/en.json` → `messages/fr.json`. `.json`, `.yaml`, `.yml`, `.po`/`.pot` (gettext), `.arb` (Flutter)
|
|
31
|
+
- **Layouts**: `locales/en/*.json` (nested folders too) → `locales/fr/...`, or `messages/en.json` → `messages/fr.json`. `.json`, `.yaml`, `.yml`, `.po`/`.pot` (gettext), `.arb` (Flutter), `.strings` (Apple) and `.properties` (Java) are all supported, mixed freely in the same project. Common folders (`locales`, `src/locales`, `public/locales`, `src/i18n`, `messages`, `lang`, `config/locales` (Rails), ...) are found automatically, or pass `--input`. Framework-specific naming is also recognized: `flutter gen-l10n`'s `app_en.arb`/`app_fr.arb` filename prefix, Apple's `en.lproj`/`fr.lproj` directory-suffix convention (`--input en.lproj` or just the project root both work), and Java `ResourceBundle`'s `messages_en.properties`/`messages_fr.properties` filename prefix (region-qualified as `messages_pt_BR.properties`, with an underscore — Java's own convention, not this project's usual BCP-47 hyphen), on both the source and the generated target side.
|
|
32
32
|
- **Android**: `res/values/strings.xml` → `res/values-fr/strings.xml` (or `values-pt-rBR` for a region). `<string>`, `<string-array>` and `<plurals>` are all translated; `translatable="false"` strings and `<!-- ... -->` comments are kept as-is. Resource-qualifier scripts/variants (`values-b+sr+Latn`) aren't supported — only a plain language or language-region qualifier.
|
|
33
|
+
- **Apple `.xcstrings`** (Xcode 15+'s String Catalog, replacing `.strings`): every locale lives inside this one file, so point `--input` straight at it (e.g. `--input Localizable.xcstrings`; it isn't auto-detected, since a project can have several). `shouldTranslate: false` entries, and Xcode's per-device variants (`variations.iphone`/`.ipad`) or multi-variable strings (`substitutions`), are left untouched rather than mistranslated — only a single cardinal plural (`variations.plural`) is supported. A key with no explicit source-language text falls back to the key itself, matching Xcode's own convention.
|
|
33
34
|
- **Placeholders are never translated or lost**: `{name}`, `{{count}}`, `%s`, `%1$s`, `%(name)s`, `%{name}`, `${name}`, `:attribute`, `$t(key)`, `@:key`, `<b>…</b>`, `<0>…</0>`, ` `, URLs, e-mails.
|
|
34
35
|
- **ICU MessageFormat**: `{count, plural, one {# file} other {# files}}`, `select`, nesting. Only the text is translated.
|
|
35
|
-
- **i18next plurals**: adds the forms the target language needs (`_few`, `_many` for Arabic, Russian, Polish...), so the app doesn't fall back to the English string — each added form copies the `_other` translation verbatim rather than inflecting it, so review and adjust the wording of `_zero`/`_two`/`_few`/`_many` by hand for languages where those forms use a different noun/adjective form. The `_one` form is translated with its `{{count}}` temporarily replaced by a literal "1", since the model otherwise sometimes picks the target's plural noun form despite the English source being grammatically singular. Rails-style nested plurals (`item:` with `one:`/`other:` children instead of `item_one`/`item_other`) get the same missing-forms treatment — Android's `<plurals>`/`<item quantity="...">` groups are the same shape and get it too.
|
|
36
|
+
- **i18next plurals**: adds the forms the target language needs (`_few`, `_many` for Arabic, Russian, Polish...), so the app doesn't fall back to the English string — each added form copies the `_other` translation verbatim rather than inflecting it, so review and adjust the wording of `_zero`/`_two`/`_few`/`_many` by hand for languages where those forms use a different noun/adjective form. The `_one` form is translated with its `{{count}}` temporarily replaced by a literal "1", since the model otherwise sometimes picks the target's plural noun form despite the English source being grammatically singular. Rails-style nested plurals (`item:` with `one:`/`other:` children instead of `item_one`/`item_other`) get the same missing-forms treatment — Android's `<plurals>`/`<item quantity="...">` groups and `.xcstrings`' `variations.plural` are the same shape and get it too.
|
|
36
37
|
- **Incremental, and edit-aware**: only missing or changed keys are translated; untouched translations are kept, so re-runs are no-ops. A small `locale-ai.lock.json` (commit it, like a package lock) remembers a hash of each source string and of the output last written for it per target locale — so editing a source string gets it re-translated automatically (it isn't enough for the target to already hold *some* text), while editing a target string by hand is detected and reported for review instead of being silently overwritten on the next run. `--overwrite` bypasses the lock and re-translates everything.
|
|
37
38
|
- **Structure**: key order, numbers, booleans, arrays, indentation and trailing newline are preserved.
|
|
38
39
|
|
|
39
|
-
**Format notes**: for `.yaml`/`.yml`, `#` comments (a translator note, an inline `# ...` after a value) survive a translation run — the target file is edited in place rather than rebuilt from scratch, so an existing comment's own value is updated without disturbing the comment, and a brand-new key is appended with no comment of its own. A newly added plural-category key (see above) is likewise appended rather than inserted between existing ones, so its position in the file won't always match the canonical `zero`/`one`/`two`/`few`/`many`/`other` order. For `.po`/`.pot`, `msgid` is the stable key (never changed) and `msgstr` is the translated value, as gettext expects — the English source's `msgstr` holds the source text when it's filled in (most generators do), and falls back to `msgid` itself when it's empty, which is normal for a `.pot` template or a project that just keeps the English text in `msgid`. That fallback only ever applies to the source file: an empty `msgstr` in an *existing target* still means "not translated yet", never "same as msgid". `msgid_plural`/`msgstr[n]` plural entries are translated too (the singular and plural source forms), then expanded to however many `msgstr[n]` the target language's plural rules need (6 for Arabic, 1 for Japanese, ...) — not full grammatical inflection for in-between categories (`few`/`many`/...), which get the plural-sourced translation verbatim, the same simplification `addPluralForms` makes for i18next. `Plural-Forms` in the header is computed for the target language, not copied from the source; an existing target file's header, and any entry's own comments (a translator's note, a hand-added `#, fuzzy`), are otherwise left exactly as last edited instead of being regenerated from the source on every rerun. `--fuzzy` flags every entry a run actually (re)translated with gettext's standard `#, fuzzy` "needs review" marker, merging onto an existing `#,` line rather than adding a second one. A `.pot` source always writes a `.po` target, never a `.pot`. For `.arb`, `@key` metadata and `@@locale` are never translated; `@@locale` is set to the target locale on output. For `.strings`, a file saved as UTF-16 (Xcode's traditional default) is read like a UTF-8 one, an unquoted key (`title = "Settings";`, without quotes around `title`) is read the same as a quoted one, and `\U00e9`-style escapes are decoded — output is always written as UTF-8 with quoted keys, regardless of how the source was saved. For Android `strings.xml`, `translatable="false"` strings and each element's own `<!-- ... -->` comment are kept on first write, but — unlike YAML/`.po`/`.strings` — a rerun regenerates the file from scratch rather than editing an existing target in place, so a hand-edited comment in an already-translated file won't survive a rerun.
|
|
40
|
+
**Format notes**: for `.yaml`/`.yml`, `#` comments (a translator note, an inline `# ...` after a value) survive a translation run — the target file is edited in place rather than rebuilt from scratch, so an existing comment's own value is updated without disturbing the comment, and a brand-new key is appended with no comment of its own. A newly added plural-category key (see above) is likewise appended rather than inserted between existing ones, so its position in the file won't always match the canonical `zero`/`one`/`two`/`few`/`many`/`other` order. For `.po`/`.pot`, `msgid` is the stable key (never changed) and `msgstr` is the translated value, as gettext expects — the English source's `msgstr` holds the source text when it's filled in (most generators do), and falls back to `msgid` itself when it's empty, which is normal for a `.pot` template or a project that just keeps the English text in `msgid`. That fallback only ever applies to the source file: an empty `msgstr` in an *existing target* still means "not translated yet", never "same as msgid". `msgid_plural`/`msgstr[n]` plural entries are translated too (the singular and plural source forms), then expanded to however many `msgstr[n]` the target language's plural rules need (6 for Arabic, 1 for Japanese, ...) — not full grammatical inflection for in-between categories (`few`/`many`/...), which get the plural-sourced translation verbatim, the same simplification `addPluralForms` makes for i18next. `Plural-Forms` in the header is computed for the target language, not copied from the source; an existing target file's header, and any entry's own comments (a translator's note, a hand-added `#, fuzzy`), are otherwise left exactly as last edited instead of being regenerated from the source on every rerun. `--fuzzy` flags every entry a run actually (re)translated with gettext's standard `#, fuzzy` "needs review" marker, merging onto an existing `#,` line rather than adding a second one. A `.pot` source always writes a `.po` target, never a `.pot`. For `.arb`, `@key` metadata and `@@locale` are never translated; `@@locale` is set to the target locale on output. For `.strings`, a file saved as UTF-16 (Xcode's traditional default) is read like a UTF-8 one, an unquoted key (`title = "Settings";`, without quotes around `title`) is read the same as a quoted one, and `\U00e9`-style escapes are decoded — output is always written as UTF-8 with quoted keys, regardless of how the source was saved. For Android `strings.xml`, `translatable="false"` strings and each element's own `<!-- ... -->` comment are kept on first write, but — unlike YAML/`.po`/`.strings` — a rerun regenerates the file from scratch rather than editing an existing target in place, so a hand-edited comment in an already-translated file won't survive a rerun. For `.xcstrings`, every target locale is written into the same one catalog file (each locale's run re-reads it first, so several locales share it safely in one invocation); an entry's `state` (e.g. a translator's `needs_review` flag) is only touched when the value it's attached to actually changes, so hand review survives a rerun. For `.properties`, `#`/`!` comments and blank lines survive a rerun exactly like YAML's — the target file is edited in place rather than rebuilt from scratch; a `\`-continued multi-line value is read correctly but gets collapsed onto one line if it's ever actually rewritten (an untouched one keeps its original layout), and a line with no unescaped `=`/`:` at all is left completely alone rather than guessed at. `.properties` has no plural convention of its own, so ICU `{count, plural, ...}` inside a value is handled the same generic way it is everywhere else.
|
|
40
41
|
|
|
41
42
|
## Why locale-ai?
|
|
42
43
|
|
package/dist/cli.js
CHANGED
|
@@ -34,7 +34,8 @@ Translate options:
|
|
|
34
34
|
--from <locale> Source locale (default: en)
|
|
35
35
|
--to <locales> Target locale(s), comma separated: fr,es,pt-BR
|
|
36
36
|
--input <dir> Folder containing <from>/ or <from>.json/.yaml/.yml/.po/.pot/.arb/.strings,
|
|
37
|
-
|
|
37
|
+
an Android res/ folder (values/strings.xml) (auto-detected),
|
|
38
|
+
or a path straight to a .xcstrings catalog (not auto-detected)
|
|
38
39
|
--overwrite Re-translate strings that already have a translation
|
|
39
40
|
--dry-run Show what would be translated without downloading or writing
|
|
40
41
|
--fast Greedy decoding: ~2x faster, slightly lower quality
|
|
@@ -173,8 +174,10 @@ async function cmdTranslate(argv) {
|
|
|
173
174
|
const sources = sourceFiles(layout, from.tag).map((f) => ({ file: f, doc: readLocaleDoc(f, from.tag, true, values.fuzzy) }));
|
|
174
175
|
const singularHints = new Set(sources.flatMap(({ doc }) => [...pluralOneStrings(doc.data)]));
|
|
175
176
|
log(`locale-ai ${VERSION}`);
|
|
176
|
-
log(
|
|
177
|
-
|
|
177
|
+
log(layout.kind === "xcstrings"
|
|
178
|
+
? `Found ${from.tag} strings in ${rel(layout.file)}`
|
|
179
|
+
: `Found ${sources.length} ${from.tag} file${sources.length === 1 ? "" : "s"} in ${rel(layout.root)}/` +
|
|
180
|
+
(layout.kind === "dir" ? `${from.tag}${layout.dirSuffix ?? ""}/` : ""));
|
|
178
181
|
let failures = 0;
|
|
179
182
|
const jobs = [];
|
|
180
183
|
for (const locale of targets) {
|
package/dist/locales.js
CHANGED
|
@@ -2,7 +2,14 @@ import fs from "node:fs";
|
|
|
2
2
|
import path from "node:path";
|
|
3
3
|
import { Document, isMap, isScalar, parseDocument } from "yaml";
|
|
4
4
|
import { parseLocale } from "./lang.js";
|
|
5
|
-
const EXTENSIONS = [".json", ".yaml", ".yml", ".po", ".pot", ".arb", ".strings"];
|
|
5
|
+
const EXTENSIONS = [".json", ".yaml", ".yml", ".po", ".pot", ".arb", ".strings", ".properties"];
|
|
6
|
+
/**
|
|
7
|
+
* Java's `ResourceBundle` convention names a region-qualified `.properties` file
|
|
8
|
+
* "fr_FR", not this project's BCP-47 "fr-FR" everywhere else — so the tag is
|
|
9
|
+
* underscore-joined only at the filename boundary, for this one extension.
|
|
10
|
+
*/
|
|
11
|
+
const propertiesTag = (ext, tag) => (ext === ".properties" ? tag.replace("-", "_") : tag);
|
|
12
|
+
const tagFromPropertiesQualifier = (ext, qualifier) => (ext === ".properties" ? qualifier.replace("_", "-") : qualifier);
|
|
6
13
|
const CANDIDATES = [
|
|
7
14
|
"locales",
|
|
8
15
|
"src/locales",
|
|
@@ -45,11 +52,12 @@ const isDir = (p) => fs.existsSync(p) && fs.statSync(p).isDirectory();
|
|
|
45
52
|
const isFile = (p) => fs.existsSync(p) && fs.statSync(p).isFile();
|
|
46
53
|
/** The source-locale file's extension, if `<root>/<from><ext>` exists for one of EXTENSIONS. */
|
|
47
54
|
function fileExtAt(root, from) {
|
|
48
|
-
return EXTENSIONS.find((ext) => isFile(path.join(root, `${from}${ext}`)));
|
|
55
|
+
return EXTENSIONS.find((ext) => isFile(path.join(root, `${propertiesTag(ext, from)}${ext}`)));
|
|
49
56
|
}
|
|
50
57
|
/**
|
|
51
58
|
* A prefixed source-locale file, if `<root>/<prefix>_<from><ext>` exists for one of
|
|
52
|
-
* EXTENSIONS — the convention `flutter gen-l10n` uses by default ("app_en.arb")
|
|
59
|
+
* EXTENSIONS — the convention `flutter gen-l10n` uses by default ("app_en.arb"), and
|
|
60
|
+
* also how a Java `ResourceBundle` basename ("messages_en.properties") is found.
|
|
53
61
|
*/
|
|
54
62
|
function prefixedFileAt(root, from) {
|
|
55
63
|
if (!isDir(root))
|
|
@@ -58,7 +66,7 @@ function prefixedFileAt(root, from) {
|
|
|
58
66
|
if (!isFile(path.join(root, entry)))
|
|
59
67
|
continue;
|
|
60
68
|
for (const ext of EXTENSIONS) {
|
|
61
|
-
const suffix = `_${from}${ext}`;
|
|
69
|
+
const suffix = `_${propertiesTag(ext, from)}${ext}`;
|
|
62
70
|
if (entry.endsWith(suffix) && entry.length > suffix.length) {
|
|
63
71
|
return { ext, prefix: entry.slice(0, -suffix.length) };
|
|
64
72
|
}
|
|
@@ -89,7 +97,7 @@ function layoutAt(root, from) {
|
|
|
89
97
|
return undefined;
|
|
90
98
|
}
|
|
91
99
|
const dirNameFor = (layout, tag) => `${tag}${layout.dirSuffix ?? ""}`;
|
|
92
|
-
const fileNameFor = (layout, tag, ext) => `${layout.prefix ? `${layout.prefix}_` : ""}${tag}${ext}`;
|
|
100
|
+
const fileNameFor = (layout, tag, ext) => `${layout.prefix ? `${layout.prefix}_` : ""}${propertiesTag(ext, tag)}${ext}`;
|
|
93
101
|
export function detectLayout(cwd, from, input) {
|
|
94
102
|
if (input) {
|
|
95
103
|
const abs = path.resolve(cwd, input);
|
|
@@ -102,13 +110,30 @@ export function detectLayout(cwd, from, input) {
|
|
|
102
110
|
if (path.basename(abs) === "values" && isFile(path.join(abs, "strings.xml"))) {
|
|
103
111
|
return { kind: "android", root: path.dirname(abs) };
|
|
104
112
|
}
|
|
113
|
+
if (abs.endsWith(".xcstrings") && isFile(abs)) {
|
|
114
|
+
let raw;
|
|
115
|
+
try {
|
|
116
|
+
raw = JSON.parse(readText(abs));
|
|
117
|
+
}
|
|
118
|
+
catch (e) {
|
|
119
|
+
throw new Error(`Invalid .xcstrings (JSON) in ${input}: ${e.message}`);
|
|
120
|
+
}
|
|
121
|
+
const sourceLanguage = raw?.sourceLanguage;
|
|
122
|
+
if (typeof sourceLanguage === "string" && sourceLanguage !== from) {
|
|
123
|
+
throw new Error(`${input} has sourceLanguage "${sourceLanguage}", not "${from}" — pass --from ${sourceLanguage}.`);
|
|
124
|
+
}
|
|
125
|
+
return { kind: "xcstrings", root: path.dirname(abs), file: abs };
|
|
126
|
+
}
|
|
105
127
|
const base = path.basename(abs);
|
|
106
|
-
const directExt = EXTENSIONS.find((ext) => base === `${from}${ext}`);
|
|
128
|
+
const directExt = EXTENSIONS.find((ext) => base === `${propertiesTag(ext, from)}${ext}`);
|
|
107
129
|
if (directExt && isFile(abs))
|
|
108
130
|
return { kind: "file", root: path.dirname(abs), ext: directExt };
|
|
109
|
-
const prefixedExt = EXTENSIONS.find((ext) =>
|
|
131
|
+
const prefixedExt = EXTENSIONS.find((ext) => {
|
|
132
|
+
const suffix = `_${propertiesTag(ext, from)}${ext}`;
|
|
133
|
+
return base.endsWith(suffix) && base.length > suffix.length;
|
|
134
|
+
});
|
|
110
135
|
if (prefixedExt && isFile(abs)) {
|
|
111
|
-
const suffix = `_${from}${prefixedExt}`;
|
|
136
|
+
const suffix = `_${propertiesTag(prefixedExt, from)}${prefixedExt}`;
|
|
112
137
|
return { kind: "file", root: path.dirname(abs), ext: prefixedExt, prefix: base.slice(0, -suffix.length) };
|
|
113
138
|
}
|
|
114
139
|
const l = layoutAt(abs, from);
|
|
@@ -139,6 +164,8 @@ function listLocaleFiles(dir) {
|
|
|
139
164
|
export function sourceFiles(layout, from) {
|
|
140
165
|
if (layout.kind === "android")
|
|
141
166
|
return [path.join(layout.root, "values", "strings.xml")];
|
|
167
|
+
if (layout.kind === "xcstrings")
|
|
168
|
+
return [layout.file];
|
|
142
169
|
return layout.kind === "dir"
|
|
143
170
|
? listLocaleFiles(path.join(layout.root, dirNameFor(layout, from)))
|
|
144
171
|
: [path.join(layout.root, fileNameFor(layout, from, layout.ext))];
|
|
@@ -148,6 +175,8 @@ const outputExt = (ext) => (ext === ".pot" ? ".po" : ext);
|
|
|
148
175
|
export function targetFile(layout, from, to, src) {
|
|
149
176
|
if (layout.kind === "android")
|
|
150
177
|
return path.join(layout.root, `values-${androidQualifier(to)}`, "strings.xml");
|
|
178
|
+
if (layout.kind === "xcstrings")
|
|
179
|
+
return layout.file;
|
|
151
180
|
if (layout.kind === "file")
|
|
152
181
|
return path.join(layout.root, fileNameFor(layout, to, outputExt(layout.ext)));
|
|
153
182
|
const rel = path.relative(path.join(layout.root, dirNameFor(layout, from)), src);
|
|
@@ -166,6 +195,25 @@ function looksLikeLocale(tag) {
|
|
|
166
195
|
}
|
|
167
196
|
/** Locale tags already translated alongside `from` in this layout (for `locale-ai check`). */
|
|
168
197
|
export function existingTargets(layout, from) {
|
|
198
|
+
if (layout.kind === "xcstrings") {
|
|
199
|
+
let raw;
|
|
200
|
+
try {
|
|
201
|
+
raw = JSON.parse(readText(layout.file));
|
|
202
|
+
}
|
|
203
|
+
catch {
|
|
204
|
+
return [];
|
|
205
|
+
}
|
|
206
|
+
const strings = isObj(raw) && isObj(raw.strings) ? raw.strings : {};
|
|
207
|
+
const tags = new Set();
|
|
208
|
+
for (const entry of Object.values(strings)) {
|
|
209
|
+
if (!isObj(entry) || !isObj(entry.localizations))
|
|
210
|
+
continue;
|
|
211
|
+
for (const tag of Object.keys(entry.localizations))
|
|
212
|
+
tags.add(tag);
|
|
213
|
+
}
|
|
214
|
+
tags.delete(from);
|
|
215
|
+
return [...tags].filter(looksLikeLocale).sort();
|
|
216
|
+
}
|
|
169
217
|
if (layout.kind === "android") {
|
|
170
218
|
return fs
|
|
171
219
|
.readdirSync(layout.root, { withFileTypes: true })
|
|
@@ -179,9 +227,9 @@ export function existingTargets(layout, from) {
|
|
|
179
227
|
const prefix = layout.prefix ? `${layout.prefix}_` : "";
|
|
180
228
|
return fs
|
|
181
229
|
.readdirSync(layout.root)
|
|
182
|
-
.filter((f) => f.startsWith(prefix) && f.endsWith(ext)
|
|
183
|
-
.map((f) => f.slice(prefix.length, -ext.length))
|
|
184
|
-
.filter(looksLikeLocale)
|
|
230
|
+
.filter((f) => f.startsWith(prefix) && f.endsWith(ext))
|
|
231
|
+
.map((f) => tagFromPropertiesQualifier(ext, f.slice(prefix.length, -ext.length)))
|
|
232
|
+
.filter((tag) => tag !== from && looksLikeLocale(tag))
|
|
185
233
|
.sort();
|
|
186
234
|
}
|
|
187
235
|
const suffix = layout.dirSuffix ?? "";
|
|
@@ -330,6 +378,109 @@ function readStringsDoc(file, text) {
|
|
|
330
378
|
};
|
|
331
379
|
return { data, serialize };
|
|
332
380
|
}
|
|
381
|
+
/** Java `.properties`: `\n`, `\t`, `\r`, `\f`, `\\` and `\uXXXX` escapes, plus an escaped separator/leading-space. */
|
|
382
|
+
function unescapeProperties(s) {
|
|
383
|
+
return s.replace(/\\(u[0-9a-fA-F]{4}|.)/gs, (_m, esc) => {
|
|
384
|
+
if (esc[0] === "u" && esc.length === 5)
|
|
385
|
+
return String.fromCharCode(parseInt(esc.slice(1), 16));
|
|
386
|
+
return { n: "\n", t: "\t", r: "\r", f: "\f" }[esc] ?? esc;
|
|
387
|
+
});
|
|
388
|
+
}
|
|
389
|
+
function escapeProperties(s) {
|
|
390
|
+
let out = s.replace(/\\/g, "\\\\").replace(/\n/g, "\\n").replace(/\t/g, "\\t").replace(/\r/g, "\\r");
|
|
391
|
+
if (out[0] === " ")
|
|
392
|
+
out = `\\${out}`;
|
|
393
|
+
return out;
|
|
394
|
+
}
|
|
395
|
+
/** True when `s` ends in an odd run of backslashes — Java's line-continuation rule. */
|
|
396
|
+
const endsWithContinuation = (s) => (/\\*$/.exec(s)[0].length % 2 === 1);
|
|
397
|
+
/** Splits `key<ws>?[=:]<ws>?value` at the first unescaped `=`/`:`, skipping `\X` pairs. Undefined if there's none (a key-only or malformed line). */
|
|
398
|
+
function splitPropertyLine(line) {
|
|
399
|
+
let i = 0;
|
|
400
|
+
while (i < line.length) {
|
|
401
|
+
const c = line[i];
|
|
402
|
+
if (c === "\\") {
|
|
403
|
+
i += 2;
|
|
404
|
+
continue;
|
|
405
|
+
}
|
|
406
|
+
if (c === "=" || c === ":") {
|
|
407
|
+
const key = unescapeProperties(line.slice(0, i).trimEnd());
|
|
408
|
+
let v = i + 1;
|
|
409
|
+
while (v < line.length && (line[v] === " " || line[v] === "\t"))
|
|
410
|
+
v++;
|
|
411
|
+
return { key, valueStart: v };
|
|
412
|
+
}
|
|
413
|
+
i++;
|
|
414
|
+
}
|
|
415
|
+
return undefined;
|
|
416
|
+
}
|
|
417
|
+
/** Parses `.properties` text into passthrough lines (comments, blank lines, unparsable lines) and key/value entries. */
|
|
418
|
+
function parsePropertiesBlocks(text) {
|
|
419
|
+
const physical = text.split(/\r?\n/);
|
|
420
|
+
const blocks = [];
|
|
421
|
+
const data = newObj();
|
|
422
|
+
let i = 0;
|
|
423
|
+
while (i < physical.length) {
|
|
424
|
+
const line = physical[i];
|
|
425
|
+
const trimmedStart = line.replace(/^[ \t\f]+/, "");
|
|
426
|
+
if (trimmedStart === "" || trimmedStart[0] === "#" || trimmedStart[0] === "!") {
|
|
427
|
+
blocks.push({ text: line });
|
|
428
|
+
i++;
|
|
429
|
+
continue;
|
|
430
|
+
}
|
|
431
|
+
const split = splitPropertyLine(line);
|
|
432
|
+
if (!split) {
|
|
433
|
+
blocks.push({ text: line });
|
|
434
|
+
i++;
|
|
435
|
+
continue;
|
|
436
|
+
}
|
|
437
|
+
let valueRaw = line.slice(split.valueStart);
|
|
438
|
+
while (endsWithContinuation(valueRaw) && i + 1 < physical.length) {
|
|
439
|
+
valueRaw = valueRaw.slice(0, -1) + physical[++i].replace(/^[ \t\f]+/, "");
|
|
440
|
+
}
|
|
441
|
+
data[split.key] = unescapeProperties(valueRaw);
|
|
442
|
+
blocks.push({ key: split.key, prefix: line.slice(0, split.valueStart) });
|
|
443
|
+
i++;
|
|
444
|
+
}
|
|
445
|
+
return { blocks, data };
|
|
446
|
+
}
|
|
447
|
+
/**
|
|
448
|
+
* Java `ResourceBundle` `.properties`: a flat `key=value` (or `key: value`) list, one
|
|
449
|
+
* per line, `#`/`!` line comments, `\`-continued multi-line values, and `\uXXXX`/`\n`/
|
|
450
|
+
* `\t` escapes. There's no plural or array convention of its own — a value containing
|
|
451
|
+
* ICU `{count, plural, ...}` syntax is still handled generically by the ICU extractor
|
|
452
|
+
* elsewhere, same as any other format.
|
|
453
|
+
*
|
|
454
|
+
* Unlike Android/ARB/`.xcstrings` (which regenerate the whole file from the source's
|
|
455
|
+
* own structure every run), this edits an existing target file's lines in place, the
|
|
456
|
+
* same as YAML/`.po`: untouched entries, comments and blank lines keep their exact
|
|
457
|
+
* text and position, and a value that didn't change is never rewritten. A multi-line
|
|
458
|
+
* (`\`-continued) value is always collapsed to one line when it's rewritten, though —
|
|
459
|
+
* only an *unchanged* continued value keeps its original multi-line layout. A line
|
|
460
|
+
* with no unescaped `=`/`:` at all (so not a recognizable `key=value` pair) is passed
|
|
461
|
+
* through untouched rather than guessed at.
|
|
462
|
+
*/
|
|
463
|
+
function readPropertiesDoc(file, text) {
|
|
464
|
+
const { blocks, data } = parsePropertiesBlocks(text);
|
|
465
|
+
const serialize = (d, _targetTag, existingText) => {
|
|
466
|
+
const obj = isObj(d) ? d : newObj();
|
|
467
|
+
const baseBlocks = existingText !== undefined ? parsePropertiesBlocks(existingText).blocks : blocks;
|
|
468
|
+
const seen = new Set();
|
|
469
|
+
const lines = baseBlocks.map((b) => {
|
|
470
|
+
if (!("key" in b))
|
|
471
|
+
return b.text;
|
|
472
|
+
seen.add(b.key);
|
|
473
|
+
return `${b.prefix}${escapeProperties(typeof obj[b.key] === "string" ? obj[b.key] : "")}`;
|
|
474
|
+
});
|
|
475
|
+
for (const k of Object.keys(obj)) {
|
|
476
|
+
if (seen.has(k) || typeof obj[k] !== "string")
|
|
477
|
+
continue;
|
|
478
|
+
lines.push(`${k}=${escapeProperties(obj[k])}`);
|
|
479
|
+
}
|
|
480
|
+
return lines.join("\n");
|
|
481
|
+
};
|
|
482
|
+
return { data, serialize };
|
|
483
|
+
}
|
|
333
484
|
/** Android's `&entity;` plus its own backslash escapes (`\'`, `\"`, `\n`, `\t`) and `<![CDATA[...]]>`. */
|
|
334
485
|
function unescapeAndroidXml(s) {
|
|
335
486
|
return s
|
|
@@ -478,6 +629,119 @@ function readArbDoc(file, text) {
|
|
|
478
629
|
};
|
|
479
630
|
return { data, serialize };
|
|
480
631
|
}
|
|
632
|
+
/** True when `entry` uses a per-device variant (`variations.iphone`/`.ipad`/...) for `tag`, not just a plural. */
|
|
633
|
+
function xcstringsHasDeviceVariant(entry) {
|
|
634
|
+
const locs = entry.localizations;
|
|
635
|
+
if (!isObj(locs))
|
|
636
|
+
return false;
|
|
637
|
+
return Object.values(locs).some((loc) => isObj(loc) && isObj(loc.variations) && Object.keys(loc.variations).some((k) => k !== "plural"));
|
|
638
|
+
}
|
|
639
|
+
/** `shouldTranslate: false`, multi-variable `substitutions`, or any per-device variant — left untouched. */
|
|
640
|
+
function xcstringsSkip(entry) {
|
|
641
|
+
return !isObj(entry) || entry.shouldTranslate === false || entry.substitutions !== undefined || xcstringsHasDeviceVariant(entry);
|
|
642
|
+
}
|
|
643
|
+
/** An entry's value for one locale: a plain string, a `{zero,one,...}` plural object, or undefined if untranslated. */
|
|
644
|
+
function xcstringsValue(entry, tag) {
|
|
645
|
+
const loc = entry.localizations?.[tag];
|
|
646
|
+
if (!loc)
|
|
647
|
+
return undefined;
|
|
648
|
+
const plural = loc.variations?.plural;
|
|
649
|
+
if (plural) {
|
|
650
|
+
const out = newObj();
|
|
651
|
+
for (const cat of CATEGORIES) {
|
|
652
|
+
const v = plural[cat]?.stringUnit?.value;
|
|
653
|
+
if (typeof v === "string")
|
|
654
|
+
out[cat] = v;
|
|
655
|
+
}
|
|
656
|
+
return Object.keys(out).length ? out : undefined;
|
|
657
|
+
}
|
|
658
|
+
const v = loc.stringUnit?.value;
|
|
659
|
+
return typeof v === "string" ? v : undefined;
|
|
660
|
+
}
|
|
661
|
+
/**
|
|
662
|
+
* Apple's `.xcstrings` String Catalog (Xcode 15+'s default, replacing `.strings`):
|
|
663
|
+
* unlike every other format here, ALL locales — source included — live in ONE JSON
|
|
664
|
+
* file, keyed by the string's own identifier under `strings`. Each entry's
|
|
665
|
+
* `localizations.<tag>` holds either a plain `stringUnit` or, for a plural string, a
|
|
666
|
+
* `variations.plural` object whose zero/one/two/few/many/other categories are exactly
|
|
667
|
+
* the shape `addNestedPluralForms` already understands (see Android/Rails), so it's
|
|
668
|
+
* reused unchanged. An entry with no `localizations.<sourceLanguage>` at all falls
|
|
669
|
+
* back to its own key as the source text, matching Xcode's own convention for a
|
|
670
|
+
* string that was never given explicit source content.
|
|
671
|
+
*
|
|
672
|
+
* `shouldTranslate: false`, Xcode's per-device variants (`variations.iphone`/`.ipad`),
|
|
673
|
+
* and multi-variable strings (`substitutions`, used with `%#@name@`-style format
|
|
674
|
+
* references) are all left completely untouched rather than mistranslated — a bigger
|
|
675
|
+
* matrix than a single cardinal plural isn't supported yet.
|
|
676
|
+
*
|
|
677
|
+
* `targetFile`/`sourceFiles` point every locale at this same one file, so each
|
|
678
|
+
* locale's turn re-reads it fresh (via `existingText`) before writing — the same
|
|
679
|
+
* read-then-write-fresh sequencing the CLI already does per locale, which is what
|
|
680
|
+
* lets several locales share one file safely in a single run without clobbering each
|
|
681
|
+
* other. A key's translated value is only ever replaced, never its `state`/metadata,
|
|
682
|
+
* when the value hasn't actually changed — so a translator's own "needs_review" flag
|
|
683
|
+
* on an untouched entry survives a rerun.
|
|
684
|
+
*/
|
|
685
|
+
function readXcstringsDoc(file, text, tag, isSource) {
|
|
686
|
+
let raw;
|
|
687
|
+
try {
|
|
688
|
+
raw = JSON.parse(text);
|
|
689
|
+
}
|
|
690
|
+
catch (e) {
|
|
691
|
+
throw new Error(`Invalid .xcstrings (JSON) in ${file}: ${e.message}`);
|
|
692
|
+
}
|
|
693
|
+
const strings = isObj(raw.strings) ? raw.strings : {};
|
|
694
|
+
const data = newObj();
|
|
695
|
+
for (const key of Object.keys(strings)) {
|
|
696
|
+
const entry = strings[key];
|
|
697
|
+
if (xcstringsSkip(entry))
|
|
698
|
+
continue;
|
|
699
|
+
const v = xcstringsValue(entry, tag);
|
|
700
|
+
if (v !== undefined)
|
|
701
|
+
data[key] = v;
|
|
702
|
+
else if (isSource)
|
|
703
|
+
data[key] = key;
|
|
704
|
+
}
|
|
705
|
+
const indent = /^\{\r?\n([ \t]+)/.exec(text)?.[1] ?? 2;
|
|
706
|
+
const keepNewline = finalNewline(text);
|
|
707
|
+
const serialize = (d, targetTag, existingText) => {
|
|
708
|
+
const obj = isObj(d) ? d : newObj();
|
|
709
|
+
let out = raw;
|
|
710
|
+
if (existingText !== undefined) {
|
|
711
|
+
try {
|
|
712
|
+
out = JSON.parse(existingText);
|
|
713
|
+
}
|
|
714
|
+
catch {
|
|
715
|
+
out = raw;
|
|
716
|
+
}
|
|
717
|
+
}
|
|
718
|
+
const outStrings = isObj(out.strings) ? out.strings : (out.strings = newObj());
|
|
719
|
+
for (const key of Object.keys(obj)) {
|
|
720
|
+
const entry = outStrings[key];
|
|
721
|
+
if (xcstringsSkip(entry))
|
|
722
|
+
continue;
|
|
723
|
+
const newVal = obj[key];
|
|
724
|
+
const curVal = xcstringsValue(entry, targetTag);
|
|
725
|
+
if (JSON.stringify(curVal) === JSON.stringify(newVal))
|
|
726
|
+
continue;
|
|
727
|
+
const locs = isObj(entry.localizations) ? entry.localizations : (entry.localizations = newObj());
|
|
728
|
+
if (isObj(newVal)) {
|
|
729
|
+
const plural = newObj();
|
|
730
|
+
for (const cat of CATEGORIES) {
|
|
731
|
+
if (typeof newVal[cat] === "string") {
|
|
732
|
+
plural[cat] = { stringUnit: { state: "translated", value: newVal[cat] } };
|
|
733
|
+
}
|
|
734
|
+
}
|
|
735
|
+
locs[targetTag] = { variations: { plural } };
|
|
736
|
+
}
|
|
737
|
+
else if (typeof newVal === "string") {
|
|
738
|
+
locs[targetTag] = { stringUnit: { state: "translated", value: newVal } };
|
|
739
|
+
}
|
|
740
|
+
}
|
|
741
|
+
return withNewline(JSON.stringify(out, null, indent), keepNewline);
|
|
742
|
+
};
|
|
743
|
+
return { data, serialize };
|
|
744
|
+
}
|
|
481
745
|
const poKey = (e) => (e.msgctxt ?? "") + "\u0004" + e.msgid;
|
|
482
746
|
/**
|
|
483
747
|
* gettext's well-established Plural-Forms formulas, keyed by language. Copying the
|
|
@@ -756,8 +1020,12 @@ export function readLocaleDoc(file, tag, isSource = false, fuzzy = false) {
|
|
|
756
1020
|
return readArbDoc(file, text);
|
|
757
1021
|
if (ext === ".strings")
|
|
758
1022
|
return readStringsDoc(file, text);
|
|
1023
|
+
if (ext === ".properties")
|
|
1024
|
+
return readPropertiesDoc(file, text);
|
|
759
1025
|
if (ext === ".xml")
|
|
760
1026
|
return readAndroidXmlDoc(file, text);
|
|
1027
|
+
if (ext === ".xcstrings")
|
|
1028
|
+
return readXcstringsDoc(file, text, tag, isSource);
|
|
761
1029
|
return readJsonDoc(file, text);
|
|
762
1030
|
}
|
|
763
1031
|
const isObj = (v) => typeof v === "object" && v !== null && !Array.isArray(v);
|