locale-ai 0.6.15 → 0.6.16

package/README.md

@@ -29,13 +29,14 @@ Download 577 MB? (Y/n)
 ## What it handles
 
 - **Layouts**: `locales/en/*.json` (nested folders too) → `locales/fr/...`, or `messages/en.json` → `messages/fr.json`. `.json`, `.yaml`, `.yml`, `.po`/`.pot` (gettext), `.arb` (Flutter) and `.strings` (Apple) are all supported, mixed freely in the same project. Common folders (`locales`, `src/locales`, `public/locales`, `src/i18n`, `messages`, `lang`, `config/locales` (Rails), ...) are found automatically, or pass `--input`. Framework-specific naming is also recognized: `flutter gen-l10n`'s `app_en.arb`/`app_fr.arb` filename prefix, and Apple's `en.lproj`/`fr.lproj` directory-suffix convention (`--input en.lproj` or just the project root both work), on both the source and the generated target side.
+- **Android**: `res/values/strings.xml` → `res/values-fr/strings.xml` (or `values-pt-rBR` for a region). `<string>`, `<string-array>` and `<plurals>` are all translated; `translatable="false"` strings and `<!-- ... -->` comments are kept as-is. Resource-qualifier scripts/variants (`values-b+sr+Latn`) aren't supported — only a plain language or language-region qualifier.
 - **Placeholders are never translated or lost**: `{name}`, `{{count}}`, `%s`, `%1$s`, `%(name)s`, `%{name}`, `${name}`, `:attribute`, `$t(key)`, `@:key`, `<b>…</b>`, `<0>…</0>`, `&nbsp;`, URLs, e-mails.
 - **ICU MessageFormat**: `{count, plural, one {# file} other {# files}}`, `select`, nesting. Only the text is translated.
-- **i18next plurals**: adds the forms the target language needs (`_few`, `_many` for Arabic, Russian, Polish...), so the app doesn't fall back to the English string — each added form copies the `_other` translation verbatim rather than inflecting it, so review and adjust the wording of `_zero`/`_two`/`_few`/`_many` by hand for languages where those forms use a different noun/adjective form. The `_one` form is translated with its `{{count}}` temporarily replaced by a literal "1", since the model otherwise sometimes picks the target's plural noun form despite the English source being grammatically singular. Rails-style nested plurals (`item:` with `one:`/`other:` children instead of `item_one`/`item_other`) get the same missing-forms treatment.
+- **i18next plurals**: adds the forms the target language needs (`_few`, `_many` for Arabic, Russian, Polish...), so the app doesn't fall back to the English string — each added form copies the `_other` translation verbatim rather than inflecting it, so review and adjust the wording of `_zero`/`_two`/`_few`/`_many` by hand for languages where those forms use a different noun/adjective form. The `_one` form is translated with its `{{count}}` temporarily replaced by a literal "1", since the model otherwise sometimes picks the target's plural noun form despite the English source being grammatically singular. Rails-style nested plurals (`item:` with `one:`/`other:` children instead of `item_one`/`item_other`) get the same missing-forms treatment — Android's `<plurals>`/`<item quantity="...">` groups are the same shape and get it too.
 - **Incremental, and edit-aware**: only missing or changed keys are translated; untouched translations are kept, so re-runs are no-ops. A small `locale-ai.lock.json` (commit it, like a package lock) remembers a hash of each source string and of the output last written for it per target locale — so editing a source string gets it re-translated automatically (it isn't enough for the target to already hold *some* text), while editing a target string by hand is detected and reported for review instead of being silently overwritten on the next run. `--overwrite` bypasses the lock and re-translates everything.
 - **Structure**: key order, numbers, booleans, arrays, indentation and trailing newline are preserved.
 
-**Format notes**: for `.yaml`/`.yml`, `#` comments (a translator note, an inline `# ...` after a value) survive a translation run — the target file is edited in place rather than rebuilt from scratch, so an existing comment's own value is updated without disturbing the comment, and a brand-new key is appended with no comment of its own. A newly added plural-category key (see above) is likewise appended rather than inserted between existing ones, so its position in the file won't always match the canonical `zero`/`one`/`two`/`few`/`many`/`other` order. For `.po`/`.pot`, `msgid` is the stable key (never changed) and `msgstr` is the translated value, as gettext expects — the English source's `msgstr` holds the source text when it's filled in (most generators do), and falls back to `msgid` itself when it's empty, which is normal for a `.pot` template or a project that just keeps the English text in `msgid`. That fallback only ever applies to the source file: an empty `msgstr` in an *existing target* still means "not translated yet", never "same as msgid". `msgid_plural`/`msgstr[n]` plural entries are translated too (the singular and plural source forms), then expanded to however many `msgstr[n]` the target language's plural rules need (6 for Arabic, 1 for Japanese, ...) — not full grammatical inflection for in-between categories (`few`/`many`/...), which get the plural-sourced translation verbatim, the same simplification `addPluralForms` makes for i18next. `Plural-Forms` in the header is computed for the target language, not copied from the source; an existing target file's header, and any entry's own comments (a translator's note, a hand-added `#, fuzzy`), are otherwise left exactly as last edited instead of being regenerated from the source on every rerun. `--fuzzy` flags every entry a run actually (re)translated with gettext's standard `#, fuzzy` "needs review" marker, merging onto an existing `#,` line rather than adding a second one. A `.pot` source always writes a `.po` target, never a `.pot`. For `.arb`, `@key` metadata and `@@locale` are never translated; `@@locale` is set to the target locale on output. For `.strings`, a file saved as UTF-16 (Xcode's traditional default) is read like a UTF-8 one, an unquoted key (`title = "Settings";`, without quotes around `title`) is read the same as a quoted one, and `\U00e9`-style escapes are decoded — output is always written as UTF-8 with quoted keys, regardless of how the source was saved.
+**Format notes**: for `.yaml`/`.yml`, `#` comments (a translator note, an inline `# ...` after a value) survive a translation run — the target file is edited in place rather than rebuilt from scratch, so an existing comment's own value is updated without disturbing the comment, and a brand-new key is appended with no comment of its own. A newly added plural-category key (see above) is likewise appended rather than inserted between existing ones, so its position in the file won't always match the canonical `zero`/`one`/`two`/`few`/`many`/`other` order. For `.po`/`.pot`, `msgid` is the stable key (never changed) and `msgstr` is the translated value, as gettext expects — the English source's `msgstr` holds the source text when it's filled in (most generators do), and falls back to `msgid` itself when it's empty, which is normal for a `.pot` template or a project that just keeps the English text in `msgid`. That fallback only ever applies to the source file: an empty `msgstr` in an *existing target* still means "not translated yet", never "same as msgid". `msgid_plural`/`msgstr[n]` plural entries are translated too (the singular and plural source forms), then expanded to however many `msgstr[n]` the target language's plural rules need (6 for Arabic, 1 for Japanese, ...) — not full grammatical inflection for in-between categories (`few`/`many`/...), which get the plural-sourced translation verbatim, the same simplification `addPluralForms` makes for i18next. `Plural-Forms` in the header is computed for the target language, not copied from the source; an existing target file's header, and any entry's own comments (a translator's note, a hand-added `#, fuzzy`), are otherwise left exactly as last edited instead of being regenerated from the source on every rerun. `--fuzzy` flags every entry a run actually (re)translated with gettext's standard `#, fuzzy` "needs review" marker, merging onto an existing `#,` line rather than adding a second one. A `.pot` source always writes a `.po` target, never a `.pot`. For `.arb`, `@key` metadata and `@@locale` are never translated; `@@locale` is set to the target locale on output. For `.strings`, a file saved as UTF-16 (Xcode's traditional default) is read like a UTF-8 one, an unquoted key (`title = "Settings";`, without quotes around `title`) is read the same as a quoted one, and `\U00e9`-style escapes are decoded — output is always written as UTF-8 with quoted keys, regardless of how the source was saved. For Android `strings.xml`, `translatable="false"` strings and each element's own `<!-- ... -->` comment are kept on first write, but — unlike YAML/`.po`/`.strings` — a rerun regenerates the file from scratch rather than editing an existing target in place, so a hand-edited comment in an already-translated file won't survive a rerun.
 
 ## Why locale-ai?
 

package/dist/cli.js

@@ -33,7 +33,8 @@ Usage:
 Translate options:
   --from <locale>       Source locale (default: en)
   --to <locales>        Target locale(s), comma separated: fr,es,pt-BR
-  --input <dir>         Folder containing <from>/ or <from>.json/.yaml/.yml/.po/.pot/.arb/.strings (auto-detected)
+  --input <dir>         Folder containing <from>/ or <from>.json/.yaml/.yml/.po/.pot/.arb/.strings,
+                        or an Android res/ folder (values/strings.xml) (auto-detected)
   --overwrite           Re-translate strings that already have a translation
   --dry-run             Show what would be translated without downloading or writing
   --fast                Greedy decoding: ~2x faster, slightly lower quality

package/dist/locales.js

@@ -24,8 +24,23 @@ const CANDIDATES = [
     "app/i18n",
     "static/locales",
     "config/locales",
+    "app/src/main/res",
+    "src/main/res",
+    "res",
     ".",
 ];
+/** Android's `values-<qualifier>` folder suffix for a locale tag: "fr" -> "fr", "pt-BR" -> "pt-rBR". */
+function androidQualifier(tag) {
+    const l = parseLocale(tag);
+    if (l.script)
+        throw new Error(`Android resource qualifiers for scripts (got "${tag}") aren't supported; use a plain language or language-region tag.`);
+    return l.region ? `${l.lang}-r${l.region}` : l.lang;
+}
+/** Reverse of androidQualifier: "fr" -> "fr", "pt-rBR" -> "pt-BR". Undefined for a qualifier it can't parse (script/BCP-47 "b+..." variants). */
+function tagFromAndroidQualifier(qualifier) {
+    const m = /^([a-z]{2,3})(?:-r([A-Za-z]{2}))?$/.exec(qualifier);
+    return m ? (m[2] ? `${m[1]}-${m[2].toUpperCase()}` : m[1]) : undefined;
+}
 const isDir = (p) => fs.existsSync(p) && fs.statSync(p).isDirectory();
 const isFile = (p) => fs.existsSync(p) && fs.statSync(p).isFile();
 /** The source-locale file's extension, if `<root>/<from><ext>` exists for one of EXTENSIONS. */
@@ -51,6 +66,10 @@ function prefixedFileAt(root, from) {
     }
     return undefined;
 }
+/** Android's default (untagged) `values/strings.xml` resource folder, if present. */
+function androidLayoutAt(root) {
+    return isFile(path.join(root, "values", "strings.xml")) ? { kind: "android", root } : undefined;
+}
 function layoutAt(root, from) {
     const dir = path.join(root, from);
     if (isDir(dir) && listLocaleFiles(dir).length > 0)
@@ -58,6 +77,9 @@ function layoutAt(root, from) {
     const lprojDir = path.join(root, `${from}.lproj`);
     if (isDir(lprojDir) && listLocaleFiles(lprojDir).length > 0)
         return { kind: "dir", root, dirSuffix: ".lproj" };
+    const android = androidLayoutAt(root);
+    if (android)
+        return android;
     const ext = fileExtAt(root, from);
     if (ext)
         return { kind: "file", root, ext };
@@ -77,6 +99,9 @@ export function detectLayout(cwd, from, input) {
         if (path.basename(abs) === `${from}.lproj` && isDir(abs)) {
             return { kind: "dir", root: path.dirname(abs), dirSuffix: ".lproj" };
         }
+        if (path.basename(abs) === "values" && isFile(path.join(abs, "strings.xml"))) {
+            return { kind: "android", root: path.dirname(abs) };
+        }
         const base = path.basename(abs);
         const directExt = EXTENSIONS.find((ext) => base === `${from}${ext}`);
         if (directExt && isFile(abs))
@@ -112,6 +137,8 @@ function listLocaleFiles(dir) {
     return out.sort();
 }
 export function sourceFiles(layout, from) {
+    if (layout.kind === "android")
+        return [path.join(layout.root, "values", "strings.xml")];
     return layout.kind === "dir"
         ? listLocaleFiles(path.join(layout.root, dirNameFor(layout, from)))
         : [path.join(layout.root, fileNameFor(layout, from, layout.ext))];
@@ -119,6 +146,8 @@ export function sourceFiles(layout, from) {
 /** A translated file is a real PO, never a template, regardless of what the source was. */
 const outputExt = (ext) => (ext === ".pot" ? ".po" : ext);
 export function targetFile(layout, from, to, src) {
+    if (layout.kind === "android")
+        return path.join(layout.root, `values-${androidQualifier(to)}`, "strings.xml");
     if (layout.kind === "file")
         return path.join(layout.root, fileNameFor(layout, to, outputExt(layout.ext)));
     const rel = path.relative(path.join(layout.root, dirNameFor(layout, from)), src);
@@ -137,6 +166,14 @@ function looksLikeLocale(tag) {
 }
 /** Locale tags already translated alongside `from` in this layout (for `locale-ai check`). */
 export function existingTargets(layout, from) {
+    if (layout.kind === "android") {
+        return fs
+            .readdirSync(layout.root, { withFileTypes: true })
+            .filter((e) => e.isDirectory() && /^values-/.test(e.name) && isFile(path.join(layout.root, e.name, "strings.xml")))
+            .map((e) => tagFromAndroidQualifier(e.name.slice("values-".length)))
+            .filter((tag) => tag !== undefined && tag !== from)
+            .sort();
+    }
     if (layout.kind === "file") {
         const ext = layout.ext;
         const prefix = layout.prefix ? `${layout.prefix}_` : "";
@@ -293,6 +330,119 @@ function readStringsDoc(file, text) {
     };
     return { data, serialize };
 }
+/** Android's `&entity;` plus its own backslash escapes (`\'`, `\"`, `\n`, `\t`) and `<![CDATA[...]]>`. */
+function unescapeAndroidXml(s) {
+    return s
+        .replace(/<!\[CDATA\[([\s\S]*?)\]\]>/g, (_m, inner) => inner)
+        .replace(/&lt;/g, "<")
+        .replace(/&gt;/g, ">")
+        .replace(/&quot;/g, '"')
+        .replace(/&apos;/g, "'")
+        .replace(/&amp;/g, "&")
+        .replace(/\\(.)/g, (_m, c) => ({ n: "\n", t: "\t" })[c] ?? c);
+}
+function escapeAndroidXml(s) {
+    return s
+        .replace(/&/g, "&amp;")
+        .replace(/</g, "&lt;")
+        .replace(/>/g, "&gt;")
+        .replace(/'/g, "\\'")
+        .replace(/"/g, '\\"')
+        .replace(/\n/g, "\\n")
+        .replace(/\t/g, "\\t");
+}
+/**
+ * Android `strings.xml`: `<string>`/`<string-array>`/`<plurals>` elements under a
+ * single `<resources>` root, each optionally preceded by an `<!-- ... -->` translator
+ * comment and marked `translatable="false"` for text that must stay verbatim (an app ID,
+ * a format string nobody localizes). A `<plurals>` element's `quantity="one|other|..."`
+ * items become a nested one/other/... object — the same shape Rails-style nested
+ * plurals use, so `addNestedPluralForms` fills in whatever categories the target
+ * language needs with zero format-specific code.
+ *
+ * This is a simpler round-trip than YAML/.po/.strings: comments are kept from the
+ * source on first write, but a rerun regenerates the whole file from `data` rather
+ * than editing an existing target in place (so a translator's own XML comment edits in
+ * an already-translated file won't survive a rerun) — the same level of fidelity ARB
+ * already has in this project.
+ */
+function readAndroidXmlDoc(file, text) {
+    const openMatch = /<resources\b[^>]*>/.exec(text);
+    const closeMatch = /<\/resources>/.exec(text);
+    if (!openMatch || !closeMatch)
+        throw new Error(`Invalid Android strings.xml in ${file}: missing a <resources> element.`);
+    const before = text.slice(0, openMatch.index + openMatch[0].length);
+    const after = text.slice(closeMatch.index);
+    const body = text.slice(openMatch.index + openMatch[0].length, closeMatch.index);
+    const data = newObj();
+    const entries = [];
+    const elementRe = /(?:(<!--[\s\S]*?-->)\s*\r?\n\s*)?<(string-array|string|plurals)\b([^>]*)(?:\/>|>([\s\S]*?)<\/\2>)/g;
+    for (const m of body.matchAll(elementRe)) {
+        const [, comment, tag, attrsRaw, inner] = m;
+        const nameMatch = /\bname="([^"]*)"/.exec(attrsRaw);
+        if (!nameMatch)
+            continue;
+        const name = unescapeAndroidXml(nameMatch[1]);
+        const translatable = !/\btranslatable\s*=\s*"false"/.test(attrsRaw);
+        if (tag === "string") {
+            entries.push({ comment, name, translatable, kind: "string" });
+            if (translatable)
+                data[name] = unescapeAndroidXml((inner ?? "").trim());
+        }
+        else if (tag === "string-array") {
+            entries.push({ comment, name, translatable, kind: "array" });
+            const items = [...(inner ?? "").matchAll(/<item\b[^>]*>([\s\S]*?)<\/item>/g)].map((im) => unescapeAndroidXml(im[1].trim()));
+            if (translatable)
+                data[name] = items;
+        }
+        else {
+            entries.push({ comment, name, translatable, kind: "plural" });
+            if (translatable) {
+                const plural = newObj();
+                for (const im of (inner ?? "").matchAll(/<item\s+quantity="(\w+)"[^>]*>([\s\S]*?)<\/item>/g)) {
+                    plural[im[1]] = unescapeAndroidXml(im[2].trim());
+                }
+                data[name] = plural;
+            }
+        }
+    }
+    if (entries.length === 0 && body.trim() !== "") {
+        throw new Error(`No <string>/<string-array>/<plurals> entries found in ${file}`);
+    }
+    const indent = /\r?\n([ \t]+)</.exec(body)?.[1] ?? "    ";
+    const keepNewline = finalNewline(text);
+    const raw = newObj();
+    for (const m of body.matchAll(elementRe))
+        raw[/\bname="([^"]*)"/.exec(m[3])[1]] = m[0];
+    const serialize = (d) => {
+        const obj = isObj(d) ? d : newObj();
+        const blocks = [];
+        for (const e of entries) {
+            const attr = e.translatable ? "" : ' translatable="false"';
+            let tag;
+            if (!e.translatable) {
+                tag = raw[e.name].replace(/^<!--[\s\S]*?-->\s*\r?\n\s*/, "");
+            }
+            else if (e.kind === "string") {
+                const v = typeof obj[e.name] === "string" ? obj[e.name] : "";
+                tag = `<string name="${escapeAndroidXml(e.name)}"${attr}>${escapeAndroidXml(v)}</string>`;
+            }
+            else if (e.kind === "array") {
+                const items = Array.isArray(obj[e.name]) ? obj[e.name] : [];
+                const lines = items.map((v) => `${indent}${indent}<item>${escapeAndroidXml(typeof v === "string" ? v : "")}</item>`);
+                tag = `<string-array name="${escapeAndroidXml(e.name)}"${attr}>\n${lines.join("\n")}\n${indent}</string-array>`;
+            }
+            else {
+                const plural = isObj(obj[e.name]) ? obj[e.name] : newObj();
+                const lines = CATEGORIES.filter((c) => typeof plural[c] === "string").map((c) => `${indent}${indent}<item quantity="${c}">${escapeAndroidXml(plural[c])}</item>`);
+                tag = `<plurals name="${escapeAndroidXml(e.name)}"${attr}>\n${lines.join("\n")}\n${indent}</plurals>`;
+            }
+            blocks.push(e.comment ? `${indent}${e.comment}\n${indent}${tag}` : `${indent}${tag}`);
+        }
+        return withNewline(`${before}\n${blocks.join("\n")}\n${after}`, keepNewline);
+    };
+    return { data, serialize };
+}
 /** ARB (Flutter): plain JSON, but "@key" / "@@locale" entries are metadata, not translatable content. */
 function readArbDoc(file, text) {
     let raw;
@@ -606,6 +756,8 @@ export function readLocaleDoc(file, tag, isSource = false, fuzzy = false) {
         return readArbDoc(file, text);
     if (ext === ".strings")
         return readStringsDoc(file, text);
+    if (ext === ".xml")
+        return readAndroidXmlDoc(file, text);
     return readJsonDoc(file, text);
 }
 const isObj = (v) => typeof v === "object" && v !== null && !Array.isArray(v);

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "locale-ai",
-  "version": "0.6.15",
+  "version": "0.6.16",
   "description": "Translate your i18n locale files locally on CPU. Free, offline, no API keys.",
   "author": "Taha KHABOUSS",
   "repository": {