ai-i18n-tools 1.0.0

package/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Waldemar Scudeller Jr.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

package/README.md

@@ -0,0 +1,157 @@
+# ai-i18n-tools
+
+CLI and programmatic toolkit for internationalising JavaScript/TypeScript applications and documentation sites. Extracts UI strings, translates them with LLMs via OpenRouter, and generates locale-ready JSON files for i18next, plus pipelines for markdown, Docusaurus JSON, and (via `features.translateSVG`, `translate-svg`, and the `svg` block) standalone SVG assets.
+
+
+<small>**Read in other languages:** </small>
+<small id="lang-list">[en-GB](./README.md) · [de](./translated-docs/README.de.md) · [es](./translated-docs/README.es.md) · [fr](./translated-docs/README.fr.md) · [hi](./translated-docs/README.hi.md) · [ja](./translated-docs/README.ja.md) · [ko](./translated-docs/README.ko.md) · [pt-BR](./translated-docs/README.pt-BR.md) · [zh-CN](./translated-docs/README.zh-CN.md) · [zh-TW](./translated-docs/README.zh-TW.md)</small>
+
+## Two core workflows
+
+**Workflow 1 - UI Translation** (React, Next.js, Node.js, any i18next project)
+
+Scans source files for `t("…")` calls, builds a master catalog (`strings.json` with optional per-locale **`models`** metadata), translates missing entries per locale via OpenRouter, and writes flat JSON files (`de.json`, `pt-BR.json`, …) ready for i18next.
+
+**Workflow 2 - Document translation** (Markdown, Docusaurus JSON)
+
+Translates `.md` and `.mdx` from each `documentations` block’s `contentPaths` and JSON label files from that block’s `jsonSource` when enabled. Supports Docusaurus-style and flat locale-suffixed layouts per block (`documentations[].markdownOutput`). Shared root `cacheDir` holds the SQLite cache so only new or changed segments are sent to the LLM. **SVG:** enable `features.translateSVG`, add the top-level `svg` block, then use `translate-svg` (also run from `sync` when both are set).
+
+Both workflows share a single `ai-i18n-tools.config.json` file and can be used independently or together. Standalone SVG translation uses `features.translateSVG` plus the top-level `svg` block and runs through `translate-svg` (or the SVG stage inside `sync`).
+
+---
+
+## Installation
+
+The published package is **ESM-only** (`"type": "module"`). Use `import` from Node.js, bundlers, or `import()` — **`require('ai-i18n-tools')` is not supported.**
+
+```bash
+npm install ai-i18n-tools
+# or
+pnpm add ai-i18n-tools
+```
+
+Set your OpenRouter API key:
+
+```bash
+export OPENROUTER_API_KEY=sk-or-v1-your-key-here
+```
+
+---
+
+## Quick start
+
+### Workflow 1 - UI strings
+
+```bash
+# 1. Create config
+npx ai-i18n-tools init
+
+# 2. Extract t("…") calls from source
+npx ai-i18n-tools extract
+
+# 3. Translate to all target locales
+npx ai-i18n-tools translate-ui
+```
+
+Wire i18next in your app using the helpers from `'ai-i18n-tools/runtime'`:
+
+```js
+import i18n from 'i18next';
+import { initReactI18next } from 'react-i18next';
+import uiLanguages from './locales/ui-languages.json';
+import {
+  defaultI18nInitOptions,
+  wrapI18nWithKeyTrim,
+  makeLoadLocale,
+  applyDirection,
+} from 'ai-i18n-tools/runtime';
+
+// Must match sourceLocale in ai-i18n-tools.config.json
+export const SOURCE_LOCALE = 'en-GB';
+
+void i18n.use(initReactI18next).init(defaultI18nInitOptions(SOURCE_LOCALE));
+wrapI18nWithKeyTrim(i18n);
+i18n.on('languageChanged', applyDirection);
+applyDirection(i18n.language);
+
+const localeLoaders = Object.fromEntries(
+  uiLanguages
+    .filter(({ code }) => code !== SOURCE_LOCALE)
+    .map(({ code }) => [code, () => import(`./locales/${code}.json`)])
+);
+export const loadLocale = makeLoadLocale(i18n, localeLoaders, SOURCE_LOCALE);
+export default i18n;
+```
+
+### Workflow 2 - Documentation
+
+```bash
+# 1. Create config for Docusaurus
+npx ai-i18n-tools init -t ui-docusaurus
+
+# 2. Translate all docs
+npx ai-i18n-tools translate-docs
+
+# 3. Check status
+npx ai-i18n-tools status
+```
+
+### Both workflows
+
+```bash
+npx ai-i18n-tools sync   # Extract UI strings, then translate UI strings, SVG, and docs
+```
+
+---
+
+## Runtime helpers
+
+Exported from `'ai-i18n-tools/runtime'` - work in any JS environment, no i18next import required:
+
+| Helper | Description |
+|---|---|
+| `defaultI18nInitOptions(sourceLocale)` | Standard i18next init options for key-as-default setups. |
+| `wrapI18nWithKeyTrim(i18n)` | Wrap `i18n.t` so keys are trimmed before lookup. |
+| `makeLoadLocale(i18n, loaders, sourceLocale)` | Factory for async locale file loading. |
+| `getTextDirection(lng)` | Returns `'ltr'` or `'rtl'` for a BCP-47 code. |
+| `applyDirection(lng, element?)` | Sets `dir` attribute on `document.documentElement`. |
+| `getUILanguageLabel(lang, t)` | Display label for a language menu row (with i18n). |
+| `getUILanguageLabelNative(lang)` | Display label without calling `t()` (header-style). |
+| `interpolateTemplate(str, vars)` | Low-level `{{var}}` substitution on a plain string (used internally; app code should use `t()` instead). |
+| `flipUiArrowsForRtl(text, isRtl)` | Flip `→` to `←` for RTL layouts. |
+
+---
+
+## CLI commands
+
+```
+ai-i18n-tools init [-t ui-markdown|ui-docusaurus]   Create config file
+ai-i18n-tools extract                               Scan source for t("…") calls
+ai-i18n-tools translate-docs [--locale <code>]      Translate documentation (markdown, JSON); see docs for
+                                                    --force-update, --force, --stats, --clear-cache,
+                                                    --prompt-format (xml | json-array | json-object)
+ai-i18n-tools translate-svg [--locale <code>]       Standalone SVG assets (features.translateSVG + config.svg); see --no-cache
+ai-i18n-tools translate-ui [--locale <code>]        Translate UI strings only; see --force, --dry-run
+ai-i18n-tools export-ui-xliff [--locale <code>]     Export UI strings to XLIFF 2.0 (one file per locale); see --untranslated-only, -o
+ai-i18n-tools sync                                  Extract UI strings, then translate UI strings, SVG, and docs
+ai-i18n-tools status                                Translation status per file × locale
+ai-i18n-tools editor                                Open cache/glossary web editor
+ai-i18n-tools cleanup [--dry-run] [--no-backup] [--backup <path>]   Runs sync --force-update, then cleans stale + orphaned cache rows; backs up SQLite by default
+ai-i18n-tools glossary-generate                     Create empty glossary CSV template
+```
+
+All commands accept `-c <config>` (default: `ai-i18n-tools.config.json`), `-v` (verbose), and optional `-w` / `--write-logs [path]` to append console output to a log file (default: under the translation cache directory).
+
+---
+
+## Documentation
+
+- [Getting Started](docs/GETTING_STARTED.md) - full setup guide for both workflows, all CLI flags, and config field reference.
+- [Package Overview](docs/PACKAGE_OVERVIEW.md) - architecture, internals, programmatic API, and extension points.
+- [AI Agent Context](docs/ai-i18n-tools-context.md) - concise project context for agents and maintainers making code or config changes.
+
+---
+
+## License
+
+MIT © [Waldemar Scudeller Jr.](https://github.com/wsj-br)

package/dist/api/openrouter.d.ts

@@ -0,0 +1,115 @@
+import type { I18nConfig } from "../core/types.js";
+import { type BatchTranslationResult, type ChatResponse, type Segment, type TranslationResult } from "../core/types.js";
+import { type DocumentBatchResponseFormat, type DocumentPromptContentType } from "../core/prompt-builder.js";
+import type { Logger } from "../utils/logger.js";
+/** Thrown when every model in the chain fails for {@link OpenRouterClient.translateDocumentBatch}. */
+export declare class DocumentBatchAllModelsFailedError extends Error {
+    readonly details: {
+        systemPrompt: string;
+        userContent: string;
+        lastModel: string;
+        lastError: unknown;
+        /** HTTP response body text when the model returned content but parsing failed. */
+        lastRawAssistantContent?: string;
+    };
+    constructor(message: string, details: {
+        systemPrompt: string;
+        userContent: string;
+        lastModel: string;
+        lastError: unknown;
+        /** HTTP response body text when the model returned content but parsing failed. */
+        lastRawAssistantContent?: string;
+    });
+}
+export interface OpenRouterClientOptions {
+    config: Pick<I18nConfig, "openrouter" | "sourceLocale" | "localeDisplayNames">;
+    apiKey?: string;
+    /**
+     * When set and non-empty, use this ordered model list instead of resolving from `config.openrouter`
+     * (e.g. UI translation with `ui.preferredModel` prepended to the global list).
+     */
+    translationModels?: string[];
+    /** Append request/response JSON when set. */
+    debugTrafficFilePath?: string | null;
+    logger?: Logger;
+    httpReferer?: string;
+    xTitle?: string;
+}
+/**
+ * OpenRouter chat client with ordered `translationModels` fallback chain.
+ */
+export declare class OpenRouterClient {
+    private readonly apiKey;
+    private readonly baseUrl;
+    private readonly modelsToTry;
+    private readonly maxTokens;
+    private readonly temperature;
+    private readonly debugTrafficFilePath;
+    private readonly logger?;
+    private readonly localeDisplayNames;
+    private readonly sourceLanguageLabel;
+    private readonly httpReferer;
+    private readonly xTitle;
+    constructor(opts: OpenRouterClientOptions);
+    getConfiguredModels(): readonly string[];
+    /**
+     * BCP-47 locale id plus English display name for LLM prompts (e.g. `pt-BR: Brazilian Portuguese`).
+     * Uses a colon so it reads as a clear key–value / label field (common in prompt instructions);
+     * it does not clash with hyphens inside tags (`zh-CN`, `pt-BR`) the way a bare `-` can.
+     * Order: `localeDisplayNames` from config, else `englishLanguageNameForLocale` (Intl), else raw code.
+     */
+    private languageLabelForPrompt;
+    private appendDebugLog;
+    private handleRateLimit;
+    /** Match doc-translate log lines: two-space indent, locale, filename. */
+    private warnModelSwitch;
+    private extractUsage;
+    private toOpenRouterMessages;
+    /** Single HTTP call for one model (with one 429 retry). */
+    private fetchCompletion;
+    chat(messages: Array<{
+        role: "system" | "user" | "assistant";
+        content: string;
+    }>, options?: {
+        startModelIndex?: number;
+        docLogContext?: {
+            locale: string;
+            relativePath: string;
+        };
+    }): Promise<ChatResponse>;
+    stripTranslateTags(content: string): string;
+    translateDocumentSegment(content: string, targetLocale: string, glossaryHints: string[], options?: {
+        startModelIndex?: number;
+        contentType?: DocumentPromptContentType;
+        docLogContext?: {
+            locale: string;
+            relativePath: string;
+        };
+    }): Promise<TranslationResult>;
+    translateDocumentBatch(segments: Segment[], locale: string, glossaryHints?: string[], options?: {
+        startModelIndex?: number;
+        contentType?: DocumentPromptContentType;
+        responseFormat?: DocumentBatchResponseFormat;
+        /** When set, log model fallback warnings (translate-docs style). */
+        docLogContext?: {
+            relativePath: string;
+        };
+    }): Promise<BatchTranslationResult>;
+    /**
+     * UI strings: translate a batch of strings and return a JSON array response, with model fallback chain.
+     */
+    translateUIBatch(texts: string[], targetLocale: string, options?: {
+        startModelIndex?: number;
+        glossaryHints?: string[];
+    }): Promise<{
+        translations: string[];
+        model: string;
+        usage: {
+            inputTokens: number;
+            outputTokens: number;
+            totalTokens: number;
+        };
+        cost?: number;
+    }>;
+}
+//# sourceMappingURL=openrouter.d.ts.map

package/dist/api/openrouter.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"openrouter.d.ts","sourceRoot":"","sources":["../../src/api/openrouter.ts"],"names":[],"mappings":"AAEA,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,kBAAkB,CAAC;AACnD,OAAO,EACL,KAAK,sBAAsB,EAC3B,KAAK,YAAY,EACjB,KAAK,OAAO,EACZ,KAAK,iBAAiB,EAEvB,MAAM,kBAAkB,CAAC;AAM1B,OAAO,EAQL,KAAK,2BAA2B,EAChC,KAAK,yBAAyB,EAC/B,MAAM,2BAA2B,CAAC;AACnC,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,oBAAoB,CAAC;AA0CjD,sGAAsG;AACtG,qBAAa,iCAAkC,SAAQ,KAAK;aAGxC,OAAO,EAAE;QACvB,YAAY,EAAE,MAAM,CAAC;QACrB,WAAW,EAAE,MAAM,CAAC;QACpB,SAAS,EAAE,MAAM,CAAC;QAClB,SAAS,EAAE,OAAO,CAAC;QACnB,kFAAkF;QAClF,uBAAuB,CAAC,EAAE,MAAM,CAAC;KAClC;gBARD,OAAO,EAAE,MAAM,EACC,OAAO,EAAE;QACvB,YAAY,EAAE,MAAM,CAAC;QACrB,WAAW,EAAE,MAAM,CAAC;QACpB,SAAS,EAAE,MAAM,CAAC;QAClB,SAAS,EAAE,OAAO,CAAC;QACnB,kFAAkF;QAClF,uBAAuB,CAAC,EAAE,MAAM,CAAC;KAClC;CAKJ;AAED,MAAM,WAAW,uBAAuB;IACtC,MAAM,EAAE,IAAI,CAAC,UAAU,EAAE,YAAY,GAAG,cAAc,GAAG,oBAAoB,CAAC,CAAC;IAC/E,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB;;;OAGG;IACH,iBAAiB,CAAC,EAAE,MAAM,EAAE,CAAC;IAC7B,6CAA6C;IAC7C,oBAAoB,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IACrC,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,WAAW,CAAC,EAAE,MAAM,CAAC;IACrB,MAAM,CAAC,EAAE,MAAM,CAAC;CACjB;AAED;;GAEG;AACH,qBAAa,gBAAgB;IAC3B,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAS;IAChC,OAAO,CAAC,QAAQ,CAAC,OAAO,CAAS;IACjC,OAAO,CAAC,QAAQ,CAAC,WAAW,CAAW;IACvC,OAAO,CAAC,QAAQ,CAAC,SAAS,CAAS;IACnC,OAAO,CAAC,QAAQ,CAAC,WAAW,CAAS;IACrC,OAAO,CAAC,QAAQ,CAAC,oBAAoB,CAAgB;IACrD,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAC,CAAS;IACjC,OAAO,CAAC,QAAQ,CAAC,kBAAkB,CAAyB;IAC5D,OAAO,CAAC,QAAQ,CAAC,mBAAmB,CAAS;IAC7C,OAAO,CAAC,QAAQ,CAAC,WAAW,CAAS;IACrC,OAAO,CAAC,QAAQ,CAAC,MAAM,CAAS;gBAEpB,IAAI,EAAE,uBAAuB;IAmCzC,mBAAmB,IAAI,SAAS,MAAM,EAAE;IAIxC;;;;;OAKG;IACH,OAAO,CAAC,sBAAsB;IAa9B,OAAO,CAAC,cAAc;YAcR,eAAe;IAQ7B,yEAAyE;IACzE,OAAO,CAAC,eAAe;IAcvB,OAAO,CAAC,YAAY;IAepB,OAAO,CAAC,oBAAoB;IAoB5B,2DAA2D;YAC7C,eAAe;IAkFvB,IAAI,CACR,QAAQ,EAAE,KAAK,CAAC;QAAE,IAAI,EAAE,QAAQ,GAAG,MAAM,GAAG,WAAW,CAAC;QAAC,OAAO,EAAE,MAAM,CAAA;KAAE,CAAC,EAC3E,OAAO,CAAC,EAAE;QACR,eAAe,CAAC,EAAE,MAAM,CAAC;QACzB,aAAa,CAAC,EAAE;YAAE,MAAM,EAAE,MAAM,CAAC;YAAC,YAAY,EAAE,MAAM,CAAA;SAAE,CAAC;KAC1D,GACA,OAAO,CAAC,YAAY,CAAC;IA+BxB,kBAAkB,CAAC,OAAO,EAAE,MAAM,GAAG,MAAM;IAOrC,wBAAwB,CAC5B,OAAO,EAAE,MAAM,EACf,YAAY,EAAE,MAAM,EACpB,aAAa,EAAE,MAAM,EAAE,EACvB,OAAO,CAAC,EAAE;QACR,eAAe,CAAC,EAAE,MAAM,CAAC;QACzB,WAAW,CAAC,EAAE,yBAAyB,CAAC;QACxC,aAAa,CAAC,EAAE;YAAE,MAAM,EAAE,MAAM,CAAC;YAAC,YAAY,EAAE,MAAM,CAAA;SAAE,CAAC;KAC1D,GACA,OAAO,CAAC,iBAAiB,CAAC;IAiCvB,sBAAsB,CAC1B,QAAQ,EAAE,OAAO,EAAE,EACnB,MAAM,EAAE,MAAM,EACd,aAAa,GAAE,MAAM,EAAO,EAC5B,OAAO,CAAC,EAAE;QACR,eAAe,CAAC,EAAE,MAAM,CAAC;QACzB,WAAW,CAAC,EAAE,yBAAyB,CAAC;QACxC,cAAc,CAAC,EAAE,2BAA2B,CAAC;QAC7C,oEAAoE;QACpE,aAAa,CAAC,EAAE;YAAE,YAAY,EAAE,MAAM,CAAA;SAAE,CAAC;KAC1C,GACA,OAAO,CAAC,sBAAsB,CAAC;IA0HlC;;OAEG;IACG,gBAAgB,CACpB,KAAK,EAAE,MAAM,EAAE,EACf,YAAY,EAAE,MAAM,EACpB,OAAO,CAAC,EAAE;QAAE,eAAe,CAAC,EAAE,MAAM,CAAC;QAAC,aAAa,CAAC,EAAE,MAAM,EAAE,CAAA;KAAE,GAC/D,OAAO,CAAC;QACT,YAAY,EAAE,MAAM,EAAE,CAAC;QACvB,KAAK,EAAE,MAAM,CAAC;QACd,KAAK,EAAE;YAAE,WAAW,EAAE,MAAM,CAAC;YAAC,YAAY,EAAE,MAAM,CAAC;YAAC,WAAW,EAAE,MAAM,CAAA;SAAE,CAAC;QAC1E,IAAI,CAAC,EAAE,MAAM,CAAC;KACf,CAAC;CA4CH"}